aboutsummaryrefslogtreecommitdiff
path: root/xorg-server/hw/dmx/doc/scaled.xml
diff options
context:
space:
mode:
authormarha <marha@users.sourceforge.net>2011-11-25 08:22:48 +0100
committermarha <marha@users.sourceforge.net>2011-11-25 08:22:48 +0100
commita0b4a1330be6a36ad095222d2ea83927cd33514d (patch)
tree616920ca608751e843c92c9815069f43789e3097 /xorg-server/hw/dmx/doc/scaled.xml
parent45710577f374972946a8eb37833a9c94e5a299bf (diff)
downloadvcxsrv-a0b4a1330be6a36ad095222d2ea83927cd33514d.tar.gz
vcxsrv-a0b4a1330be6a36ad095222d2ea83927cd33514d.tar.bz2
vcxsrv-a0b4a1330be6a36ad095222d2ea83927cd33514d.zip
mesa xserver pixman git update 25 nov 2011
Diffstat (limited to 'xorg-server/hw/dmx/doc/scaled.xml')
-rw-r--r--xorg-server/hw/dmx/doc/scaled.xml1452
1 files changed, 727 insertions, 725 deletions
diff --git a/xorg-server/hw/dmx/doc/scaled.xml b/xorg-server/hw/dmx/doc/scaled.xml
index 575cafd9d..30c9b9f50 100644
--- a/xorg-server/hw/dmx/doc/scaled.xml
+++ b/xorg-server/hw/dmx/doc/scaled.xml
@@ -1,725 +1,727 @@
-<?xml version="1.0" encoding="ISO-8859-1"?>
-<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN"
- "http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd" [
-]>
- <article>
-
- <articleinfo>
- <!-- Title information -->
- <title>Scaled Window Support in DMX</title>
- <authorgroup>
- <author><firstname>Kevin E.</firstname><surname>Martin</surname></author>
- <author><firstname>Rickard E.</firstname><surname>Faith</surname></author>
- </authorgroup>
- <pubdate>15 October 2003 (created 19 September 2003)</pubdate>
- <abstract>
- <para>
- This document investigates the possibility of adding scaled window
- support to the DMX X server, thereby allowing a window or some
- selected part of the logical DMX area to be displayed using a
- scaling factor. For example, this might allow the contents of a
- window to be magnified for easier viewing. In particular, scaling
- for the VNC client is explored. <emphasis remap="it">Copyright 2003
- by Red Hat, Inc., Raleigh, North Carolina</emphasis>
- </para>
- </abstract>
- </articleinfo>
-
- <!-- Begin the document -->
- <sect1><title>Introduction</title>
- <sect2><title>DMX</title>
- <para>
- The DMX X server (Xdmx) is a proxy server that is designed
- to allow X servers on multiple machines to be combined into
- a single multi-headed X server. Combined with Xinerama,
- these heads can appear as a single very high-resolution
- screen. Typical applications include the creation of a
- video wall with 16 1280x1024 displays arranged in a
- rectangle, for a total resolution of of 5120x4096.
- </para>
- </sect2>
- <sect2><title>Problem Statement</title>
- <para>
- Applications displayed on a physically large video wall that
- provides high pixel-resolution may be difficult to see,
- especially if the application is designed for use on a
- typical desktop computer with a relatively small display
- located close to the human operator. The goal of this paper
- is to describe and discuss solutions to this problem.
- </para>
- <para>
- The original driving problem for this work is to provide
- scaling for the <command>vncviewer</command> application when
- displayed using DMX (VNC scaling is currently available only
- with the Windows client, and there is no plan to extend that
- capability to other clients). While this specific problem
- will be addressed in this paper, the general solution space
- will also be explored, since this may lead to a good
- solution not only for <command>vncviewer</command> but also for
- other applications.
- </para>
- </sect2>
- <sect2><title>Task</title>
- <para>
- For reference, here is the original description of the task
- this paper addresses:
- <itemizedlist>
- <listitem><para>Scaled window support (for VNC)
- <itemizedlist>
- <listitem><para>
- Investigate possibility of implementing a "scaled
- window" extension:
- <itemizedlist>
- <listitem><para>
- Add XCreateScaledWindow call that could be used
- in place of XCreateWindow
- </para></listitem>
- <listitem><para>
- All primitives drawn to scaled window would be
- scaled by appropriate (integral?) scaling factor
- </para></listitem>
- </itemizedlist>
- </para></listitem>
- <listitem><para>
- Alternate approach: special case VNC support
- </para></listitem>
- </itemizedlist>
- </para></listitem>
- </itemizedlist>
- </para>
- </sect2>
- </sect1>
-
- <sect1><title>Previous Work</title>
- <para>
- This section reviews relevant previous work.
- </para>
- <sect2><title>VNC</title>
- <sect3><title>Scaling under VNC</title>
- <para>
- When using the <command>vncviewer</command> program for Windows, it
- is possible to specify a scaling factor (as numerator and
- denominator). When scaling is in effect, the viewer
- software uses StretchBlt (instead of BitBlt) to display
- the pixels for the user. When this call is made, the
- viewer already has received all of the pixel information
- (at full unscaled resolution).
- </para>
- <para>
- The scaling in VNC is primitive. It does not conserve
- bandwidth, it does not treat textual information
- differently (i.e., by using a suitably scaled font), and
- it does not provide any anti-aliasing other than that
- provided by the underlying (Windows-only) system library.
- </para>
- </sect3>
- </sect2>
- <sect2><title>The X Video Extension</title>
- <para>
- The X Video Extension is a widely-available extension to the
- X11 protocol that provides support for streaming video.
- Integral to this support is the ability to arbitrarily scale
- the output. In version 2.2 of the X Video specification,
- support for scaled still images was provided, using both
- shared memory and traditional transport. The API for this
- support uses calls that are quite similar to XCreateWindow,
- XPutImage, and XShmPutImage. Currently, most of the drivers
- implemented in XFree86 only support data in various YUV
- formats. However, several modern video adaptors support RGB
- as well.
- </para>
- <para>
- Note, though, that the target output for this scaling is an
- overlay plane -- so X Video provides functionality that is
- fundamentally different from that provided by the Windows
- StrechBlt call.
- </para>
- </sect2>
- </sect1>
-
- <sect1><title>Possible Solutions</title>
- <para>
- This section briefly discusses possible solutions, including
- major advantages and disadvantages from both the
- implementation and the end-user programmer standpoint.
- </para>
- <sect2><title>VNC-like Scaling</title>
- <sect3><title>Software Scaling</title>
- <para>
- The <command>vncviewer</command> application could be modified to
- provide software scaling. This is not a general solution,
- but it does solve one of the goals of this work.
- </para>
- <para>
- A prototype of this solution was implemented and a patch
- against <filename>vnc-3.3.7-unixsrc</filename> is available in the
- <filename>dmx/external</filename> directory. Because of limited time
- available for this work, all of the edge cases were not
- considered and the solution works well mainly for integer
- scaling.
- </para>
- <para>
- Currently, <command>vncviewer</command> writes to the X display
- with XPutImage, XCopyArea, and XFillRectangle. All
- instances of these calls have to be aware of scaling
- and must round correctly. In the prototype solution,
- rounding is incorrect and can cause artifacts.
- </para>
- <para>
- A better solution would be to cache all updates to the
- desktop image in <command>vncviewer</command> and only send the
- damaged area to the X display with XPutImage. This would
- allow the damaged area to be computed so that rounding
- errors do not create artifacts. This method is probably
- similar to what is used in the Window client. (The whole
- VNC suite is being re-written in C++ and the forthcoming
- version 4 has not been evaluated.)
- </para>
- </sect3>
- <sect3><title>Scaling with the X Video Extension</title>
- <para>
- The scaling in the Windows <command>vncviewer</command> application
- makes use of a scaled blit that is supplied by the
- underlying system library. Several video cards currently
- provide support for a scaled blit, and some X servers
- (including XFree86) expose this capability to applications
- via the XvPutImage interface of the X Video Extension.
- The capability exposed by XvPutImage results in the scaled
- image being drawn to an overlay plane. Most video cards
- also provide support for a scaled blit into the normal
- output planes, but this is not exposed via XvPutImage.
- </para>
- <para>
- The <command>vncviewer</command> program could be modified to use
- the X Video Extension to provide scaling under X11 that is
- similar to the scaling currently provided under Windows.
- Unfortunately, Xdmx does not currently export the X Video
- Extension, so this would not provide an immediate solution
- usable with DMX.
- </para>
- <para>
- A very early-stage proof-of-concept prototype was
- implemented and a preliminary patch against
- <filename>vnc-3.3.7-unixsrc</filename> is available in the
- <filename>dmx/external</filename> directory. This prototype was
- implemented to better understand the problems that must be
- solved to make this solution viable:
- <itemizedlist>
- <listitem><para>
- As noted under the software scaling section above,
- <command>vncviewer</command> writes to the X display with
- several different calls. These calls write to the
- normal output planes and are compatible with
- XvPutImage, which writes to an overlay plane. To
- eliminate artifacts caused by this problem,
- <command>vncviewer</command> should be modified so that a cached
- copy of the desktop is available, either as a
- client-side image or a server-side off-screen pixmap,
- so that XvPutImage would be the only method for
- writing to the X display.
- </para></listitem>
- <listitem>
- <para>
- Although several modern graphics adaptors support
- hardware scaling using an RGB format (e.g., ATI
- Radeon, nVidia, etc.), XFree86 drivers typically
- only implement YUV formats. YUV generally compress
- the pixel information in some way. For example, two
- commonly implemented formats, YUY2 and UYVY provide
- intensity information for every RGB pixel, but only
- provide chroma and luminance information for pairs
- of horizontal pixels. Since VNC uses
- pixel-resolution for communicating updates on the
- wire, additional artifacts are introduced (because
- there may not be enough information from the wire to
- update a pair of pixels).
- </para>
- <para>
- Further, the well-known problem with YUV encoding
- is even more evident when the image is a desktop
- instead of a movie. For example, consider a
- 1-pixel-wide vertical window border. If the border
- changes in color but not intensity (e.g., because a
- window manager uses color to indicate focus), there
- may or may not be a change in the YUY2 image,
- depending on the algorithm used for RGB to YUV
- conversion and on how the border pixel is ordered in
- the pair of pixels used by the algorithm.
- </para>
- <para>
- Many of these artifacts could be eliminated if
- <command>vncviewer</command> cached a complete RGB image of
- the desktop, and only did the conversion to YUV for
- properly aligned areas of damage. The remaining artifacts
- could be eliminated if an RGB format was used with X
- Video (which may require the extension of existing
- XFree86 drivers to support RGB).
- </para>
- </listitem>
- <listitem><para>
- Most modern video cards support exactly one overlay
- plane that is suitable for use with X Video.
- Therefore, only one application can use X Video at any
- given time. This is a severe limitation in a desktop
- environment.
- </para></listitem>
- </itemizedlist>
- </para>
- <sect4><title>Implementing the X Video Extension for DMX</title>
- <para>
- The user-level API for X Video is fairly simple, but the
- underlying support required for the full specification
- is large. However, since the API provides a method to
- query supported capabilities, a usable subset of X
- Video can be implemented that would support XvPutImage
- and little else. This would require support for the
- following:
- <itemizedlist>
- <listitem><para>
- X Video Extension API calls, including the
- following:
- <itemizedlist>
- <listitem><para>XvQueryExtension</para></listitem>
- <listitem><para>XvQueryAdaptors</para></listitem>
- <listitem><para>XvQueryPortAttributes</para></listitem>
- <listitem><para>XvFreeAdaptorInfo</para></listitem>
- <listitem><para>XvListImageFormats</para></listitem>
- <listitem><para>XvGrabPort</para></listitem>
- <listitem><para>XvCreateImage</para></listitem>
- <listitem><para>XvPutImage</para></listitem>
- <listitem><para>XvShmCreateImage</para></listitem>
- <listitem><para>XvShmPutImage</para></listitem>
- </itemizedlist>
- </para></listitem>
- <listitem><para>
- Support for querying back-end X Video Extension
- capabilities.
- </para></listitem>
- <listitem><para>
- Support for sending the image to the back-ends.
- Because X Video requires sending full images, there
- may be a trade-off between bandwidth limitations and
- additional complexity to divide the image up such
- that is scales properly.
- </para></listitem>
- <listitem><para>
- Possible support for a software fall-back. For
- example, if all of the back-ends do not support the X
- Video Extension, software scaling can be implemented
- such that the image is sent to the back-end with
- XPutImage. This pathway would have poor
- performance.
- </para></listitem>
- </itemizedlist>
- </para>
- </sect4>
- <sect4><title>Supporting RGB formats for the X Video Extension</title>
- <para>
- Assuming an XFree86 driver already supports the X Video
- Extension, and assuming the target hardware supports an
- RGB format, then adding support for that format is
- relatively simple and straightforward.
- </para>
- </sect4>
- </sect3>
- <sect3><title>Scaling with an XPutImageScaled Extension</title>
- <para>
- Instead of (or in addition to) implementing the X Video
- Extension in DMX, one obvious solution would be to
- implement a new extension that provides access to
- hardware-assisted scaled blits, similar to the StretchBlt
- call available under Windows. This call would scale RGB
- images and would not use the overlay plane (unlike the X
- Video Extension).
- </para>
- <para>
- This approach has many of the same advantages and
- disadvantages as the XCopyAreaScaled Extension, discussed
- in the next section. Discussion of XPutImageScaled is
- deferred in favor of XCopyAreaScaled for the following
- reasons:
- <itemizedlist>
- <listitem><para>
- XPutImageScaled can be emulated with XCopyAreaScaled
- by first using XPutImage to copy the image to an
- off-screen pixmap, and then calling XCopyAreaScaled
- between that off-screen pixmap and the target
- drawable.
- </para></listitem>
- <listitem><para>
- Since XCopyAreaScaled would copy between two areas of
- on-screen or off-screen memory, it has additional uses
- and can be viewed as efficiently providing a superset
- of XPutImageScaled functionality.
- </para></listitem>
- </itemizedlist>
- </para>
- </sect3>
- <sect3><title>Scaling with an XCopyAreaScaled Extension</title>
- <para>
- As noted in the previous section, because XCopyAreaScaled
- provides a superset of the functionality provided by
- XPutImageScaled, we will consider this extension instead.
- </para>
- <para>
- First, XCopyAreaScaled would provide for RGB scaling
- between pixmaps (i.e., on-screen or off-screen areas of
- memory that reside on the video card). Unlike the X Video
- Extension, which writes into an overlay plane,
- XCopyAreaScaled would write into the non-overlay areas of
- the screen. Key points to consider are as follows:
- <itemizedlist>
- <listitem><para>
- Because different planes are involved, the two scaling
- operations are usually implemented in hardware
- differently, so an XCopyAreaScaled extension could be
- added in a manner that would neither conflict with nor
- interact with the X Video extension in any way.
- </para></listitem>
- <listitem><para>
- The XCopyAreaScaled extension provides new
- functionality that the X Video Extension does not
- provide. Based on anecdotal feedback, we believe that
- many people outside the DMX and VNC communities would
- be excited about this extension.
- </para></listitem>
- <listitem><para>
- The main drawback to this extension is that it is new
- and needs to be implemented at the driver level in
- XFree86 for each video card to be supported. At the
- present time, it is more likely that the X Video
- Extension will be implemented for a particular piece
- hardware because the X Video extension has multimedia
- uses. However, over time, we would expect the
- XCopyAreaScaled extension to be implemented along with
- the X Video extension, especially if it becomes
- popular.
- </para></listitem>
- <listitem><para>
- Another drawback is that not all modern cards provide
- support for a simple scaled blit operation. However,
- these cards usually do provide a 3D pipeline which
- could be used to provide this functionality in a
- manner that is transparent to the client application
- that is using the XCopyAreaScaled extension. However,
- this implementation pathway would make this extension
- somewhat more difficult to implement on certain cards.
- </para></listitem>
- </itemizedlist>
- </para>
- </sect3>
- <sect3><title>Scaling with OpenGL</title>
- <para>
- Another general solution to the scaling problem is to use
- the texture scaling found in all 3D hardware. This
- ability is already exposed through OpenGL and can be
- exploited by clients without X server modification (i.e.,
- other than the ability to support OpenGL). An application
- using OpenGL would transmit the non-scaled image to the X
- server as a texture, and would then display a single
- non-transformed rect using that texture. This also works
- around the single overlay problem with the X Video
- Extension as well as the need to implement additional
- scaled primitive extensions.
- </para>
- <para>
- The downside is that most OpenGL implementations require
- power of 2 texture sizes and this can be very wasteful of
- memory if, for example, the application needs to scale a
- 1025x1025 image, which would require a 2048x2048 texture
- area (even a 640x480 image would require a 1024x512
- texture). Another downside is that some OpenGL
- implementations have a limited about of texture memory and
- cannot handle textures that are very large. For example,
- they might limit the texture size to 1024x1024.
- </para>
- </sect3>
- </sect2>
- <sect2><title>Application-transparent Scaling for DMX
- </title><sect3><title>Back-end Scaling Without Disconnect/Reconnect</title>
- <para>
- VNC does scaling on the client side (in the
- <command>vncviewer</command> application). Implementing a similar
- solution for DMX would require support in the back-end X
- servers and, therefore, is not a general solution.
- </para>
- <para>
- XFree86 already implements some support for "scaling" that
- could be used with DMX: if, in the XF86Config file,
- multiple Modes are listed in the Display Subsection of the
- Screen Section, then pressing Ctrl-Alt-Plus and
- Ctrl-Alt-Minus can be used to iterate through the listed
- modes. The display dimensions will change to the
- dimensions in the Modes line, but the logical dimensions
- of the X server (i.e., the dimensions that Xdmx knows
- about) will not change.
- </para>
- <para>
- Further, the dimensions of the XFree86 display are under
- software control (via the XFree86-VidModeExtension), so
- the Xdmx server could change the screen dimensions on a
- per-display basis, thereby scaling the information on part
- of that display.
- </para>
- <para>
- However, this scaling appears to have limited use. For
- example, assume a 4 by 4 display wall consisting of 16
- 1280x1024 displays. If all of the back-end servers were
- simultaneously configured to display 640x480, the left
- hand corner of each display would be magnified, but the
- composite result would be unreadable. Magnifying one
- display at a time could be usable, but could have limited
- utility, since the result would still be no larger than a
- single display.
- </para>
- </sect3>
- <sect3><title>Back-end Scaling With Disconnect/Reconnect</title>
- <para>
- Disconnect and reconnect features are not currently
- supported in DMX, but are scheduled to be implemented in
- the future. These features, combined with the
- XFree86-VidModeExtension Extension, would allow an
- application to do the following:
- <itemizedlist>
- <listitem><para>
- Disconnect a specific back-end server (via the DMX
- Extension),
- </para></listitem>
- <listitem><para>
- reconfigure the XFree86 back-end server resolution,
- and
- </para></listitem>
- <listitem><para>
- reconnect the back-end server to DMX -- at a new
- origin with the new screen resolution.
- </para></listitem>
- </itemizedlist>
- </para>
- <para>
- For example, consider a display wall consisting of 16
- 1280x1024 displays with a total resolution of 5120x4096.
- All of the screens could be disconnected, repositioned,
- and reconnected each at a resolution of 640x480. The
- total resolution of the display wall would be 2560x1920,
- allowing a view of a selected area approximately
- one-fourth of the size of the DMX display. This change
- would be completely application independent (except,
- perhaps, for a DMX-aware window manager). When work at
- the increased resolution was completed, the back-end
- servers could be disconnected, reconfigured, and
- reconnected for the original 5120x4096 view.
- </para>
- <para>
- Support for this type of scaling can be implemented in a
- DMX-aware X11 client assuming the DMX server support
- arbitrary disconnect and reconnect semantics. Because
- this application cannot be written before
- disconnect/reconnect is implemented, this solution will
- not be discussed further in this paper.
- </para>
- </sect3>
- <sect3><title>Server-side Scaling</title>
- <para>
- In earlier versions of DMX, a frame buffer was maintained
- on the server side, and XPutImage was used to move the
- information from the server to the client (similar to some
- early VNC implementations). The use of a server-side
- frame buffer would allow the server to do scaling, but is
- not a recommended solution because of overall performance
- issues and server-side memory issues (i.e., the frame
- buffer would be very large for large display walls).
- </para>
- <para>
- Exploration of this path is not recommended.
- </para>
- </sect3>
- </sect2>
- <sect2><title>XCreateScaledWindow API</title>
- <para>
- The implementation of X Video Extension in DMX, and the use
- of XvPutImage by applications requiring scaling requires
- significant changes in DMX Further, XvPutImage is,
- essentially a scaled blit, and it is only useful for
- applications which are already using (or can be modified to
- use) XPutImage. Therefore, a more general API will be
- discussed as another possibility.
- </para>
- <para>
- X applications typically create windows with the
- XCreateWindow call. A new extension could provide an
- XCreateScaledWindow call that could be used in place of the
- XCreateWindow call and be otherwise transparent to the
- application. This would allow applications, even those that
- do not depend on XPutImage, to take advantage of window
- scaling. In this section we describe how the call would
- work, what transparency it provides, and how to solve the
- potential problems that transparency creates.
- </para>
- <sect3><title>XCreateWindow</title>
- <para>
- The XCreateWindow call takes width and height as
- parameters. An XCreateScaledWindow call could take all
- the same parameters, with the addition of a scaling factor.
- </para>
- </sect3>
- <sect3><title>XSetWindowAttributes</title>
- <para>
- An X11 window has several attributes that would have to be
- scaled:
- <itemizedlist>
- <listitem><para>Background and border pixmaps</para></listitem>
- <listitem><para>Border width</para></listitem>
- <listitem><para>Cursor</para></listitem>
- </itemizedlist>
- </para>
- </sect3>
- <sect3><title>XGetWindowAttributes, XGetGeometry</title>
- <para>
- For transparency, calls that query the window attributes
- should return unscaled information. This suggests that
- all unscaled pixmaps and window attributes should be
- cached.
- </para>
- <para>
- Unfortunately, a window manager requires the scaled
- geometry to properly decorate the window. The X server
- can probably determine which client is acting as the
- window manager (e.g., because that client will select
- events that are used exclusively by the window manager).
- However, other Scaled Window Extension aware clients may
- also need to determine the scaled geometry. Therefore, at
- least two additional extension calls should be
- implemented: XGetScaledWindowAttributes and
- XGetScaledGeometry.
- </para>
- </sect3>
- <sect3><title>Popup and Child window positions</title>
- <para>
- Some applications may position popup and child windows
- based on an unscaled notion of the main window geometry.
- In this case, additional modifications to the client would
- be required.
- </para>
- </sect3>
- <sect3><title>Events</title>
- <para>
- Most events (e.g., for mouse motion) return information
- about the coordinates at which the even occurred. These
- coordinates would have to be modified so that unscaled
- values were presented to the client.
- </para>
- </sect3>
- <sect3><title>Implementation</title>
- <para>
- There are many implementation issues, some of which are
- similar to the issues involved in implementing the X Video
- Extension for DMX. The window contents must be scaled,
- either by performing all operations to a frame buffer and
- then writing the image to the display (perhaps using
- hardware scaling support), or by modifying all of the
- various drawing operations to perform scaling. Because of
- the complexity involved, the frame buffer option is
- recommended.
- </para>
- </sect3>
- </sect2>
- </sect1>
-
- <sect1><title>Conclusion and Recommendations
- </title><para>
- We recommend a three phase implementation strategy, based on
- how an application could be written to take advantage of
- scaling:
- <orderedlist>
- <listitem>
- <para>
- The XCopyAreaScaled extension should be implemented, since
- this is the ideal solution for applications like VNC, and
- since making use of this extension will require minimal
- changes to applications that already use XPutImage or
- XCopyArea.
- </para>
- <para>
- The initial implementation work would include the design
- of the X protocol extension, writing this up in the
- usual format for extension documentation, implementation
- of the protocol transport pieces in XFree86,
- implementation of a software fall-back in XFree86 and
- DMX, one example hardware implementation for XFree86,
- and implementation of support for this extension in DMX.
- </para>
- <para>
- We suggest implementing the extension first on the ATI
- Radeon cards. However, since these cards do not provide
- a 2D scaled blit primitive, the implementation would
- have to make use of the 3D texture engine to emulate a
- scaled blit. This is recommended, since other modern
- graphics cards also do not provide a simple 2D scaled
- blit operation and an example of the more difficult
- implementation pathway would be helpful to others.
- </para>
- </listitem>
- <listitem>
- <para>
- Until XCopyAreaScaled is widely supported, applications
- that require scaling will have to fall back to another
- scaling method. We suggest OpenGL as the first fall-back
- method because it is widely available and supported by
- DMX.
- </para>
- <para>
- A project centered around OpenGL-based scaling would
- implement this scaling in VNC as an example. This work
- would include re-writing the <command>vncviewer</command>
- rendering engine to cache a master copy of the desktop
- image for all operations.
- </para>
- </listitem>
- <listitem>
- <para>
- Since OpenGL is not implemented everywhere, and may not
- provide hardware-assisted performance in every
- implementation, an application that requires scaling
- should also fall back to using the X Video Extension.
- </para>
- <para>
- This project would add support for the X Video Extension
- to DMX and would add support to VNC to take advantage of
- this extension without introducing artifacts. This
- would require modifying the <command>vncviewer</command> rendering
- engine to cache a master copy of the desktop image for
- all operations. This project should also add support
- for the RGB format to at least one XFree86 driver (e.g.,
- ATI Radeon).
- </para>
- <para>
- The X Video Extension is one of the few popular
- extensions that DMX does not support. We recommend
- implementing the X Video Extension even if scaling is
- the specific goal of that work.
- </para>
- </listitem>
- </orderedlist>
- </para>
- <para>
- We do <emphasis>not</emphasis> recommend implementation of the
- XCreateScaledWindow extension because of the complexity
- involved. We do <emphasis>not</emphasis> recommend implementation of the
- XPutImageScaled extension because it requires the same amount
- of work as the XCopyAreaScaled extension, but provides less
- functionality. Further, server-side scaling with a large
- frame buffer is <emphasis>not</emphasis> recommended because of the
- performance implications.
- </para>
- <para>
- The back-end scaling, especially with disconnect/reconnect
- support should be explored in the future after
- disconnect/reconnect is implemented, but not at the present
- time.
- </para>
- </sect1>
-
- </article>
- <!-- Local Variables: -->
- <!-- fill-column: 72 -->
- <!-- End: -->
+<?xml version="1.0" encoding="ISO-8859-1"?>
+<!DOCTYPE article PUBLIC "-//OASIS//DTD DocBook XML V4.3//EN"
+ "http://www.oasis-open.org/docbook/xml/4.3/docbookx.dtd" [
+ <!ENTITY % defs SYSTEM "/xserver/doc/xml/xserver.ent"> %defs;
+]>
+ <article>
+
+ <articleinfo>
+ <!-- Title information -->
+ <title>Scaled Window Support in DMX</title>
+ <authorgroup>
+ <author><firstname>Kevin E.</firstname><surname>Martin</surname></author>
+ <author><firstname>Rickard E.</firstname><surname>Faith</surname></author>
+ </authorgroup>
+ <pubdate>15 October 2003 (created 19 September 2003)</pubdate>
+ <releaseinfo>X Server Version &xserver.version;</releaseinfo>
+ <abstract>
+ <para>
+ This document investigates the possibility of adding scaled window
+ support to the DMX X server, thereby allowing a window or some
+ selected part of the logical DMX area to be displayed using a
+ scaling factor. For example, this might allow the contents of a
+ window to be magnified for easier viewing. In particular, scaling
+ for the VNC client is explored. <emphasis remap="it">Copyright 2003
+ by Red Hat, Inc., Raleigh, North Carolina</emphasis>
+ </para>
+ </abstract>
+ </articleinfo>
+
+ <!-- Begin the document -->
+ <sect1><title>Introduction</title>
+ <sect2><title>DMX</title>
+ <para>
+ The DMX X server (Xdmx) is a proxy server that is designed
+ to allow X servers on multiple machines to be combined into
+ a single multi-headed X server. Combined with Xinerama,
+ these heads can appear as a single very high-resolution
+ screen. Typical applications include the creation of a
+ video wall with 16 1280x1024 displays arranged in a
+ rectangle, for a total resolution of of 5120x4096.
+ </para>
+ </sect2>
+ <sect2><title>Problem Statement</title>
+ <para>
+ Applications displayed on a physically large video wall that
+ provides high pixel-resolution may be difficult to see,
+ especially if the application is designed for use on a
+ typical desktop computer with a relatively small display
+ located close to the human operator. The goal of this paper
+ is to describe and discuss solutions to this problem.
+ </para>
+ <para>
+ The original driving problem for this work is to provide
+ scaling for the <command>vncviewer</command> application when
+ displayed using DMX (VNC scaling is currently available only
+ with the Windows client, and there is no plan to extend that
+ capability to other clients). While this specific problem
+ will be addressed in this paper, the general solution space
+ will also be explored, since this may lead to a good
+ solution not only for <command>vncviewer</command> but also for
+ other applications.
+ </para>
+ </sect2>
+ <sect2><title>Task</title>
+ <para>
+ For reference, here is the original description of the task
+ this paper addresses:
+ <itemizedlist>
+ <listitem><para>Scaled window support (for VNC)
+ <itemizedlist>
+ <listitem><para>
+ Investigate possibility of implementing a "scaled
+ window" extension:
+ <itemizedlist>
+ <listitem><para>
+ Add XCreateScaledWindow call that could be used
+ in place of XCreateWindow
+ </para></listitem>
+ <listitem><para>
+ All primitives drawn to scaled window would be
+ scaled by appropriate (integral?) scaling factor
+ </para></listitem>
+ </itemizedlist>
+ </para></listitem>
+ <listitem><para>
+ Alternate approach: special case VNC support
+ </para></listitem>
+ </itemizedlist>
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </sect2>
+ </sect1>
+
+ <sect1><title>Previous Work</title>
+ <para>
+ This section reviews relevant previous work.
+ </para>
+ <sect2><title>VNC</title>
+ <sect3><title>Scaling under VNC</title>
+ <para>
+ When using the <command>vncviewer</command> program for Windows, it
+ is possible to specify a scaling factor (as numerator and
+ denominator). When scaling is in effect, the viewer
+ software uses StretchBlt (instead of BitBlt) to display
+ the pixels for the user. When this call is made, the
+ viewer already has received all of the pixel information
+ (at full unscaled resolution).
+ </para>
+ <para>
+ The scaling in VNC is primitive. It does not conserve
+ bandwidth, it does not treat textual information
+ differently (i.e., by using a suitably scaled font), and
+ it does not provide any anti-aliasing other than that
+ provided by the underlying (Windows-only) system library.
+ </para>
+ </sect3>
+ </sect2>
+ <sect2><title>The X Video Extension</title>
+ <para>
+ The X Video Extension is a widely-available extension to the
+ X11 protocol that provides support for streaming video.
+ Integral to this support is the ability to arbitrarily scale
+ the output. In version 2.2 of the X Video specification,
+ support for scaled still images was provided, using both
+ shared memory and traditional transport. The API for this
+ support uses calls that are quite similar to XCreateWindow,
+ XPutImage, and XShmPutImage. Currently, most of the drivers
+ implemented in XFree86 only support data in various YUV
+ formats. However, several modern video adaptors support RGB
+ as well.
+ </para>
+ <para>
+ Note, though, that the target output for this scaling is an
+ overlay plane -- so X Video provides functionality that is
+ fundamentally different from that provided by the Windows
+ StrechBlt call.
+ </para>
+ </sect2>
+ </sect1>
+
+ <sect1><title>Possible Solutions</title>
+ <para>
+ This section briefly discusses possible solutions, including
+ major advantages and disadvantages from both the
+ implementation and the end-user programmer standpoint.
+ </para>
+ <sect2><title>VNC-like Scaling</title>
+ <sect3><title>Software Scaling</title>
+ <para>
+ The <command>vncviewer</command> application could be modified to
+ provide software scaling. This is not a general solution,
+ but it does solve one of the goals of this work.
+ </para>
+ <para>
+ A prototype of this solution was implemented and a patch
+ against <filename>vnc-3.3.7-unixsrc</filename> is available in the
+ <filename>dmx/external</filename> directory. Because of limited time
+ available for this work, all of the edge cases were not
+ considered and the solution works well mainly for integer
+ scaling.
+ </para>
+ <para>
+ Currently, <command>vncviewer</command> writes to the X display
+ with XPutImage, XCopyArea, and XFillRectangle. All
+ instances of these calls have to be aware of scaling
+ and must round correctly. In the prototype solution,
+ rounding is incorrect and can cause artifacts.
+ </para>
+ <para>
+ A better solution would be to cache all updates to the
+ desktop image in <command>vncviewer</command> and only send the
+ damaged area to the X display with XPutImage. This would
+ allow the damaged area to be computed so that rounding
+ errors do not create artifacts. This method is probably
+ similar to what is used in the Window client. (The whole
+ VNC suite is being re-written in C++ and the forthcoming
+ version 4 has not been evaluated.)
+ </para>
+ </sect3>
+ <sect3><title>Scaling with the X Video Extension</title>
+ <para>
+ The scaling in the Windows <command>vncviewer</command> application
+ makes use of a scaled blit that is supplied by the
+ underlying system library. Several video cards currently
+ provide support for a scaled blit, and some X servers
+ (including XFree86) expose this capability to applications
+ via the XvPutImage interface of the X Video Extension.
+ The capability exposed by XvPutImage results in the scaled
+ image being drawn to an overlay plane. Most video cards
+ also provide support for a scaled blit into the normal
+ output planes, but this is not exposed via XvPutImage.
+ </para>
+ <para>
+ The <command>vncviewer</command> program could be modified to use
+ the X Video Extension to provide scaling under X11 that is
+ similar to the scaling currently provided under Windows.
+ Unfortunately, Xdmx does not currently export the X Video
+ Extension, so this would not provide an immediate solution
+ usable with DMX.
+ </para>
+ <para>
+ A very early-stage proof-of-concept prototype was
+ implemented and a preliminary patch against
+ <filename>vnc-3.3.7-unixsrc</filename> is available in the
+ <filename>dmx/external</filename> directory. This prototype was
+ implemented to better understand the problems that must be
+ solved to make this solution viable:
+ <itemizedlist>
+ <listitem><para>
+ As noted under the software scaling section above,
+ <command>vncviewer</command> writes to the X display with
+ several different calls. These calls write to the
+ normal output planes and are compatible with
+ XvPutImage, which writes to an overlay plane. To
+ eliminate artifacts caused by this problem,
+ <command>vncviewer</command> should be modified so that a cached
+ copy of the desktop is available, either as a
+ client-side image or a server-side off-screen pixmap,
+ so that XvPutImage would be the only method for
+ writing to the X display.
+ </para></listitem>
+ <listitem>
+ <para>
+ Although several modern graphics adaptors support
+ hardware scaling using an RGB format (e.g., ATI
+ Radeon, nVidia, etc.), XFree86 drivers typically
+ only implement YUV formats. YUV generally compress
+ the pixel information in some way. For example, two
+ commonly implemented formats, YUY2 and UYVY provide
+ intensity information for every RGB pixel, but only
+ provide chroma and luminance information for pairs
+ of horizontal pixels. Since VNC uses
+ pixel-resolution for communicating updates on the
+ wire, additional artifacts are introduced (because
+ there may not be enough information from the wire to
+ update a pair of pixels).
+ </para>
+ <para>
+ Further, the well-known problem with YUV encoding
+ is even more evident when the image is a desktop
+ instead of a movie. For example, consider a
+ 1-pixel-wide vertical window border. If the border
+ changes in color but not intensity (e.g., because a
+ window manager uses color to indicate focus), there
+ may or may not be a change in the YUY2 image,
+ depending on the algorithm used for RGB to YUV
+ conversion and on how the border pixel is ordered in
+ the pair of pixels used by the algorithm.
+ </para>
+ <para>
+ Many of these artifacts could be eliminated if
+ <command>vncviewer</command> cached a complete RGB image of
+ the desktop, and only did the conversion to YUV for
+ properly aligned areas of damage. The remaining artifacts
+ could be eliminated if an RGB format was used with X
+ Video (which may require the extension of existing
+ XFree86 drivers to support RGB).
+ </para>
+ </listitem>
+ <listitem><para>
+ Most modern video cards support exactly one overlay
+ plane that is suitable for use with X Video.
+ Therefore, only one application can use X Video at any
+ given time. This is a severe limitation in a desktop
+ environment.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ <sect4><title>Implementing the X Video Extension for DMX</title>
+ <para>
+ The user-level API for X Video is fairly simple, but the
+ underlying support required for the full specification
+ is large. However, since the API provides a method to
+ query supported capabilities, a usable subset of X
+ Video can be implemented that would support XvPutImage
+ and little else. This would require support for the
+ following:
+ <itemizedlist>
+ <listitem><para>
+ X Video Extension API calls, including the
+ following:
+ <itemizedlist>
+ <listitem><para>XvQueryExtension</para></listitem>
+ <listitem><para>XvQueryAdaptors</para></listitem>
+ <listitem><para>XvQueryPortAttributes</para></listitem>
+ <listitem><para>XvFreeAdaptorInfo</para></listitem>
+ <listitem><para>XvListImageFormats</para></listitem>
+ <listitem><para>XvGrabPort</para></listitem>
+ <listitem><para>XvCreateImage</para></listitem>
+ <listitem><para>XvPutImage</para></listitem>
+ <listitem><para>XvShmCreateImage</para></listitem>
+ <listitem><para>XvShmPutImage</para></listitem>
+ </itemizedlist>
+ </para></listitem>
+ <listitem><para>
+ Support for querying back-end X Video Extension
+ capabilities.
+ </para></listitem>
+ <listitem><para>
+ Support for sending the image to the back-ends.
+ Because X Video requires sending full images, there
+ may be a trade-off between bandwidth limitations and
+ additional complexity to divide the image up such
+ that is scales properly.
+ </para></listitem>
+ <listitem><para>
+ Possible support for a software fall-back. For
+ example, if all of the back-ends do not support the X
+ Video Extension, software scaling can be implemented
+ such that the image is sent to the back-end with
+ XPutImage. This pathway would have poor
+ performance.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </sect4>
+ <sect4><title>Supporting RGB formats for the X Video Extension</title>
+ <para>
+ Assuming an XFree86 driver already supports the X Video
+ Extension, and assuming the target hardware supports an
+ RGB format, then adding support for that format is
+ relatively simple and straightforward.
+ </para>
+ </sect4>
+ </sect3>
+ <sect3><title>Scaling with an XPutImageScaled Extension</title>
+ <para>
+ Instead of (or in addition to) implementing the X Video
+ Extension in DMX, one obvious solution would be to
+ implement a new extension that provides access to
+ hardware-assisted scaled blits, similar to the StretchBlt
+ call available under Windows. This call would scale RGB
+ images and would not use the overlay plane (unlike the X
+ Video Extension).
+ </para>
+ <para>
+ This approach has many of the same advantages and
+ disadvantages as the XCopyAreaScaled Extension, discussed
+ in the next section. Discussion of XPutImageScaled is
+ deferred in favor of XCopyAreaScaled for the following
+ reasons:
+ <itemizedlist>
+ <listitem><para>
+ XPutImageScaled can be emulated with XCopyAreaScaled
+ by first using XPutImage to copy the image to an
+ off-screen pixmap, and then calling XCopyAreaScaled
+ between that off-screen pixmap and the target
+ drawable.
+ </para></listitem>
+ <listitem><para>
+ Since XCopyAreaScaled would copy between two areas of
+ on-screen or off-screen memory, it has additional uses
+ and can be viewed as efficiently providing a superset
+ of XPutImageScaled functionality.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </sect3>
+ <sect3><title>Scaling with an XCopyAreaScaled Extension</title>
+ <para>
+ As noted in the previous section, because XCopyAreaScaled
+ provides a superset of the functionality provided by
+ XPutImageScaled, we will consider this extension instead.
+ </para>
+ <para>
+ First, XCopyAreaScaled would provide for RGB scaling
+ between pixmaps (i.e., on-screen or off-screen areas of
+ memory that reside on the video card). Unlike the X Video
+ Extension, which writes into an overlay plane,
+ XCopyAreaScaled would write into the non-overlay areas of
+ the screen. Key points to consider are as follows:
+ <itemizedlist>
+ <listitem><para>
+ Because different planes are involved, the two scaling
+ operations are usually implemented in hardware
+ differently, so an XCopyAreaScaled extension could be
+ added in a manner that would neither conflict with nor
+ interact with the X Video extension in any way.
+ </para></listitem>
+ <listitem><para>
+ The XCopyAreaScaled extension provides new
+ functionality that the X Video Extension does not
+ provide. Based on anecdotal feedback, we believe that
+ many people outside the DMX and VNC communities would
+ be excited about this extension.
+ </para></listitem>
+ <listitem><para>
+ The main drawback to this extension is that it is new
+ and needs to be implemented at the driver level in
+ XFree86 for each video card to be supported. At the
+ present time, it is more likely that the X Video
+ Extension will be implemented for a particular piece
+ hardware because the X Video extension has multimedia
+ uses. However, over time, we would expect the
+ XCopyAreaScaled extension to be implemented along with
+ the X Video extension, especially if it becomes
+ popular.
+ </para></listitem>
+ <listitem><para>
+ Another drawback is that not all modern cards provide
+ support for a simple scaled blit operation. However,
+ these cards usually do provide a 3D pipeline which
+ could be used to provide this functionality in a
+ manner that is transparent to the client application
+ that is using the XCopyAreaScaled extension. However,
+ this implementation pathway would make this extension
+ somewhat more difficult to implement on certain cards.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ </sect3>
+ <sect3><title>Scaling with OpenGL</title>
+ <para>
+ Another general solution to the scaling problem is to use
+ the texture scaling found in all 3D hardware. This
+ ability is already exposed through OpenGL and can be
+ exploited by clients without X server modification (i.e.,
+ other than the ability to support OpenGL). An application
+ using OpenGL would transmit the non-scaled image to the X
+ server as a texture, and would then display a single
+ non-transformed rect using that texture. This also works
+ around the single overlay problem with the X Video
+ Extension as well as the need to implement additional
+ scaled primitive extensions.
+ </para>
+ <para>
+ The downside is that most OpenGL implementations require
+ power of 2 texture sizes and this can be very wasteful of
+ memory if, for example, the application needs to scale a
+ 1025x1025 image, which would require a 2048x2048 texture
+ area (even a 640x480 image would require a 1024x512
+ texture). Another downside is that some OpenGL
+ implementations have a limited about of texture memory and
+ cannot handle textures that are very large. For example,
+ they might limit the texture size to 1024x1024.
+ </para>
+ </sect3>
+ </sect2>
+ <sect2><title>Application-transparent Scaling for DMX
+ </title><sect3><title>Back-end Scaling Without Disconnect/Reconnect</title>
+ <para>
+ VNC does scaling on the client side (in the
+ <command>vncviewer</command> application). Implementing a similar
+ solution for DMX would require support in the back-end X
+ servers and, therefore, is not a general solution.
+ </para>
+ <para>
+ XFree86 already implements some support for "scaling" that
+ could be used with DMX: if, in the XF86Config file,
+ multiple Modes are listed in the Display Subsection of the
+ Screen Section, then pressing Ctrl-Alt-Plus and
+ Ctrl-Alt-Minus can be used to iterate through the listed
+ modes. The display dimensions will change to the
+ dimensions in the Modes line, but the logical dimensions
+ of the X server (i.e., the dimensions that Xdmx knows
+ about) will not change.
+ </para>
+ <para>
+ Further, the dimensions of the XFree86 display are under
+ software control (via the XFree86-VidModeExtension), so
+ the Xdmx server could change the screen dimensions on a
+ per-display basis, thereby scaling the information on part
+ of that display.
+ </para>
+ <para>
+ However, this scaling appears to have limited use. For
+ example, assume a 4 by 4 display wall consisting of 16
+ 1280x1024 displays. If all of the back-end servers were
+ simultaneously configured to display 640x480, the left
+ hand corner of each display would be magnified, but the
+ composite result would be unreadable. Magnifying one
+ display at a time could be usable, but could have limited
+ utility, since the result would still be no larger than a
+ single display.
+ </para>
+ </sect3>
+ <sect3><title>Back-end Scaling With Disconnect/Reconnect</title>
+ <para>
+ Disconnect and reconnect features are not currently
+ supported in DMX, but are scheduled to be implemented in
+ the future. These features, combined with the
+ XFree86-VidModeExtension Extension, would allow an
+ application to do the following:
+ <itemizedlist>
+ <listitem><para>
+ Disconnect a specific back-end server (via the DMX
+ Extension),
+ </para></listitem>
+ <listitem><para>
+ reconfigure the XFree86 back-end server resolution,
+ and
+ </para></listitem>
+ <listitem><para>
+ reconnect the back-end server to DMX -- at a new
+ origin with the new screen resolution.
+ </para></listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ For example, consider a display wall consisting of 16
+ 1280x1024 displays with a total resolution of 5120x4096.
+ All of the screens could be disconnected, repositioned,
+ and reconnected each at a resolution of 640x480. The
+ total resolution of the display wall would be 2560x1920,
+ allowing a view of a selected area approximately
+ one-fourth of the size of the DMX display. This change
+ would be completely application independent (except,
+ perhaps, for a DMX-aware window manager). When work at
+ the increased resolution was completed, the back-end
+ servers could be disconnected, reconfigured, and
+ reconnected for the original 5120x4096 view.
+ </para>
+ <para>
+ Support for this type of scaling can be implemented in a
+ DMX-aware X11 client assuming the DMX server support
+ arbitrary disconnect and reconnect semantics. Because
+ this application cannot be written before
+ disconnect/reconnect is implemented, this solution will
+ not be discussed further in this paper.
+ </para>
+ </sect3>
+ <sect3><title>Server-side Scaling</title>
+ <para>
+ In earlier versions of DMX, a frame buffer was maintained
+ on the server side, and XPutImage was used to move the
+ information from the server to the client (similar to some
+ early VNC implementations). The use of a server-side
+ frame buffer would allow the server to do scaling, but is
+ not a recommended solution because of overall performance
+ issues and server-side memory issues (i.e., the frame
+ buffer would be very large for large display walls).
+ </para>
+ <para>
+ Exploration of this path is not recommended.
+ </para>
+ </sect3>
+ </sect2>
+ <sect2><title>XCreateScaledWindow API</title>
+ <para>
+ The implementation of X Video Extension in DMX, and the use
+ of XvPutImage by applications requiring scaling requires
+ significant changes in DMX Further, XvPutImage is,
+ essentially a scaled blit, and it is only useful for
+ applications which are already using (or can be modified to
+ use) XPutImage. Therefore, a more general API will be
+ discussed as another possibility.
+ </para>
+ <para>
+ X applications typically create windows with the
+ XCreateWindow call. A new extension could provide an
+ XCreateScaledWindow call that could be used in place of the
+ XCreateWindow call and be otherwise transparent to the
+ application. This would allow applications, even those that
+ do not depend on XPutImage, to take advantage of window
+ scaling. In this section we describe how the call would
+ work, what transparency it provides, and how to solve the
+ potential problems that transparency creates.
+ </para>
+ <sect3><title>XCreateWindow</title>
+ <para>
+ The XCreateWindow call takes width and height as
+ parameters. An XCreateScaledWindow call could take all
+ the same parameters, with the addition of a scaling factor.
+ </para>
+ </sect3>
+ <sect3><title>XSetWindowAttributes</title>
+ <para>
+ An X11 window has several attributes that would have to be
+ scaled:
+ <itemizedlist>
+ <listitem><para>Background and border pixmaps</para></listitem>
+ <listitem><para>Border width</para></listitem>
+ <listitem><para>Cursor</para></listitem>
+ </itemizedlist>
+ </para>
+ </sect3>
+ <sect3><title>XGetWindowAttributes, XGetGeometry</title>
+ <para>
+ For transparency, calls that query the window attributes
+ should return unscaled information. This suggests that
+ all unscaled pixmaps and window attributes should be
+ cached.
+ </para>
+ <para>
+ Unfortunately, a window manager requires the scaled
+ geometry to properly decorate the window. The X server
+ can probably determine which client is acting as the
+ window manager (e.g., because that client will select
+ events that are used exclusively by the window manager).
+ However, other Scaled Window Extension aware clients may
+ also need to determine the scaled geometry. Therefore, at
+ least two additional extension calls should be
+ implemented: XGetScaledWindowAttributes and
+ XGetScaledGeometry.
+ </para>
+ </sect3>
+ <sect3><title>Popup and Child window positions</title>
+ <para>
+ Some applications may position popup and child windows
+ based on an unscaled notion of the main window geometry.
+ In this case, additional modifications to the client would
+ be required.
+ </para>
+ </sect3>
+ <sect3><title>Events</title>
+ <para>
+ Most events (e.g., for mouse motion) return information
+ about the coordinates at which the even occurred. These
+ coordinates would have to be modified so that unscaled
+ values were presented to the client.
+ </para>
+ </sect3>
+ <sect3><title>Implementation</title>
+ <para>
+ There are many implementation issues, some of which are
+ similar to the issues involved in implementing the X Video
+ Extension for DMX. The window contents must be scaled,
+ either by performing all operations to a frame buffer and
+ then writing the image to the display (perhaps using
+ hardware scaling support), or by modifying all of the
+ various drawing operations to perform scaling. Because of
+ the complexity involved, the frame buffer option is
+ recommended.
+ </para>
+ </sect3>
+ </sect2>
+ </sect1>
+
+ <sect1><title>Conclusion and Recommendations
+ </title><para>
+ We recommend a three phase implementation strategy, based on
+ how an application could be written to take advantage of
+ scaling:
+ <orderedlist>
+ <listitem>
+ <para>
+ The XCopyAreaScaled extension should be implemented, since
+ this is the ideal solution for applications like VNC, and
+ since making use of this extension will require minimal
+ changes to applications that already use XPutImage or
+ XCopyArea.
+ </para>
+ <para>
+ The initial implementation work would include the design
+ of the X protocol extension, writing this up in the
+ usual format for extension documentation, implementation
+ of the protocol transport pieces in XFree86,
+ implementation of a software fall-back in XFree86 and
+ DMX, one example hardware implementation for XFree86,
+ and implementation of support for this extension in DMX.
+ </para>
+ <para>
+ We suggest implementing the extension first on the ATI
+ Radeon cards. However, since these cards do not provide
+ a 2D scaled blit primitive, the implementation would
+ have to make use of the 3D texture engine to emulate a
+ scaled blit. This is recommended, since other modern
+ graphics cards also do not provide a simple 2D scaled
+ blit operation and an example of the more difficult
+ implementation pathway would be helpful to others.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Until XCopyAreaScaled is widely supported, applications
+ that require scaling will have to fall back to another
+ scaling method. We suggest OpenGL as the first fall-back
+ method because it is widely available and supported by
+ DMX.
+ </para>
+ <para>
+ A project centered around OpenGL-based scaling would
+ implement this scaling in VNC as an example. This work
+ would include re-writing the <command>vncviewer</command>
+ rendering engine to cache a master copy of the desktop
+ image for all operations.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Since OpenGL is not implemented everywhere, and may not
+ provide hardware-assisted performance in every
+ implementation, an application that requires scaling
+ should also fall back to using the X Video Extension.
+ </para>
+ <para>
+ This project would add support for the X Video Extension
+ to DMX and would add support to VNC to take advantage of
+ this extension without introducing artifacts. This
+ would require modifying the <command>vncviewer</command> rendering
+ engine to cache a master copy of the desktop image for
+ all operations. This project should also add support
+ for the RGB format to at least one XFree86 driver (e.g.,
+ ATI Radeon).
+ </para>
+ <para>
+ The X Video Extension is one of the few popular
+ extensions that DMX does not support. We recommend
+ implementing the X Video Extension even if scaling is
+ the specific goal of that work.
+ </para>
+ </listitem>
+ </orderedlist>
+ </para>
+ <para>
+ We do <emphasis>not</emphasis> recommend implementation of the
+ XCreateScaledWindow extension because of the complexity
+ involved. We do <emphasis>not</emphasis> recommend implementation of the
+ XPutImageScaled extension because it requires the same amount
+ of work as the XCopyAreaScaled extension, but provides less
+ functionality. Further, server-side scaling with a large
+ frame buffer is <emphasis>not</emphasis> recommended because of the
+ performance implications.
+ </para>
+ <para>
+ The back-end scaling, especially with disconnect/reconnect
+ support should be explored in the future after
+ disconnect/reconnect is implemented, but not at the present
+ time.
+ </para>
+ </sect1>
+
+ </article>
+ <!-- Local Variables: -->
+ <!-- fill-column: 72 -->
+ <!-- End: -->