aboutsummaryrefslogtreecommitdiff
path: root/nx-X11/programs/Xserver/hw/dmx/doc/scaled.txt
diff options
context:
space:
mode:
Diffstat (limited to 'nx-X11/programs/Xserver/hw/dmx/doc/scaled.txt')
-rw-r--r--nx-X11/programs/Xserver/hw/dmx/doc/scaled.txt579
1 files changed, 0 insertions, 579 deletions
diff --git a/nx-X11/programs/Xserver/hw/dmx/doc/scaled.txt b/nx-X11/programs/Xserver/hw/dmx/doc/scaled.txt
deleted file mode 100644
index d30105dd2..000000000
--- a/nx-X11/programs/Xserver/hw/dmx/doc/scaled.txt
+++ /dev/null
@@ -1,579 +0,0 @@
- Scaled Window Support in DMX
- Rickard E. Faith and Kevin E. Martin
- 15 October 2003 (created 19 September 2003)
-
- This document investigates the possibility of adding scaled window
- support to the DMX X server, thereby allowing a window or some
- selected part of the logical DMX area to be displayed using a scaling
- factor. For example, this might allow the contents of a window to be
- magnified for easier viewing. In particular, scaling for the VNC
- client is explored. _C_o_p_y_r_i_g_h_t _2_0_0_3 _b_y _R_e_d _H_a_t_, _I_n_c_._, _R_a_l_e_i_g_h_, _N_o_r_t_h
- _C_a_r_o_l_i_n_a
-
- ______________________________________________________________________
-
- Table of Contents
-
-
- 1. Introduction
- 1.1 DMX
- 1.2 Problem Statement
- 1.3 Task
-
- 2. Previous Work
- 2.1 VNC
- 2.1.1 Scaling under VNC
- 2.2 The X Video Extension
-
- 3. Possible Solutions
- 3.1 VNC-like Scaling
- 3.1.1 Software Scaling
- 3.1.2 Scaling with the X Video Extension
- 3.1.2.1 Implementing the X Video Extension for DMX
- 3.1.2.2 Supporting RGB formats for the X Video Extension
- 3.1.3 Scaling with an XPutImageScaled Extension
- 3.1.4 Scaling with an XCopyAreaScaled Extension
- 3.1.5 Scaling with OpenGL
- 3.2 Application-transparent Scaling for DMX
- 3.2.1 Back-end Scaling Without Disconnect/Reconnect
- 3.2.2 Back-end Scaling With Disconnect/Reconnect
- 3.2.3 Server-side Scaling
- 3.3 XCreateScaledWindow API
- 3.3.1 XCreateWindow
- 3.3.2 XSetWindowAttributes
- 3.3.3 XGetWindowAttributes, XGetGeometry
- 3.3.4 Popup and Child window positions
- 3.3.5 Events
- 3.3.6 Implementation
-
- 4. Conclusion and Recommendations
-
-
- ______________________________________________________________________
-
- 11.. IInnttrroodduuccttiioonn
-
- 11..11.. DDMMXX
-
- The DMX X server (Xdmx) is a proxy server that is designed to allow X
- servers on multiple machines to be combined into a single multi-headed
- X server. Combined with Xinerama, these heads can appear as a single
- very high-resolution screen. Typical applications include the
- creation of a video wall with 16 1280x1024 displays arranged in a
- rectangle, for a total resolution of of 5120x4096.
-
-
-
- 11..22.. PPrroobblleemm SSttaatteemmeenntt
-
- Applications displayed on a physically large video wall that provides
- high pixel-resolution may be difficult to see, especially if the
- application is designed for use on a typical desktop computer with a
- relatively small display located close to the human operator. The
- goal of this paper is to describe and discuss solutions to this
- problem.
-
- The original driving problem for this work is to provide scaling for
- the vncviewer application when displayed using DMX (VNC scaling is
- currently available only with the Windows client, and there is no plan
- to extend that capability to other clients). While this specific
- problem will be addressed in this paper, the general solution space
- will also be explored, since this may lead to a good solution not only
- for vncviewer but also for other applications.
-
- 11..33.. TTaasskk
-
- For reference, here is the original description of the task this paper
- addresses:
-
- +o Scaled window support (for VNC)
-
- +o Investigate possibility of implementing a "scaled window"
- extension:
-
- +o Add XCreateScaledWindow call that could be used in place of
- XCreateWindow
-
- +o All primitives drawn to scaled window would be scaled by
- appropriate (integral?) scaling factor
-
- +o Alternate approach: special case VNC support
-
- 22.. PPrreevviioouuss WWoorrkk
-
- This section reviews relevant previous work.
-
- 22..11.. VVNNCC
-
- 22..11..11.. SSccaalliinngg uunnddeerr VVNNCC
-
- When using the vncviewer program for Windows, it is possible to
- specify a scaling factor (as numerator and denominator). When scaling
- is in effect, the viewer software uses StretchBlt (instead of BitBlt)
- to display the pixels for the user. When this call is made, the
- viewer already has received all of the pixel information (at full
- unscaled resolution).
-
- The scaling in VNC is primitive. It does not conserve bandwidth, it
- does not treat textual information differently (i.e., by using a
- suitably scaled font), and it does not provide any anti-aliasing other
- than that provided by the underlying (Windows-only) system library.
-
- 22..22.. TThhee XX VViiddeeoo EExxtteennssiioonn
-
- The X Video Extension is a widely-available extension to the X11
- protocol that provides support for streaming video. Integral to this
- support is the ability to arbitrarily scale the output. In version
- 2.2 of the X Video specification, support for scaled still images was
- provided, using both shared memory and traditional transport. The API
- for this support uses calls that are quite similar to XCreateWindow,
- XPutImage, and XShmPutImage. Currently, most of the drivers
- implemented in XFree86 only support data in various YUV formats.
- However, several modern video adaptors support RGB as well.
- Note, though, that the target output for this scaling is an overlay
- plane -- so X Video provides functionality that is fundamentally
- different from that provided by the Windows StrechBlt call.
-
- 33.. PPoossssiibbllee SSoolluuttiioonnss
-
- This section briefly discusses possible solutions, including major
- advantages and disadvantages from both the implementation and the end-
- user programmer standpoint.
-
- 33..11.. VVNNCC--lliikkee SSccaalliinngg
-
- 33..11..11.. SSooffttwwaarree SSccaalliinngg
-
- The vncviewer application could be modified to provide software
- scaling. This is not a general solution, but it does solve one of the
- goals of this work.
-
- A prototype of this solution was implemented and a patch against
- vnc-3.3.7-unixsrc is available in the dmx/external directory. Because
- of limited time available for this work, all of the edge cases were
- not considered and the solution works well mainly for integer scaling.
-
- Currently, vncviewer writes to the X display with XPutImage,
- XCopyArea, and XFillRectangle. All instances of these calls have to
- be aware of scaling and must round correctly. In the prototype
- solution, rounding is incorrect and can cause artifacts.
-
- A better solution would be to cache all updates to the desktop image
- in vncviewer and only send the damaged area to the X display with
- XPutImage. This would allow the damaged area to be computed so that
- rounding errors do not create artifacts. This method is probably
- similar to what is used in the Window client. (The whole VNC suite is
- being re-written in C++ and the forthcoming version 4 has not been
- evaluated.)
-
- 33..11..22.. SSccaalliinngg wwiitthh tthhee XX VViiddeeoo EExxtteennssiioonn
-
- The scaling in the Windows vncviewer application makes use of a scaled
- blit that is supplied by the underlying system library. Several video
- cards currently provide support for a scaled blit, and some X servers
- (including XFree86) expose this capability to applications via the
- XvPutImage interface of the X Video Extension. The capability exposed
- by XvPutImage results in the scaled image being drawn to an overlay
- plane. Most video cards also provide support for a scaled blit into
- the normal output planes, but this is not exposed via XvPutImage.
-
- The vncviewer program could be modified to use the X Video Extension
- to provide scaling under X11 that is similar to the scaling currently
- provided under Windows. Unfortunately, Xdmx does not currently export
- the X Video Extension, so this would not provide an immediate solution
- usable with DMX.
-
- A very early-stage proof-of-concept prototype was implemented and a
- preliminary patch against vnc-3.3.7-unixsrc is available in the
- dmx/external directory. This prototype was implemented to better
- understand the problems that must be solved to make this solution
- viable:
-
- +o As noted under the software scaling section above, vncviewer writes
- to the X display with several different calls. These calls write
- to the normal output planes and are compatible with XvPutImage,
- which writes to an overlay plane. To eliminate artifacts caused by
- this problem, vncviewer should be modified so that a cached copy of
- the desktop is available, either as a client-side image or a
- server-side off-screen pixmap, so that XvPutImage would be the only
- method for writing to the X display.
-
- +o
-
- Although several modern graphics adaptors support hardware scaling
- using an RGB format (e.g., ATI Radeon, nVidia, etc.), XFree86
- drivers typically only implement YUV formats. YUV generally
- compress the pixel information in some way. For example, two
- commonly implemented formats, YUY2 and UYVY provide intensity
- information for every RGB pixel, but only provide chroma and
- luminance information for pairs of horizontal pixels. Since VNC
- uses pixel-resolution for communicating updates on the wire,
- additional artifacts are introduced (because there may not be
- enough information from the wire to update a pair of pixels).
-
- Further, the well-known problem with YUV encoding is even more
- evident when the image is a desktop instead of a movie. For
- example, consider a 1-pixel-wide vertical window border. If the
- border changes in color but not intensity (e.g., because a window
- manager uses color to indicate focus), there may or may not be a
- change in the YUY2 image, depending on the algorithm used for RGB
- to YUV conversion and on how the border pixel is ordered in the
- pair of pixels used by the algorithm.
-
- Many of these artifacts could be eliminated if vncviewer cached a
- complete RGB image of the desktop, and only did the conversion to
- YUV for properly aligned areas of damage. The remaining artifacts
- could be eliminated if an RGB format was used with X Video (which
- may require the extension of existing XFree86 drivers to support
- RGB).
-
- +o Most modern video cards support exactly one overlay plane that is
- suitable for use with X Video. Therefore, only one application can
- use X Video at any given time. This is a severe limitation in a
- desktop environment.
-
- 33..11..22..11.. IImmpplleemmeennttiinngg tthhee XX VViiddeeoo EExxtteennssiioonn ffoorr DDMMXX
-
- The user-level API for X Video is fairly simple, but the underlying
- support required for the full specification is large. However, since
- the API provides a method to query supported capabilities, a usable
- subset of X Video can be implemented that would support XvPutImage and
- little else. This would require support for the following:
-
- +o X Video Extension API calls, including the following:
-
- +o XvQueryExtension
-
- +o XvQueryAdaptors
-
- +o XvQueryPortAttributes
-
- +o XvFreeAdaptorInfo
-
- +o XvListImageFormats
-
- +o XvGrabPort
-
- +o XvCreateImage
-
- +o XvPutImage
-
- +o XvShmCreateImage
-
- +o XvShmPutImage
-
- +o Support for querying back-end X Video Extension capabilities.
-
- +o Support for sending the image to the back-ends. Because X Video
- requires sending full images, there may be a trade-off between
- bandwidth limitations and additional complexity to divide the image
- up such that is scales properly.
-
- +o Possible support for a software fall-back. For example, if all of
- the back-ends do not support the X Video Extension, software
- scaling can be implemented such that the image is sent to the back-
- end with XPutImage. This pathway would have poor performance.
-
- 33..11..22..22.. SSuuppppoorrttiinngg RRGGBB ffoorrmmaattss ffoorr tthhee XX VViiddeeoo EExxtteennssiioonn
-
- Assuming an XFree86 driver already supports the X Video Extension, and
- assuming the target hardware supports an RGB format, then adding
- support for that format is relatively simple and straightforward.
-
- 33..11..33.. SSccaalliinngg wwiitthh aann XXPPuuttIImmaaggeeSSccaalleedd EExxtteennssiioonn
-
- Instead of (or in addition to) implementing the X Video Extension in
- DMX, one obvious solution would be to implement a new extension that
- provides access to hardware-assisted scaled blits, similar to the
- StretchBlt call available under Windows. This call would scale RGB
- images and would not use the overlay plane (unlike the X Video
- Extension).
-
- This approach has many of the same advantages and disadvantages as the
- XCopyAreaScaled Extension, discussed in the next section. Discussion
- of XPutImageScaled is deferred in favor of XCopyAreaScaled for the
- following reasons:
-
- +o XPutImageScaled can be emulated with XCopyAreaScaled by first using
- XPutImage to copy the image to an off-screen pixmap, and then
- calling XCopyAreaScaled between that off-screen pixmap and the
- target drawable.
-
- +o Since XCopyAreaScaled would copy between two areas of on-screen or
- off-screen memory, it has additional uses and can be viewed as
- efficiently providing a superset of XPutImageScaled functionality.
-
- 33..11..44.. SSccaalliinngg wwiitthh aann XXCCooppyyAArreeaaSSccaalleedd EExxtteennssiioonn
-
- As noted in the previous section, because XCopyAreaScaled provides a
- superset of the functionality provided by XPutImageScaled, we will
- consider this extension instead.
-
- First, XCopyAreaScaled would provide for RGB scaling between pixmaps
- (i.e., on-screen or off-screen areas of memory that reside on the
- video card). Unlike the X Video Extension, which writes into an
- overlay plane, XCopyAreaScaled would write into the non-overlay areas
- of the screen. Key points to consider are as follows:
-
- +o Because different planes are involved, the two scaling operations
- are usually implemented in hardware differently, so an
- XCopyAreaScaled extension could be added in a manner that would
- neither conflict with nor interact with the X Video extension in
- any way.
-
- +o The XCopyAreaScaled extension provides new functionality that the X
- Video Extension does not provide. Based on anecdotal feedback, we
- believe that many people outside the DMX and VNC communities would
- be excited about this extension.
-
- +o The main drawback to this extension is that it is new and needs to
- be implemented at the driver level in XFree86 for each video card
- to be supported. At the present time, it is more likely that the X
- Video Extension will be implemented for a particular piece hardware
- because the X Video extension has multimedia uses. However, over
- time, we would expect the XCopyAreaScaled extension to be
- implemented along with the X Video extension, especially if it
- becomes popular.
-
- +o Another drawback is that not all modern cards provide support for a
- simple scaled blit operation. However, these cards usually do
- provide a 3D pipeline which could be used to provide this
- functionality in a manner that is transparent to the client
- application that is using the XCopyAreaScaled extension. However,
- this implementation pathway would make this extension somewhat more
- difficult to implement on certain cards.
-
- 33..11..55.. SSccaalliinngg wwiitthh OOppeennGGLL
-
- Another general solution to the scaling problem is to use the texture
- scaling found in all 3D hardware. This ability is already exposed
- through OpenGL and can be exploited by clients without X server
- modification (i.e., other than the ability to support OpenGL). An
- application using OpenGL would transmit the non-scaled image to the X
- server as a texture, and would then display a single non-transformed
- rect using that texture. This also works around the single overlay
- problem with the X Video Extension as well as the need to implement
- additional scaled primitive extensions.
-
- The downside is that most OpenGL implementations require power of 2
- texture sizes and this can be very wasteful of memory if, for example,
- the application needs to scale a 1025x1025 image, which would require
- a 2048x2048 texture area (even a 640x480 image would require a
- 1024x512 texture). Another downside is that some OpenGL
- implementations have a limited about of texture memory and cannot
- handle textures that are very large. For example, they might limit
- the texture size to 1024x1024.
-
- 33..22.. AApppplliiccaattiioonn--ttrraannssppaarreenntt SSccaalliinngg ffoorr DDMMXX
-
- 33..22..11.. BBaacckk--eenndd SSccaalliinngg WWiitthhoouutt DDiissccoonnnneecctt//RReeccoonnnneecctt
-
- VNC does scaling on the client side (in the vncviewer application).
- Implementing a similar solution for DMX would require support in the
- back-end X servers and, therefore, is not a general solution.
-
- XFree86 already implements some support for "scaling" that could be
- used with DMX: if, in the XF86Config file, multiple Modes are listed
- in the Display Subsection of the Screen Section, then pressing Ctrl-
- Alt-Plus and Ctrl-Alt-Minus can be used to iterate through the listed
- modes. The display dimensions will change to the dimensions in the
- Modes line, but the logical dimensions of the X server (i.e., the
- dimensions that Xdmx knows about) will not change.
-
- Further, the dimensions of the XFree86 display are under software
- control (via the XFree86-VidModeExtension), so the Xdmx server could
- change the screen dimensions on a per-display basis, thereby scaling
- the information on part of that display.
-
- However, this scaling appears to have limited use. For example,
- assume a 4 by 4 display wall consisting of 16 1280x1024 displays. If
- all of the back-end servers were simultaneously configured to display
- 640x480, the left hand corner of each display would be magnified, but
- the composite result would be unreadable. Magnifying one display at a
- time could be usable, but could have limited utility, since the result
- would still be no larger than a single display.
-
-
- 33..22..22.. BBaacckk--eenndd SSccaalliinngg WWiitthh DDiissccoonnnneecctt//RReeccoonnnneecctt
-
- Disconnect and reconnect features are not currently supported in DMX,
- but are scheduled to be implemented in the future. These features,
- combined with the XFree86-VidModeExtension Extension, would allow an
- application to do the following:
-
- +o Disconnect a specific back-end server (via the DMX Extension),
-
- +o reconfigure the XFree86 back-end server resolution, and
-
- +o reconnect the back-end server to DMX -- at a new origin with the
- new screen resolution.
-
- For example, consider a display wall consisting of 16 1280x1024
- displays with a total resolution of 5120x4096. All of the screens
- could be disconnected, repositioned, and reconnected each at a
- resolution of 640x480. The total resolution of the display wall would
- be 2560x1920, allowing a view of a selected area approximately one-
- fourth of the size of the DMX display. This change would be
- completely application independent (except, perhaps, for a DMX-aware
- window manager). When work at the increased resolution was completed,
- the back-end servers could be disconnected, reconfigured, and
- reconnected for the original 5120x4096 view.
-
- Support for this type of scaling can be implemented in a DMX-aware X11
- client assuming the DMX server support arbitrary disconnect and
- reconnect semantics. Because this application cannot be written
- before disconnect/reconnect is implemented, this solution will not be
- discussed further in this paper.
-
- 33..22..33.. SSeerrvveerr--ssiiddee SSccaalliinngg
-
- In earlier versions of DMX, a frame buffer was maintained on the
- server side, and XPutImage was used to move the information from the
- server to the client (similar to some early VNC implementations). The
- use of a server-side frame buffer would allow the server to do
- scaling, but is not a recommended solution because of overall
- performance issues and server-side memory issues (i.e., the frame
- buffer would be very large for large display walls).
-
- Exploration of this path is not recommended.
-
- 33..33.. XXCCrreeaatteeSSccaalleeddWWiinnddooww AAPPII
-
- The implementation of X Video Extension in DMX, and the use of
- XvPutImage by applications requiring scaling requires significant
- changes in DMX Further, XvPutImage is, essentially a scaled blit, and
- it is only useful for applications which are already using (or can be
- modified to use) XPutImage. Therefore, a more general API will be
- discussed as another possibility.
-
- X applications typically create windows with the XCreateWindow call.
- A new extension could provide an XCreateScaledWindow call that could
- be used in place of the XCreateWindow call and be otherwise
- transparent to the application. This would allow applications, even
- those that do not depend on XPutImage, to take advantage of window
- scaling. In this section we describe how the call would work, what
- transparency it provides, and how to solve the potential problems that
- transparency creates.
-
- 33..33..11.. XXCCrreeaatteeWWiinnddooww
-
- The XCreateWindow call takes width and height as parameters. An
- XCreateScaledWindow call could take all the same parameters, with the
- addition of a scaling factor.
- 33..33..22.. XXSSeettWWiinnddoowwAAttttrriibbuutteess
-
- An X11 window has several attributes that would have to be scaled:
-
- +o Background and border pixmaps
-
- +o Border width
-
- +o Cursor
-
- 33..33..33.. XXGGeettWWiinnddoowwAAttttrriibbuutteess,, XXGGeettGGeeoommeettrryy
-
- For transparency, calls that query the window attributes should return
- unscaled information. This suggests that all unscaled pixmaps and
- window attributes should be cached.
-
- Unfortunately, a window manager requires the scaled geometry to
- properly decorate the window. The X server can probably determine
- which client is acting as the window manager (e.g., because that
- client will select events that are used exclusively by the window
- manager). However, other Scaled Window Extension aware clients may
- also need to determine the scaled geometry. Therefore, at least two
- additional extension calls should be implemented:
- XGetScaledWindowAttributes and XGetScaledGeometry.
-
- 33..33..44.. PPooppuupp aanndd CChhiilldd wwiinnddooww ppoossiittiioonnss
-
- Some applications may position popup and child windows based on an
- unscaled notion of the main window geometry. In this case, additional
- modifications to the client would be required.
-
- 33..33..55.. EEvveennttss
-
- Most events (e.g., for mouse motion) return information about the
- coordinates at which the even occurred. These coordinates would have
- to be modified so that unscaled values were presented to the client.
-
- 33..33..66.. IImmpplleemmeennttaattiioonn
-
- There are many implementation issues, some of which are similar to the
- issues involved in implementing the X Video Extension for DMX. The
- window contents must be scaled, either by performing all operations to
- a frame buffer and then writing the image to the display (perhaps
- using hardware scaling support), or by modifying all of the various
- drawing operations to perform scaling. Because of the complexity
- involved, the frame buffer option is recommended.
-
- 44.. CCoonncclluussiioonn aanndd RReeccoommmmeennddaattiioonnss
-
- We recommend a three phase implementation strategy, based on how an
- application could be written to take advantage of scaling:
-
- 1.
-
- The XCopyAreaScaled extension should be implemented, since this is
- the ideal solution for applications like VNC, and since making use
- of this extension will require minimal changes to applications that
- already use XPutImage or XCopyArea.
-
- The initial implementation work would include the design of the X
- protocol extension, writing this up in the usual format for
- extension documentation, implementation of the protocol transport
- pieces in XFree86, implementation of a software fall-back in
- XFree86 and DMX, one example hardware implementation for XFree86,
- and implementation of support for this extension in DMX.
-
- We suggest implementing the extension first on the ATI Radeon
- cards. However, since these cards do not provide a 2D scaled blit
- primitive, the implementation would have to make use of the 3D
- texture engine to emulate a scaled blit. This is recommended,
- since other modern graphics cards also do not provide a simple 2D
- scaled blit operation and an example of the more difficult
- implementation pathway would be helpful to others.
-
- 2.
-
- Until XCopyAreaScaled is widely supported, applications that
- require scaling will have to fall back to another scaling method.
- We suggest OpenGL as the first fall-back method because it is
- widely available and supported by DMX.
-
- A project centered around OpenGL-based scaling would implement this
- scaling in VNC as an example. This work would include re-writing
- the vncviewer rendering engine to cache a master copy of the
- desktop image for all operations.
-
- 3.
-
- Since OpenGL is not implemented everywhere, and may not provide
- hardware-assisted performance in every implementation, an
- application that requires scaling should also fall back to using
- the X Video Extension.
-
- This project would add support for the X Video Extension to DMX and
- would add support to VNC to take advantage of this extension
- without introducing artifacts. This would require modifying the
- vncviewer rendering engine to cache a master copy of the desktop
- image for all operations. This project should also add support for
- the RGB format to at least one XFree86 driver (e.g., ATI Radeon).
-
- The X Video Extension is one of the few popular extensions that DMX
- does not support. We recommend implementing the X Video Extension
- even if scaling is the specific goal of that work.
-
- We do nnoott recommend implementation of the XCreateScaledWindow
- extension because of the complexity involved. We do nnoott recommend
- implementation of the XPutImageScaled extension because it requires
- the same amount of work as the XCopyAreaScaled extension, but provides
- less functionality. Further, server-side scaling with a large frame
- buffer is nnoott recommended because of the performance implications.
-
- The back-end scaling, especially with disconnect/reconnect support
- should be explored in the future after disconnect/reconnect is
- implemented, but not at the present time.
-
-
-