diff options
Diffstat (limited to 'xorg-server/hw/dmx/doc/scaled.txt')
| -rw-r--r-- | xorg-server/hw/dmx/doc/scaled.txt | 579 |
1 files changed, 0 insertions, 579 deletions
diff --git a/xorg-server/hw/dmx/doc/scaled.txt b/xorg-server/hw/dmx/doc/scaled.txt deleted file mode 100644 index d30105dd2..000000000 --- a/xorg-server/hw/dmx/doc/scaled.txt +++ /dev/null @@ -1,579 +0,0 @@ - Scaled Window Support in DMX - Rickard E. Faith and Kevin E. Martin - 15 October 2003 (created 19 September 2003) - - This document investigates the possibility of adding scaled window - support to the DMX X server, thereby allowing a window or some - selected part of the logical DMX area to be displayed using a scaling - factor. For example, this might allow the contents of a window to be - magnified for easier viewing. In particular, scaling for the VNC - client is explored. _C_o_p_y_r_i_g_h_t _2_0_0_3 _b_y _R_e_d _H_a_t_, _I_n_c_._, _R_a_l_e_i_g_h_, _N_o_r_t_h - _C_a_r_o_l_i_n_a - - ______________________________________________________________________ - - Table of Contents - - - 1. Introduction - 1.1 DMX - 1.2 Problem Statement - 1.3 Task - - 2. Previous Work - 2.1 VNC - 2.1.1 Scaling under VNC - 2.2 The X Video Extension - - 3. Possible Solutions - 3.1 VNC-like Scaling - 3.1.1 Software Scaling - 3.1.2 Scaling with the X Video Extension - 3.1.2.1 Implementing the X Video Extension for DMX - 3.1.2.2 Supporting RGB formats for the X Video Extension - 3.1.3 Scaling with an XPutImageScaled Extension - 3.1.4 Scaling with an XCopyAreaScaled Extension - 3.1.5 Scaling with OpenGL - 3.2 Application-transparent Scaling for DMX - 3.2.1 Back-end Scaling Without Disconnect/Reconnect - 3.2.2 Back-end Scaling With Disconnect/Reconnect - 3.2.3 Server-side Scaling - 3.3 XCreateScaledWindow API - 3.3.1 XCreateWindow - 3.3.2 XSetWindowAttributes - 3.3.3 XGetWindowAttributes, XGetGeometry - 3.3.4 Popup and Child window positions - 3.3.5 Events - 3.3.6 Implementation - - 4. Conclusion and Recommendations - - - ______________________________________________________________________ - - 11.. IInnttrroodduuccttiioonn - - 11..11.. DDMMXX - - The DMX X server (Xdmx) is a proxy server that is designed to allow X - servers on multiple machines to be combined into a single multi-headed - X server. Combined with Xinerama, these heads can appear as a single - very high-resolution screen. Typical applications include the - creation of a video wall with 16 1280x1024 displays arranged in a - rectangle, for a total resolution of of 5120x4096. - - - - 11..22.. PPrroobblleemm SSttaatteemmeenntt - - Applications displayed on a physically large video wall that provides - high pixel-resolution may be difficult to see, especially if the - application is designed for use on a typical desktop computer with a - relatively small display located close to the human operator. The - goal of this paper is to describe and discuss solutions to this - problem. - - The original driving problem for this work is to provide scaling for - the vncviewer application when displayed using DMX (VNC scaling is - currently available only with the Windows client, and there is no plan - to extend that capability to other clients). While this specific - problem will be addressed in this paper, the general solution space - will also be explored, since this may lead to a good solution not only - for vncviewer but also for other applications. - - 11..33.. TTaasskk - - For reference, here is the original description of the task this paper - addresses: - - +o Scaled window support (for VNC) - - +o Investigate possibility of implementing a "scaled window" - extension: - - +o Add XCreateScaledWindow call that could be used in place of - XCreateWindow - - +o All primitives drawn to scaled window would be scaled by - appropriate (integral?) scaling factor - - +o Alternate approach: special case VNC support - - 22.. PPrreevviioouuss WWoorrkk - - This section reviews relevant previous work. - - 22..11.. VVNNCC - - 22..11..11.. SSccaalliinngg uunnddeerr VVNNCC - - When using the vncviewer program for Windows, it is possible to - specify a scaling factor (as numerator and denominator). When scaling - is in effect, the viewer software uses StretchBlt (instead of BitBlt) - to display the pixels for the user. When this call is made, the - viewer already has received all of the pixel information (at full - unscaled resolution). - - The scaling in VNC is primitive. It does not conserve bandwidth, it - does not treat textual information differently (i.e., by using a - suitably scaled font), and it does not provide any anti-aliasing other - than that provided by the underlying (Windows-only) system library. - - 22..22.. TThhee XX VViiddeeoo EExxtteennssiioonn - - The X Video Extension is a widely-available extension to the X11 - protocol that provides support for streaming video. Integral to this - support is the ability to arbitrarily scale the output. In version - 2.2 of the X Video specification, support for scaled still images was - provided, using both shared memory and traditional transport. The API - for this support uses calls that are quite similar to XCreateWindow, - XPutImage, and XShmPutImage. Currently, most of the drivers - implemented in XFree86 only support data in various YUV formats. - However, several modern video adaptors support RGB as well. - Note, though, that the target output for this scaling is an overlay - plane -- so X Video provides functionality that is fundamentally - different from that provided by the Windows StrechBlt call. - - 33.. PPoossssiibbllee SSoolluuttiioonnss - - This section briefly discusses possible solutions, including major - advantages and disadvantages from both the implementation and the end- - user programmer standpoint. - - 33..11.. VVNNCC--lliikkee SSccaalliinngg - - 33..11..11.. SSooffttwwaarree SSccaalliinngg - - The vncviewer application could be modified to provide software - scaling. This is not a general solution, but it does solve one of the - goals of this work. - - A prototype of this solution was implemented and a patch against - vnc-3.3.7-unixsrc is available in the dmx/external directory. Because - of limited time available for this work, all of the edge cases were - not considered and the solution works well mainly for integer scaling. - - Currently, vncviewer writes to the X display with XPutImage, - XCopyArea, and XFillRectangle. All instances of these calls have to - be aware of scaling and must round correctly. In the prototype - solution, rounding is incorrect and can cause artifacts. - - A better solution would be to cache all updates to the desktop image - in vncviewer and only send the damaged area to the X display with - XPutImage. This would allow the damaged area to be computed so that - rounding errors do not create artifacts. This method is probably - similar to what is used in the Window client. (The whole VNC suite is - being re-written in C++ and the forthcoming version 4 has not been - evaluated.) - - 33..11..22.. SSccaalliinngg wwiitthh tthhee XX VViiddeeoo EExxtteennssiioonn - - The scaling in the Windows vncviewer application makes use of a scaled - blit that is supplied by the underlying system library. Several video - cards currently provide support for a scaled blit, and some X servers - (including XFree86) expose this capability to applications via the - XvPutImage interface of the X Video Extension. The capability exposed - by XvPutImage results in the scaled image being drawn to an overlay - plane. Most video cards also provide support for a scaled blit into - the normal output planes, but this is not exposed via XvPutImage. - - The vncviewer program could be modified to use the X Video Extension - to provide scaling under X11 that is similar to the scaling currently - provided under Windows. Unfortunately, Xdmx does not currently export - the X Video Extension, so this would not provide an immediate solution - usable with DMX. - - A very early-stage proof-of-concept prototype was implemented and a - preliminary patch against vnc-3.3.7-unixsrc is available in the - dmx/external directory. This prototype was implemented to better - understand the problems that must be solved to make this solution - viable: - - +o As noted under the software scaling section above, vncviewer writes - to the X display with several different calls. These calls write - to the normal output planes and are compatible with XvPutImage, - which writes to an overlay plane. To eliminate artifacts caused by - this problem, vncviewer should be modified so that a cached copy of - the desktop is available, either as a client-side image or a - server-side off-screen pixmap, so that XvPutImage would be the only - method for writing to the X display. - - +o - - Although several modern graphics adaptors support hardware scaling - using an RGB format (e.g., ATI Radeon, nVidia, etc.), XFree86 - drivers typically only implement YUV formats. YUV generally - compress the pixel information in some way. For example, two - commonly implemented formats, YUY2 and UYVY provide intensity - information for every RGB pixel, but only provide chroma and - luminance information for pairs of horizontal pixels. Since VNC - uses pixel-resolution for communicating updates on the wire, - additional artifacts are introduced (because there may not be - enough information from the wire to update a pair of pixels). - - Further, the well-known problem with YUV encoding is even more - evident when the image is a desktop instead of a movie. For - example, consider a 1-pixel-wide vertical window border. If the - border changes in color but not intensity (e.g., because a window - manager uses color to indicate focus), there may or may not be a - change in the YUY2 image, depending on the algorithm used for RGB - to YUV conversion and on how the border pixel is ordered in the - pair of pixels used by the algorithm. - - Many of these artifacts could be eliminated if vncviewer cached a - complete RGB image of the desktop, and only did the conversion to - YUV for properly aligned areas of damage. The remaining artifacts - could be eliminated if an RGB format was used with X Video (which - may require the extension of existing XFree86 drivers to support - RGB). - - +o Most modern video cards support exactly one overlay plane that is - suitable for use with X Video. Therefore, only one application can - use X Video at any given time. This is a severe limitation in a - desktop environment. - - 33..11..22..11.. IImmpplleemmeennttiinngg tthhee XX VViiddeeoo EExxtteennssiioonn ffoorr DDMMXX - - The user-level API for X Video is fairly simple, but the underlying - support required for the full specification is large. However, since - the API provides a method to query supported capabilities, a usable - subset of X Video can be implemented that would support XvPutImage and - little else. This would require support for the following: - - +o X Video Extension API calls, including the following: - - +o XvQueryExtension - - +o XvQueryAdaptors - - +o XvQueryPortAttributes - - +o XvFreeAdaptorInfo - - +o XvListImageFormats - - +o XvGrabPort - - +o XvCreateImage - - +o XvPutImage - - +o XvShmCreateImage - - +o XvShmPutImage - - +o Support for querying back-end X Video Extension capabilities. - - +o Support for sending the image to the back-ends. Because X Video - requires sending full images, there may be a trade-off between - bandwidth limitations and additional complexity to divide the image - up such that is scales properly. - - +o Possible support for a software fall-back. For example, if all of - the back-ends do not support the X Video Extension, software - scaling can be implemented such that the image is sent to the back- - end with XPutImage. This pathway would have poor performance. - - 33..11..22..22.. SSuuppppoorrttiinngg RRGGBB ffoorrmmaattss ffoorr tthhee XX VViiddeeoo EExxtteennssiioonn - - Assuming an XFree86 driver already supports the X Video Extension, and - assuming the target hardware supports an RGB format, then adding - support for that format is relatively simple and straightforward. - - 33..11..33.. SSccaalliinngg wwiitthh aann XXPPuuttIImmaaggeeSSccaalleedd EExxtteennssiioonn - - Instead of (or in addition to) implementing the X Video Extension in - DMX, one obvious solution would be to implement a new extension that - provides access to hardware-assisted scaled blits, similar to the - StretchBlt call available under Windows. This call would scale RGB - images and would not use the overlay plane (unlike the X Video - Extension). - - This approach has many of the same advantages and disadvantages as the - XCopyAreaScaled Extension, discussed in the next section. Discussion - of XPutImageScaled is deferred in favor of XCopyAreaScaled for the - following reasons: - - +o XPutImageScaled can be emulated with XCopyAreaScaled by first using - XPutImage to copy the image to an off-screen pixmap, and then - calling XCopyAreaScaled between that off-screen pixmap and the - target drawable. - - +o Since XCopyAreaScaled would copy between two areas of on-screen or - off-screen memory, it has additional uses and can be viewed as - efficiently providing a superset of XPutImageScaled functionality. - - 33..11..44.. SSccaalliinngg wwiitthh aann XXCCooppyyAArreeaaSSccaalleedd EExxtteennssiioonn - - As noted in the previous section, because XCopyAreaScaled provides a - superset of the functionality provided by XPutImageScaled, we will - consider this extension instead. - - First, XCopyAreaScaled would provide for RGB scaling between pixmaps - (i.e., on-screen or off-screen areas of memory that reside on the - video card). Unlike the X Video Extension, which writes into an - overlay plane, XCopyAreaScaled would write into the non-overlay areas - of the screen. Key points to consider are as follows: - - +o Because different planes are involved, the two scaling operations - are usually implemented in hardware differently, so an - XCopyAreaScaled extension could be added in a manner that would - neither conflict with nor interact with the X Video extension in - any way. - - +o The XCopyAreaScaled extension provides new functionality that the X - Video Extension does not provide. Based on anecdotal feedback, we - believe that many people outside the DMX and VNC communities would - be excited about this extension. - - +o The main drawback to this extension is that it is new and needs to - be implemented at the driver level in XFree86 for each video card - to be supported. At the present time, it is more likely that the X - Video Extension will be implemented for a particular piece hardware - because the X Video extension has multimedia uses. However, over - time, we would expect the XCopyAreaScaled extension to be - implemented along with the X Video extension, especially if it - becomes popular. - - +o Another drawback is that not all modern cards provide support for a - simple scaled blit operation. However, these cards usually do - provide a 3D pipeline which could be used to provide this - functionality in a manner that is transparent to the client - application that is using the XCopyAreaScaled extension. However, - this implementation pathway would make this extension somewhat more - difficult to implement on certain cards. - - 33..11..55.. SSccaalliinngg wwiitthh OOppeennGGLL - - Another general solution to the scaling problem is to use the texture - scaling found in all 3D hardware. This ability is already exposed - through OpenGL and can be exploited by clients without X server - modification (i.e., other than the ability to support OpenGL). An - application using OpenGL would transmit the non-scaled image to the X - server as a texture, and would then display a single non-transformed - rect using that texture. This also works around the single overlay - problem with the X Video Extension as well as the need to implement - additional scaled primitive extensions. - - The downside is that most OpenGL implementations require power of 2 - texture sizes and this can be very wasteful of memory if, for example, - the application needs to scale a 1025x1025 image, which would require - a 2048x2048 texture area (even a 640x480 image would require a - 1024x512 texture). Another downside is that some OpenGL - implementations have a limited about of texture memory and cannot - handle textures that are very large. For example, they might limit - the texture size to 1024x1024. - - 33..22.. AApppplliiccaattiioonn--ttrraannssppaarreenntt SSccaalliinngg ffoorr DDMMXX - - 33..22..11.. BBaacckk--eenndd SSccaalliinngg WWiitthhoouutt DDiissccoonnnneecctt//RReeccoonnnneecctt - - VNC does scaling on the client side (in the vncviewer application). - Implementing a similar solution for DMX would require support in the - back-end X servers and, therefore, is not a general solution. - - XFree86 already implements some support for "scaling" that could be - used with DMX: if, in the XF86Config file, multiple Modes are listed - in the Display Subsection of the Screen Section, then pressing Ctrl- - Alt-Plus and Ctrl-Alt-Minus can be used to iterate through the listed - modes. The display dimensions will change to the dimensions in the - Modes line, but the logical dimensions of the X server (i.e., the - dimensions that Xdmx knows about) will not change. - - Further, the dimensions of the XFree86 display are under software - control (via the XFree86-VidModeExtension), so the Xdmx server could - change the screen dimensions on a per-display basis, thereby scaling - the information on part of that display. - - However, this scaling appears to have limited use. For example, - assume a 4 by 4 display wall consisting of 16 1280x1024 displays. If - all of the back-end servers were simultaneously configured to display - 640x480, the left hand corner of each display would be magnified, but - the composite result would be unreadable. Magnifying one display at a - time could be usable, but could have limited utility, since the result - would still be no larger than a single display. - - - 33..22..22.. BBaacckk--eenndd SSccaalliinngg WWiitthh DDiissccoonnnneecctt//RReeccoonnnneecctt - - Disconnect and reconnect features are not currently supported in DMX, - but are scheduled to be implemented in the future. These features, - combined with the XFree86-VidModeExtension Extension, would allow an - application to do the following: - - +o Disconnect a specific back-end server (via the DMX Extension), - - +o reconfigure the XFree86 back-end server resolution, and - - +o reconnect the back-end server to DMX -- at a new origin with the - new screen resolution. - - For example, consider a display wall consisting of 16 1280x1024 - displays with a total resolution of 5120x4096. All of the screens - could be disconnected, repositioned, and reconnected each at a - resolution of 640x480. The total resolution of the display wall would - be 2560x1920, allowing a view of a selected area approximately one- - fourth of the size of the DMX display. This change would be - completely application independent (except, perhaps, for a DMX-aware - window manager). When work at the increased resolution was completed, - the back-end servers could be disconnected, reconfigured, and - reconnected for the original 5120x4096 view. - - Support for this type of scaling can be implemented in a DMX-aware X11 - client assuming the DMX server support arbitrary disconnect and - reconnect semantics. Because this application cannot be written - before disconnect/reconnect is implemented, this solution will not be - discussed further in this paper. - - 33..22..33.. SSeerrvveerr--ssiiddee SSccaalliinngg - - In earlier versions of DMX, a frame buffer was maintained on the - server side, and XPutImage was used to move the information from the - server to the client (similar to some early VNC implementations). The - use of a server-side frame buffer would allow the server to do - scaling, but is not a recommended solution because of overall - performance issues and server-side memory issues (i.e., the frame - buffer would be very large for large display walls). - - Exploration of this path is not recommended. - - 33..33.. XXCCrreeaatteeSSccaalleeddWWiinnddooww AAPPII - - The implementation of X Video Extension in DMX, and the use of - XvPutImage by applications requiring scaling requires significant - changes in DMX Further, XvPutImage is, essentially a scaled blit, and - it is only useful for applications which are already using (or can be - modified to use) XPutImage. Therefore, a more general API will be - discussed as another possibility. - - X applications typically create windows with the XCreateWindow call. - A new extension could provide an XCreateScaledWindow call that could - be used in place of the XCreateWindow call and be otherwise - transparent to the application. This would allow applications, even - those that do not depend on XPutImage, to take advantage of window - scaling. In this section we describe how the call would work, what - transparency it provides, and how to solve the potential problems that - transparency creates. - - 33..33..11.. XXCCrreeaatteeWWiinnddooww - - The XCreateWindow call takes width and height as parameters. An - XCreateScaledWindow call could take all the same parameters, with the - addition of a scaling factor. - 33..33..22.. XXSSeettWWiinnddoowwAAttttrriibbuutteess - - An X11 window has several attributes that would have to be scaled: - - +o Background and border pixmaps - - +o Border width - - +o Cursor - - 33..33..33.. XXGGeettWWiinnddoowwAAttttrriibbuutteess,, XXGGeettGGeeoommeettrryy - - For transparency, calls that query the window attributes should return - unscaled information. This suggests that all unscaled pixmaps and - window attributes should be cached. - - Unfortunately, a window manager requires the scaled geometry to - properly decorate the window. The X server can probably determine - which client is acting as the window manager (e.g., because that - client will select events that are used exclusively by the window - manager). However, other Scaled Window Extension aware clients may - also need to determine the scaled geometry. Therefore, at least two - additional extension calls should be implemented: - XGetScaledWindowAttributes and XGetScaledGeometry. - - 33..33..44.. PPooppuupp aanndd CChhiilldd wwiinnddooww ppoossiittiioonnss - - Some applications may position popup and child windows based on an - unscaled notion of the main window geometry. In this case, additional - modifications to the client would be required. - - 33..33..55.. EEvveennttss - - Most events (e.g., for mouse motion) return information about the - coordinates at which the even occurred. These coordinates would have - to be modified so that unscaled values were presented to the client. - - 33..33..66.. IImmpplleemmeennttaattiioonn - - There are many implementation issues, some of which are similar to the - issues involved in implementing the X Video Extension for DMX. The - window contents must be scaled, either by performing all operations to - a frame buffer and then writing the image to the display (perhaps - using hardware scaling support), or by modifying all of the various - drawing operations to perform scaling. Because of the complexity - involved, the frame buffer option is recommended. - - 44.. CCoonncclluussiioonn aanndd RReeccoommmmeennddaattiioonnss - - We recommend a three phase implementation strategy, based on how an - application could be written to take advantage of scaling: - - 1. - - The XCopyAreaScaled extension should be implemented, since this is - the ideal solution for applications like VNC, and since making use - of this extension will require minimal changes to applications that - already use XPutImage or XCopyArea. - - The initial implementation work would include the design of the X - protocol extension, writing this up in the usual format for - extension documentation, implementation of the protocol transport - pieces in XFree86, implementation of a software fall-back in - XFree86 and DMX, one example hardware implementation for XFree86, - and implementation of support for this extension in DMX. - - We suggest implementing the extension first on the ATI Radeon - cards. However, since these cards do not provide a 2D scaled blit - primitive, the implementation would have to make use of the 3D - texture engine to emulate a scaled blit. This is recommended, - since other modern graphics cards also do not provide a simple 2D - scaled blit operation and an example of the more difficult - implementation pathway would be helpful to others. - - 2. - - Until XCopyAreaScaled is widely supported, applications that - require scaling will have to fall back to another scaling method. - We suggest OpenGL as the first fall-back method because it is - widely available and supported by DMX. - - A project centered around OpenGL-based scaling would implement this - scaling in VNC as an example. This work would include re-writing - the vncviewer rendering engine to cache a master copy of the - desktop image for all operations. - - 3. - - Since OpenGL is not implemented everywhere, and may not provide - hardware-assisted performance in every implementation, an - application that requires scaling should also fall back to using - the X Video Extension. - - This project would add support for the X Video Extension to DMX and - would add support to VNC to take advantage of this extension - without introducing artifacts. This would require modifying the - vncviewer rendering engine to cache a master copy of the desktop - image for all operations. This project should also add support for - the RGB format to at least one XFree86 driver (e.g., ATI Radeon). - - The X Video Extension is one of the few popular extensions that DMX - does not support. We recommend implementing the X Video Extension - even if scaling is the specific goal of that work. - - We do nnoott recommend implementation of the XCreateScaledWindow - extension because of the complexity involved. We do nnoott recommend - implementation of the XPutImageScaled extension because it requires - the same amount of work as the XCopyAreaScaled extension, but provides - less functionality. Further, server-side scaling with a large frame - buffer is nnoott recommended because of the performance implications. - - The back-end scaling, especially with disconnect/reconnect support - should be explored in the future after disconnect/reconnect is - implemented, but not at the present time. - - - |