From f4a658e55b7cf4ae95169c638dee54b7f1ea6a6f Mon Sep 17 00:00:00 2001 From: Arthur Huillet Date: Wed, 1 Feb 2017 15:02:41 +0100 Subject: _XDefaultError: set XlibDisplayIOError flag before calling exit _XReply isn't reentrant, and it can lead to deadlocks when the default error handler is called: _XDefaultError calls exit(1). It is called indirectly by _XReply when a X protocol error comes in that isn't filtered/handled by an extension or the application. This means that if the application (or one of its loaded shared libraries such as the NVIDIA OpenGL driver) has registered any _fini destructor, _fini will get called while still on the call stack of _XReply. If the destructor interacts with the X server and calls _XReply, it will hit a deadlock, looping on the following in _XReply: ConditionWait(dpy, dpy->xcb->reply_notify); It is legal for an application to make Xlib calls during _fini, and that is useful for an OpenGL driver to avoid resource leaks on the X server side, for example in the dlopen/dlclose case. However, the driver can not readily tell whether its _fini is being called because Xlib called exit, or for another reason (dlclose), so it is hard to cleanly work around this issue in the driver. This change makes it so _XReply effectively becomes a no-op when called after _XDefaultError was called, as though an XIOError had happened. The dpy connection isn't broken at that point, but any call to _XReply is going to hang. This is a bit of a kludge, because the more correct solution would be to make _XReply reentrant, maybe by broadcasting the reply_notify condition before calling the default error handler. However, such a change would carry a grater risk of introducing regressions in Xlib. This change will drop some valid requests on the floor, but this should not matter, as it will only do so in the case where the application is dying: X will clean up after it once exit() is done running. There is the case of XSetCloseDownMode(RETAIN_PERMANENT), but an application using that and wishing to clean up resources in _fini would currently be hitting a deadlock, which is hardly a better situation. Signed-off-by: Aaron Plattner Reviewed-by: Jamey Sharp --- nx-X11/lib/X11/XlibInt.c | 10 ++++++++++ 1 file changed, 10 insertions(+) (limited to 'nx-X11') diff --git a/nx-X11/lib/X11/XlibInt.c b/nx-X11/lib/X11/XlibInt.c index 912c52989..5afc65149 100644 --- a/nx-X11/lib/X11/XlibInt.c +++ b/nx-X11/lib/X11/XlibInt.c @@ -3604,6 +3604,16 @@ int _XDefaultError( XErrorEvent *event) { if (_XPrintDefaultError (dpy, event, stderr) == 0) return 0; + + /* + * Store in dpy flags that the client is exiting on an unhandled XError + * (pretend it is an IOError, since the application is dying anyway it + * does not make a difference). + * This is useful for _XReply not to hang if the application makes Xlib + * calls in _fini as part of process termination. + */ + dpy->flags |= XlibDisplayIOError; + exit(1); /*NOTREACHED*/ } -- cgit v1.2.3