This reverts af7b89d7 commit. It causes an oops on X server shutdown here, and for the reporter of bug #7629 as well.