Some clients (like xdotool) can send multiple events for the same
key. In some instances the number of presses and releases can even
be mismatched leading to stuck keys. While we clean up pressed keys
when the client vanishes, this doesnt help when it doesnt immediately
disconnect (for example Xwayland keeps client connections live
longer than the actual x client to reuse them).
Also fixes a copy paste mistake where key presses where also written
to buttons.