somebody forgot to actually erase 😞

this would eventually result in a crash as the container would have
dangling pointers as remove only moves around the entries so the
container would be the same size before and after remove but one of the
pointers would then get deleted in the eventloop

BUG: 428746
FIXED-IN: 5.20.3


(cherry picked from commit fd099a75)