Summary:
virtual desktops can be destroyed, active clients update, but deleted
keeps a cache.

Someone needs to do cleanup to avoid dangly pointers.

Test Plan:
I couldn't find a case of someone calling desktops mid way through an
animation, so it's only a hypothetical bug.

Reviewers: #kwin, zzag

Reviewed By: #kwin, zzag

Subscribers: graesslin, zzag, kwin

Tags: #kwin

Differential Revision: https://phabricator.kde.org/D17247