pulse-server: use a work-queue to schedule destroy

Use the context work queue to schedule destroys from callbacks.
This is better because we can pass the destroyed object around and
implement just the action we need to do on it.