The main motivation behind the split is to simplify client buffer code
and allow adding new features easier, for example referencing the shm
pool when a shm buffer is destroyed, or monitoring for readable linux
dmabuf file descriptors, etc.

Also, a referenced ClientBuffer cannot be destroyed, unlike the old
BufferInterface.