ScreenCast : fix edge-case for format modifier fixation
The current implementation for DMA-Buf sharing on kwin is slightly broken.
When a client connects to the kwin screencast's stream, it supplies a SPA_PARAM_EnumFormat
list, containing entries for every supported format, and the drm modifier list for that format (for dmabuf sharing).
Usually, the SPA_FORMAT_VIDEO_modifier
property from the consumer has the SPA_POD_PROP_FLAG_DONT_FIXATE
flag set. Kwin then receives the full choice list, and gets to pick a relevant modifier.
In situations where the DONT_FIXATE flag is missing, pipewire chooses an arbitrary match between the consumer and kwin. In that case, kwin currently assumes DRM_FORMAT_MOD_INVALID is to be used no matter what, which goes against the comment right above it within the screencaststream.cpp file. Even worse, if DRM_FORMAT_MOD_INVALID is not supported by the consumer, this also causes the param_changed callback to freak out, as one of the conditions to update the stream params is !receivedModifiers.contains(m_dmabufParams->modifier
. Since m_dmabufParams->modifier contains DRM_FORMAT_MOD_INVALID, which is not among the modifiers kwin receives, the params are changed continuously and no buffer sharing can happen.
--
I'd be happy to supply code to reproduce what I attempted to describe above if needed. I'm not aware of any program currently affected, but I found this out when messing around myself.
Relevant docs are here : https://docs.pipewire.org/page_dma_buf.html
If the SPA_PARAM_Format contains a modifier key, without the flag SPA_POD_PROP_FLAG_DONT_FIXATE, it should only contain one value in the SPA_CHOICE_Enum. In this case announce the SPA_PARAM_Buffers accordingly to the selected format and modifier.