Vulkan brings some advantages like async compute, better defined behavior and in general more control over the specifics of what's happening. This is a rough plan on how to get there, and how to do it in steps, without having one giant merge request # Basic infrastructure - [x] Vulkan devices and swapchains: added in !8926 - [x] dmabuf imports: lots of boilerplate, dealt with in !8926 - [ ] texture uploading and updating: it's not very complex on its own, but synchronization can make the simplest task a challenge! Uploading works like this: - [ ] if the data is from a QImage, we first copy to a staging buffer with the CPU. For synchronization, we could - allocate a buffer per upload, pass it along with the `RenderTarget` or something to be deleted once the command buffer is done executing - allocate a buffer per upload, add a fence, get a fence fd, delete the buffer once the fd is signaled - store a list of buffers per texture, with a fence for each. The fence would signal when copying is done, and on the next upload, the idle staging buffers are re-used or deleted - [ ] if the data is from a shm `GraphicsBuffer`, we can instead use `VK_KHR_external_memory_fd` to just import it into memory - we need to keep the `GraphicsBuffer` referenced until the copy is complete / the fence fd is signaled. We could get the fence fd here, and add some helper to keep the buffer referenced until the fence is readable - [ ] then we do an asynchronous buffer -> texture copy with the GPU. This copy can only start once previous read accesses have completed... - a timeline semaphore could be used. The renderer already needs to handle a release point per `SurfaceItem`, extending that to other items should be easy - [ ] scene: port the item renderer, ignore all effects for now. - [ ] the transfer queue should be enough for just copying without color management or any transforms - [ ] shaders are required for making it practically useful - [ ] shaders - [ ] compiling to SPIR-V at build time would be ideal, with specialization constants we could still have different variants - [ ] effects currently use GLSL. If we want backwards compatibility, we need to build SPIR-V at runtime as well # ItemRenderer We have basically two options for how to structure the renderer: 1. for each item, do one render call (just like with our OpenGL renderer) - advantages: - we can use graphics, transfer or compute - scene and effects possibly don't need to change as much - disadvantages: - if effects need a different queue type, special and possibly expensive synchronization would be required - doing many render calls may leave a bunch of performance on the table 2. the scene calls `renderItem(m_containerItem.get())`, the item renderer constructs a list of all items and renders them in a single invokation with a compute shader - advantages: - it's perfect for async compute, which may improve performance and latency - instead of collecting a lot of `RenderNode` for intermediary data representation, we could directly push data into a Vulkan buffer - having one feature-complete shader should be much nicer to deal with than the mess of micro-optimized shaders we have in the OpenGL code path - clipping is easily done by just not invoking the compute shader on parts we don't want to modify - disadvantages: - effects that modify the framebuffer directly or render windows multiple times or whatever are not possible with this. We wanted to get away from that anyways, so that may be okay, but we still need lots of new effects API to make it work - the blur effect would need to work completely differently... the item renderer would need to render items up to the `BackgroundEffectItem`, tell it to copy the area behind the item and update its texture before the item renderer can continue. Same as with other effects, we wanted this anyways, but it needs to be done. - same as with option 1, if effects use a different queue family, special synchronization may be required. Depending on how much we want backwards compatibility with GLSL shaders from effects, this might be a problem, or no problem at all. While option 1 sounds simpler at first, I feel like it may make it more difficult to move to option 2 later, so I'll go for option 2. As for the moment we ignore all effects anyways, it should be fine. # Libkwineffects - [ ] add infrastructure for Vulkan objects - [ ] add infrastructure for Vulkan <-> OpenGL interop - [ ] port offscreen effects to use Vulkan <-> OpenGL interop - [ ] import Vulkan textures into QtQuick - [ ] port effects: - [ ] blur to Vulkan - [ ] colorpicker to Vulkan - [ ] invert to Vulkan - [ ] magnifier to Vulkan - [ ] mouseclick to QML - [ ] mousemark to QML - [x] screenedge to items - [ ] screentransform to Vulkan - [ ] showpaint to Vulkan - [x] ~~snaphelper to QML~~ deleted - [ ] touchpoints to QML - [x] trackmouse to items - [ ] zoom to Vulkan - [ ] startupfeedback to Vulkan or QML - [ ] thumbnailaside to Items # Other plugins - [ ] qpa: port to Vulkan, or use OpenGL interop - [ ] screencast: already uses the renderer, with small amounts of OpenGL glue code. Should be pretty simple to port to Vulkan - [ ] screenshot: same as screencast. Some common abstraction for rendering a scene would help # output backends - [ ] virtual backend: couldn't be simpler - [ ] wayland backend: should be really simple as well - [ ] nested x11 backend: in theory, same as the Wayland backend - [ ] drm backend: more complex. Best course is probably to first simplify the backend by moving shadow buffer stuff to the compositor with some generic infrastructure, then port the drm backend to Vulkan # Longer term goals - port effects from Vulkan <-> OpenGL interop to use Vulkan directly?