In-process workers: socket-less connection (with zero-copy)
In-process WorkerThread workers (file, admin by default) previously talked to the application over a real QLocalSocket pair, even though both ends share an address space. This MR adds a ThreadConnectionBackend that exchanges Tasks directly through a shared, mutex-protected channel: a queued invocation wakes the application (event loop), a wait condition wakes the worker (polled), back-pressure is bounded worker→application, and the payload is handed over without a copy — the application reads the worker's QByteArray directly.
Besides being faster, this removes the waitForBytesWritten() back-pressure deadlock class for in-process workers (a joined worker thread could otherwise wedge against the main thread).
Performance
Release build. Worker→application transport, micro-benchmark with a fresh buffer allocated per send (so the zero-copy path still pays to produce the data it moves):
| message size | socket | thread (this MR) | speedup |
|---|---|---|---|
| 256 B | 14 MB/s | 97 MB/s | 6.8× |
| 4 KiB | 101 MB/s | 441 MB/s | 4.4× |
| 64 KiB | 247 MB/s | 997 MB/s | 4.0× |
| 512 KiB | 121 MB/s | 831 MB/s | 6.9× |
The gain comes from dropping the socket's syscalls and double-buffering in favour of sharing the worker's QByteArray directly. File content travels in 512 KiB chunks, i.e. the high end of the table.
End-to-end file reads (KIO::get/storedGet) are faster as well, but there the bottleneck shifts to disk and the application's own data handling (e.g. StoredTransferJob accumulating the whole file), so the transport is no longer the limiting factor — the transport micro-benchmark above is the representative figure.
Note: file_copy (local→local) is unaffected — it never traverses the IPC (the worker uses copy_file_range internally); the gain is on the data-streaming path.
Contract change
Because the thread backend delivers asynchronously and no longer copies, in-process workers must pass an owned QByteArray (not a fromRawData() view over a transient/reused buffer). kio_file is updated accordingly (get()/read() read into owned buffers); the admin worker doesn't use fromRawData. Out-of-process workers are unaffected — the socket backend still serializes synchronously on send. The invariant is enforced by the ASan CI (jobtest).
Commits
- make ConnectionBackend an abstract transport with a socket backend
- add ThreadConnectionBackend for in-process workers
- run in-process workers over ThreadConnectionBackend instead of a socket
- add ThreadConnectionBackend unit test
This is an optimization that was made possible since file:/ started using threads. I have been meaning to do this for a long while.
This benefit mostly listing directory where a lot of data needs to be sent over the wire and to a lesser extent copying many small files.
That's a logical continuation of !740 (merged) This address my 2022 concern: !740 (comment 396387)
AI: Claude opus 4.8