file: add a batch-copy command to the file worker
Batch copy of many small files
On master, CopyJob copies many files one KIO::file_copy job at a time. Each
file is its own job and scheduler dispatch plus a serialized command round-trip
to the worker, which dominates the wall time for small files where the data copy
itself is tiny. This MR adds a file-worker command that copies a list of files in
one round-trip, used by CopyJob.
What changes
- File worker: a new
special()command copies a list of local (source, destination) pairs in one round-trip. It copies withcopy_file_rangeand a read/write fallback, and preserves the same metadata ascopy()(permissions, timestamps, ownership, POSIX ACL, extended attributes). It only creates fresh files. An existing destination (or the source itself via a hardlink, or a symlink) is reported back as a conflict and left tocopy(). The engine is infile_unix.cpp.file_win.cppreturnsERR_UNSUPPORTED_ACTION. - CopyJob: when the next files to copy are a run of plain local-to-local
files on a local filesystem, it dispatches them as one batch instead of one
file_copyjob each. Conflicts and per-item errors are requeued and handled by the existing per-file path (conflict dialog, skip, rename), so progress and undo are unchanged.
Performance
Release build. copybatchbench, 5000 x 4 KiB files, serial
KIO::file_copy (master) vs the batch command:
| storage / cache | serial | batch | speedup |
|---|---|---|---|
| tmpfs, warm | ~4k to 6k files/s | ~60k files/s | ~10x to 15x |
| SSD, warm | ~3.3k files/s | ~24k files/s | ~7x |
| SSD, cold | ~2.6k files/s | ~12k files/s | ~5x |
Edited by Méven Car