Skip to content

Draft: writetransaction: add approximate pending data tracking

Harald Sitter requested to merge work/sitter/kf5-initial-commiting into kf5

and use it to regularly commit firstrun documents.

this reduces the memory footprint substantially when initially scanning directories with a large number of documents. previously these would all get pushed into the same transaction potentially exhausting memory before the transaction is actually complete (even more so now that we have resource constraining when used through systemd)

getting the approximate size takes a lot of guessing cause I don't want to track all the data down to the last byte, a rough idea of how much memory is probably being used is enough for our optimization purposes since this is effectively a somewhat more dynamic commit-every-N-loops kind of behavior.

Merge request reports