Suggestion: "In memory area/range" on the timeline.
Kdenlive has huge problems with larger projects because of system memory usage. RAM is expensive, and with a little management these issues may be alleviated. As always, there is a trade-off between speed and cost, and no one single solution will make everyone happy.
The situation: As far as I can tell, Kdenlive insists on loading every clip that it knows about into memory. Due to the high bitrate of some of the footage out there, memory is consumed pretty quickly. This effectively limits project size to the available RAM. <- Big problem!
As an example, the entry level Sony A7S II can produce 100mb/s footage all day long. That's 1Gb every 10 seconds, or 100Gb in 16 minutes.
There are two independent ways that I can imagine that this issue can be alleviated:
- Proxy clips.
When a proxy is enabled for a clip, only load the proxy into memory, and explicitly unload the original clip. Kdenlive then only works with the proxy, and only on rendering time is the original read.
Because much lower resolution and higher compression can be achieved on proxy clips, their memory footprint can be much lower than the original footage. (This may already be the case, I don't use proxies much)
- "In memory" range from the cursor.
Conceivably, one could define a range in time from the cursor. Any footage outside of this range can be dropped from memory, and footage that scrolls into this range can be pre-loaded before the cursor gets there when playing. In large projects, one typically works on a small area at a time. In this small area on the timeline is the only place where having the footage in memory is an advantage.
When the system starts swapping because some footage from far away is hogging the memory, all these advantages are lost. It would make sense to drop the stale footage from memory to avoid running out of memory, no?
Yes, this plan would make Kdenlive less responsive when madly clicking around in the timeline as a lot of footage loading would happen. This is why this should be configurable... maybe some range in minutes, or completely disabled for the memory kings out there.
Also, some visual cue that the footage is not in memory anymore, like fading or greying out the clips on the timeline would be nice. Think "Fog of war" in video games style.
Additionally, unload the clips in the project bin from memory if they have not been accessed in some definable range of time. I'm ok with the slight delay of re-accessing a clip that I have not looked at in some time. I'm not ok with my entire system grinding to a halt while it's swapping madly because it's trying to load 100GB of footage into 20GB of RAM when opening a project. This is what thumbnails are for.
For both of these solutions, there is something that the user has to enable, so if you DO have a workstation with 512GB of RAM or more, and you would totally like to avoid the lags of loading footage, just don't turn on the memory-saving features.
Discuss. ;)