[Fix] Implement chunk based file transfer
Currently, share plugin will load all the files been transferred into app's memory. This is problematic because iOS limits how much memory each app can use, so transferring large files will cause the app to run out of memory and crash. To make the matter worse, somehow bridging between NSData
and Data
caused Swift to make a copy of NSData
's contents, doubling the memory needed to transfer files.
This merge requests implements chunk based file transfer, where only a small portion of file contents is loaded into memory, while most remain on disk, for both sending and receiving. To simplify the logic, urls to payloads (i.e. file contents) are now stored in separate arrays for sending and receiving, so locking mechanism is simpler: use _pendingLSockets
for locking shared resources for sending out files, and _pendingRSockets
for receiving. Thus, it should also fix the "refresh discovery freezes app" issue that people have reported.
During the implementation process I've identified some potential issues to be fixed, but they remain TODOs to keep the diff/change in behavior more contained.
Testing
The app's memory usage remained around 30MB before, during, and after the file transfer process. I was able to successfully receive large files such as Xcode.xip (around 7GB) with up to 20MiB/s transfer speed, receive multiple files transferred together, send a single/multiple file/photos/videos between KDE Connect iOS and KDE Connect Qt macOS client, and between iOS and iOS (with and without chunked file transfer)
Transfer speed dropped to around 10 MiB/s during when I tried to transfer another large file, but I suspect this is likely due to network condition instead of problem in the implementation. The following shows CPU, memory, disk, and network usage: