Skip to content

[Fix] Implement chunk based file transfer

Apollo Zhu requested to merge work/apollozhu/file-transfer into master

Currently, share plugin will load all the files been transferred into app's memory. This is problematic because iOS limits how much memory each app can use, so transferring large files will cause the app to run out of memory and crash. To make the matter worse, somehow bridging between NSData and Data caused Swift to make a copy of NSData's contents, doubling the memory needed to transfer files.

This merge requests implements chunk based file transfer, where only a small portion of file contents is loaded into memory, while most remain on disk, for both sending and receiving. To simplify the logic, urls to payloads (i.e. file contents) are now stored in separate arrays for sending and receiving, so locking mechanism is simpler: use _pendingLSockets for locking shared resources for sending out files, and _pendingRSockets for receiving. Thus, it should also fix the "refresh discovery freezes app" issue that people have reported.

During the implementation process I've identified some potential issues to be fixed, but they remain TODOs to keep the diff/change in behavior more contained.

Testing

The app's memory usage remained around 30MB before, during, and after the file transfer process. I was able to successfully receive large files such as Xcode.xip (around 7GB) with up to 20MiB/s transfer speed, receive multiple files transferred together, send a single/multiple file/photos/videos between KDE Connect iOS and KDE Connect Qt macOS client, and between iOS and iOS (with and without chunked file transfer)

Transfer Speed can be up to 20 MiB per second

Transfer speed dropped to around 10 MiB/s during when I tried to transfer another large file, but I suspect this is likely due to network condition instead of problem in the implementation. The following shows CPU, memory, disk, and network usage:

Xcode

Edited by Apollo Zhu

Merge request reports