tag_scan: Fix painful rescan of music metadata on startup.
For the longest time, JuK has suffered from a problem where its intended behavior to load music metadata from a cached database, instead of re-scanning every individual track on startup, was not working right. There has been debugging lines in JuK since all the way back to 2013 trying to trace what area of startup sequence was taking up all the time, but to no avail in helping me fix the bug. The Problem =========== Recently I took a different approach, of adding a debug-only crash whenever we would load a music track tag the "slow" way, and long story short there were two bugs that each would cause slowdown: 1. Playlists aside from the CollectionList would cause every music track in that playlist to be re-scanned. What this means is that every though the music in the CollectionList would be loaded quickly, if you had that same music track in a separate Playlist, that music track would reload the same tags from disk rather than copying from the existing CollectionList item. This was especially bad for users of the old "tree mode" view, since every individual artist *and* album were rendered as individual playlists, which would therefore each re-scan the music over and over again. 2. JuK supports a "folder scan" feature, and in fact really wants the user to have at least one folder assigned. Any music identified in this folder is added to the CollectionList automatically on startup and, you guessed it, causes the music track information to be loaded from disk, even if the music was already in the CollectionList! :( The net effect is that most music would be re-scanned on startup unless you were a user who used CollectionList exclusively, and had most of your music not visible to the folder scanner. The Solution ============ Due to how painful this problem has been, I had ended up adding a threaded solution for the folder scan process. This didn't help make things any faster but at least the GUI wasn't frozen. But now that the threading code is present I judged it would be easier and safer to make the central object holding track metadata (CollectionList's m_itemsDict) available in thread-safe fashion. This then permitted me to check for whether a track has already been loaded when performing folder scan, and to check whether a track has already been loaded when creating a new (non-CollectionList) Playlist. In either event if the track already exists, then we copy the FileHandle rather than create a new one. The combination speeds up loading significantly, taking anywhere from 60% to 70% off of the total time to load on my system, with mostly a CollectionList under folder scan and few additional playlists. In this configuration I go from about 5.4 seconds to 1.5 seconds with cold caches. The difference should be even more stark on systems where disk I/O is expensive, or where there are a great number of tracks in playlists outside of the CollectionList. I consider this a bugfix (and there are even multiple bug reports) so I will backport shortly. CHANGELOG:Reduce startup time by 60-70% or more. CCBUG:428772 CCBUG:317666 FIXED-IN:21.04 (cherry picked from commit d6b28a9b)
Showing with 90 additions and 33 deletions