Skip to content
  • Michael Pyne's avatar
    tag_scan: Fix painful rescan of music metadata on startup. · d6b28a9b
    Michael Pyne authored
    For the longest time, JuK has suffered from a problem where its intended
    behavior to load music metadata from a cached database, instead of
    re-scanning every individual track on startup, was not working right.
    
    There has been debugging lines in JuK since all the way back to 2013
    trying to trace what area of startup sequence was taking up all the
    time, but to no avail in helping me fix the bug.
    
    The Problem
    ===========
    
    Recently I took a different approach, of adding a debug-only crash
    whenever we would load a music track tag the "slow" way, and long story
    short there were two bugs that each would cause slowdown:
    
    1. Playlists aside from the CollectionList would cause every music track
       in that playlist to be re-scanned. What this means is that every
       though the music in the CollectionList would be loaded quickly, if
       you had that same music track in a separate Playlist, that music
       track would reload the same tags from disk rather than copying from
       the existing CollectionList item.  This was especially bad for users
       of the old "tree mode" view, since every individual artist *and*
       album were rendered as individual playlists, which would therefore
       each re-scan the music over and over again.
    2. JuK supports a "folder scan" feature, and in fact really wants the
       user to have at least one folder assigned. Any music identified in
       this folder is added to the CollectionList automatically on startup
       and, you guessed it, causes the music track information to be loaded
       from disk, even if the music was already in the CollectionList! :(
    
    The net effect is that most music would be re-scanned on startup unless
    you were a user who used CollectionList exclusively, and had most of
    your music not visible to the folder scanner.
    
    The Solution
    ============
    
    Due to how painful this problem has been, I had ended up adding a
    threaded solution for the folder scan process. This didn't help make
    things any faster but at least the GUI wasn't frozen. But now that the
    threading code is present I judged it would be easier and safer to make
    the central object holding track metadata (CollectionList's m_itemsDict)
    available in thread-safe fashion.
    
    This then permitted me to check for whether a track has already been
    loaded when performing folder scan, and to check whether a track has
    already been loaded when creating a new (non-CollectionList) Playlist.
    In either event if the track already exists, then we copy the FileHandle
    rather than create a new one.
    
    The combination speeds up loading significantly, taking anywhere from
    60% to 70% off of the total time to load on my system, with mostly a
    CollectionList under folder scan and few additional playlists. In this
    configuration I go from about 5.4 seconds to 1.5 seconds with cold
    caches. The difference should be even more stark on systems where disk
    I/O is expensive, or where there are a great number of tracks in
    playlists outside of the CollectionList.
    
    I consider this a bugfix (and there are even multiple bug reports) so I
    will backport shortly.
    
    CHANGELOG:Reduce startup time by 60-70% or more.
    BUG:428772
    BUG:317666
    FIXED-IN:21.04
    d6b28a9b