I've written before about our metrics system and discussed at Akademy. I now want to move forwards with a replacement. # Why the current system doesn't work - We didn't allow an upgrade path. Because we asked the user ahead of time "Can I collect your screen resolution", when we wanted something new (like DPI) we can't just add it. Even though no user in the world would ok with screen resolution but not DPI, it's not right to just change what the user agreed to, and we didn't want to pester them with an annoying prompt. The result was that we did nothing. - Because we needed user consent ahead of time it meant any new keys need complicated descriptions ahead of time. - Changing keys also meant changing the backend which no-one knew how to do/had access - The library is *way* too complex for our needs. It's not designed around dumping a config key, and we're constantly fighting against it - The code ended up scattered across repositories with a hack to unify things - The backend visualisation tools are rubbish (the default ones fallover, the grafana ones require too much manual difficult SQL) and made it difficult to extract anything - Time based graphs look cool, but they're not useful. The only things that are useful of "this is the state now", which was doable but meant fighting against the system - Allowing user's to pick a setting was making things complicated. We have metrics that show users just enabled everything. # Version two As discussed last Akademy we want to move to a more survey like system. Once/Twice a year we prompt the user "we would like to submit information on your hardware/configs". Because we are prompting every time, we can easily change the data every time. We're not having to show them what information we will gather, we can show them what we will send. This solves the upgrade problem. Code-wise it can be one binary in plasma that gathers the information. I'm happy for projects like Discover or even Kate can just add code here. The part everyone wanted was just config keys, that doesn't need anything in process. If they do need anything they can write data into .cache that this process can pick up when it eventually runs. For storage I want to use just dump the JSON, there's lots of databases that can do that optimally. For visualisations I want to use Apache Superset. It's amazing. I tested this with the existing data and it scales just fine. {width=900 height=417} Not only does it allow for drag and drop graphs (above), but also for easy cross-filtering (below) i.e "how many panels do people who have 2 screens have". "are people using a touchpad more likely to disable middle click paste" etc. just by clicking on various parts of the graphs. {width=900 height=310} It allows extensive permissions so we can allow everyone to see the graphs without the raw data - and it has OAuth login. # Implementations I've made the systems that we've talked about. Client side: https://invent.kde.org/davidedmundson/telemetry2 The client side is very simple: https://invent.kde.org/davidedmundson/telemetry2/-/blob/master/src/backend/ConfigGenerator.cpp?ref_type=heads Anyone can read it, anyone can vet it. It's deliberately dumb and deliberately simple and I'd like to keep it that way. I've also hooked up a temporary UI which submits this data to a temporary server. This is currently my personal home server, please don't abuse it too much! Backend: This is all done, it's very simple. I need to talk to sysadmin. Ping me if you want to play with the backend side or set up your own. # Things to do The configuration backend needs to be fleshed out with just more-and-more data we want to fetch, please yolo any MRs into it. My UI needs some love from someone into QML. A key discussion point we didn't finalise at Akademy is I'm not quite sure what the entry point would be: The KCM? A surprise popup? Something integrated with the donations prompt?