# Description The goal is to make your computer understand how your files, contacts, and tasks are connected, saving you time and effort. The burden of keeping the location and connections between different pieces of information (content of documents, notes, pictures, calendars, contacts, bookmarks, conversations, tags, …) should shift from the human to the machine. Plasma should do the heavy lifting of the menial data management, and present it to the user so that the human can let their creative juices flow. Core Ideas: - **Learning from Your Habits:** Instead of forcing you to organize everything manually, your computer can naturally learn from what you do. - **Smart Activities:** KDE Plasma has a feature called "Activities" to group your work environments. If you search for something while in the "Work" Activity, the system will automatically prioritize your work files and contacts. - **Asking the Right Tool:** If you search for a photo using KRunner, it won't try to scan your photos itself. It will instantly ask your photo app and Baloo for the answer, pulling the best results together. - **Modularity:** Instead of relying on a single heavy database, we want to use small, smart, mostly pre-existing tools that talk to each other smoothly, while keeping everything local, safe and private. That way the user can tailor the experience to their liking. This includes turning individual features or everything off, of course. - **Voice Commands:** Instead of sending your voice to a cloud server, a tiny helper on your machine can translate your spoken command directly into a system action. - **Auto-Organizing:** A small helper could quietly label your saved screenshots as "receipts" or "notes" in the background so you can find them easily later. - **Playing Nice with Others:** We will provide standard ways for external apps to ask the system simple questions without interfering with the core of your computer's system. - **Enabling More Creative Workflows:** While not directly in scope of this goal, a knock-on effect of contextualising and exposing rich data could be that more creative workflows for different tasks or even desktop concepts emerge down the line. # What will it take Here are a few architectural approaches we could explore: ## Algorithmic Context & Heuristic Linking (Zero-AI Overhead) We can achieve deep contextual awareness purely through deterministic algorithms and statistical heuristics: **Activity-Driven Dynamic Ranking:** KActivities already isolate workflows. Baloo and KRunner can dynamically adjust search scoring based on the active Activity, prioritizing code repositories in a "Development" activity, and slide decks or calendar items in a "Work" activity. **Temporal Co-Occurrence Graphs:** Rather than asking users to manually tag relationships, Plasma can observe temporal patterns. If a document, a web URL, and an address book contact are routinely opened within the same working session, Plasma can establish an implicit "soft link." Searching for one naturally surfaces the others as related context. **Smart Clipboard & Seamless Vision:** KDE's screenshot tool, Spectacle, already features Optical Character Recognition (OCR) to extract text from images. We can deeply integrate this capability into Klipper (the Plasma clipboard). By combining OCR with pattern recognition, the system can instantly identify when you copy a URL, an event date, or a tracking number even directly from an image capture and automatically offer quick actions for your browser or calendar without requiring heavy AI and using simpler ML technics or models to handle it. Another idea to contextualise copied content is a clipboard “file” / scrapbook that is associated with a specific Activity or even a specific document. ## Federated Search & Query Routing Instead of trying to centralize all desktop metadata into one giant master index, Plasma should rely on a federated query architecture. **On-the-Fly Metadata Correlation:** Data stays where it naturally belongs (Akonadi for PIM, Baloo for file indexing, browser profiles for web history). When a search is performed in KRunner, it acts as a federated coordinator. If a document's metadata contains an author name, KRunner correlates that author against Akonadi contacts on the fly, presenting unified results without requiring a persistent master database. **Leveraging Domain-Specific Indices** (e.g., digiKam): Individual KDE applications already implement powerful local discovery. For instance, digiKam provides local semantic and similarity search for photos. Plasma can query digiKam's local index through standard interfaces instead of re-indexing image content at the shell level. ## Specialized, Low-Resource Edge ML Where machine learning adds clear value, it should be limited to small, specialized, on-device models running via lightweight runtimes (such as ONNX Runtime or LiteRT or ncnn). **Offline Voice & System Intention** (e.g. Kelam): Building upon offline dictation tools like Kelam (which uses local Whisper models for speech-to-text), Plasma can map natural spoken commands to desktop actions. A compact, local intent classifier translates spoken phrases directly into existing Plasma or KWin D-Bus calls entirely on-device. **Background Automated Tagging:** Low-priority background workers can process incoming downloads or screenshots with tiny, quantized vision models, applying basic semantic labels (e.g., "invoice", "receipt", "screenshot") to file attributes so Baloo can index them instantly. ## Standardized Desktop Action Interfaces For users running local AI tools or custom scripts (such as Alpaka for local LLMs), Plasma can provide clean, structured access to system capabilities: **Introspectable D-Bus Action Registry:** Expose well-defined D-Bus methods for common desktop actions (querying active windows, setting focus, managing notifications, retrieving contextual file lists). **Pluggable Runner Extensions:** Allow external local clients to act as bidirectional KRunner providers or consumers, enabling local models to query desktop context and trigger system actions cleanly and safely. If we rely on our existing Inter-Process Communication (IPC) and federated indexing, we can shift the burden of data management away from the user while keeping the desktop responsive and entirely private. ## Human-centric UX All the backend intelligence in the world is useless if the interface only makes sense to developers – a trap that previous semantic desktop efforts often fell into. The user-facing side must be organic, intuitive, and entirely frictionless: **Natural Language Interaction:** Users should be able to type or speak exactly what they mean into KRunner (e.g. "Find the PDF John sent me yesterday") without needing to memorize complex search operators or navigate rigid database structures. **Contextual Surfacing:** Instead of dumping a raw, unorganized list of files, search results and system prompts should be grouped by context (e.g, "Files related to my upcoming meetings this week") and presenting meaningful context (e.g. immediate snippets/previews of what was searched), offering the user a cohesive working set rather than scattered data points. **Invisible Assistance:** The UX should not force the user into a separate "data management" application. The smart features should live directly inside the tools they already use (like the clipboard or screenshot tool), offering one-click actions exactly when they are needed. # How we know we succeed Different KDE (and other) data silos will be connected, resulting in context-richer information and knowledge graphs. Users will find it much more natural to interact with Plasma and the information they have at hand, finding what they are looking for not just faster, but in a manner that invites unburdened creativity. Instead of spending time and effort manually sorting, tagging and searching via separate apps (plus fighting against bugs), user will be able to rely on Plasma to simply handle their data. As a very important side-goal, users who rely on voice commands – whether due disability or otherwise – would have a much better time interacting with their computer through Plasma. This is an important improvement of accessibility. Several (old) Baloo, Akonadi etc. bugs get triaged and fixed. # Relevant links - **digiKam:** Recently added a feature that lets you search your photos using natural, everyday language. The search runs entirely on your computer to keep your photos private. https://www.digikam.org/news/2026-08-20-advanced_search_improvements_with_llm/ - **Kdenlive:** The 25.04.0 release integrated powerful local tools like Whisper (for audio) and SAM2 (for video tracking) to process media directly on your machine. https://kdenlive.org/news/releases/25.04.0/ - **Kelam:** This tool allows you to speak to your computer and dictate text across the system. It works completely offline. https://invent.kde.org/osezer/kelam - **OpenVoiceOS (ex Mycroft):** A natural-language voice control system that was integrated with KDE before https://www.openvoiceos.org/ https://store.kde.org/browse?cat=608&ord=latest - **Alpaka:** A clean interface that lets you talk to local AI models directly on your desktop, proving we can have smart assistants without relying on the cloud. https://invent.kde.org/utilities/alpaka - **NEPOMUK:** A two-decades old KDE solution to a similar problem, good to learn from: https://en.wikipedia.org/wiki/NEPOMUK_(software) - **Recoll:** An alternative to Baloo, which indexes also some non-KDE app data and provides previews/snippets on search – good for inspiration: https://recoll.org/ - **FAISS:** A library for efficient similarity search and clustering of dense vectors: https://faiss.ai/ - **LocalSearch:** Another approach to local search: https://github.com/neural-koala/LocalSearch - **Scott Jenson: How the Desktop UX needs to evolve to keep up with Local-first:** a great historic overview and exploration of different UX improvements of how richer data and local models can bring a breakthrough in desktops and their usability: https://www.youtube.com/watch?v=-IOLRcFC6OY ## Champions - Matija Šuklje (@matijasuklje, hook) - Onuralp Sezer (@osezer) - … ## Contributors I am willing to put work into this: - Onuralp Sezer (@osezer) - Benson Muite (@kbkmde) - Ivan Čukić (@ivan) - Maciej Bonin (@pillowtrucker) - Stefan Brüns (@bruns) - … ## Supporters I am interested: - Jure Repinc (@repinc, jlp) - David Narváez (@narvaez, dmaggot) - Matthew Vanderpool (@mvpool) - …