Skip to content

taskswitcher: enhance gesture with flick speed tracking and orthogonal movement tracking

This MR contains work to track the speed with which the taskswitcher gesture has occurred to allow different handling of a "fast flick" (eg: go to homescreen instead of taskswitcher).

Features

The center piece of the overhaul is tracking recent velocity (averaged over the last ~50ms) of the gesture to determine behavior:

  • Swipe: A normal swipe is a "slow" finger move
  • Flick: A "fast" swipe

The determining factor is the speed at the end of the gesture (= when lifting the finger up), anything before is irrelevant

  • Swipe up goes to task switcher
  • Flick up goes to homescreen directly instead of task switcher
    • this is suppressed when already on homescreen - it makes no sense there
      • Note: On Android that is actually mapped to "going to the first page of the home screen if not there already" - I don't care much for that and it can lead to nothing happening if you're already there, I think I prefer the consistency of always opening the task switcher vs a "quick way to get to the main (first) page of the homescreen" - especially since what that means depends a lot on the active home screen.
  • added orthogonal gesture movement tracking (2-axis gestures)
    • gesture properly follows finger movement in all directions now making it feel much more responsive and nice
    • diagonal flick for quick task switch
    • horizontal only movement (while holding, not releasing) triggers task scrub gesture for scrubbing through larger number of task in the switcher backlog
  • Complete refactor of state keeping logic, not just in relation to gestures
    • state keeper fully in one place in C++ except for
      • a few derived values that are very directly UI related which are kept in TaskSwitcherHelpers.qml
      • and directly touch gesture related state which are kept separate to maybe allow upstreaming the new touch border gesture handling to KWin in the future
done TODO items
  • currentTaskIndex tracking is majorly broken
  • old TaskSwitcherState.qml is still around, should be removed
  • clamping minimum scale for some reason also clamps y position which makes the task not track the finger when moving past the "full activation" point
  • switching between tasks when task switcher is open is super crap
  • task switching when switcher is fully open works now, but flicks don't work - you have to fully move at least halfway to the next task for it to "snap" to that task when you let go, there is 0 momentum and I don't know why -> Flickable is just broken and refuses to flick on touch input, same behavior on master with old implementation, so I reimplemented the same workaround from before
  • the gestures don't work until the notification drawer was invoked at least once after every screen geometry change (resolution change, screen rotation etc)
  • keyboard shortcut for toggling task switcher is kinda hacky right now and feels awful when closing (some delay noticable, maybe to do with animations?) NOTE: I disabled it for now because the hacky implementation interfered with/created bugs in other parts
  • task scrub mode has broken offsets/positions (can scroll past the end of the list, but not reach all elements) when opening task index is not 0
  • fast flick velocity threshold is a mostly arbitrary number and I think it also depends on pixel density - ideally we want a mm (or cm or whatever) per ms, not logical pixels per ms

Design decisions I made during development:

  • Changed the y opening animation from OutBack (with a bounce) to a simple OutExpo (quickly move towards and then approach slowly) as the bouncy look and feel from the OutBack animation felt wrong to me in that place
  • To fit better with the new docked mode:
    • the task switcher now shows the screen geometry behind a task that is not fullscreened. This allows to consistently press (and see) the adjacent tasks which might not always be the case if it's a thin window on a landscape screen.
      • This is not great overall yet - actually it just checks the windows actual aspect ratio, so if it's smaller but same aspect ratio as screen it shows as if it's a full screen task - also it maximizes and centers the window thumbnail in the task switcher so it really isn't ideal, I just don't know what the behavior should ideally be. This change at least doesn't break the switcher when in docked mode.
    • on opening an app from task switcher it doesn't get force-fullscreened anymore - either it already is fullscreen (because fullscreen in docked mode or just not in docked mode) or it should just keep its geometry (this could be gated behind checking if docked mode is on or not, but I think it's fine without)

TODO before merge

  • IMPORTANT: Because of regular crashing on interaction I've commented out any code that uses WindowFilterModel::data() which seems to be the cause for that. Since master also uses this and experiences these crashes (at least on both dev machines I tested), I don't think this MR necessarily needs to be held up for that issue (especially since it feels like it could be upstream). I removed this to have an easier time with testing my code without constant crashes, but before merging this would need to be put in again as it breaks: skipping windows with "skipSwitcher" flag set, and showing the correct task when opening the switcher in certain situations.
    • seems to be a bug with WindowFilterModel::data sometimes returning an invalid/null element? But not in C++, it just doesn't make its way to QML.
    • feels like it could be a timing thing (but maybe not?) when reverting the skipSwitcher filter rule on tasksModel or adding some lengthy debug console log in TaskList.qml:minimizeAll() before trying to access item.window properties it seems to be fine. When keeping skipSwitcher and having no lengthy debug output I get crashes rather reliably every ~3rd to 5th time invoking the task switcher.
    • While this does seem to be reproducible on master and thus "the same", if it is timing based it may be that my code triggers the error more often technically making it a regression (though also easier to track down?)
    • Interestingly I don't seem to be able to reproduce this on-device, my PinePhone doesn't have this behavior at all. Maybe a bug with nested KWin session?
  • task scrub mode may need to have a dynamic amount of items to scrub through based on display size/orientation (maybe keep distance for task scrubbing the same on portrait vs landscape to aid muscle memory)? or instead of using percentages/pixel distances use mm distance based on dpi?
    • is task scrub mode even something we want? Since I prefer the "quick switch gesture" only being able to switch between the two most recent windows (because of immediate reordering by recency), this leaves our task switcher in gesture mode with no quick "shortcut gesture" to switch to older tasks, task scrub mode makes this possible in IMO a nice way, but would love input there.
  • get rid of all code comments marked with TODO! (with an exclamation mark)
  • One last pass on removing all superfluous console logs, includes and unused functions. I cleaned that up a lot already, but not completely (one std::cout is still left in on purpose until last minute because of a very rare gesture bug I saw twice and I'm not sure if my fix really got it)

Other

  • I've only ever tested on my laptop with touch screen (ie: effectively a large tablet form factor), if this merges and we get it on nightly I'll definitely try on the PinePhone, there I will closely scrutinize whether the undoYThreshold and task scrub mode number of tasks feels good to use on a small portrait mode display. This may lead to followup MRs to tweak those essentially arbitrarily chosen values.

TODOs for later MRs

  • consider moving velocity calculation/generic 2-dimensional gesture tracking to kwin if it's robust and generic enough for that to be useful on their end. (maybe do that later after it has proven itself for a while on our side)
  • (optional?) vibration feedback to tell if it goes to taskswitcher or homescreen? android has that (at least mine) with a tight vibration when task switcher is invoked vs homescree
  • flick to homescreen animation kinda looks bad
  • task sorting by recency
  • polish task scrub mode if we intend to keep it
    • task scrub mode can "run out of space" for the gesture and thus can only scrub through a maximum number of tasks (currently 8 either direction in my implementation), but we could add a timer if reaching the end that goes through tasks eg: one per second (acceleration?) to "unlock" the full task list
    • task scrub mode should have some visual cue when it's activated (eg: fully "opening" the switcher y position, maybe some extra UI with window thumbnails)
  • Positioning after screen geometry change (resize, rotation) of the task switcher is kinda broken (task may end off-screen etc) - fixed by interacting with it or re-opening and not a huge issue IMO
  • some bound for y position above the "fully opened position" (maybe make it more and more "reluctant" to follow the finger upwards the further up you go - I think that just makes it feel a bit nicer
  • make it behave better with docked mode (currently some animations look a bit scuffed)
  • make it behave better with multi screen (currently it's mirrored on all screens, we probably only want it on the screen that currently has focus?)
  • navbar buttons when not in gesture mode seem to have slightly different button icon sizes than the system wide navbar (is there no better way to add the navbar than to essentially reimplement it?)
Edited by Luis Büchi

Merge request reports