Encoder API ideas
I learned a lot about video encoding while working on !90 (merged) and it made me realize how basic the recording part of KPipeWire is. I knew I could start proposing things to add, but I also realized that the scope of KPipeWire was unclear. How much should it do? How much is too much? I get the impression that it's supposed to be basic from how basic the available encoder related APIs are, but it's not explicitly stated anywhere. Video encoding and container formats are very complex subjects, so it could just be because there wasn't enough time nor any apps that were directly in need of more than what was created.
While working on !90 (merged), I learned that the most important controls for an encoder are usually for quality<->speed, quality<->compression or speed<->compression. The first two are commonly used for lossy encoding since you have to balance quality, speed and compression depending on how the content will be stored and used. The third is more common for lossless encoding since the quality isn't supposed to change. Considering this, PipeWireBaseEncodedStream::setQuality
isn't enough for controlling the encoder. It's also unclear what the tradeoffs for more or less quality are unless you examine the source code related to the encoder you want to use. If we want to keep the API fairly simple, we need controls for balancing speed, quality and compression. Maybe they could be set as ratios (1 speed:2 quality:3 compression
, 33% speed:67% quality:100% compression
or 17% speed, 33% quality, 50% compression
adding up to 100%) and then those would be interpreted into something appropriate for each encoder. Another option would be to have a function that can set ffmpeg options directly by sending a map of strings based on ffmpeg CLI options. That way someone who knows ffmpeg well enough can set any options they want directly instead of only having the default options we set and the basic encoder API.
It's worth saying right now that there are currently no apps I know of that would use what I suggested (Spectacle doesn't even use setQuality
). I still think it's worth posting this because I think the problem with the API being too basic would become evident to anyone who actually wants to tweak the encoder settings or allow a user to do that. If we're not serious about supporting those 2 usecases, we should deprecate setQuality
because it's not really good enough for that. Even functions for setting speed, quality and compression may not always be enough to make getting good results intuitive, but it's a lot better than just setting quality and hoping you get the right tradeoff or having to read source code.