AI tools integration
Some new important tools provided by AI are out today in the open source world too.
I've begun to list them and to test what I'm finding to evaluate a possible integration in Kdenlive.
These tools, are mostly python scripts which use AI and models for the most different purposes.
Before integrating more python plugins, we should have the python executable embedded in our package, see
https://invent.kde.org/multimedia/kdenlive/-/tree/work/embeded-python2?ref_type=heads
It requires MSVC on Windows as python on MinGW is too complicated.
Interesting tools could be:
1. Speech to text (we have already found and integrated Vosk)
2. Object detection (categorization)
3. Super resolution (The one I'm using now for testing is Realesrgan https://github.com/xinntao/Real-ESRGAN but there are a bunch of other options like the OpenCV suer resolution tool and othrs)
4. Color matching (Color-matcher https://github.com/hahnec/color-matcher integrated with the G'MIC "CLUT from After - Before Layers" does a great job in real-time)
5. Frameboost (RIFE https://github.com/hzwer/arXiv2020-RIFE combined with the speed ramp could be a very great tool)
6. Denoiser (I'm still testing some and I'll come back to this in a next update - BTW Realesrgan do somehow the job too but there are more specific scripts)
7. Voice restoration (VoiceFixer https://haoheliu.github.io/demopage-voicefixer/ is able to upscale the audio resolution, to denoise, to de-clip and to remove reverb and echo.)
Other less important tools but anyway to consider are:
1. Segmentation (for converting 2D footage to stereoscopic one, for advanced 2ndary color correction and depth-of-field reduction in post-production. I've not yet a clear idea about this tool as segmentation is a complex process for its approach to depth but also to semantic recognition of the objects)
2. Style transfer (just as an artistic effect but, anyway, I've found this https://github.com/aksh-ai/style-transfer)
3. Colorize (not very important but it could be a plus - There are several project but probably the most popular is Deoldify so far)
This post it's only a note. I'm going to continue to test these scripts and see how they perform before to discuss a possible integration/implementation in Kdenlive
issue