AI tools integration
Some new important tools provided by AI are out today in the open source world too. I've begun to list them and to test what I'm finding to evaluate a possible integration in Kdenlive.
These tools, are mostly python scripts which use AI and models for the most different purposes. Interesting tools could be:
- Speech to text (we have already found and integrated Vosk)
- Super resolution (The one I'm using now for testing is Realesrgan https://github.com/xinntao/Real-ESRGAN but there are a bunch of other options like the OpenCV suer resolution tool and othrs)
- Color matching (Color-matcher https://github.com/hahnec/color-matcher integrated with the G'MIC "CLUT from After - Before Layers" does a great job in real-time)
- Frameboost (RIFE https://github.com/hzwer/arXiv2020-RIFE combined with the speed ramp could be a very great tool)
- Denoiser (I'm still testing some and I'll come back to this in a next update - BTW Realesrgan do somehow the job too but there are more specific scripts)
- Voice restoration (VoiceFixer https://haoheliu.github.io/demopage-voicefixer/ is able to upscale the audio resolution, to denoise, to de-clip and to remove reverb and echo.)
Other less important tools but anyway to consider are:
- Segmentation (for converting 2D footage to stereoscopic one, for advanced 2ndary color correction and depth-of-field reduction in post-production. I've not yet a clear idea about this tool as segmentation is a complex process for its approach to depth but also to semantic recognition of the objects)
- Style transfer (just as an artistic effect but, anyway, I've found this https://github.com/aksh-ai/style-transfer)
- Colorize (not very important but it could be a plus - There are several project but probably the most popular is Deoldify so far)
This post it's only a note. I'm going to continue to test these scripts and see how they perform before to discuss a possible integration/implementation in Kdenlive
Edited by Massimo Stella