Instead of removing QMenu, we can take all its actions and repopulate our model and thus allowing to recursively filter through submenus nicely :)
This is mostly working with the few menus I tried(Color Scheme / External Tools) except for the "Color Themes" menu item, not sure why there are no actions() in it. EDIT, its working for that too now it seems, not sure why it wasn't before.