wayland/frog_colormanagement: improve mastering luminance handling

Before we only used max_cll and max_fall, but with some videos they are not provided and only mastering display luminances are set

This is important for !6138 (merged) and doing better tone mapping in general