some color refactors to make night light more correct

For details see the individual commits, but this makes night light be applied in linear space instead of gamma 2.2 - even if no color management features are enabled - and removes some unused code.

I also have a patch that goes on top of this, which makes night light do whitepoint adaptation as well, making it correct for color management workflows, but it's a bit bigger and might need some discussion about what we/users really want, so I'll do that as a separate MR.