rounding in mix op ($1071) · Snippets · GitLab

Wolthera van Hövell @woltherav · Jul 25, 2020

So, I tested it. Works as expected in 8 and 16bit integer, breaks everything in float.

BTW, is this expected to be calculated as: (totalAlpha + (sumOfWeights / 2)) / sumOfWeights; or ((totalAlpha + sumOfWeights) / 2) / sumOfWeights;

EDIT2: Shouldn't we be leveraging KoColorSpaceMaths for this?

Edited Jul 25, 2020 by Wolthera van Hövell

So, I tested it. Works as expected in 8 and 16bit integer, breaks everything in float. BTW, is this expected to be calculated as: `(totalAlpha + (sumOfWeights / 2)) / sumOfWeights;` or `((totalAlpha + sumOfWeights) / 2) / sumOfWeights;` EDIT2: Shouldn't we be leveraging KoColorSpaceMaths for this?

Please register or to comment