render/vulkan: clip negative values before applying transfer function

Not all eotf or eotf inverse are well defined for values outside the
intended domain, so just ignore it and clamp it away.

An alternative solution would be to use sign preserving pow here (i.e.
sign(x) * pow(abs(x), p)), however I'm not sure that makes sense or is
defined anywhere. Negative values here are likely a result of colors
being outside the gamut range, so clipping them to 0 is more correct
than mirroring from 0.
This commit is contained in:
llyyr 2025-10-06 08:10:49 +05:30
parent 879243e370
commit 604fcdb1db
2 changed files with 6 additions and 0 deletions

View file

@ -72,6 +72,9 @@ void main() {
rgb = mat3(data.matrix) * rgb;
if (OUTPUT_TRANSFORM != OUTPUT_TRANSFORM_IDENTITY) {
rgb = max(rgb, vec3(0));
}
if (OUTPUT_TRANSFORM == OUTPUT_TRANSFORM_LUT_3D) {
// Apply 3D LUT
vec3 pos = data.lut_3d_offset + rgb * data.lut_3d_scale;

View file

@ -66,6 +66,9 @@ void main() {
rgb = in_color.rgb / alpha;
}
if (TEXTURE_TRANSFORM != TEXTURE_TRANSFORM_IDENTITY) {
rgb = max(rgb, vec3(0));
}
if (TEXTURE_TRANSFORM == TEXTURE_TRANSFORM_SRGB) {
rgb = srgb_color_to_linear(rgb);
} else if (TEXTURE_TRANSFORM == TEXTURE_TRANSFORM_ST2084_PQ) {