diff --git a/HEIF_IMPLEMENTATION_PLAN.md b/HEIF_IMPLEMENTATION_PLAN.md index 1019e98c3..64f9509be 100644 --- a/HEIF_IMPLEMENTATION_PLAN.md +++ b/HEIF_IMPLEMENTATION_PLAN.md @@ -67,7 +67,8 @@ This snapshot pins or classifies the available references and failures; it does | Managed implementation | Normative behavior | Reviewed implementation reference | Use | | --- | --- | --- | --- | -| `Av1YuvConverter.ConvertToRgb`, `ConvertFromRgb`, scalar row conversion, chroma reconstruction, and chroma downsampling | ITU-T H.273 (V4) equations 27-57 and 76-78, including chromaticity-derived equations 39-47; AV1 section 6.4.2 chroma sample positions | Official ITU-T H.273 (V4) (07/2024); libavif `src/reformat.c` and `src/colr.c` at `092276ce89098ead06db80975173191e5fee1826`; libaom `aom/aom_image.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Scalar behavioral oracle for full/limited-range conversion at 8, 10, and 12 bits. Decode and encode cover identity, YCgCo, coefficient-based NCL, chromaticity-derived NCL, and SMPTE ST 2085 across monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with AV1 chroma sample positioning. Chromaticity derivation uses every defined H.273 primary and matches libavif's BT.709 fallback for unspecified or reserved primaries. Later SIMD paths must preserve this scalar behavior. | +| `Av1YuvConverter.ConvertToRgb`, `ConvertFromRgb`, scalar row conversion, chroma reconstruction, and chroma downsampling | ITU-T H.273 (V4) equations 27-78, including chromaticity-derived equations 39-47 and constant-luminance equations 66-75; AV1 section 6.4.2 chroma sample positions | Official ITU-T H.273 (V4) (07/2024); libavif `src/reformat.c` and `src/colr.c` at `092276ce89098ead06db80975173191e5fee1826`; libaom `aom/aom_image.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Scalar behavioral oracle for full/limited-range conversion at 8, 10, and 12 bits. Decode and encode cover identity, YCgCo, coefficient-based NCL, both fixed and chromaticity-derived constant/non-constant-luminance systems, and SMPTE ST 2085 across monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with AV1 chroma sample positioning. Chromaticity derivation uses every defined H.273 primary and matches libavif's BT.709 fallback for unspecified or reserved primaries. Later SIMD paths must preserve this scalar behavior. | +| `Av1TransferFunctions` | ITU-T H.273 (V4) Table 3 transfer characteristics 1-18 | Official ITU-T H.273 (V4) (07/2024); libavif `src/colr.c` at `092276ce89098ead06db80975173191e5fee1826` | Apply every AV1-signallable transfer function required by constant-luminance and ICtCp color conversion. Retain the H.273 normalized PQ and HLG definitions; do not import libavif's display-oriented 203-nit scaling or HLG OOTF into codec sample interpretation. Use libavif's midpoint convention only for the non-bijective zero code of the two logarithmic curves. | | `Av1FrameBuffer` high-bit-depth sample layout and `Av1YuvConverter` 10/12-bit packed-pixel conversion | AV1 section 6.4.1 bit depth and H.273 sample-range scaling | libaom `aom_scale/yv12config.h`, `av1/common/idct.c`, and `av1/common/reconintra.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f`; libavif `src/avif.c` and `src/reformat.c` at `092276ce89098ead06db80975173191e5fee1826` | Establish two-byte native sample storage with sample-unit strides for 10/12-bit reconstruction and use ImageSharp's existing `Rgb48` pixel-operation paths in both directions so packed-pixel staging does not reduce high-bit-depth samples to eight bits. | | `Av1PredictionDecoder`, `Av1HighBitDepthPredictor`, `Av1ChromaFromLumaContext`, `Av1PartitionInfo`, and the scalar DC, directional, Paeth, smooth, filter-intra, and chroma-from-luma predictors | AV1 sections 7.11.2 and 7.11.2.3 intra prediction | libaom `aom_dsp/intrapred.c`, `av1/common/reconintra.c`, `av1/common/av1_common_int.h`, `av1/common/blockd.h`, `av1/common/cfl.c`, and `av1/common/cfl.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Behavioral oracle for luma/chroma mode-neighbor addressing, directional upsampling, Paeth selection, smooth normalization, filter-intra taps, high-bit-depth clipping, chroma-from-luma storage/subsampling, and chroma-from-luma row strides. Existing managed scalar tables and predictors remain the implementation base. The WIP rectangular byte-pipeline smooth digest expectations encode width/height-swapped weights and must be replaced only from an independently generated oracle, not regenerated from this implementation. | | `Av1TileReader` palette mode/color-map parsing, `Av1SymbolDecoder` palette distributions, `Av1BlockModeInfo` palette state, and `Av1PredictionDecoder` palette reconstruction | AV1 sections 5.11.46, 5.11.49, and 7.11.2 palette prediction | libaom `av1/decoder/decodemv.c`, `av1/decoder/detokenize.c`, `av1/decoder/decoder.h`, `av1/common/pred_common.c`, `av1/common/pred_common.h`, and `av1/common/entropymode.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Reuse the existing tile range decoder and frame-owned mode map while porting the normative palette cache merge, high-bit-depth color deltas, tile-adaptive mode/size/index distributions, diagonal color-map traversal, edge padding, and direct palette-sample reconstruction. This is AV1 still-image compression syntax and does not add retained video reference state or any ISO BMFF surface. | @@ -130,7 +131,7 @@ This assessment is based on the current source after the upstream ImageSharp mer - Palette mode now reads the normative luma/chroma mode and size CDFs, neighbor color caches, high-bit-depth color syntax, diagonal color-index maps, clipped-edge padding, and direct sample prediction through the existing reconstruction pipeline. The scalar implementation matches the pinned libaom source, but an independently encoded palette AVIF fixture is still required before this path is independently verified. - Loop-restoration unit parsing records tile-local switchable/Wiener/self-guided filter selections and coefficients in frame-owned plane grids, including super-resolution-adjusted unit corners and the corrected conditional 64x64-superblock unit-size bit. The active restoration stage implements the normative unit geometry, striped deblocked boundaries, Wiener filtering, self-guided projection, and 8/10/12-bit clipping, while reusing the existing SIMD-backed super-resolution and adjacent multiply/add primitives. Independently encoded fixtures covering every parameter set, plane layout, bit depth, and frame-edge geometry are still required. Other normative independently decodable still-image syntax paths still contain `NotImplementedException` or equivalent unsupported branches. Tile-local palette CDF adaptation is present; the remaining still-image frame-context behavior requires a separate source audit without introducing sequence playback state. - The frame buffer now establishes two-byte native sample storage, logical plane rows, and sample-unit block strides for 10/12-bit frames. The active intra-prediction, inverse-transform, and block-reconstruction path selects native 16-bit samples for 10/12-bit frames and has focused pipeline wiring coverage. Chroma-from-luma storage, subsampling, parameter derivation, U/V sharing, and 8/10/12-bit prediction are active; independently encoded high-bit-depth and chroma-from-luma AVIF conformance files are still required. -- `Av1YuvConverter` now consumes the signaled range, H.273 matrix coefficients implemented so far, subsampling, and chroma sample position for 8, 10, and 12-bit output. Its high-bit-depth decode and encode paths use allocator-backed `Rgb48` rows and the existing `PixelOperations` conversions, avoiding the former eight-bit intermediate. Encoder conversion covers monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with libavif-compatible box averaging. Identity, YCgCo, the fixed non-constant-luminance matrices, chromaticity-derived non-constant luminance for every defined primary, and SMPTE ST 2085 are active in both directions. BT.2020 constant luminance, chromaticity-derived constant luminance, and ICtCp remain incomplete and require the signaled H.273 transfer functions rather than reuse of the Kr/Kb path. +- `Av1YuvConverter` now consumes the signaled range, H.273 matrix coefficients implemented so far, transfer characteristics where the matrix definition requires them, subsampling, and chroma sample position for 8, 10, and 12-bit output. Its high-bit-depth decode and encode paths use allocator-backed `Rgb48` rows and the existing `PixelOperations` conversions, avoiding the former eight-bit intermediate. Encoder conversion covers monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with libavif-compatible box averaging. Identity, YCgCo, the fixed non-constant-luminance matrices, both fixed and chromaticity-derived constant/non-constant-luminance systems, and SMPTE ST 2085 are active in both directions. ICtCp remains incomplete. Limited-range YCgCo also remains unsupported because the existing libavif-derived range pipeline scales YUV rather than the R/G/B signals required by H.273 equations 27-29. - The inverse-transform path allocates arrays in a per-transform hot path. - The production prediction, transform, and nonlinear self-guided paths remain predominantly scalar. Normative super-resolution and Wiener horizontal products now reuse ImageSharp's cross-platform adjacent multiply/add SIMD helper with exact scalar fallbacks; further SIMD work must preserve these scalar reconstruction oracles. diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1TransferFunctions.cs b/src/ImageSharp/Formats/Heif/Av1/Av1TransferFunctions.cs new file mode 100644 index 000000000..add4ea9c1 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Av1TransferFunctions.cs @@ -0,0 +1,444 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1; + +/// +/// Applies the H.273 transfer characteristics used by AV1 color conversion. +/// +internal static class Av1TransferFunctions +{ + /// + /// The BT.709 and BT.2020 nonlinear scale factor. + /// + private const float Bt709Alpha = 1.09929682680944F; + + /// + /// The BT.709 and BT.2020 linear-domain transition point. + /// + private const float Bt709Beta = 0.018053968510807F; + + /// + /// The SMPTE ST 240 nonlinear scale factor. + /// + private const float Smpte240Alpha = 1.111572195921731F; + + /// + /// The SMPTE ST 240 linear-domain transition point. + /// + private const float Smpte240Beta = 0.022821585529445F; + + /// + /// The sRGB nonlinear scale factor. + /// + private const float SrgbAlpha = 1.0550107189475866F; + + /// + /// The sRGB linear-domain transition point. + /// + private const float SrgbBeta = 0.0030412825601275209F; + + /// + /// The SMPTE ST 2084 first rational constant. + /// + private const float PqC1 = 0.8359375F; + + /// + /// The SMPTE ST 2084 numerator scale. + /// + private const float PqC2 = 18.8515625F; + + /// + /// The SMPTE ST 2084 denominator scale. + /// + private const float PqC3 = 18.6875F; + + /// + /// The SMPTE ST 2084 outer exponent. + /// + private const float PqM = 78.84375F; + + /// + /// The SMPTE ST 2084 inner exponent. + /// + private const float PqN = 0.1593017578125F; + + /// + /// The SMPTE ST 428 luminance normalization factor. + /// + private const float Smpte428Scale = 0.91655527974030934F; + + /// + /// The HLG logarithmic scale. + /// + private const float HlgA = 0.17883277F; + + /// + /// The HLG logarithmic offset. + /// + private const float HlgB = 0.28466892F; + + /// + /// The HLG output offset. + /// + private const float HlgC = 0.55991073F; + + /// + /// Converts a nonlinear signal value to its H.273 linear-domain value. + /// + /// The signaled transfer characteristics. + /// The nonlinear signal value. + /// The corresponding linear-domain value. + public static float ToLinear(ObuTransferCharacteristics transferCharacteristics, float value) + { + switch (transferCharacteristics) + { + case ObuTransferCharacteristics.Bt709: + case ObuTransferCharacteristics.Bt601: + case ObuTransferCharacteristics.Bt202010Bit: + case ObuTransferCharacteristics.Bt202012Bit: + return ToLinearBt709(value); + case ObuTransferCharacteristics.Bt470M: + return MathF.Pow(Math.Clamp(value, 0F, 1F), 2.2F); + case ObuTransferCharacteristics.Bt470BG: + return MathF.Pow(Math.Clamp(value, 0F, 1F), 2.8F); + case ObuTransferCharacteristics.Smpte240: + return ToLinearSmpte240(value); + case ObuTransferCharacteristics.Linear: + return Math.Clamp(value, 0F, 1F); + case ObuTransferCharacteristics.Log100: + // Zero represents an interval rather than one linear value. The midpoint matches libavif and + // minimizes the worst-case round-trip error when constant-luminance content is decoded. + return value <= 0F ? 0.005F : MathF.Pow(10F, 2F * (MathF.Min(value, 1F) - 1F)); + case ObuTransferCharacteristics.Log100Sqrt10: + return value <= 0F ? 0.00158113883F : MathF.Pow(10F, 2.5F * (MathF.Min(value, 1F) - 1F)); + case ObuTransferCharacteristics.Iec61966: + return ToLinearIec61966(value); + case ObuTransferCharacteristics.Bt1361: + return ToLinearBt1361(value); + case ObuTransferCharacteristics.Srgb: + return ToLinearSrgb(value); + case ObuTransferCharacteristics.Smpte2084: + return ToLinearPq(value); + case ObuTransferCharacteristics.Smpte428: + return MathF.Pow(MathF.Max(value, 0F), 2.6F) / Smpte428Scale; + case ObuTransferCharacteristics.Hlg: + return ToLinearHlg(value); + default: + // H.273 leaves unspecified and reserved transfer values to the application. Match libavif's + // deterministic BT.709 fallback for still-image conversion. + return ToLinearBt709(value); + } + } + + /// + /// Converts a linear signal value to its H.273 nonlinear-domain value. + /// + /// The signaled transfer characteristics. + /// The linear signal value. + /// The corresponding nonlinear-domain value. + public static float ToGamma(ObuTransferCharacteristics transferCharacteristics, float value) + { + switch (transferCharacteristics) + { + case ObuTransferCharacteristics.Bt709: + case ObuTransferCharacteristics.Bt601: + case ObuTransferCharacteristics.Bt202010Bit: + case ObuTransferCharacteristics.Bt202012Bit: + return ToGammaBt709(value); + case ObuTransferCharacteristics.Bt470M: + return MathF.Pow(Math.Clamp(value, 0F, 1F), 1F / 2.2F); + case ObuTransferCharacteristics.Bt470BG: + return MathF.Pow(Math.Clamp(value, 0F, 1F), 1F / 2.8F); + case ObuTransferCharacteristics.Smpte240: + return ToGammaSmpte240(value); + case ObuTransferCharacteristics.Linear: + return Math.Clamp(value, 0F, 1F); + case ObuTransferCharacteristics.Log100: + return value <= 0.01F ? 0F : 1F + (MathF.Log10(MathF.Min(value, 1F)) / 2F); + case ObuTransferCharacteristics.Log100Sqrt10: + return value <= 0.00316227766F ? 0F : 1F + (MathF.Log10(MathF.Min(value, 1F)) / 2.5F); + case ObuTransferCharacteristics.Iec61966: + return ToGammaIec61966(value); + case ObuTransferCharacteristics.Bt1361: + return ToGammaBt1361(value); + case ObuTransferCharacteristics.Srgb: + return ToGammaSrgb(value); + case ObuTransferCharacteristics.Smpte2084: + return ToGammaPq(value); + case ObuTransferCharacteristics.Smpte428: + return MathF.Pow(Smpte428Scale * MathF.Max(value, 0F), 1F / 2.6F); + case ObuTransferCharacteristics.Hlg: + return ToGammaHlg(value); + default: + return ToGammaBt709(value); + } + } + + /// + /// Applies the inverse BT.709-family opto-electronic transfer function. + /// + /// The nonlinear signal value. + /// The linear signal value. + private static float ToLinearBt709(float value) + { + if (value < 0F) + { + return 0F; + } + + if (value < 4.5F * Bt709Beta) + { + return value / 4.5F; + } + + return value < 1F + ? MathF.Pow((value + (Bt709Alpha - 1F)) / Bt709Alpha, 1F / 0.45F) + : 1F; + } + + /// + /// Applies the BT.709-family opto-electronic transfer function. + /// + /// The linear signal value. + /// The nonlinear signal value. + private static float ToGammaBt709(float value) + { + if (value < 0F) + { + return 0F; + } + + if (value < Bt709Beta) + { + return value * 4.5F; + } + + return value < 1F + ? (Bt709Alpha * MathF.Pow(value, 0.45F)) - (Bt709Alpha - 1F) + : 1F; + } + + /// + /// Applies the inverse SMPTE ST 240 opto-electronic transfer function. + /// + /// The nonlinear signal value. + /// The linear signal value. + private static float ToLinearSmpte240(float value) + { + if (value < 0F) + { + return 0F; + } + + if (value < 4F * Smpte240Beta) + { + return value / 4F; + } + + return value < 1F + ? MathF.Pow((value + (Smpte240Alpha - 1F)) / Smpte240Alpha, 1F / 0.45F) + : 1F; + } + + /// + /// Applies the SMPTE ST 240 opto-electronic transfer function. + /// + /// The linear signal value. + /// The nonlinear signal value. + private static float ToGammaSmpte240(float value) + { + if (value < 0F) + { + return 0F; + } + + if (value < Smpte240Beta) + { + return value * 4F; + } + + return value < 1F + ? (Smpte240Alpha * MathF.Pow(value, 0.45F)) - (Smpte240Alpha - 1F) + : 1F; + } + + /// + /// Applies the inverse extended IEC 61966-2-4 transfer function. + /// + /// The nonlinear signal value. + /// The linear signal value. + private static float ToLinearIec61966(float value) + { + if (value < -4.5F * Bt709Beta) + { + return -MathF.Pow((value - (Bt709Alpha - 1F)) / -Bt709Alpha, 1F / 0.45F); + } + + return value < 4.5F * Bt709Beta + ? value / 4.5F + : MathF.Pow((value + (Bt709Alpha - 1F)) / Bt709Alpha, 1F / 0.45F); + } + + /// + /// Applies the extended IEC 61966-2-4 transfer function. + /// + /// The linear signal value. + /// The nonlinear signal value. + private static float ToGammaIec61966(float value) + { + if (value < -Bt709Beta) + { + return (-Bt709Alpha * MathF.Pow(-value, 0.45F)) + (Bt709Alpha - 1F); + } + + return value < Bt709Beta + ? value * 4.5F + : (Bt709Alpha * MathF.Pow(value, 0.45F)) - (Bt709Alpha - 1F); + } + + /// + /// Applies the inverse extended BT.1361 transfer function. + /// + /// The nonlinear signal value. + /// The linear signal value. + private static float ToLinearBt1361(float value) + { + if (value < -0.25F) + { + return -0.25F; + } + + if (value < 0F) + { + return MathF.Pow((value - 0.02482420670236F) / -0.27482420670236F, 1F / 0.45F) / -4F; + } + + return ToLinearBt709(value); + } + + /// + /// Applies the extended BT.1361 transfer function. + /// + /// The linear signal value. + /// The nonlinear signal value. + private static float ToGammaBt1361(float value) + { + if (value < -0.25F) + { + return -0.25F; + } + + if (value < 0F) + { + return (-0.27482420670236F * MathF.Pow(-4F * value, 0.45F)) + 0.02482420670236F; + } + + return ToGammaBt709(value); + } + + /// + /// Applies the inverse extended IEC 61966-2-1 transfer function. + /// + /// The nonlinear signal value. + /// The linear signal value. + private static float ToLinearSrgb(float value) + { + if (value < -12.92F * SrgbBeta) + { + return -MathF.Pow((value - (SrgbAlpha - 1F)) / -SrgbAlpha, 2.4F); + } + + return value < 12.92F * SrgbBeta + ? value / 12.92F + : MathF.Pow((value + (SrgbAlpha - 1F)) / SrgbAlpha, 2.4F); + } + + /// + /// Applies the extended IEC 61966-2-1 transfer function. + /// + /// The linear signal value. + /// The nonlinear signal value. + private static float ToGammaSrgb(float value) + { + if (value < -SrgbBeta) + { + return (-SrgbAlpha * MathF.Pow(-value, 1F / 2.4F)) + (SrgbAlpha - 1F); + } + + return value < SrgbBeta + ? value * 12.92F + : (SrgbAlpha * MathF.Pow(value, 1F / 2.4F)) - (SrgbAlpha - 1F); + } + + /// + /// Applies the inverse SMPTE ST 2084 perceptual-quantizer transfer function. + /// + /// The nonlinear signal value. + /// The normalized linear signal value. + private static float ToLinearPq(float value) + { + if (value <= 0F) + { + return 0F; + } + + float nonlinearPower = MathF.Pow(MathF.Min(value, 1F), 1F / PqM); + float numerator = MathF.Max(nonlinearPower - PqC1, 0F); + float denominator = PqC2 - (PqC3 * nonlinearPower); + return MathF.Pow(numerator / denominator, 1F / PqN); + } + + /// + /// Applies the SMPTE ST 2084 perceptual-quantizer transfer function. + /// + /// The normalized linear signal value. + /// The nonlinear signal value. + private static float ToGammaPq(float value) + { + if (value <= 0F) + { + return 0F; + } + + float linearPower = MathF.Pow(MathF.Min(value, 1F), PqN); + return MathF.Pow((PqC1 + (PqC2 * linearPower)) / (1F + (PqC3 * linearPower)), PqM); + } + + /// + /// Applies the inverse HLG opto-electronic transfer function. + /// + /// The nonlinear signal value. + /// The normalized linear signal value. + private static float ToLinearHlg(float value) + { + if (value <= 0F) + { + return 0F; + } + + return value <= 0.5F + ? (value * value) / 3F + : (MathF.Exp((MathF.Min(value, 1F) - HlgC) / HlgA) + HlgB) / 12F; + } + + /// + /// Applies the HLG opto-electronic transfer function. + /// + /// The normalized linear signal value. + /// The nonlinear signal value. + private static float ToGammaHlg(float value) + { + if (value <= 0F) + { + return 0F; + } + + float bounded = MathF.Min(value, 1F); + return bounded <= 1F / 12F + ? MathF.Sqrt(3F * bounded) + : (HlgA * MathF.Log((12F * bounded) - HlgB)) + HlgC; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1YuvConverter.cs b/src/ImageSharp/Formats/Heif/Av1/Av1YuvConverter.cs index afd61456f..a41142f54 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Av1YuvConverter.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Av1YuvConverter.cs @@ -49,6 +49,11 @@ internal static class Av1YuvConverter /// The SMPTE ST 2085 YDzDx color transform. /// Smpte2085, + + /// + /// A constant-luminance transform using the signaled transfer characteristics. + /// + ConstantLuminance, } /// @@ -73,6 +78,11 @@ internal static class Av1YuvConverter out float chromaScale, out float sampleMaximum); + ObuTransferCharacteristics transferCharacteristics = frameBuffer.ColorConfig.TransferCharacteristics; + ConstantLuminanceScales constantLuminanceScales = mode == ConversionMode.ConstantLuminance + ? new ConstantLuminanceScales(transferCharacteristics, kr, kb) + : default; + Buffer2DRegion yPlane = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0); bool isMonochrome = frameBuffer.ColorFormat == Av1ColorFormat.Yuv400; int subX = frameBuffer.ColorConfig.SubSamplingX ? 1 : 0; @@ -128,6 +138,8 @@ internal static class Av1YuvConverter kr, kg, kb, + transferCharacteristics, + in constantLuminanceScales, lumaBias, lumaScale, chromaBias, @@ -154,6 +166,8 @@ internal static class Av1YuvConverter kr, kg, kb, + transferCharacteristics, + in constantLuminanceScales, lumaBias, lumaScale, chromaBias, @@ -200,6 +214,11 @@ internal static class Av1YuvConverter out float chromaScale, out float sampleMaximum); + ObuTransferCharacteristics transferCharacteristics = frameBuffer.ColorConfig.TransferCharacteristics; + ConstantLuminanceScales constantLuminanceScales = mode == ConversionMode.ConstantLuminance + ? new ConstantLuminanceScales(transferCharacteristics, kr, kb) + : default; + bool isMonochrome = frameBuffer.ColorFormat == Av1ColorFormat.Yuv400; int subX = frameBuffer.ColorConfig.SubSamplingX ? 1 : 0; int subY = frameBuffer.ColorConfig.SubSamplingY ? 1 : 0; @@ -282,6 +301,8 @@ internal static class Av1YuvConverter kr, kg, kb, + transferCharacteristics, + in constantLuminanceScales, lumaBias, lumaScale, chromaBias, @@ -301,6 +322,8 @@ internal static class Av1YuvConverter kr, kg, kb, + transferCharacteristics, + in constantLuminanceScales, lumaBias, lumaScale, chromaBias, @@ -322,6 +345,8 @@ internal static class Av1YuvConverter kr, kg, kb, + transferCharacteristics, + in constantLuminanceScales, lumaBias, lumaScale, chromaBias, @@ -341,6 +366,8 @@ internal static class Av1YuvConverter kr, kg, kb, + transferCharacteristics, + in constantLuminanceScales, lumaBias, lumaScale, chromaBias, @@ -412,12 +439,21 @@ internal static class Av1YuvConverter kr = 0.2627F; kb = 0.0593F; break; + case ObuMatrixCoefficients.Bt2020ConstantLuminance: + mode = ConversionMode.ConstantLuminance; + kr = 0.2627F; + kb = 0.0593F; + break; case ObuMatrixCoefficients.Smpte2085: mode = ConversionMode.Smpte2085; break; case ObuMatrixCoefficients.ChromaticityDerivedNonConstantLuminance: GetChromaticityDerivedCoefficients(frameBuffer.ColorConfig.ColorPrimaries, out kr, out kb); break; + case ObuMatrixCoefficients.ChromaticityDerivedConstantLuminance: + mode = ConversionMode.ConstantLuminance; + GetChromaticityDerivedCoefficients(frameBuffer.ColorConfig.ColorPrimaries, out kr, out kb); + break; default: throw new NotSupportedException($"AV1 matrix coefficients '{frameBuffer.ColorConfig.MatrixCoefficients}' are not currently supported."); } @@ -619,6 +655,8 @@ internal static class Av1YuvConverter /// The red luma coefficient. /// The green luma coefficient. /// The blue luma coefficient. + /// The signaled transfer characteristics. + /// The constant-luminance chroma scales. /// The encoded luma bias. /// The encoded luma range. /// The encoded chroma midpoint. @@ -640,6 +678,8 @@ internal static class Av1YuvConverter float kr, float kg, float kb, + ObuTransferCharacteristics transferCharacteristics, + in ConstantLuminanceScales constantLuminanceScales, float lumaBias, float lumaScale, float chromaBias, @@ -689,6 +729,24 @@ internal static class Av1YuvConverter b = ((2F * cb) + y) / 0.986566F; r = (2F * cr) + (0.991902F * y); break; + case ConversionMode.ConstantLuminance: + // H.273 equations 66 to 75 define luma in linear light, while the stored luma and + // difference signals remain nonlinear. Reconstruct red and blue before solving green. + float nonlinearBlue = y + + (2F * (cb <= 0F ? constantLuminanceScales.NegativeBlue : constantLuminanceScales.PositiveBlue) * cb); + + float nonlinearRed = y + + (2F * (cr <= 0F ? constantLuminanceScales.NegativeRed : constantLuminanceScales.PositiveRed) * cr); + + float linearY = Av1TransferFunctions.ToLinear(transferCharacteristics, y); + float linearBlue = Av1TransferFunctions.ToLinear(transferCharacteristics, nonlinearBlue); + float linearRed = Av1TransferFunctions.ToLinear(transferCharacteristics, nonlinearRed); + float linearGreen = (linearY - (kr * linearRed) - (kb * linearBlue)) / kg; + + r = nonlinearRed; + g = Av1TransferFunctions.ToGamma(transferCharacteristics, linearGreen); + b = nonlinearBlue; + break; default: r = y + (2F * (1F - kr) * cr); g = y - (2F * ((kr * (1F - kr) * cr) + (kb * (1F - kb) * cb)) / kg); @@ -807,6 +865,8 @@ internal static class Av1YuvConverter /// The red luma coefficient. /// The green luma coefficient. /// The blue luma coefficient. + /// The signaled transfer characteristics. + /// The constant-luminance chroma scales. /// The encoded luma bias. /// The encoded luma range. /// The encoded chroma midpoint. @@ -821,6 +881,8 @@ internal static class Av1YuvConverter float kr, float kg, float kb, + ObuTransferCharacteristics transferCharacteristics, + in ConstantLuminanceScales constantLuminanceScales, float lumaBias, float lumaScale, float chromaBias, @@ -831,7 +893,17 @@ internal static class Av1YuvConverter { for (int x = 0; x < source.Length; x++) { - ConvertRgbToYuv(source[x], mode, kr, kg, kb, out float y, out float cb, out float cr); + ConvertRgbToYuv( + source[x], + mode, + kr, + kg, + kb, + transferCharacteristics, + in constantLuminanceScales, + out float y, + out float cb, + out float cr); yDestination[x] = ToSample((y * lumaScale) + lumaBias, sampleMaximum); if (!uDestination.IsEmpty) @@ -865,6 +937,8 @@ internal static class Av1YuvConverter /// The red luma coefficient. /// The green luma coefficient. /// The blue luma coefficient. + /// The signaled transfer characteristics. + /// The constant-luminance chroma scales. /// The encoded luma bias. /// The encoded luma range. /// The encoded chroma midpoint. @@ -881,6 +955,8 @@ internal static class Av1YuvConverter float kr, float kg, float kb, + ObuTransferCharacteristics transferCharacteristics, + in ConstantLuminanceScales constantLuminanceScales, float lumaBias, float lumaScale, float chromaBias, @@ -902,7 +978,18 @@ internal static class Av1YuvConverter for (int column = 0; column < columnCount; column++) { int sourceIndex = x + column; - ConvertRgbToYuv(source[sourceIndex], mode, kr, kg, kb, out float y, out float cb, out float cr); + ConvertRgbToYuv( + source[sourceIndex], + mode, + kr, + kg, + kb, + transferCharacteristics, + in constantLuminanceScales, + out float y, + out float cb, + out float cr); + yDestination[sourceIndex] = ToSample((y * lumaScale) + lumaBias, sampleMaximum); cbSum += cb; crSum += cr; @@ -929,6 +1016,8 @@ internal static class Av1YuvConverter /// The red luma coefficient. /// The green luma coefficient. /// The blue luma coefficient. + /// The signaled transfer characteristics. + /// The constant-luminance chroma scales. /// The normalized luma result. /// The normalized blue-difference chroma result. /// The normalized red-difference chroma result. @@ -939,6 +1028,8 @@ internal static class Av1YuvConverter float kr, float kg, float kb, + ObuTransferCharacteristics transferCharacteristics, + in ConstantLuminanceScales constantLuminanceScales, out float y, out float cb, out float cr) @@ -983,6 +1074,24 @@ internal static class Av1YuvConverter y = g; cb = ((0.986566F * b) - y) * 0.5F; cr = (r - (0.991902F * y)) * 0.5F; + break; + case ConversionMode.ConstantLuminance: + // The packed RGB values are nonlinear signal components. H.273 constant luminance derives Y + // after applying the inverse transfer curve to each component. + float linearRed = Av1TransferFunctions.ToLinear(transferCharacteristics, r); + float linearGreen = Av1TransferFunctions.ToLinear(transferCharacteristics, g); + float linearBlue = Av1TransferFunctions.ToLinear(transferCharacteristics, b); + float linearY = (kr * linearRed) + (kg * linearGreen) + (kb * linearBlue); + y = Av1TransferFunctions.ToGamma(transferCharacteristics, linearY); + + float blueDifference = b - y; + float redDifference = r - y; + cb = blueDifference / + (2F * (blueDifference <= 0F ? constantLuminanceScales.NegativeBlue : constantLuminanceScales.PositiveBlue)); + + cr = redDifference / + (2F * (redDifference <= 0F ? constantLuminanceScales.NegativeRed : constantLuminanceScales.PositiveRed)); + break; default: y = (kr * r) + (kg * g) + (kb * b); @@ -1030,4 +1139,47 @@ internal static class Av1YuvConverter ushort highBitDepthResult = (ushort)sample; return Unsafe.As(ref highBitDepthResult); } + + /// + /// Stores the H.273 chroma normalization constants for constant-luminance conversion. + /// + private readonly struct ConstantLuminanceScales + { + /// + /// Initializes a new instance of the struct. + /// + /// The signaled transfer characteristics. + /// The red luma coefficient. + /// The blue luma coefficient. + public ConstantLuminanceScales( + ObuTransferCharacteristics transferCharacteristics, + float kr, + float kb) + { + this.NegativeBlue = Av1TransferFunctions.ToGamma(transferCharacteristics, 1F - kb); + this.PositiveBlue = 1F - Av1TransferFunctions.ToGamma(transferCharacteristics, kb); + this.NegativeRed = Av1TransferFunctions.ToGamma(transferCharacteristics, 1F - kr); + this.PositiveRed = 1F - Av1TransferFunctions.ToGamma(transferCharacteristics, kr); + } + + /// + /// Gets the scale for a non-positive blue difference. + /// + public float NegativeBlue { get; } + + /// + /// Gets the scale for a positive blue difference. + /// + public float PositiveBlue { get; } + + /// + /// Gets the scale for a non-positive red difference. + /// + public float NegativeRed { get; } + + /// + /// Gets the scale for a positive red difference. + /// + public float PositiveRed { get; } + } }