Support additional AV1 color matrices

HEIF_IMPLEMENTATION_PLAN.md

@@ -67,7 +67,7 @@ This snapshot pins or classifies the available references and failures; it does
 
 | Managed implementation | Normative behavior | Reviewed implementation reference | Use |
 | --- | --- | --- | --- |
-| `Av1YuvConverter.ConvertToRgb`, `ConvertFromRgb`, scalar row conversion, chroma reconstruction, and chroma downsampling | H.273 formulas 20-31 and the identity, YCgCo, and non-constant-luminance matrix formulas; AV1 section 6.4.2 chroma sample positions | libavif `src/reformat.c` and `src/colr.c` at `092276ce89098ead06db80975173191e5fee1826`; libaom `aom/aom_image.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Scalar behavioral oracle for full/limited-range conversion at 8, 10, and 12 bits. Decode covers monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with AV1 chroma sample positioning. Encode covers the same plane layouts, using libavif's actual-edge-count box average for subsampled chroma. Later SIMD paths must preserve this scalar behavior. |
+| `Av1YuvConverter.ConvertToRgb`, `ConvertFromRgb`, scalar row conversion, chroma reconstruction, and chroma downsampling | ITU-T H.273 (V4) equations 27-57 and 76-78, including chromaticity-derived equations 39-47; AV1 section 6.4.2 chroma sample positions | Official ITU-T H.273 (V4) (07/2024); libavif `src/reformat.c` and `src/colr.c` at `092276ce89098ead06db80975173191e5fee1826`; libaom `aom/aom_image.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Scalar behavioral oracle for full/limited-range conversion at 8, 10, and 12 bits. Decode and encode cover identity, YCgCo, coefficient-based NCL, chromaticity-derived NCL, and SMPTE ST 2085 across monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with AV1 chroma sample positioning. Chromaticity derivation uses every defined H.273 primary and matches libavif's BT.709 fallback for unspecified or reserved primaries. Later SIMD paths must preserve this scalar behavior. |
 | `Av1FrameBuffer` high-bit-depth sample layout and `Av1YuvConverter` 10/12-bit packed-pixel conversion | AV1 section 6.4.1 bit depth and H.273 sample-range scaling | libaom `aom_scale/yv12config.h`, `av1/common/idct.c`, and `av1/common/reconintra.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f`; libavif `src/avif.c` and `src/reformat.c` at `092276ce89098ead06db80975173191e5fee1826` | Establish two-byte native sample storage with sample-unit strides for 10/12-bit reconstruction and use ImageSharp's existing `Rgb48` pixel-operation paths in both directions so packed-pixel staging does not reduce high-bit-depth samples to eight bits. |
 | `Av1PredictionDecoder`, `Av1HighBitDepthPredictor`, `Av1ChromaFromLumaContext`, `Av1PartitionInfo`, and the scalar DC, directional, Paeth, smooth, filter-intra, and chroma-from-luma predictors | AV1 sections 7.11.2 and 7.11.2.3 intra prediction | libaom `aom_dsp/intrapred.c`, `av1/common/reconintra.c`, `av1/common/av1_common_int.h`, `av1/common/blockd.h`, `av1/common/cfl.c`, and `av1/common/cfl.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Behavioral oracle for luma/chroma mode-neighbor addressing, directional upsampling, Paeth selection, smooth normalization, filter-intra taps, high-bit-depth clipping, chroma-from-luma storage/subsampling, and chroma-from-luma row strides. Existing managed scalar tables and predictors remain the implementation base. The WIP rectangular byte-pipeline smooth digest expectations encode width/height-swapped weights and must be replaced only from an independently generated oracle, not regenerated from this implementation. |
 | `Av1TileReader` palette mode/color-map parsing, `Av1SymbolDecoder` palette distributions, `Av1BlockModeInfo` palette state, and `Av1PredictionDecoder` palette reconstruction | AV1 sections 5.11.46, 5.11.49, and 7.11.2 palette prediction | libaom `av1/decoder/decodemv.c`, `av1/decoder/detokenize.c`, `av1/decoder/decoder.h`, `av1/common/pred_common.c`, `av1/common/pred_common.h`, and `av1/common/entropymode.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Reuse the existing tile range decoder and frame-owned mode map while porting the normative palette cache merge, high-bit-depth color deltas, tile-adaptive mode/size/index distributions, diagonal color-map traversal, edge padding, and direct palette-sample reconstruction. This is AV1 still-image compression syntax and does not add retained video reference state or any ISO BMFF surface. |
@@ -130,7 +130,7 @@ This assessment is based on the current source after the upstream ImageSharp mer
 - Palette mode now reads the normative luma/chroma mode and size CDFs, neighbor color caches, high-bit-depth color syntax, diagonal color-index maps, clipped-edge padding, and direct sample prediction through the existing reconstruction pipeline. The scalar implementation matches the pinned libaom source, but an independently encoded palette AVIF fixture is still required before this path is independently verified.
 - Loop-restoration unit parsing records tile-local switchable/Wiener/self-guided filter selections and coefficients in frame-owned plane grids, including super-resolution-adjusted unit corners and the corrected conditional 64x64-superblock unit-size bit. The active restoration stage implements the normative unit geometry, striped deblocked boundaries, Wiener filtering, self-guided projection, and 8/10/12-bit clipping, while reusing the existing SIMD-backed super-resolution and adjacent multiply/add primitives. Independently encoded fixtures covering every parameter set, plane layout, bit depth, and frame-edge geometry are still required. Other normative independently decodable still-image syntax paths still contain `NotImplementedException` or equivalent unsupported branches. Tile-local palette CDF adaptation is present; the remaining still-image frame-context behavior requires a separate source audit without introducing sequence playback state.
 - The frame buffer now establishes two-byte native sample storage, logical plane rows, and sample-unit block strides for 10/12-bit frames. The active intra-prediction, inverse-transform, and block-reconstruction path selects native 16-bit samples for 10/12-bit frames and has focused pipeline wiring coverage. Chroma-from-luma storage, subsampling, parameter derivation, U/V sharing, and 8/10/12-bit prediction are active; independently encoded high-bit-depth and chroma-from-luma AVIF conformance files are still required.
-- `Av1YuvConverter` now consumes the signaled range, supported H.273 matrix coefficients, subsampling, and chroma sample position for 8, 10, and 12-bit output. Its high-bit-depth decode and encode paths use allocator-backed `Rgb48` rows and the existing `PixelOperations<TPixel>` conversions, avoiding the former eight-bit intermediate. Encoder conversion covers monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with libavif-compatible box averaging. Constant-luminance and chromaticity-derived matrices and ICtCp remain incomplete.
+- `Av1YuvConverter` now consumes the signaled range, H.273 matrix coefficients implemented so far, subsampling, and chroma sample position for 8, 10, and 12-bit output. Its high-bit-depth decode and encode paths use allocator-backed `Rgb48` rows and the existing `PixelOperations<TPixel>` conversions, avoiding the former eight-bit intermediate. Encoder conversion covers monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with libavif-compatible box averaging. Identity, YCgCo, the fixed non-constant-luminance matrices, chromaticity-derived non-constant luminance for every defined primary, and SMPTE ST 2085 are active in both directions. BT.2020 constant luminance, chromaticity-derived constant luminance, and ICtCp remain incomplete and require the signaled H.273 transfer functions rather than reuse of the Kr/Kb path.
 - The inverse-transform path allocates arrays in a per-transform hot path.
 - The production prediction, transform, and nonlinear self-guided paths remain predominantly scalar. Normative super-resolution and Wiener horizontal products now reuse ImageSharp's cross-platform adjacent multiply/add SIMD helper with exact scalar fallbacks; further SIMD work must preserve these scalar reconstruction oracles.
 

src/ImageSharp/Formats/Heif/Av1/Av1YuvConverter.cs

@@ -44,6 +44,11 @@ internal static class Av1YuvConverter
         /// The reversible-style YCgCo color transform.
         /// </summary>
         YCgCo,
+
+        /// <summary>
+        /// The SMPTE ST 2085 YDzDx color transform.
+        /// </summary>
+        Smpte2085,
     }
 
     /// <summary>
@@ -407,6 +412,12 @@ internal static class Av1YuvConverter
                 kr = 0.2627F;
                 kb = 0.0593F;
                 break;
+            case ObuMatrixCoefficients.Smpte2085:
+                mode = ConversionMode.Smpte2085;
+                break;
+            case ObuMatrixCoefficients.ChromaticityDerivedNonConstantLuminance:
+                GetChromaticityDerivedCoefficients(frameBuffer.ColorConfig.ColorPrimaries, out kr, out kb);
+                break;
             default:
                 throw new NotSupportedException($"AV1 matrix coefficients '{frameBuffer.ColorConfig.MatrixCoefficients}' are not currently supported.");
         }
@@ -438,6 +449,156 @@ internal static class Av1YuvConverter
         chromaScale = isFullRange ? sampleMaximum : 224F * depthScale;
     }
 
+    /// <summary>
+    /// Computes the luma coefficients defined by the signaled H.273 primary chromaticities.
+    /// </summary>
+    /// <param name="colorPrimaries">The signaled color-primary code point.</param>
+    /// <param name="kr">The resulting red luma coefficient.</param>
+    /// <param name="kb">The resulting blue luma coefficient.</param>
+    private static void GetChromaticityDerivedCoefficients(
+        ObuColorPrimaries colorPrimaries,
+        out float kr,
+        out float kb)
+    {
+        float redX;
+        float redY;
+        float greenX;
+        float greenY;
+        float blueX;
+        float blueY;
+        float whiteX;
+        float whiteY;
+
+        switch (colorPrimaries)
+        {
+            case ObuColorPrimaries.Bt470M:
+                redX = 0.67F;
+                redY = 0.33F;
+                greenX = 0.21F;
+                greenY = 0.71F;
+                blueX = 0.14F;
+                blueY = 0.08F;
+                whiteX = 0.310F;
+                whiteY = 0.316F;
+                break;
+            case ObuColorPrimaries.Bt470BG:
+                redX = 0.64F;
+                redY = 0.33F;
+                greenX = 0.29F;
+                greenY = 0.60F;
+                blueX = 0.15F;
+                blueY = 0.06F;
+                whiteX = 0.3127F;
+                whiteY = 0.3290F;
+                break;
+            case ObuColorPrimaries.Bt601:
+            case ObuColorPrimaries.Smpte240:
+                redX = 0.630F;
+                redY = 0.340F;
+                greenX = 0.310F;
+                greenY = 0.595F;
+                blueX = 0.155F;
+                blueY = 0.070F;
+                whiteX = 0.3127F;
+                whiteY = 0.3290F;
+                break;
+            case ObuColorPrimaries.GenericFilm:
+                redX = 0.681F;
+                redY = 0.319F;
+                greenX = 0.243F;
+                greenY = 0.692F;
+                blueX = 0.145F;
+                blueY = 0.049F;
+                whiteX = 0.310F;
+                whiteY = 0.316F;
+                break;
+            case ObuColorPrimaries.Bt2020:
+                redX = 0.708F;
+                redY = 0.292F;
+                greenX = 0.170F;
+                greenY = 0.797F;
+                blueX = 0.131F;
+                blueY = 0.046F;
+                whiteX = 0.3127F;
+                whiteY = 0.3290F;
+                break;
+            case ObuColorPrimaries.Xyz:
+                redX = 1F;
+                redY = 0F;
+                greenX = 0F;
+                greenY = 1F;
+                blueX = 0F;
+                blueY = 0F;
+                whiteX = 1F / 3F;
+                whiteY = 1F / 3F;
+                break;
+            case ObuColorPrimaries.Smpte431:
+                redX = 0.680F;
+                redY = 0.320F;
+                greenX = 0.265F;
+                greenY = 0.690F;
+                blueX = 0.150F;
+                blueY = 0.060F;
+                whiteX = 0.314F;
+                whiteY = 0.351F;
+                break;
+            case ObuColorPrimaries.Smpte432:
+                redX = 0.680F;
+                redY = 0.320F;
+                greenX = 0.265F;
+                greenY = 0.690F;
+                blueX = 0.150F;
+                blueY = 0.060F;
+                whiteX = 0.3127F;
+                whiteY = 0.3290F;
+                break;
+            case ObuColorPrimaries.Ebu3213:
+                redX = 0.630F;
+                redY = 0.340F;
+                greenX = 0.295F;
+                greenY = 0.605F;
+                blueX = 0.155F;
+                blueY = 0.077F;
+                whiteX = 0.3127F;
+                whiteY = 0.3290F;
+                break;
+            default:
+                // Unspecified and reserved code points have no chromaticities to derive. Match libavif's established
+                // still-image fallback so such files retain the same deterministic BT.709 interpretation.
+                redX = 0.64F;
+                redY = 0.33F;
+                greenX = 0.30F;
+                greenY = 0.60F;
+                blueX = 0.15F;
+                blueY = 0.06F;
+                whiteX = 0.3127F;
+                whiteY = 0.3290F;
+                break;
+        }
+
+        float redZ = 1F - (redX + redY);
+        float greenZ = 1F - (greenX + greenY);
+        float blueZ = 1F - (blueX + blueY);
+        float whiteZ = 1F - (whiteX + whiteY);
+
+        // H.273 equations 39 and 40 solve the RGB-to-XYZ primary matrix at the signaled white point. Keeping
+        // the expanded determinant matches libavif and avoids introducing a general matrix inversion dependency.
+        float denominator = whiteY *
+            ((redX * ((greenY * blueZ) - (blueY * greenZ))) +
+             (greenX * ((blueY * redZ) - (redY * blueZ))) +
+             (blueX * ((redY * greenZ) - (greenY * redZ))));
+
+        kr = (redY *
+            ((whiteX * ((greenY * blueZ) - (blueY * greenZ))) +
+             (whiteY * ((blueX * greenZ) - (greenX * blueZ))) +
+             (whiteZ * ((greenX * blueY) - (blueX * greenY))))) / denominator;
+
+        kb = (blueY *
+            ((whiteX * ((redY * greenZ) - (greenY * redZ))) +
+             (whiteY * ((greenX * redZ) - (redX * greenZ))) +
+             (whiteZ * ((redX * greenY) - (greenX * redY))))) / denominator;
+    }
+
     /// <summary>
     /// Converts one YUV row to packed RGB using the resolved H.273 conversion state.
     /// </summary>
@@ -521,6 +682,13 @@ internal static class Av1YuvConverter
                         g = y + cb;
                         b = temporary - cr;
                         break;
+                    case ConversionMode.Smpte2085:
+                        // H.273 equations 76 to 78 store green as luma and use the ST 2085 scale factors for
+                        // the blue and red difference components.
+                        g = y;
+                        b = ((2F * cb) + y) / 0.986566F;
+                        r = (2F * cr) + (0.991902F * y);
+                        break;
                     default:
                         r = y + (2F * (1F - kr) * cr);
                         g = y - (2F * ((kr * (1F - kr) * cr) + (kb * (1F - kb) * cb)) / kg);
@@ -810,6 +978,12 @@ internal static class Av1YuvConverter
                 cb = (0.5F * g) - (0.25F * (r + b));
                 cr = 0.5F * (r - b);
                 break;
+            case ConversionMode.Smpte2085:
+                // ST 2085 uses green directly as luma, so this path must remain separate from Kr/Kb YCbCr.
+                y = g;
+                cb = ((0.986566F * b) - y) * 0.5F;
+                cr = (r - (0.991902F * y)) * 0.5F;
+                break;
             default:
                 y = (kr * r) + (kg * g) + (kb * b);
                 cb = (b - y) / (2F * (1F - kb));