Parse AV1 loop restoration units

HEIF_IMPLEMENTATION_PLAN.md

@@ -73,7 +73,8 @@ This snapshot pins or classifies the available references and failures; it does
 | `Av1TileReader` palette mode/color-map parsing, `Av1SymbolDecoder` palette distributions, `Av1BlockModeInfo` palette state, and `Av1PredictionDecoder` palette reconstruction | AV1 sections 5.11.46, 5.11.49, and 7.11.2 palette prediction | libaom `av1/decoder/decodemv.c`, `av1/decoder/detokenize.c`, `av1/decoder/decoder.h`, `av1/common/pred_common.c`, `av1/common/pred_common.h`, and `av1/common/entropymode.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Reuse the existing tile range decoder and frame-owned mode map while porting the normative palette cache merge, high-bit-depth color deltas, tile-adaptive mode/size/index distributions, diagonal color-map traversal, edge padding, and direct palette-sample reconstruction. This is AV1 still-image compression syntax and does not add retained video reference state or any ISO BMFF surface. |
 | `Av1LoopFilterKernels`, `Av1LoopFilterContext`, and `Av1LoopFilterDecoder` | AV1 section 7.14 deblocking loop filter | libaom `aom_dsp/loopfilter.c` and `av1/common/av1_loopfilter.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Port the scalar 4-, 6-, 8-, and 14-tap low/high-bit-depth filters, sharpness thresholds, still-frame intra filter-level derivation, transform-edge selection, and plane traversal before enabling the stage. Later SIMD must preserve the scalar result. This is normative AV1 image reconstruction and adds neither generic ISO BMFF models nor retained video reference state. |
 | `Av1CdefDecoder`, `Av1CdefKernels`, and CDEF-unit strength storage | AV1 sections 7.15.2 through 7.15.4 constrained directional enhancement filtering | libaom `av1/common/cdef.c`, `av1/common/cdef_block.c`, `av1/common/cdef.h`, and `av1/common/cdef_block.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Port the scalar direction search, variance adjustment, constrained primary/secondary taps, subsampling direction conversion, skipped-8x8 selection, and frame-edge sentinel behavior. Use a frame-owned source snapshot so filtering never consumes already modified samples. This is normative AV1 still-image reconstruction and introduces no ISO BMFF, track, timing, or sequence-playback surface. |
-| `Av1SuperResolutionDecoder`, `Av1SuperResolutionKernels`, frame-size derivation, and decoded-image dimensions | AV1 sections 7.16 and 7.17 normative super-resolution upscaling | libaom `av1/common/resize.c`, `av1/common/resize.h`, `av1/common/convolve.c`, and `aom_dsp/aom_filter.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Port the fixed 64-phase, 8-tap horizontal filter, phase/step derivation, replicated frame edges, chroma width rounding, signed rounding, and 8/10/12-bit clipping. Reuse ImageSharp's existing cross-platform `Vector128_.MultiplyAddAdjacent` helper for the exact eight-coefficient dot product with a scalar fallback. Generic image resizing is not normative AV1 super-resolution. This adds no track, timing, fragment, animation, or generic ISO BMFF model. |
+| `Av1SuperResolutionDecoder`, `Av1SuperResolutionKernels`, frame-size derivation, and decoded-image dimensions | AV1 section 7.16 normative super-resolution upscaling | libaom `av1/common/resize.c`, `av1/common/resize.h`, `av1/common/convolve.c`, and `aom_dsp/aom_filter.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Port the fixed 64-phase, 8-tap horizontal filter, phase/step derivation, replicated frame edges, chroma width rounding, signed rounding, and 8/10/12-bit clipping. Reuse ImageSharp's existing cross-platform `Vector128_.MultiplyAddAdjacent` helper for the exact eight-coefficient dot product with a scalar fallback. Generic image resizing is not normative AV1 super-resolution. This adds no track, timing, fragment, animation, or generic ISO BMFF model. |
+| `Av1TileReader` loop-restoration unit syntax, `Av1SymbolDecoder` restoration distributions/subexponential codes, and `Av1FrameInfo` unit storage | AV1 section 5.11.57 `read_lr` and `read_lr_unit` syntax | libaom `av1/decoder/decodeframe.c`, `av1/common/restoration.c`, `av1/common/restoration.h`, `av1/common/entropymode.c`, `aom_dsp/binary_codes_reader.c`, and `aom_dsp/recenter.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Decode tile-local switchable/Wiener/self-guided selections, finite reference-subexponential coefficients, chroma Wiener windows, self-guided parameter sets, super-resolution-adjusted unit corners, and the AV1 nearest-unit-count rule into frame-owned per-plane grids. This is compressed still-image syntax and adds no movie, track, timing, fragment, audio, or sequence surface. |
 | `Av1FrameInfo`, `Av1TileReader`, and `Av1BlockDecoder` transform/coefficient storage | AV1 section 5.11.39 coefficient syntax and section 7.11.2 reconstruction | libaom `av1/decoder/decodetxb.c` and `av1/decoder/decoder.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Preserve separate luma and chroma transform coefficients at monotonically advancing per-plane offsets within each superblock so reconstruction consumes the same transform-block order produced by tile parsing. |
 | `Av1InverseQuantizer` and `Av1InverseQuantizationLookup` | AV1 section 7.12.3 inverse quantization | libaom `aom_dsp/aom_dsp_common.h`, `av1/common/quant_common.c`, and `av1/decoder/decodetxb.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Select the per-segment matrix level, alias 64-pixel transform dimensions to their adjusted matrices, retain a flat level-15 matrix, and apply the five-bit inverse-matrix weight scale. The large managed lookup remains a single process-wide table. |
 | `Av1Inverse2dTransformer` and `Av1InverseTransformerFactory` | AV1 section 7.11.2 inverse transform and reconstruction | libaom `av1/common/av1_inv_txfm1d.c`, `av1/common/av1_inv_txfm2d.c`, and `av1/common/idct.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Scalar transform oracle for coefficient-row traversal, intermediate layout, stage ranges, clipping, and high-bit-depth sample addition. The managed 16-bit overload is also used as a parity oracle for the byte overload. |
@@ -123,7 +124,7 @@ This assessment is based on the current source after the upstream ImageSharp mer
 - The single-still `Av1Decoder` path now parses tile state before allocating and reconstructing one independently decodable frame, and it disposes the reconstruction planes after pixel conversion. It deliberately does not retain animation/video reference frames or implement `show_existing_frame` playback state.
 - The reconstruction pipeline now records plane-relative transform geometry, preserves tile-local delta-Q and delta-LF predictors, derives segmentation and reference-adjusted filter levels, and runs the exact scalar low/high-bit-depth AV1 4-, 6-, 8-, and 14-tap deblocking kernels in normative vertical-then-horizontal order. It then applies scalar CDEF direction search, luma variance adjustment, primary and secondary constrained taps, chroma direction conversion, high-bit-depth scaling, skipped-block selection, and frame-edge sentinel handling from immutable per-plane snapshots. Active super-resolution now derives the Appendix A bounded coded width and applies the exact 64-phase, 8-tap horizontal filter with aligned reconstruction-edge input, 8/10/12-bit clipping, and the existing cross-platform `Vector128_.MultiplyAddAdjacent` helper. The available full-decode AV1 fixture does not exercise filtered transform edges, active CDEF strengths, or super-resolution, so independent 8-, 10-, and 12-bit AVIF vectors remain required before these stages have external pixel-level verification. Loop restoration and padding remain disabled even though they are normative when signaled, not optional quality improvements.
 - Palette mode now reads the normative luma/chroma mode and size CDFs, neighbor color caches, high-bit-depth color syntax, diagonal color-index maps, clipped-edge padding, and direct sample prediction through the existing reconstruction pipeline. The scalar implementation matches the pinned libaom source, but an independently encoded palette AVIF fixture is still required before this path is independently verified.
-- Loop restoration and other normative independently decodable still-image syntax paths still contain `NotImplementedException` or equivalent unsupported branches. Tile-local palette CDF adaptation is now present; the remaining still-image frame-context behavior requires a separate source audit without introducing sequence playback state.
+- Loop-restoration unit parsing now records tile-local switchable/Wiener/self-guided filter selections and coefficients in frame-owned plane grids, including super-resolution-adjusted unit corners and the corrected conditional 64x64-superblock unit-size bit. The normative Wiener/self-guided filtering and deblocked stripe-boundary preservation are still disabled, so active loop-restoration images are not yet complete. Other normative independently decodable still-image syntax paths still contain `NotImplementedException` or equivalent unsupported branches. Tile-local palette CDF adaptation is now present; the remaining still-image frame-context behavior requires a separate source audit without introducing sequence playback state.
 - The frame buffer now establishes two-byte native sample storage, logical plane rows, and sample-unit block strides for 10/12-bit frames. The active intra-prediction, inverse-transform, and block-reconstruction path selects native 16-bit samples for 10/12-bit frames and has focused pipeline wiring coverage. Independently encoded high-bit-depth AVIF conformance files are still required; chroma-from-luma storage and the other normative reconstruction stages listed below remain incomplete.
 - `Av1YuvConverter` now consumes the signaled range, supported H.273 matrix coefficients, subsampling, and chroma sample position for 8, 10, and 12-bit output. Its high-bit-depth decode and encode paths use allocator-backed `Rgb48` rows and the existing `PixelOperations<TPixel>` conversions, avoiding the former eight-bit intermediate. Encoder conversion covers monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with libavif-compatible box averaging. Constant-luminance and chromaticity-derived matrices and ICtCp remain incomplete.
 - The inverse-transform path allocates arrays in a per-transform hot path.

src/ImageSharp/Formats/Heif/Av1/Entropy/Av1DefaultDistributions.cs

@@ -113,6 +113,21 @@ internal static class Av1DefaultDistributions
     /// </summary>
     public static Av1Distribution IntraBlockCopy => new(30531);
 
+    /// <summary>
+    /// Gets the distribution selecting none, Wiener, or self-guided filtering for a switchable restoration unit.
+    /// </summary>
+    public static Av1Distribution SwitchableRestoration => new(9413, 22581);
+
+    /// <summary>
+    /// Gets the distribution selecting whether a Wiener restoration unit is filtered.
+    /// </summary>
+    public static Av1Distribution WienerRestoration => new(11570);
+
+    /// <summary>
+    /// Gets the distribution selecting whether a self-guided restoration unit is filtered.
+    /// </summary>
+    public static Av1Distribution SgrProjectionRestoration => new(16855);
+
     /// <summary>
     /// Gets the luma palette-mode distributions indexed by block-size and neighboring-palette contexts.
     /// </summary>

src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolDecoder.cs

@@ -23,6 +23,21 @@ internal ref struct Av1SymbolDecoder
     /// </summary>
     private readonly Av1Distribution tileIntraBlockCopy;
 
+    /// <summary>
+    /// The tile-adaptive switchable loop-restoration distribution.
+    /// </summary>
+    private readonly Av1Distribution switchableRestoration;
+
+    /// <summary>
+    /// The tile-adaptive Wiener loop-restoration distribution.
+    /// </summary>
+    private readonly Av1Distribution wienerRestoration;
+
+    /// <summary>
+    /// The tile-adaptive self-guided loop-restoration distribution.
+    /// </summary>
+    private readonly Av1Distribution sgrProjectionRestoration;
+
     /// <summary>
     /// The tile-adaptive luma palette-mode distributions.
     /// </summary>
@@ -190,6 +205,9 @@ internal ref struct Av1SymbolDecoder
         // Every tile starts from its own frame-context copy. Sharing these objects would let one image's adaptive
         // updates change the initial probabilities used to decode the next tile or image.
         this.tileIntraBlockCopy = Av1DefaultDistributions.IntraBlockCopy.CreateCopy();
+        this.switchableRestoration = Av1DefaultDistributions.SwitchableRestoration.CreateCopy();
+        this.wienerRestoration = Av1DefaultDistributions.WienerRestoration.CreateCopy();
+        this.sgrProjectionRestoration = Av1DefaultDistributions.SgrProjectionRestoration.CreateCopy();
         this.paletteYMode = Av1Distribution.CreateCopy(Av1DefaultDistributions.PaletteYMode);
         this.paletteUvMode = Av1Distribution.CreateCopy(Av1DefaultDistributions.PaletteUvMode);
         this.paletteYSize = Av1Distribution.CreateCopy(Av1DefaultDistributions.PaletteYSize);
@@ -265,6 +283,105 @@ internal ref struct Av1SymbolDecoder
         return (value << 1) - threshold + r.ReadLiteral(1);
     }
 
+    /// <summary>
+    /// Reads a finite subexponential value recentered around a preceding value.
+    /// </summary>
+    /// <param name="valueCount">The number of values in the coded domain.</param>
+    /// <param name="k">The initial subexponential group-size exponent.</param>
+    /// <param name="reference">The preceding value expressed in the zero-based coded domain.</param>
+    /// <returns>The decoded zero-based value.</returns>
+    public int ReadReferenceSubexponential(int valueCount, int k, int reference)
+    {
+        int value = this.ReadSubexponential(valueCount, k);
+        if ((reference << 1) <= valueCount)
+        {
+            return InverseRecenter(reference, value);
+        }
+
+        return valueCount - 1 - InverseRecenter(valueCount - 1 - reference, value);
+    }
+
+    /// <summary>
+    /// Reads the filter type selected for a switchable loop-restoration unit.
+    /// </summary>
+    /// <returns>The decoded unit filter type.</returns>
+    public Av1RestorationFilterType ReadSwitchableRestorationType()
+    {
+        ref Av1SymbolReader r = ref this.reader;
+        return (Av1RestorationFilterType)r.ReadSymbol(this.switchableRestoration);
+    }
+
+    /// <summary>
+    /// Reads whether a Wiener loop-restoration unit applies its filter.
+    /// </summary>
+    /// <returns><see langword="true"/> when Wiener filtering is selected; otherwise, <see langword="false"/>.</returns>
+    public bool ReadWienerRestoration()
+    {
+        ref Av1SymbolReader r = ref this.reader;
+        return r.ReadSymbol(this.wienerRestoration) != 0;
+    }
+
+    /// <summary>
+    /// Reads whether a self-guided loop-restoration unit applies its filter.
+    /// </summary>
+    /// <returns><see langword="true"/> when self-guided filtering is selected; otherwise, <see langword="false"/>.</returns>
+    public bool ReadSgrProjectionRestoration()
+    {
+        ref Av1SymbolReader r = ref this.reader;
+        return r.ReadSymbol(this.sgrProjectionRestoration) != 0;
+    }
+
+    /// <summary>
+    /// Reads a finite subexponential code from the tile entropy stream.
+    /// </summary>
+    /// <param name="valueCount">The number of values in the coded domain.</param>
+    /// <param name="k">The initial subexponential group-size exponent.</param>
+    /// <returns>The decoded zero-based value.</returns>
+    private int ReadSubexponential(int valueCount, int k)
+    {
+        int group = 0;
+        int groupStart = 0;
+        while (true)
+        {
+            int bitCount = group == 0 ? k : k + group - 1;
+            int groupSize = 1 << bitCount;
+            if (valueCount <= groupStart + (3 * groupSize))
+            {
+                // The final group absorbs the remaining alphabet through truncated-binary coding
+                // once fewer than three full subexponential groups remain.
+                return this.ReadUniform(valueCount - groupStart) + groupStart;
+            }
+
+            if (this.ReadLiteral(1) == 0)
+            {
+                return this.ReadLiteral(bitCount) + groupStart;
+            }
+
+            group++;
+            groupStart += groupSize;
+        }
+    }
+
+    /// <summary>
+    /// Maps a non-negative recentered code back around its reference value.
+    /// </summary>
+    /// <param name="reference">The center of the coded value order.</param>
+    /// <param name="value">The recentered non-negative value.</param>
+    /// <returns>The value in its original non-negative domain.</returns>
+    private static int InverseRecenter(int reference, int value)
+    {
+        if (value > (reference << 1))
+        {
+            return value;
+        }
+
+        // Even and odd codes alternate above and below the reference so nearby values receive
+        // the shortest finite-subexponential representations.
+        return (value & 1) == 0
+            ? (value >> 1) + reference
+            : reference - ((value + 1) >> 1);
+    }
+
     /// <summary>
     /// Reads whether the current luma block uses palette prediction.
     /// </summary>

src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuReader.cs

@@ -1591,12 +1591,11 @@ internal class ObuReader
             {
                 frameHeader.LoopRestorationParameters.UnitShift++;
             }
-            else
+            else if (frameHeader.LoopRestorationParameters.UnitShift != 0)
             {
-                if (reader.ReadBoolean())
-                {
-                    frameHeader.LoopRestorationParameters.UnitShift += (int)reader.ReadLiteral(1);
-                }
+                // A 64x64-superblock frame signals the extra size bit only after selecting a
+                // restoration unit larger than 64 samples with the first size bit.
+                frameHeader.LoopRestorationParameters.UnitShift += (int)reader.ReadLiteral(1);
             }
 
             frameHeader.LoopRestorationParameters.Items[0].Size = Av1Constants.RestorationMaxTileSize >> (2 - frameHeader.LoopRestorationParameters.UnitShift);

src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.cs

@@ -105,6 +105,16 @@ internal partial class Av1FrameInfo
     /// </summary>
     private readonly int[] deltaLoopFilter;
 
+    /// <summary>
+    /// Stores raster-ordered loop-restoration units for each color plane.
+    /// </summary>
+    private readonly Av1LoopRestorationUnit[][] loopRestorationUnits = [[], [], []];
+
+    /// <summary>
+    /// Stores the number of loop-restoration unit columns for each color plane.
+    /// </summary>
+    private readonly int[] loopRestorationUnitColumns = new int[Av1Constants.MaxPlanes];
+
     /// <summary>
     /// Initializes a new instance of the <see cref="Av1FrameInfo"/> class.
     /// </summary>
@@ -173,6 +183,43 @@ internal partial class Av1FrameInfo
     /// </summary>
     public int SuperblockModeInfoSize => this.modeInfoSizePerSuperblock;
 
+    /// <summary>
+    /// Allocates the loop-restoration unit grid described by the active frame header.
+    /// </summary>
+    /// <param name="sequenceHeader">The sequence header defining the plane count and chroma subsampling.</param>
+    /// <param name="frameHeader">The frame header defining upscaled dimensions and restoration-unit sizes.</param>
+    public void InitializeLoopRestoration(ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader)
+    {
+        ObuColorConfig colorConfig = sequenceHeader.ColorConfig;
+        for (int planeIndex = 0; planeIndex < colorConfig.PlaneCount; planeIndex++)
+        {
+            ObuLoopRestorationItem item = frameHeader.LoopRestorationParameters.Items[planeIndex];
+            if (item.Type == ObuRestorationType.None)
+            {
+                continue;
+            }
+
+            Av1Plane plane = (Av1Plane)planeIndex;
+            int subsamplingX = plane != Av1Plane.Y && colorConfig.SubSamplingX ? 1 : 0;
+            int subsamplingY = plane != Av1Plane.Y && colorConfig.SubSamplingY ? 1 : 0;
+            int planeWidth = Av1Math.DivideLog2Ceiling(frameHeader.FrameSize.SuperResolutionUpscaledWidth, subsamplingX);
+            int planeHeight = Av1Math.DivideLog2Ceiling(frameHeader.FrameSize.FrameHeight, subsamplingY);
+
+            // A final unit may extend to 150 percent of the nominal size, so AV1 rounds the
+            // unit count to nearest instead of unconditionally rounding a partial unit upward.
+            int columnCount = Math.Max((planeWidth + (item.Size >> 1)) / item.Size, 1);
+            int rowCount = Math.Max((planeHeight + (item.Size >> 1)) / item.Size, 1);
+            Av1LoopRestorationUnit[] units = new Av1LoopRestorationUnit[columnCount * rowCount];
+            for (int i = 0; i < units.Length; i++)
+            {
+                units[i] = new();
+            }
+
+            this.loopRestorationUnitColumns[planeIndex] = columnCount;
+            this.loopRestorationUnits[planeIndex] = units;
+        }
+    }
+
     /// <summary>
     /// Gets the superblock view at the specified frame-grid position.
     /// </summary>
@@ -353,6 +400,26 @@ internal partial class Av1FrameInfo
         return span.Slice(i, 1 << this.deltaLoopFactorLog2);
     }
 
+    /// <summary>
+    /// Gets the number of loop-restoration unit columns allocated for a color plane.
+    /// </summary>
+    /// <param name="plane">The zero-based color-plane index.</param>
+    /// <returns>The number of restoration-unit columns.</returns>
+    public int GetLoopRestorationUnitColumnCount(int plane) => this.loopRestorationUnitColumns[plane];
+
+    /// <summary>
+    /// Gets the loop-restoration unit at a plane-relative grid position.
+    /// </summary>
+    /// <param name="plane">The zero-based color-plane index.</param>
+    /// <param name="row">The restoration-unit row.</param>
+    /// <param name="column">The restoration-unit column.</param>
+    /// <returns>The decoded restoration-unit information.</returns>
+    public Av1LoopRestorationUnit GetLoopRestorationUnit(int plane, int row, int column)
+    {
+        int index = (row * this.loopRestorationUnitColumns[plane]) + column;
+        return this.loopRestorationUnits[plane][index];
+    }
+
     /// <summary>
     /// Stores decoded mode information and maps every 4x4 position covered by its block.
     /// </summary>

src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LoopRestorationUnit.cs

@@ -0,0 +1,45 @@
+// Copyright (c) Six Labors.
+// Licensed under the Six Labors Split License.
+
+namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
+
+/// <summary>
+/// Stores the filter selection and coefficients decoded for one AV1 loop-restoration unit.
+/// </summary>
+internal class Av1LoopRestorationUnit
+{
+    /// <summary>
+    /// Initializes a new instance of the <see cref="Av1LoopRestorationUnit"/> class.
+    /// </summary>
+    public Av1LoopRestorationUnit()
+    {
+        this.WienerVertical = new int[Av1Constants.WienerCoefficientCount];
+        this.WienerHorizontal = new int[Av1Constants.WienerCoefficientCount];
+        this.SgrProjectionCoefficients = new int[2];
+    }
+
+    /// <summary>
+    /// Gets or sets the restoration filter selected for the unit.
+    /// </summary>
+    public Av1RestorationFilterType FilterType { get; set; }
+
+    /// <summary>
+    /// Gets the three transmitted symmetric vertical Wiener coefficients.
+    /// </summary>
+    public int[] WienerVertical { get; }
+
+    /// <summary>
+    /// Gets the three transmitted symmetric horizontal Wiener coefficients.
+    /// </summary>
+    public int[] WienerHorizontal { get; }
+
+    /// <summary>
+    /// Gets or sets the self-guided filter parameter-set index.
+    /// </summary>
+    public int SgrParameterSet { get; set; }
+
+    /// <summary>
+    /// Gets the two self-guided projection coefficients.
+    /// </summary>
+    public int[] SgrProjectionCoefficients { get; }
+}

src/ImageSharp/Formats/Heif/Av1/Tiling/Av1RestorationFilterType.cs

@@ -0,0 +1,25 @@
+// Copyright (c) Six Labors.
+// Licensed under the Six Labors Split License.
+
+namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
+
+/// <summary>
+/// Identifies the filter selected for one AV1 loop-restoration unit.
+/// </summary>
+internal enum Av1RestorationFilterType
+{
+    /// <summary>
+    /// Leaves the restoration-unit samples unchanged.
+    /// </summary>
+    None,
+
+    /// <summary>
+    /// Applies the separable Wiener restoration filter.
+    /// </summary>
+    Wiener,
+
+    /// <summary>
+    /// Applies self-guided restoration projection.
+    /// </summary>
+    SgrProjection,
+}

src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs

@@ -26,6 +26,50 @@ internal class Av1TileReader : IAv1TileReader
     /// </summary>
     private static readonly int[] WienerTapsMid = [3, -7, 15];
 
+    /// <summary>
+    /// The minimum transmitted value for each independent Wiener coefficient.
+    /// </summary>
+    private static readonly int[] WienerCoefficientMinimum = [-5, -23, -17];
+
+    /// <summary>
+    /// The number of possible transmitted values for each independent Wiener coefficient.
+    /// </summary>
+    private static readonly int[] WienerCoefficientValueCount = [16, 32, 64];
+
+    /// <summary>
+    /// The subexponential group-size exponent for each independent Wiener coefficient.
+    /// </summary>
+    private static readonly int[] WienerCoefficientSubexponentialK = [1, 2, 3];
+
+    /// <summary>
+    /// The two self-guided filter radii selected by each parameter-set index.
+    /// </summary>
+    private static readonly int[][] SgrProjectionRadii =
+    [
+        [2, 1], [2, 1], [2, 1], [2, 1], [2, 1], [2, 1], [2, 1], [2, 1],
+        [2, 1], [2, 1], [0, 1], [0, 1], [0, 1], [0, 1], [2, 0], [2, 0]
+    ];
+
+    /// <summary>
+    /// The minimum value of the first self-guided projection coefficient.
+    /// </summary>
+    private const int SgrProjectionCoefficient0Minimum = -96;
+
+    /// <summary>
+    /// The minimum value of the second self-guided projection coefficient.
+    /// </summary>
+    private const int SgrProjectionCoefficient1Minimum = -32;
+
+    /// <summary>
+    /// The number of values in either self-guided projection coefficient domain.
+    /// </summary>
+    private const int SgrProjectionCoefficientValueCount = 128;
+
+    /// <summary>
+    /// The subexponential group-size exponent for self-guided projection coefficients.
+    /// </summary>
+    private const int SgrProjectionSubexponentialK = 4;
+
     /// <summary>
     /// Maps packed coefficient sign classes to their signed contribution to the DC context.
     /// </summary>
@@ -55,7 +99,7 @@ internal class Av1TileReader : IAv1TileReader
     private int[][] referenceSgrXqd = [];
 
     /// <summary>
-    /// Stores the preceding horizontal and vertical Wiener taps for each color plane.
+    /// Stores the preceding vertical and horizontal Wiener taps for each color plane.
     /// </summary>
     private int[][][] referenceLrWiener = [];
 
@@ -123,6 +167,7 @@ internal class Av1TileReader : IAv1TileReader
 
         // FrameInfo owns all traversal-order records and coefficient storage produced by the tile readers.
         this.FrameInfo = new(this.SequenceHeader);
+        this.FrameInfo.InitializeLoopRestoration(this.SequenceHeader, this.FrameHeader);
         this.segmentIds = new int[this.FrameHeader.ModeInfoRowCount][];
         for (int y = 0; y < this.FrameHeader.ModeInfoRowCount; y++)
         {
@@ -232,7 +277,7 @@ internal class Av1TileReader : IAv1TileReader
                 this.firstTransformOffset[0] = 0;
                 this.firstTransformOffset[1] = 0;
                 this.coefficientIndex.AsSpan().Clear();
-                this.ReadLoopRestoration(modeInfoPosition, superBlockSize);
+                this.ReadLoopRestoration(ref reader, modeInfoPosition, superBlockSize);
                 this.ParsePartition(ref reader, modeInfoPosition, superBlockSize, superblockInfo, tileInfo);
 
                 // Identify-only parsing omits a frame decoder but still populates the complete syntax model.
@@ -250,21 +295,183 @@ internal class Av1TileReader : IAv1TileReader
     /// <summary>
     /// Reads loop-restoration unit syntax that begins at a superblock location.
     /// </summary>
+    /// <param name="reader">The tile symbol decoder.</param>
     /// <param name="modeInfoLocation">The superblock origin in 4x4 mode-information units.</param>
     /// <param name="superBlockSize">The superblock size.</param>
-    /// <exception cref="NotImplementedException">A color plane signals a loop-restoration filter.</exception>
-    private void ReadLoopRestoration(Point modeInfoLocation, Av1BlockSize superBlockSize)
+    private void ReadLoopRestoration(ref Av1SymbolDecoder reader, Point modeInfoLocation, Av1BlockSize superBlockSize)
     {
-        int planesCount = this.SequenceHeader.ColorConfig.PlaneCount;
+        ObuColorConfig colorConfig = this.SequenceHeader.ColorConfig;
+        int planesCount = colorConfig.PlaneCount;
         for (int plane = 0; plane < planesCount; plane++)
         {
-            if (this.FrameHeader.LoopRestorationParameters.Items[plane].Type != ObuRestorationType.None)
+            ObuLoopRestorationItem item = this.FrameHeader.LoopRestorationParameters.Items[plane];
+            if (item.Type == ObuRestorationType.None)
             {
-                throw new NotImplementedException("No loop restoration filter support.");
+                continue;
+            }
+
+            int subsamplingX = plane > 0 && colorConfig.SubSamplingX ? 1 : 0;
+            int subsamplingY = plane > 0 && colorConfig.SubSamplingY ? 1 : 0;
+            int planeHeight = Av1Math.DivideLog2Ceiling(this.FrameHeader.FrameSize.FrameHeight, subsamplingY);
+            int unitColumnCount = this.FrameInfo.GetLoopRestorationUnitColumnCount(plane);
+            int unitRowCount = Math.Max((planeHeight + (item.Size >> 1)) / item.Size, 1);
+            int superblockModeInfoSize = superBlockSize.Get4x4WideCount();
+            int modeInfoColumnEnd = modeInfoLocation.X + superblockModeInfoSize;
+            int modeInfoRowEnd = modeInfoLocation.Y + superblockModeInfoSize;
+            int modeInfoSampleWidth = (1 << Av1Constants.ModeInfoSizeLog2) >> subsamplingX;
+            int modeInfoSampleHeight = (1 << Av1Constants.ModeInfoSizeLog2) >> subsamplingY;
+            bool usesSuperResolution =
+                this.FrameHeader.FrameSize.FrameWidth != this.FrameHeader.FrameSize.SuperResolutionUpscaledWidth;
+
+            int columnNumeratorScale = usesSuperResolution
+                ? modeInfoSampleWidth * this.FrameHeader.FrameSize.SuperResolutionDenominator
+                : modeInfoSampleWidth;
+
+            int columnDenominator = usesSuperResolution
+                ? item.Size * Av1Constants.ScaleNumerator
+                : item.Size;
+
+            int rowDenominator = item.Size;
+
+            // Restoration syntax is attached to the superblock containing each unit's upper-left
+            // corner. Super-resolution changes only the horizontal corner conversion.
+            int unitColumnStart = DivideCeiling(modeInfoLocation.X * columnNumeratorScale, columnDenominator);
+            int unitColumnEnd = Math.Min(DivideCeiling(modeInfoColumnEnd * columnNumeratorScale, columnDenominator), unitColumnCount);
+            int unitRowStart = DivideCeiling(modeInfoLocation.Y * modeInfoSampleHeight, rowDenominator);
+            int unitRowEnd = Math.Min(DivideCeiling(modeInfoRowEnd * modeInfoSampleHeight, rowDenominator), unitRowCount);
+            for (int unitRow = unitRowStart; unitRow < unitRowEnd; unitRow++)
+            {
+                for (int unitColumn = unitColumnStart; unitColumn < unitColumnEnd; unitColumn++)
+                {
+                    Av1LoopRestorationUnit unit = this.FrameInfo.GetLoopRestorationUnit(plane, unitRow, unitColumn);
+                    this.ReadLoopRestorationUnit(ref reader, item.Type, plane, unit);
+                }
             }
         }
     }
 
+    /// <summary>
+    /// Reads the filter selection and coefficients for one loop-restoration unit.
+    /// </summary>
+    /// <param name="reader">The tile symbol decoder.</param>
+    /// <param name="frameType">The restoration mode allowed by the frame header.</param>
+    /// <param name="plane">The zero-based color-plane index.</param>
+    /// <param name="unit">The destination restoration-unit information.</param>
+    private void ReadLoopRestorationUnit(
+        ref Av1SymbolDecoder reader,
+        ObuRestorationType frameType,
+        int plane,
+        Av1LoopRestorationUnit unit)
+    {
+        unit.FilterType = frameType switch
+        {
+            ObuRestorationType.Switchable => reader.ReadSwitchableRestorationType(),
+            ObuRestorationType.Wiener => reader.ReadWienerRestoration()
+                ? Av1RestorationFilterType.Wiener
+                : Av1RestorationFilterType.None,
+            ObuRestorationType.SgrProj => reader.ReadSgrProjectionRestoration()
+                ? Av1RestorationFilterType.SgrProjection
+                : Av1RestorationFilterType.None,
+            _ => Av1RestorationFilterType.None,
+        };
+
+        if (unit.FilterType == Av1RestorationFilterType.Wiener)
+        {
+            this.ReadWienerFilter(ref reader, plane, unit);
+        }
+        else if (unit.FilterType == Av1RestorationFilterType.SgrProjection)
+        {
+            this.ReadSgrProjectionFilter(ref reader, plane, unit);
+        }
+    }
+
+    /// <summary>
+    /// Reads the symmetric vertical and horizontal Wiener coefficients for one restoration unit.
+    /// </summary>
+    /// <param name="reader">The tile symbol decoder.</param>
+    /// <param name="plane">The zero-based color-plane index.</param>
+    /// <param name="unit">The destination restoration-unit information.</param>
+    private void ReadWienerFilter(ref Av1SymbolDecoder reader, int plane, Av1LoopRestorationUnit unit)
+    {
+        for (int pass = 0; pass < 2; pass++)
+        {
+            int[] destination = pass == 0 ? unit.WienerVertical : unit.WienerHorizontal;
+            int firstCoefficient = plane == 0 ? 0 : 1;
+            destination[0] = 0;
+            for (int coefficient = firstCoefficient; coefficient < Av1Constants.WienerCoefficientCount; coefficient++)
+            {
+                int minimum = WienerCoefficientMinimum[coefficient];
+                int value = reader.ReadReferenceSubexponential(
+                    WienerCoefficientValueCount[coefficient],
+                    WienerCoefficientSubexponentialK[coefficient],
+                    this.referenceLrWiener[plane][pass][coefficient] - minimum);
+
+                value += minimum;
+                destination[coefficient] = value;
+                this.referenceLrWiener[plane][pass][coefficient] = value;
+            }
+        }
+    }
+
+    /// <summary>
+    /// Reads the parameter-set index and projection coefficients for one self-guided restoration unit.
+    /// </summary>
+    /// <param name="reader">The tile symbol decoder.</param>
+    /// <param name="plane">The zero-based color-plane index.</param>
+    /// <param name="unit">The destination restoration-unit information.</param>
+    private void ReadSgrProjectionFilter(ref Av1SymbolDecoder reader, int plane, Av1LoopRestorationUnit unit)
+    {
+        unit.SgrParameterSet = reader.ReadLiteral(4);
+        int[] radii = SgrProjectionRadii[unit.SgrParameterSet];
+        int[] coefficients = unit.SgrProjectionCoefficients;
+        int[] references = this.referenceSgrXqd[plane];
+        if (radii[0] == 0)
+        {
+            coefficients[0] = 0;
+            coefficients[1] = ReadSgrProjectionCoefficient(ref reader, references[1], SgrProjectionCoefficient1Minimum);
+        }
+        else if (radii[1] == 0)
+        {
+            coefficients[0] = ReadSgrProjectionCoefficient(ref reader, references[0], SgrProjectionCoefficient0Minimum);
+
+            // When the second filter is disabled, AV1 derives the missing projection coefficient
+            // so the combined projection retains its fixed seven-bit scale.
+            coefficients[1] = Av1Math.Clip3(
+                SgrProjectionCoefficient1Minimum,
+                SgrProjectionCoefficient1Minimum + SgrProjectionCoefficientValueCount - 1,
+                SgrProjectionCoefficientValueCount - coefficients[0]);
+        }
+        else
+        {
+            coefficients[0] = ReadSgrProjectionCoefficient(ref reader, references[0], SgrProjectionCoefficient0Minimum);
+            coefficients[1] = ReadSgrProjectionCoefficient(ref reader, references[1], SgrProjectionCoefficient1Minimum);
+        }
+
+        coefficients.CopyTo(references, 0);
+    }
+
+    /// <summary>
+    /// Reads one differentially coded self-guided projection coefficient.
+    /// </summary>
+    /// <param name="reader">The tile symbol decoder.</param>
+    /// <param name="reference">The preceding coefficient value for the plane.</param>
+    /// <param name="minimum">The minimum value in the coefficient domain.</param>
+    /// <returns>The decoded signed coefficient.</returns>
+    private static int ReadSgrProjectionCoefficient(ref Av1SymbolDecoder reader, int reference, int minimum)
+        => reader.ReadReferenceSubexponential(
+            SgrProjectionCoefficientValueCount,
+            SgrProjectionSubexponentialK,
+            reference - minimum) + minimum;
+
+    /// <summary>
+    /// Divides a non-negative numerator by a positive denominator and rounds upward.
+    /// </summary>
+    /// <param name="numerator">The non-negative numerator.</param>
+    /// <param name="denominator">The positive denominator.</param>
+    /// <returns>The ceiling of the quotient.</returns>
+    private static int DivideCeiling(int numerator, int denominator)
+        => (numerator + denominator - 1) / denominator;
+
     /// <summary>
     /// Decodes AV1 partition syntax and recursively visits each resulting coding block.
     /// </summary>