Add AV1 palette entropy coding

HEIF_IMPLEMENTATION_PLAN.md

@@ -850,6 +850,7 @@ Encoder verification contract:
 - [x] Chroma-from-luma mode decision now reuses the decoder's SIMD-first 4:2:0, 4:2:2, and 4:4:4 reconstructed-luma preparation and prediction kernels for both byte and high-bit-depth encoder operators. The constant DC predictor for each chroma plane is computed once and its sample refills every alpha candidate, matching libaom's per-plane DC cache instead of rebuilding the same edge average 33 times. Each block uses 512 bytes of fixed stack scratch for the maximum 8-row predictor surface plus 792 bytes for complete U/V rate and distortion tables; no allocator owner, managed object, frame copy, or persistent buffer was added. Live probability costs exactly mirror current libaom's joint-sign ownership and conditional magnitude symbols. Nine production cases independently derive exact CfL targets from decoder-visible reconstructed luma at 8, 10, and 12 bits, and three entropy cases cover two nonzero signs plus each single-zero-plane form. The exact net11 Release rebuild remains at 1,005 warnings and zero errors, all 8,959 HEIF/AV1 tests pass, and current-main `aomdec` at `a40ed1ea9e4ecc3df58a5bccb76623f2c94ae727` accepts all 29 regenerated payloads.
 - [x] Filter-intra mode decision now runs after ordinary luma modes in current-libaom order, evaluates all five recursive predictors, and refines each predictor across every legal 8x8 transform in transform-enum order. Strictly-better replacement preserves ordinary-mode and filter-mode tie order. Each filter prediction and its source residual are prepared once and reused across transform candidates, avoiding repeated recursive prediction while retaining SIMD-first predictor and subtraction operators. The stack cost is 192 bytes for eight-bit samples or 256 bytes for high-bit-depth samples; no allocator owner or managed buffer was added. Fifteen production cases force every filter mode at 8, 10, and 12 bits and prove retained filter syntax, zero-residual reconstruction, and the DCT-DCT equal-cost transform tie. The decoded-frame MD5 values selected by this checkpoint are `d7d68803763b95827483f14515281d3a` for the 8x8 10-bit gradient, `3f7e34d44c65d7797ad26b5cd4c35bf4` for the 8x8 12-bit gradient, and `9985f05790d2c9f5f28723ef86d5b89b`, `2ba2f1d0fcfef60394a5175553c7cb8b`, and `6aa7a2ed0dbf76ad2ec0c222585272d0` for the odd 4:2:0, 4:2:2, and 4:4:4 gradients. The exact net11 Release rebuild remains at 1,005 warnings and zero errors, 18 focused filter-intra, predictor-reference, syntax-cost, and allocation cases pass, all 8,974 HEIF/AV1 tests pass, and current-main `aomdec` at `a40ed1ea9e4ecc3df58a5bccb76623f2c94ae727` accepts all 29 regenerated payloads.
 - [x] Empty-transform block skip now compares the complete live rate of the two decoder-identical syntax choices after luma and every coded chroma plane have been selected. Current libaom forces all-intra blocks to non-skip; this encoder retains that behavior for every non-empty block and for equal-cost empty blocks, but emits block skip when its adapted context cost is strictly lower than non-skip plus all empty-transform coefficient costs. Costing and writing share the same above-and-left skip-context calculation, and the coefficient estimator returns after the transform-block-skip symbol without reading coefficient storage. This adds no allocation, copy, or persistent state. A focused adapted-CDF regression proves both outcomes through the production decision helper, the two production all-zero fixtures still prove the default real block path, the exact net11 Release rebuild remains at 1,005 warnings and zero errors, all 8,975 HEIF/AV1 tests pass, and current-main `aomdec` at `a40ed1ea9e4ecc3df58a5bccb76623f2c94ae727` accepts all 29 regenerated payloads.
+- [~] Palette entropy coding now mirrors current libaom's adaptive luma-mode, chroma-mode, palette-size, and spatial color-index distributions, together with its truncated-binary uniform code used by palette colors. The complete mutable palette probability graph is created once on first palette search or write, so the current palette-disabled frame path retains zero palette allocations. Three focused regressions cover every legal 2-through-8 color alphabet and every defined mode, size, and color-index context; all 1,928 entropy cases and all 8,978 HEIF/AV1 cases pass direct net11 Release VSTest. The exact Release rebuild remains at 1,005 warnings and zero errors. This checkpoint adds the exact entropy foundation only: palette candidate generation, retained color and index storage, mode decision, map tokenization, and production syntax remain incomplete, and no generated payload changed.
 - [x] The expanded checkpoint exposed a pre-existing transform-block test that asserted uninitialized pooled padding was zero. The test now initializes the complete physical luma plane with a sentinel and proves the block operation leaves both adjacent padding samples unchanged. The exact net11 Release rebuild remains at 1,005 baseline warnings and zero errors, the focused allocator-order set passes 30 of 30 cases, and the complete HEIF/AV1 namespace passes 8,859 of 8,859 direct VSTest cases with zero failures or skips.
 - [x] Combined-frame OBU output now counts the byte-aligned frame and tile-group headers, non-final tile-size fields, and owned tile payloads before emitting the OBU size. It retains only the small allocator-owned header scratch and writes each entropy-coded tile span directly from its detached owner, removing the second file-sized allocator rent and complete-payload copy. A 64 KiB regression proves exactly one sub-payload-sized byte rent with a balanced return and verifies the exact streamed tile tail; the existing two-tile round trip proves size-prefix and ordering parity. The focused writer and production-frame set passes 32 of 32 direct net11 VSTest cases, current-main `aomdec` accepts all 29 generated native-format payloads, and the complete HEIF/AV1 namespace passes 8,860 of 8,860 cases with zero failures or skips.
 - [x] Finalized fixed-block decisions now set the block-level transform-skip flag only when every retained luma and coded chroma transform has zero EOB, matching current libaom's conjunction of per-plane skip state. The previous always-false flag produced legal but redundant non-skip and zero-coefficient syntax. Monochrome and 4:2:0 regressions prove both branches from actual coefficient state; the focused decision and production-frame set passes 32 of 32 direct net11 VSTest cases. Current-main `aomdec` accepts all 29 regenerated payloads, the recorded decoded-frame MD5s are unchanged, and affected 16x16 constant 8-bit and 10-bit payloads are one byte smaller. The complete HEIF/AV1 namespace passes 8,862 of 8,862 cases with zero failures or skips.

src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolEncoder.cs

@@ -81,6 +81,11 @@ internal class Av1SymbolEncoder : IDisposable
     /// </summary>
     private readonly Av1Distribution filterIntraMode;
 
+    /// <summary>
+    /// The palette probability state, created only when screen-content coding uses it.
+    /// </summary>
+    private PaletteEntropyContext? paletteEntropyContext;
+
     /// <summary>
     /// The tile-adaptive absolute quantizer delta distribution.
     /// </summary>
@@ -235,6 +240,184 @@ internal class Av1SymbolEncoder : IDisposable
             int bitCount);
     }
 
+    /// <summary>
+    /// Writes an unsigned fixed-width literal to the tile entropy stream.
+    /// </summary>
+    /// <param name="value">The low-order literal bits.</param>
+    /// <param name="bitCount">The number of bits to write.</param>
+    public void WriteLiteral(uint value, int bitCount)
+    {
+        ref Av1SymbolWriter w = ref this.writer;
+        w.WriteLiteral(value, bitCount);
+    }
+
+    /// <summary>
+    /// Writes a uniformly coded value from a non-power-of-two alphabet.
+    /// </summary>
+    /// <param name="valueCount">The number of possible values.</param>
+    /// <param name="value">The value in the range from zero through <paramref name="valueCount"/> minus one.</param>
+    public void WriteUniform(int valueCount, int value)
+    {
+        ref Av1SymbolWriter w = ref this.writer;
+        int bitCount = Av1Math.Log2(valueCount) + 1;
+        int threshold = (1 << bitCount) - valueCount;
+        if (value < threshold)
+        {
+            // The lower values use the short prefix; every remaining value carries one final disambiguating bit.
+            w.WriteLiteral((uint)value, bitCount - 1);
+            return;
+        }
+
+        int offset = value - threshold;
+        w.WriteLiteral((uint)(threshold + (offset >> 1)), bitCount - 1);
+        w.WriteLiteral((uint)(offset & 1), 1);
+    }
+
+    /// <summary>
+    /// Gets the fixed-point rate of a uniformly coded value.
+    /// </summary>
+    /// <param name="valueCount">The number of possible values.</param>
+    /// <param name="value">The value in the range from zero through <paramref name="valueCount"/> minus one.</param>
+    /// <returns>The rate cost in 1/512-bit units.</returns>
+    public static int GetUniformCost(int valueCount, int value)
+    {
+        int bitCount = Av1Math.Log2(valueCount) + 1;
+        int threshold = (1 << bitCount) - valueCount;
+        return Av1ProbabilityCost.GetLiteralCost(value < threshold ? bitCount - 1 : bitCount);
+    }
+
+    /// <summary>
+    /// Gets the current fixed-point cost of the luma palette-mode flag.
+    /// </summary>
+    /// <param name="usePalette">Indicates whether the block uses luma palette prediction.</param>
+    /// <param name="blockSizeContext">The block-area context in the range from zero through six.</param>
+    /// <param name="neighborContext">The number of available above and left luma neighbors that use palettes.</param>
+    /// <returns>The rate cost in 1/512-bit units.</returns>
+    public int GetPaletteYModeCost(bool usePalette, int blockSizeContext, int neighborContext)
+    {
+        PaletteEntropyContext context = this.paletteEntropyContext ??= new();
+        return Av1ProbabilityCost.GetSymbolCost(
+            context.YMode[blockSizeContext][neighborContext],
+            usePalette ? 1 : 0);
+    }
+
+    /// <summary>
+    /// Writes the luma palette-mode flag.
+    /// </summary>
+    /// <param name="usePalette">Indicates whether the block uses luma palette prediction.</param>
+    /// <param name="blockSizeContext">The block-area context in the range from zero through six.</param>
+    /// <param name="neighborContext">The number of available above and left luma neighbors that use palettes.</param>
+    public void WritePaletteYMode(bool usePalette, int blockSizeContext, int neighborContext)
+    {
+        ref Av1SymbolWriter w = ref this.writer;
+        PaletteEntropyContext context = this.paletteEntropyContext ??= new();
+        w.WriteSymbol(usePalette, context.YMode[blockSizeContext][neighborContext]);
+    }
+
+    /// <summary>
+    /// Gets the current fixed-point cost of the chroma palette-mode flag.
+    /// </summary>
+    /// <param name="usePalette">Indicates whether the block uses chroma palette prediction.</param>
+    /// <param name="hasLumaPalette">Indicates whether the current block uses a luma palette.</param>
+    /// <returns>The rate cost in 1/512-bit units.</returns>
+    public int GetPaletteUvModeCost(bool usePalette, bool hasLumaPalette)
+    {
+        PaletteEntropyContext context = this.paletteEntropyContext ??= new();
+        return Av1ProbabilityCost.GetSymbolCost(
+            context.UvMode[hasLumaPalette ? 1 : 0],
+            usePalette ? 1 : 0);
+    }
+
+    /// <summary>
+    /// Writes the chroma palette-mode flag.
+    /// </summary>
+    /// <param name="usePalette">Indicates whether the block uses chroma palette prediction.</param>
+    /// <param name="hasLumaPalette">Indicates whether the current block uses a luma palette.</param>
+    public void WritePaletteUvMode(bool usePalette, bool hasLumaPalette)
+    {
+        ref Av1SymbolWriter w = ref this.writer;
+        PaletteEntropyContext context = this.paletteEntropyContext ??= new();
+        w.WriteSymbol(usePalette, context.UvMode[hasLumaPalette ? 1 : 0]);
+    }
+
+    /// <summary>
+    /// Gets the current fixed-point cost of a palette-size symbol.
+    /// </summary>
+    /// <param name="paletteSize">The palette size in the range from two through eight.</param>
+    /// <param name="blockSizeContext">The block-area context in the range from zero through six.</param>
+    /// <param name="planeType">The luma or chroma plane class.</param>
+    /// <returns>The rate cost in 1/512-bit units.</returns>
+    public int GetPaletteSizeCost(int paletteSize, int blockSizeContext, Av1PlaneType planeType)
+    {
+        PaletteEntropyContext context = this.paletteEntropyContext ??= new();
+        Av1Distribution distribution = planeType == Av1PlaneType.Y
+            ? context.YSize[blockSizeContext]
+            : context.UvSize[blockSizeContext];
+
+        return Av1ProbabilityCost.GetSymbolCost(distribution, paletteSize - 2);
+    }
+
+    /// <summary>
+    /// Writes a palette-size symbol.
+    /// </summary>
+    /// <param name="paletteSize">The palette size in the range from two through eight.</param>
+    /// <param name="blockSizeContext">The block-area context in the range from zero through six.</param>
+    /// <param name="planeType">The luma or chroma plane class.</param>
+    public void WritePaletteSize(int paletteSize, int blockSizeContext, Av1PlaneType planeType)
+    {
+        ref Av1SymbolWriter w = ref this.writer;
+        PaletteEntropyContext context = this.paletteEntropyContext ??= new();
+        Av1Distribution distribution = planeType == Av1PlaneType.Y
+            ? context.YSize[blockSizeContext]
+            : context.UvSize[blockSizeContext];
+
+        w.WriteSymbol(paletteSize - 2, distribution);
+    }
+
+    /// <summary>
+    /// Gets the current fixed-point cost of a palette color-order index.
+    /// </summary>
+    /// <param name="colorOrderIndex">The index in the context-specific palette color order.</param>
+    /// <param name="paletteSize">The number of colors in the palette.</param>
+    /// <param name="colorContext">The color-index context derived from preceding spatial indices.</param>
+    /// <param name="planeType">The luma or chroma plane class.</param>
+    /// <returns>The rate cost in 1/512-bit units.</returns>
+    public int GetPaletteColorIndexCost(
+        int colorOrderIndex,
+        int paletteSize,
+        int colorContext,
+        Av1PlaneType planeType)
+    {
+        PaletteEntropyContext context = this.paletteEntropyContext ??= new();
+        Av1Distribution distribution = planeType == Av1PlaneType.Y
+            ? context.YColorIndex[paletteSize - 2][colorContext]
+            : context.UvColorIndex[paletteSize - 2][colorContext];
+
+        return Av1ProbabilityCost.GetSymbolCost(distribution, colorOrderIndex);
+    }
+
+    /// <summary>
+    /// Writes a palette color-order index.
+    /// </summary>
+    /// <param name="colorOrderIndex">The index in the context-specific palette color order.</param>
+    /// <param name="paletteSize">The number of colors in the palette.</param>
+    /// <param name="colorContext">The color-index context derived from preceding spatial indices.</param>
+    /// <param name="planeType">The luma or chroma plane class.</param>
+    public void WritePaletteColorIndex(
+        int colorOrderIndex,
+        int paletteSize,
+        int colorContext,
+        Av1PlaneType planeType)
+    {
+        ref Av1SymbolWriter w = ref this.writer;
+        PaletteEntropyContext context = this.paletteEntropyContext ??= new();
+        Av1Distribution distribution = planeType == Av1PlaneType.Y
+            ? context.YColorIndex[paletteSize - 2][colorContext]
+            : context.UvColorIndex[paletteSize - 2][colorContext];
+
+        w.WriteSymbol(colorOrderIndex, distribution);
+    }
+
     /// <summary>
     /// Writes the frame-local intra-block-copy flag.
     /// </summary>
@@ -1224,4 +1407,40 @@ internal class Av1SymbolEncoder : IDisposable
             int bitCount)
             => Av1ProbabilityCost.GetLiteralCost(bitCount);
     }
+
+    /// <summary>
+    /// Owns the adaptive distributions used only by AV1 palette syntax.
+    /// </summary>
+    private sealed class PaletteEntropyContext
+    {
+        /// <summary>
+        /// Gets the luma palette-mode distributions.
+        /// </summary>
+        public Av1Distribution[][] YMode { get; } = Av1DefaultDistributions.PaletteYMode;
+
+        /// <summary>
+        /// Gets the chroma palette-mode distributions.
+        /// </summary>
+        public Av1Distribution[] UvMode { get; } = Av1DefaultDistributions.PaletteUvMode;
+
+        /// <summary>
+        /// Gets the luma palette-size distributions.
+        /// </summary>
+        public Av1Distribution[] YSize { get; } = Av1DefaultDistributions.PaletteYSize;
+
+        /// <summary>
+        /// Gets the chroma palette-size distributions.
+        /// </summary>
+        public Av1Distribution[] UvSize { get; } = Av1DefaultDistributions.PaletteUvSize;
+
+        /// <summary>
+        /// Gets the luma palette color-index distributions.
+        /// </summary>
+        public Av1Distribution[][] YColorIndex { get; } = Av1DefaultDistributions.PaletteYColorIndex;
+
+        /// <summary>
+        /// Gets the chroma palette color-index distributions.
+        /// </summary>
+        public Av1Distribution[][] UvColorIndex { get; } = Av1DefaultDistributions.PaletteUvColorIndex;
+    }
 }

tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EntropyTests.cs

@@ -1009,6 +1009,142 @@ public class Av1EntropyTests
         Assert.Equal(expectedValues, values);
     }
 
+    [Fact]
+    public void RoundTripUniformPaletteIndices()
+    {
+        Configuration configuration = Configuration.Default;
+        using Av1SymbolEncoder encoder = new(configuration, 64, BaseQIndex);
+
+        for (int valueCount = 2; valueCount <= Av1Constants.PaletteMaxSize; valueCount++)
+        {
+            for (int value = 0; value < valueCount; value++)
+            {
+                encoder.WriteUniform(valueCount, value);
+            }
+        }
+
+        using IMemoryOwner<byte> encoded = encoder.Exit();
+        Av1SymbolDecoder decoder = new(configuration, encoded.GetSpan(), BaseQIndex);
+        for (int valueCount = 2; valueCount <= Av1Constants.PaletteMaxSize; valueCount++)
+        {
+            for (int value = 0; value < valueCount; value++)
+            {
+                Assert.Equal(value, decoder.ReadUniform(valueCount));
+            }
+        }
+    }
+
+    [Fact]
+    public void RoundTripPaletteSymbols()
+    {
+        Configuration configuration = Configuration.Default;
+        using Av1SymbolEncoder encoder = new(configuration, 256, BaseQIndex);
+
+        for (int blockSizeContext = 0; blockSizeContext < 7; blockSizeContext++)
+        {
+            for (int neighborContext = 0; neighborContext < 3; neighborContext++)
+            {
+                encoder.WritePaletteYMode(
+                    ((blockSizeContext + neighborContext) & 1) != 0,
+                    blockSizeContext,
+                    neighborContext);
+            }
+
+            int paletteSize = blockSizeContext + 2;
+            encoder.WritePaletteSize(paletteSize, blockSizeContext, Av1PlaneType.Y);
+            encoder.WritePaletteSize(Av1Constants.PaletteMaxSize - blockSizeContext, blockSizeContext, Av1PlaneType.Uv);
+        }
+
+        encoder.WritePaletteUvMode(false, false);
+        encoder.WritePaletteUvMode(true, false);
+        encoder.WritePaletteUvMode(false, true);
+        encoder.WritePaletteUvMode(true, true);
+        for (int paletteSize = 2; paletteSize <= Av1Constants.PaletteMaxSize; paletteSize++)
+        {
+            for (int colorContext = 0; colorContext < 5; colorContext++)
+            {
+                int colorOrderIndex = (paletteSize + colorContext - 1) % paletteSize;
+                encoder.WritePaletteColorIndex(colorOrderIndex, paletteSize, colorContext, Av1PlaneType.Y);
+                encoder.WritePaletteColorIndex(colorOrderIndex, paletteSize, colorContext, Av1PlaneType.Uv);
+            }
+        }
+
+        using IMemoryOwner<byte> encoded = encoder.Exit();
+        Av1SymbolDecoder decoder = new(configuration, encoded.GetSpan(), BaseQIndex);
+        for (int blockSizeContext = 0; blockSizeContext < 7; blockSizeContext++)
+        {
+            for (int neighborContext = 0; neighborContext < 3; neighborContext++)
+            {
+                Assert.Equal(
+                    ((blockSizeContext + neighborContext) & 1) != 0,
+                    decoder.ReadPaletteYMode(blockSizeContext, neighborContext));
+            }
+
+            Assert.Equal(blockSizeContext + 2, decoder.ReadPaletteSize(blockSizeContext, Av1PlaneType.Y));
+            Assert.Equal(
+                Av1Constants.PaletteMaxSize - blockSizeContext,
+                decoder.ReadPaletteSize(blockSizeContext, Av1PlaneType.Uv));
+        }
+
+        Assert.False(decoder.ReadPaletteUvMode(false));
+        Assert.True(decoder.ReadPaletteUvMode(false));
+        Assert.False(decoder.ReadPaletteUvMode(true));
+        Assert.True(decoder.ReadPaletteUvMode(true));
+        for (int paletteSize = 2; paletteSize <= Av1Constants.PaletteMaxSize; paletteSize++)
+        {
+            for (int colorContext = 0; colorContext < 5; colorContext++)
+            {
+                int expected = (paletteSize + colorContext - 1) % paletteSize;
+                Assert.Equal(
+                    expected,
+                    decoder.ReadPaletteColorIndex(paletteSize, colorContext, Av1PlaneType.Y));
+
+                Assert.Equal(
+                    expected,
+                    decoder.ReadPaletteColorIndex(paletteSize, colorContext, Av1PlaneType.Uv));
+            }
+        }
+    }
+
+    [Fact]
+    public void PaletteSyntaxCostsMatchCurrentDistributions()
+    {
+        using Av1SymbolEncoder encoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: false);
+        Av1Distribution[][] yMode = Av1DefaultDistributions.PaletteYMode;
+        Av1Distribution[] uvMode = Av1DefaultDistributions.PaletteUvMode;
+        Av1Distribution[] ySize = Av1DefaultDistributions.PaletteYSize;
+        Av1Distribution[] uvSize = Av1DefaultDistributions.PaletteUvSize;
+        Av1Distribution[][] yColorIndex = Av1DefaultDistributions.PaletteYColorIndex;
+        Av1Distribution[][] uvColorIndex = Av1DefaultDistributions.PaletteUvColorIndex;
+
+        Assert.Equal(
+            Av1ProbabilityCost.GetSymbolCost(yMode[4][2], 1),
+            encoder.GetPaletteYModeCost(true, 4, 2));
+
+        Assert.Equal(
+            Av1ProbabilityCost.GetSymbolCost(uvMode[1], 0),
+            encoder.GetPaletteUvModeCost(false, true));
+
+        Assert.Equal(
+            Av1ProbabilityCost.GetSymbolCost(ySize[3], 4),
+            encoder.GetPaletteSizeCost(6, 3, Av1PlaneType.Y));
+
+        Assert.Equal(
+            Av1ProbabilityCost.GetSymbolCost(uvSize[5], 1),
+            encoder.GetPaletteSizeCost(3, 5, Av1PlaneType.Uv));
+
+        Assert.Equal(
+            Av1ProbabilityCost.GetSymbolCost(yColorIndex[6][4], 7),
+            encoder.GetPaletteColorIndexCost(7, 8, 4, Av1PlaneType.Y));
+
+        Assert.Equal(
+            Av1ProbabilityCost.GetSymbolCost(uvColorIndex[3][2], 4),
+            encoder.GetPaletteColorIndexCost(4, 5, 2, Av1PlaneType.Uv));
+
+        Assert.Equal(Av1ProbabilityCost.GetLiteralCost(2), Av1SymbolEncoder.GetUniformCost(5, 2));
+        Assert.Equal(Av1ProbabilityCost.GetLiteralCost(3), Av1SymbolEncoder.GetUniformCost(5, 3));
+    }
+
     [Theory]
     [MemberData(nameof(GetRangeData), 20)]
     public void RoundTripPartitionType(int context)