diff --git a/HEIF_IMPLEMENTATION_PLAN.md b/HEIF_IMPLEMENTATION_PLAN.md index fd440ef065..9596d46aac 100644 --- a/HEIF_IMPLEMENTATION_PLAN.md +++ b/HEIF_IMPLEMENTATION_PLAN.md @@ -850,6 +850,7 @@ Encoder verification contract: - [x] Chroma-from-luma mode decision now reuses the decoder's SIMD-first 4:2:0, 4:2:2, and 4:4:4 reconstructed-luma preparation and prediction kernels for both byte and high-bit-depth encoder operators. The constant DC predictor for each chroma plane is computed once and its sample refills every alpha candidate, matching libaom's per-plane DC cache instead of rebuilding the same edge average 33 times. Each block uses 512 bytes of fixed stack scratch for the maximum 8-row predictor surface plus 792 bytes for complete U/V rate and distortion tables; no allocator owner, managed object, frame copy, or persistent buffer was added. Live probability costs exactly mirror current libaom's joint-sign ownership and conditional magnitude symbols. Nine production cases independently derive exact CfL targets from decoder-visible reconstructed luma at 8, 10, and 12 bits, and three entropy cases cover two nonzero signs plus each single-zero-plane form. The exact net11 Release rebuild remains at 1,005 warnings and zero errors, all 8,959 HEIF/AV1 tests pass, and current-main `aomdec` at `a40ed1ea9e4ecc3df58a5bccb76623f2c94ae727` accepts all 29 regenerated payloads. - [x] Filter-intra mode decision now runs after ordinary luma modes in current-libaom order, evaluates all five recursive predictors, and refines each predictor across every legal 8x8 transform in transform-enum order. Strictly-better replacement preserves ordinary-mode and filter-mode tie order. Each filter prediction and its source residual are prepared once and reused across transform candidates, avoiding repeated recursive prediction while retaining SIMD-first predictor and subtraction operators. The stack cost is 192 bytes for eight-bit samples or 256 bytes for high-bit-depth samples; no allocator owner or managed buffer was added. Fifteen production cases force every filter mode at 8, 10, and 12 bits and prove retained filter syntax, zero-residual reconstruction, and the DCT-DCT equal-cost transform tie. The decoded-frame MD5 values selected by this checkpoint are `d7d68803763b95827483f14515281d3a` for the 8x8 10-bit gradient, `3f7e34d44c65d7797ad26b5cd4c35bf4` for the 8x8 12-bit gradient, and `9985f05790d2c9f5f28723ef86d5b89b`, `2ba2f1d0fcfef60394a5175553c7cb8b`, and `6aa7a2ed0dbf76ad2ec0c222585272d0` for the odd 4:2:0, 4:2:2, and 4:4:4 gradients. The exact net11 Release rebuild remains at 1,005 warnings and zero errors, 18 focused filter-intra, predictor-reference, syntax-cost, and allocation cases pass, all 8,974 HEIF/AV1 tests pass, and current-main `aomdec` at `a40ed1ea9e4ecc3df58a5bccb76623f2c94ae727` accepts all 29 regenerated payloads. - [x] Empty-transform block skip now compares the complete live rate of the two decoder-identical syntax choices after luma and every coded chroma plane have been selected. Current libaom forces all-intra blocks to non-skip; this encoder retains that behavior for every non-empty block and for equal-cost empty blocks, but emits block skip when its adapted context cost is strictly lower than non-skip plus all empty-transform coefficient costs. Costing and writing share the same above-and-left skip-context calculation, and the coefficient estimator returns after the transform-block-skip symbol without reading coefficient storage. This adds no allocation, copy, or persistent state. A focused adapted-CDF regression proves both outcomes through the production decision helper, the two production all-zero fixtures still prove the default real block path, the exact net11 Release rebuild remains at 1,005 warnings and zero errors, all 8,975 HEIF/AV1 tests pass, and current-main `aomdec` at `a40ed1ea9e4ecc3df58a5bccb76623f2c94ae727` accepts all 29 regenerated payloads. +- [~] Palette entropy coding now mirrors current libaom's adaptive luma-mode, chroma-mode, palette-size, and spatial color-index distributions, together with its truncated-binary uniform code used by palette colors. The complete mutable palette probability graph is created once on first palette search or write, so the current palette-disabled frame path retains zero palette allocations. Three focused regressions cover every legal 2-through-8 color alphabet and every defined mode, size, and color-index context; all 1,928 entropy cases and all 8,978 HEIF/AV1 cases pass direct net11 Release VSTest. The exact Release rebuild remains at 1,005 warnings and zero errors. This checkpoint adds the exact entropy foundation only: palette candidate generation, retained color and index storage, mode decision, map tokenization, and production syntax remain incomplete, and no generated payload changed. - [x] The expanded checkpoint exposed a pre-existing transform-block test that asserted uninitialized pooled padding was zero. The test now initializes the complete physical luma plane with a sentinel and proves the block operation leaves both adjacent padding samples unchanged. The exact net11 Release rebuild remains at 1,005 baseline warnings and zero errors, the focused allocator-order set passes 30 of 30 cases, and the complete HEIF/AV1 namespace passes 8,859 of 8,859 direct VSTest cases with zero failures or skips. - [x] Combined-frame OBU output now counts the byte-aligned frame and tile-group headers, non-final tile-size fields, and owned tile payloads before emitting the OBU size. It retains only the small allocator-owned header scratch and writes each entropy-coded tile span directly from its detached owner, removing the second file-sized allocator rent and complete-payload copy. A 64 KiB regression proves exactly one sub-payload-sized byte rent with a balanced return and verifies the exact streamed tile tail; the existing two-tile round trip proves size-prefix and ordering parity. The focused writer and production-frame set passes 32 of 32 direct net11 VSTest cases, current-main `aomdec` accepts all 29 generated native-format payloads, and the complete HEIF/AV1 namespace passes 8,860 of 8,860 cases with zero failures or skips. - [x] Finalized fixed-block decisions now set the block-level transform-skip flag only when every retained luma and coded chroma transform has zero EOB, matching current libaom's conjunction of per-plane skip state. The previous always-false flag produced legal but redundant non-skip and zero-coefficient syntax. Monochrome and 4:2:0 regressions prove both branches from actual coefficient state; the focused decision and production-frame set passes 32 of 32 direct net11 VSTest cases. Current-main `aomdec` accepts all 29 regenerated payloads, the recorded decoded-frame MD5s are unchanged, and affected 16x16 constant 8-bit and 10-bit payloads are one byte smaller. The complete HEIF/AV1 namespace passes 8,862 of 8,862 cases with zero failures or skips. diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolEncoder.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolEncoder.cs index f79a07dfb7..0e0e2c2939 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolEncoder.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolEncoder.cs @@ -81,6 +81,11 @@ internal class Av1SymbolEncoder : IDisposable /// private readonly Av1Distribution filterIntraMode; + /// + /// The palette probability state, created only when screen-content coding uses it. + /// + private PaletteEntropyContext? paletteEntropyContext; + /// /// The tile-adaptive absolute quantizer delta distribution. /// @@ -235,6 +240,184 @@ internal class Av1SymbolEncoder : IDisposable int bitCount); } + /// + /// Writes an unsigned fixed-width literal to the tile entropy stream. + /// + /// The low-order literal bits. + /// The number of bits to write. + public void WriteLiteral(uint value, int bitCount) + { + ref Av1SymbolWriter w = ref this.writer; + w.WriteLiteral(value, bitCount); + } + + /// + /// Writes a uniformly coded value from a non-power-of-two alphabet. + /// + /// The number of possible values. + /// The value in the range from zero through minus one. + public void WriteUniform(int valueCount, int value) + { + ref Av1SymbolWriter w = ref this.writer; + int bitCount = Av1Math.Log2(valueCount) + 1; + int threshold = (1 << bitCount) - valueCount; + if (value < threshold) + { + // The lower values use the short prefix; every remaining value carries one final disambiguating bit. + w.WriteLiteral((uint)value, bitCount - 1); + return; + } + + int offset = value - threshold; + w.WriteLiteral((uint)(threshold + (offset >> 1)), bitCount - 1); + w.WriteLiteral((uint)(offset & 1), 1); + } + + /// + /// Gets the fixed-point rate of a uniformly coded value. + /// + /// The number of possible values. + /// The value in the range from zero through minus one. + /// The rate cost in 1/512-bit units. + public static int GetUniformCost(int valueCount, int value) + { + int bitCount = Av1Math.Log2(valueCount) + 1; + int threshold = (1 << bitCount) - valueCount; + return Av1ProbabilityCost.GetLiteralCost(value < threshold ? bitCount - 1 : bitCount); + } + + /// + /// Gets the current fixed-point cost of the luma palette-mode flag. + /// + /// Indicates whether the block uses luma palette prediction. + /// The block-area context in the range from zero through six. + /// The number of available above and left luma neighbors that use palettes. + /// The rate cost in 1/512-bit units. + public int GetPaletteYModeCost(bool usePalette, int blockSizeContext, int neighborContext) + { + PaletteEntropyContext context = this.paletteEntropyContext ??= new(); + return Av1ProbabilityCost.GetSymbolCost( + context.YMode[blockSizeContext][neighborContext], + usePalette ? 1 : 0); + } + + /// + /// Writes the luma palette-mode flag. + /// + /// Indicates whether the block uses luma palette prediction. + /// The block-area context in the range from zero through six. + /// The number of available above and left luma neighbors that use palettes. + public void WritePaletteYMode(bool usePalette, int blockSizeContext, int neighborContext) + { + ref Av1SymbolWriter w = ref this.writer; + PaletteEntropyContext context = this.paletteEntropyContext ??= new(); + w.WriteSymbol(usePalette, context.YMode[blockSizeContext][neighborContext]); + } + + /// + /// Gets the current fixed-point cost of the chroma palette-mode flag. + /// + /// Indicates whether the block uses chroma palette prediction. + /// Indicates whether the current block uses a luma palette. + /// The rate cost in 1/512-bit units. + public int GetPaletteUvModeCost(bool usePalette, bool hasLumaPalette) + { + PaletteEntropyContext context = this.paletteEntropyContext ??= new(); + return Av1ProbabilityCost.GetSymbolCost( + context.UvMode[hasLumaPalette ? 1 : 0], + usePalette ? 1 : 0); + } + + /// + /// Writes the chroma palette-mode flag. + /// + /// Indicates whether the block uses chroma palette prediction. + /// Indicates whether the current block uses a luma palette. + public void WritePaletteUvMode(bool usePalette, bool hasLumaPalette) + { + ref Av1SymbolWriter w = ref this.writer; + PaletteEntropyContext context = this.paletteEntropyContext ??= new(); + w.WriteSymbol(usePalette, context.UvMode[hasLumaPalette ? 1 : 0]); + } + + /// + /// Gets the current fixed-point cost of a palette-size symbol. + /// + /// The palette size in the range from two through eight. + /// The block-area context in the range from zero through six. + /// The luma or chroma plane class. + /// The rate cost in 1/512-bit units. + public int GetPaletteSizeCost(int paletteSize, int blockSizeContext, Av1PlaneType planeType) + { + PaletteEntropyContext context = this.paletteEntropyContext ??= new(); + Av1Distribution distribution = planeType == Av1PlaneType.Y + ? context.YSize[blockSizeContext] + : context.UvSize[blockSizeContext]; + + return Av1ProbabilityCost.GetSymbolCost(distribution, paletteSize - 2); + } + + /// + /// Writes a palette-size symbol. + /// + /// The palette size in the range from two through eight. + /// The block-area context in the range from zero through six. + /// The luma or chroma plane class. + public void WritePaletteSize(int paletteSize, int blockSizeContext, Av1PlaneType planeType) + { + ref Av1SymbolWriter w = ref this.writer; + PaletteEntropyContext context = this.paletteEntropyContext ??= new(); + Av1Distribution distribution = planeType == Av1PlaneType.Y + ? context.YSize[blockSizeContext] + : context.UvSize[blockSizeContext]; + + w.WriteSymbol(paletteSize - 2, distribution); + } + + /// + /// Gets the current fixed-point cost of a palette color-order index. + /// + /// The index in the context-specific palette color order. + /// The number of colors in the palette. + /// The color-index context derived from preceding spatial indices. + /// The luma or chroma plane class. + /// The rate cost in 1/512-bit units. + public int GetPaletteColorIndexCost( + int colorOrderIndex, + int paletteSize, + int colorContext, + Av1PlaneType planeType) + { + PaletteEntropyContext context = this.paletteEntropyContext ??= new(); + Av1Distribution distribution = planeType == Av1PlaneType.Y + ? context.YColorIndex[paletteSize - 2][colorContext] + : context.UvColorIndex[paletteSize - 2][colorContext]; + + return Av1ProbabilityCost.GetSymbolCost(distribution, colorOrderIndex); + } + + /// + /// Writes a palette color-order index. + /// + /// The index in the context-specific palette color order. + /// The number of colors in the palette. + /// The color-index context derived from preceding spatial indices. + /// The luma or chroma plane class. + public void WritePaletteColorIndex( + int colorOrderIndex, + int paletteSize, + int colorContext, + Av1PlaneType planeType) + { + ref Av1SymbolWriter w = ref this.writer; + PaletteEntropyContext context = this.paletteEntropyContext ??= new(); + Av1Distribution distribution = planeType == Av1PlaneType.Y + ? context.YColorIndex[paletteSize - 2][colorContext] + : context.UvColorIndex[paletteSize - 2][colorContext]; + + w.WriteSymbol(colorOrderIndex, distribution); + } + /// /// Writes the frame-local intra-block-copy flag. /// @@ -1224,4 +1407,40 @@ internal class Av1SymbolEncoder : IDisposable int bitCount) => Av1ProbabilityCost.GetLiteralCost(bitCount); } + + /// + /// Owns the adaptive distributions used only by AV1 palette syntax. + /// + private sealed class PaletteEntropyContext + { + /// + /// Gets the luma palette-mode distributions. + /// + public Av1Distribution[][] YMode { get; } = Av1DefaultDistributions.PaletteYMode; + + /// + /// Gets the chroma palette-mode distributions. + /// + public Av1Distribution[] UvMode { get; } = Av1DefaultDistributions.PaletteUvMode; + + /// + /// Gets the luma palette-size distributions. + /// + public Av1Distribution[] YSize { get; } = Av1DefaultDistributions.PaletteYSize; + + /// + /// Gets the chroma palette-size distributions. + /// + public Av1Distribution[] UvSize { get; } = Av1DefaultDistributions.PaletteUvSize; + + /// + /// Gets the luma palette color-index distributions. + /// + public Av1Distribution[][] YColorIndex { get; } = Av1DefaultDistributions.PaletteYColorIndex; + + /// + /// Gets the chroma palette color-index distributions. + /// + public Av1Distribution[][] UvColorIndex { get; } = Av1DefaultDistributions.PaletteUvColorIndex; + } } diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EntropyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EntropyTests.cs index eb0560c593..2c2df5df8e 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EntropyTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EntropyTests.cs @@ -1009,6 +1009,142 @@ public class Av1EntropyTests Assert.Equal(expectedValues, values); } + [Fact] + public void RoundTripUniformPaletteIndices() + { + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, 64, BaseQIndex); + + for (int valueCount = 2; valueCount <= Av1Constants.PaletteMaxSize; valueCount++) + { + for (int value = 0; value < valueCount; value++) + { + encoder.WriteUniform(valueCount, value); + } + } + + using IMemoryOwner encoded = encoder.Exit(); + Av1SymbolDecoder decoder = new(configuration, encoded.GetSpan(), BaseQIndex); + for (int valueCount = 2; valueCount <= Av1Constants.PaletteMaxSize; valueCount++) + { + for (int value = 0; value < valueCount; value++) + { + Assert.Equal(value, decoder.ReadUniform(valueCount)); + } + } + } + + [Fact] + public void RoundTripPaletteSymbols() + { + Configuration configuration = Configuration.Default; + using Av1SymbolEncoder encoder = new(configuration, 256, BaseQIndex); + + for (int blockSizeContext = 0; blockSizeContext < 7; blockSizeContext++) + { + for (int neighborContext = 0; neighborContext < 3; neighborContext++) + { + encoder.WritePaletteYMode( + ((blockSizeContext + neighborContext) & 1) != 0, + blockSizeContext, + neighborContext); + } + + int paletteSize = blockSizeContext + 2; + encoder.WritePaletteSize(paletteSize, blockSizeContext, Av1PlaneType.Y); + encoder.WritePaletteSize(Av1Constants.PaletteMaxSize - blockSizeContext, blockSizeContext, Av1PlaneType.Uv); + } + + encoder.WritePaletteUvMode(false, false); + encoder.WritePaletteUvMode(true, false); + encoder.WritePaletteUvMode(false, true); + encoder.WritePaletteUvMode(true, true); + for (int paletteSize = 2; paletteSize <= Av1Constants.PaletteMaxSize; paletteSize++) + { + for (int colorContext = 0; colorContext < 5; colorContext++) + { + int colorOrderIndex = (paletteSize + colorContext - 1) % paletteSize; + encoder.WritePaletteColorIndex(colorOrderIndex, paletteSize, colorContext, Av1PlaneType.Y); + encoder.WritePaletteColorIndex(colorOrderIndex, paletteSize, colorContext, Av1PlaneType.Uv); + } + } + + using IMemoryOwner encoded = encoder.Exit(); + Av1SymbolDecoder decoder = new(configuration, encoded.GetSpan(), BaseQIndex); + for (int blockSizeContext = 0; blockSizeContext < 7; blockSizeContext++) + { + for (int neighborContext = 0; neighborContext < 3; neighborContext++) + { + Assert.Equal( + ((blockSizeContext + neighborContext) & 1) != 0, + decoder.ReadPaletteYMode(blockSizeContext, neighborContext)); + } + + Assert.Equal(blockSizeContext + 2, decoder.ReadPaletteSize(blockSizeContext, Av1PlaneType.Y)); + Assert.Equal( + Av1Constants.PaletteMaxSize - blockSizeContext, + decoder.ReadPaletteSize(blockSizeContext, Av1PlaneType.Uv)); + } + + Assert.False(decoder.ReadPaletteUvMode(false)); + Assert.True(decoder.ReadPaletteUvMode(false)); + Assert.False(decoder.ReadPaletteUvMode(true)); + Assert.True(decoder.ReadPaletteUvMode(true)); + for (int paletteSize = 2; paletteSize <= Av1Constants.PaletteMaxSize; paletteSize++) + { + for (int colorContext = 0; colorContext < 5; colorContext++) + { + int expected = (paletteSize + colorContext - 1) % paletteSize; + Assert.Equal( + expected, + decoder.ReadPaletteColorIndex(paletteSize, colorContext, Av1PlaneType.Y)); + + Assert.Equal( + expected, + decoder.ReadPaletteColorIndex(paletteSize, colorContext, Av1PlaneType.Uv)); + } + } + } + + [Fact] + public void PaletteSyntaxCostsMatchCurrentDistributions() + { + using Av1SymbolEncoder encoder = new(Configuration.Default, 64, BaseQIndex, updateCdf: false); + Av1Distribution[][] yMode = Av1DefaultDistributions.PaletteYMode; + Av1Distribution[] uvMode = Av1DefaultDistributions.PaletteUvMode; + Av1Distribution[] ySize = Av1DefaultDistributions.PaletteYSize; + Av1Distribution[] uvSize = Av1DefaultDistributions.PaletteUvSize; + Av1Distribution[][] yColorIndex = Av1DefaultDistributions.PaletteYColorIndex; + Av1Distribution[][] uvColorIndex = Av1DefaultDistributions.PaletteUvColorIndex; + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(yMode[4][2], 1), + encoder.GetPaletteYModeCost(true, 4, 2)); + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(uvMode[1], 0), + encoder.GetPaletteUvModeCost(false, true)); + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(ySize[3], 4), + encoder.GetPaletteSizeCost(6, 3, Av1PlaneType.Y)); + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(uvSize[5], 1), + encoder.GetPaletteSizeCost(3, 5, Av1PlaneType.Uv)); + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(yColorIndex[6][4], 7), + encoder.GetPaletteColorIndexCost(7, 8, 4, Av1PlaneType.Y)); + + Assert.Equal( + Av1ProbabilityCost.GetSymbolCost(uvColorIndex[3][2], 4), + encoder.GetPaletteColorIndexCost(4, 5, 2, Av1PlaneType.Uv)); + + Assert.Equal(Av1ProbabilityCost.GetLiteralCost(2), Av1SymbolEncoder.GetUniformCost(5, 2)); + Assert.Equal(Av1ProbabilityCost.GetLiteralCost(3), Av1SymbolEncoder.GetUniformCost(5, 3)); + } + [Theory] [MemberData(nameof(GetRangeData), 20)] public void RoundTripPartitionType(int context)