diff --git a/HEIF_IMPLEMENTATION_PLAN.md b/HEIF_IMPLEMENTATION_PLAN.md index 0e16688bb..5001fab66 100644 --- a/HEIF_IMPLEMENTATION_PLAN.md +++ b/HEIF_IMPLEMENTATION_PLAN.md @@ -559,7 +559,7 @@ For every item: Previously verified algorithm checkpoints remain valuable evidence, but the final decoder gate requires a fresh current-tree run after the inter and cleanup corrections. - [x] Bounded OBU framing, sequence headers, frame headers, tile groups, alignment, and trailing-bit parsing have been re-audited and verified against current libaom `main`. -- [~] Partition traversal, mode information, segmentation, delta quantization, transform-size selection, coefficient decoding, inverse quantization, and inverse transforms have historical checkpoint evidence against an obsolete pinned tree. Re-audit the current libaom `main` implementation before restoring verified status. +- [x] Partition traversal, mode information, segmentation, delta quantization, transform-size selection, coefficient decoding, inverse quantization, and inverse transforms have been re-audited and verified against current libaom `main`. - [x] Intra prediction covers directional, DC, smooth, Paeth, chroma-from-luma, filter-intra, and palette families with the established operator architecture. - [x] Intra-block copy has exact native reconstruction and feature-isolated SIMD evidence. - [x] Lossless inverse transform, loop filtering, CDEF, super-resolution, restoration, and film grain have focused checkpoint evidence. @@ -604,6 +604,51 @@ Verified bounded-OBU checkpoint evidence on 2026-08-31: `diff=lfs`, and `.gitattributes` was not edited. - [x] Release source builds pass for net10.0 and net11.0 with zero warnings and zero errors. Roslynk reports zero compiler errors, and scoped production and test analyzer verification reports no changes. +- [x] The completed checkpoint was committed as `243524c2c0b52a49d8d161fab806ab092cabe47c` with author + and committer `James Jackson-South `. + +Verified partition, mode, segmentation, quantization, and transform checkpoint evidence on 2026-08-31: + +- [x] Audited partition traversal and chroma representability against `read_partition` and the subsampled + plane-size rejection in current libaom `av1/decoder/decodeframe.c`; spatial segment-ID decoding and + corruption handling against `read_segment_id` in `av1/decoder/decodemv.c`; delta-Q syntax, resolution, + arithmetic, and clamping against `read_delta_qindex` and `read_delta_q_params` in the same file. +- [x] Audited selected and variable transform-size traversal against `read_tx_size`, `read_tx_size_vartx`, + and transform-block traversal in `av1/decoder/decodeframe.c`; coefficient syntax and arithmetic against + `av1_read_coeffs_txb` in `av1/decoder/decodetxb.c`; inverse quantization and transform application against + current `av1/decoder/decodeframe.c`, `av1/common/idct.c`, and the current libaom transform test oracle. + The observed clean `HEAD` and `origin/main` revision was + `441c439b9916474cac15d2822af47a9ad70674a8`; this is verification evidence, not a pin. +- [x] Partition decoding now rejects an invalid partition subsize and a block size that cannot represent the + current subsampled chroma plane. Spatial segmentation rejects decoded IDs above the active segment range. + Focused tests exercise both current-libaom corruption boundaries through the production tile reader. +- [x] Coefficient entropy decoding uses one allocator-owned maximum-size `Av1LevelBuffer` per tile reader. + Each transform resets and clears only its active padded geometry, so no transform creates an allocation. + Allocation tracking over all eight minimum- and maximum-quantizer frames proves exactly one coefficient + scratch allocation per frame and exactly-once return after decoder disposal. +- [x] Palette index maps are allocator-backed frame surfaces addressed row by row through `Buffer2DRegion`. + The wavefront context, stable neighbor ordering, right/bottom padding, transform offsets, and prediction + were audited against current libaom `av1/decoder/detokenize.c`, `av1/common/entropymode.c`, + `av1/decoder/decodeframe.c`, and `av1/common/reconintra.c`. A 1 KiB constrained allocator forces both + luma and chroma maps across multiple memory groups without copies or per-block allocations and proves + exactly-once disposal. +- [x] `Av1BlockModeInfo` is value storage, removing the managed object allocation formerly created for every + decoded coding block. Explicit `ModeInfoIndex` values preserve libaom's mode-info identity semantics at + prediction-unit loop-filter edges, and the frame map now uses integer offsets so more than 65,535 decoded + blocks cannot wrap its lookup identity. +- [x] Current official libaom reproduced the 39-frame all-intra reference and all four 8/10-bit minimum- and + maximum-quantizer references byte for byte. The production tests compare every native sample exactly, + cover every intra mode and seven selected transform types, execute SIMD and scalar paths through + `FeatureTestRunner`, and exercise the quantizer sequences under constrained tracked allocation. +- [x] Current official libaom decoded the 42-byte palette payload into the retained 1,089-byte YUV444 + reference at SHA-256 `E05F7C0DF06ECCF0E43869D1D7B03DAA1D635ACD26A766F8940899BE18D53251`. + The exact native test requires luma and chroma palette syntax. The established reference-output test uses + the unchanged presentation PNG at SHA-256 + `1148EBF6AA4B0F2D069D5E9B9605F6FB2A315E525F18016CDCAE23EFDD81DA84`, whose renamed path still + resolves to `diff=lfs`; `.gitattributes` was not edited. +- [x] The exact final AV1 namespace passes 8,732 of 8,732 cases on net10.0 and 8,732 of 8,732 cases on + net11.0, with zero failures or skips. Release source builds pass for net10.0 and net11.0 with zero warnings + and zero errors. Roslynk reports zero compiler errors, and scoped analyzer verification reports no changes. Decoder exit gate: diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolContextHelper.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolContextHelper.cs index 10a7df009..187059506 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolContextHelper.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolContextHelper.cs @@ -200,17 +200,19 @@ internal static class Av1SymbolContextHelper int aboveContext = 0; if (above is not null) { - aboveContext = above.ReferenceFrames[1] > Av1ReferenceFrameType.Intra - ? above.CompoundGroupIndex ? 1 : 0 - : above.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 3 : 0; + Av1BlockModeInfo aboveModeInfo = above.Value; + aboveContext = aboveModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra + ? aboveModeInfo.CompoundGroupIndex ? 1 : 0 + : aboveModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 3 : 0; } int leftContext = 0; if (left is not null) { - leftContext = left.ReferenceFrames[1] > Av1ReferenceFrameType.Intra - ? left.CompoundGroupIndex ? 1 : 0 - : left.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 3 : 0; + Av1BlockModeInfo leftModeInfo = left.Value; + leftContext = leftModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra + ? leftModeInfo.CompoundGroupIndex ? 1 : 0 + : leftModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 3 : 0; } return Math.Min(5, aboveContext + leftContext); @@ -244,17 +246,19 @@ internal static class Av1SymbolContextHelper int aboveContext = 0; if (above is not null) { - aboveContext = above.ReferenceFrames[1] > Av1ReferenceFrameType.Intra - ? above.CompoundIndex ? 1 : 0 - : above.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 1 : 0; + Av1BlockModeInfo aboveModeInfo = above.Value; + aboveContext = aboveModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra + ? aboveModeInfo.CompoundIndex ? 1 : 0 + : aboveModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 1 : 0; } int leftContext = 0; if (left is not null) { - leftContext = left.ReferenceFrames[1] > Av1ReferenceFrameType.Intra - ? left.CompoundIndex ? 1 : 0 - : left.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 1 : 0; + Av1BlockModeInfo leftModeInfo = left.Value; + leftContext = leftModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra + ? leftModeInfo.CompoundIndex ? 1 : 0 + : leftModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 1 : 0; } return aboveContext + leftContext + (forwardDistance == backwardDistance ? 3 : 0); @@ -703,8 +707,10 @@ internal static class Av1SymbolContextHelper { if (above is not null && left is not null) { - bool aboveIsIntra = above.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; - bool leftIsIntra = left.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; + Av1BlockModeInfo aboveModeInfo = above.Value; + Av1BlockModeInfo leftModeInfo = left.Value; + bool aboveIsIntra = aboveModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; + bool leftIsIntra = leftModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; // AV1 reserves context three for two intra neighbors, context one for a mixed pair, and context zero for // two inter neighbors. These values directly index intra_inter_cdf and are not probability ranks. @@ -720,12 +726,12 @@ internal static class Av1SymbolContextHelper // context zero, matching the unavailable-neighbor behavior in libaom's av1_get_intra_inter_context. if (above is not null) { - return above.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra ? 2 : 0; + return above.Value.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra ? 2 : 0; } if (left is not null) { - return left.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra ? 2 : 0; + return left.Value.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra ? 2 : 0; } return 0; @@ -743,29 +749,31 @@ internal static class Av1SymbolContextHelper // their forward/backward direction, while intra neighbors take the same branch as a non-forward reference. if (above is not null && left is not null) { - bool aboveIsCompound = above.ReferenceFrames[1] > Av1ReferenceFrameType.Intra; - bool leftIsCompound = left.ReferenceFrames[1] > Av1ReferenceFrameType.Intra; + Av1BlockModeInfo aboveModeInfo = above.Value; + Av1BlockModeInfo leftModeInfo = left.Value; + bool aboveIsCompound = aboveModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra; + bool leftIsCompound = leftModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra; if (!aboveIsCompound && !leftIsCompound) { - bool aboveIsBackward = above.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; - bool leftIsBackward = left.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; + bool aboveIsBackward = aboveModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; + bool leftIsBackward = leftModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; return aboveIsBackward == leftIsBackward ? 0 : 1; } if (!aboveIsCompound) { - bool aboveIsBackward = above.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; - bool aboveIsIntra = above.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; + bool aboveIsBackward = aboveModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; + bool aboveIsIntra = aboveModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; return 2 + (aboveIsBackward || aboveIsIntra ? 1 : 0); } if (!leftIsCompound) { - bool leftIsBackward = left.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; - bool leftIsIntra = left.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; + bool leftIsBackward = leftModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; + bool leftIsIntra = leftModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; return 2 + (leftIsBackward || leftIsIntra ? 1 : 0); } @@ -777,14 +785,15 @@ internal static class Av1SymbolContextHelper if (neighbor is not null) { - bool isCompound = neighbor.ReferenceFrames[1] > Av1ReferenceFrameType.Intra; + Av1BlockModeInfo neighborModeInfo = neighbor.Value; + bool isCompound = neighborModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra; if (isCompound) { return 3; } - return neighbor.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward ? 1 : 0; + return neighborModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward ? 1 : 0; } // With no spatial votes, AV1 uses the neutral single-versus-compound context rather than context zero. @@ -801,8 +810,10 @@ internal static class Av1SymbolContextHelper { if (above is not null && left is not null) { - bool aboveIntra = !IsInterBlock(above); - bool leftIntra = !IsInterBlock(left); + Av1BlockModeInfo aboveModeInfo = above.Value; + Av1BlockModeInfo leftModeInfo = left.Value; + bool aboveIntra = !IsInterBlock(aboveModeInfo); + bool leftIntra = !IsInterBlock(leftModeInfo); if (aboveIntra && leftIntra) { return 2; @@ -810,14 +821,14 @@ internal static class Av1SymbolContextHelper if (aboveIntra || leftIntra) { - Av1BlockModeInfo inter = aboveIntra ? left : above; + Av1BlockModeInfo inter = aboveIntra ? leftModeInfo : aboveModeInfo; return HasCompoundReference(inter) ? 1 + (2 * (HasUnidirectionalCompoundReferences(inter) ? 1 : 0)) : 2; } - bool aboveSingle = !HasCompoundReference(above); - bool leftSingle = !HasCompoundReference(left); - Av1ReferenceFrameType abovePrimary = above.ReferenceFrames[0]; - Av1ReferenceFrameType leftPrimary = left.ReferenceFrames[0]; + bool aboveSingle = !HasCompoundReference(aboveModeInfo); + bool leftSingle = !HasCompoundReference(leftModeInfo); + Av1ReferenceFrameType abovePrimary = aboveModeInfo.ReferenceFrames[0]; + Av1ReferenceFrameType leftPrimary = leftModeInfo.ReferenceFrames[0]; if (aboveSingle && leftSingle) { return 1 + (2 * (IsBackwardReference(abovePrimary) == IsBackwardReference(leftPrimary) ? 1 : 0)); @@ -825,7 +836,7 @@ internal static class Av1SymbolContextHelper if (aboveSingle || leftSingle) { - Av1BlockModeInfo compound = aboveSingle ? left : above; + Av1BlockModeInfo compound = aboveSingle ? leftModeInfo : aboveModeInfo; if (!HasUnidirectionalCompoundReferences(compound)) { return 1; @@ -834,8 +845,8 @@ internal static class Av1SymbolContextHelper return 3 + (IsBackwardReference(abovePrimary) == IsBackwardReference(leftPrimary) ? 1 : 0); } - bool aboveUnidirectional = HasUnidirectionalCompoundReferences(above); - bool leftUnidirectional = HasUnidirectionalCompoundReferences(left); + bool aboveUnidirectional = HasUnidirectionalCompoundReferences(aboveModeInfo); + bool leftUnidirectional = HasUnidirectionalCompoundReferences(leftModeInfo); if (!aboveUnidirectional && !leftUnidirectional) { return 0; @@ -850,12 +861,18 @@ internal static class Av1SymbolContextHelper } Av1BlockModeInfo? edge = above ?? left; - if (edge is null || !IsInterBlock(edge) || !HasCompoundReference(edge)) + if (edge is null) { return 2; } - return HasUnidirectionalCompoundReferences(edge) ? 4 : 0; + Av1BlockModeInfo edgeModeInfo = edge.Value; + if (!IsInterBlock(edgeModeInfo) || !HasCompoundReference(edgeModeInfo)) + { + return 2; + } + + return HasUnidirectionalCompoundReferences(edgeModeInfo) ? 4 : 0; } /// @@ -984,12 +1001,12 @@ internal static class Av1SymbolContextHelper if (above is not null) { - AddNeighborReferenceCounts(above, referenceCounts); + AddNeighborReferenceCounts(above.Value, referenceCounts); } if (left is not null) { - AddNeighborReferenceCounts(left, referenceCounts); + AddNeighborReferenceCounts(left.Value, referenceCounts); } } @@ -1144,8 +1161,8 @@ internal static class Av1SymbolContextHelper /// The context in the inclusive range zero through two. public static int GetSegmentIdPredictedContext(Av1BlockModeInfo? aboveModeInfo, Av1BlockModeInfo? leftModeInfo) { - int abovePredicted = aboveModeInfo is not null && aboveModeInfo.SegmentIdPredicted ? 1 : 0; - int leftPredicted = leftModeInfo is not null && leftModeInfo.SegmentIdPredicted ? 1 : 0; + int abovePredicted = aboveModeInfo is not null && aboveModeInfo.Value.SegmentIdPredicted ? 1 : 0; + int leftPredicted = leftModeInfo is not null && leftModeInfo.Value.SegmentIdPredicted ? 1 : 0; return abovePredicted + leftPredicted; } @@ -1306,7 +1323,8 @@ internal static class Av1SymbolContextHelper return SwitchableInterpolationFilterCount; } - ReadOnlySpan referenceFrames = modeInfo.ReferenceFrames; + Av1BlockModeInfo neighborModeInfo = modeInfo.Value; + ReadOnlySpan referenceFrames = neighborModeInfo.ReferenceFrames; // A compound neighbor contributes when either of its references matches the current primary reference. if (referenceFrames[0] != referenceFrame && referenceFrames[1] != referenceFrame) @@ -1314,6 +1332,6 @@ internal static class Av1SymbolContextHelper return SwitchableInterpolationFilterCount; } - return (int)modeInfo.InterpolationFilters[direction]; + return (int)neighborModeInfo.InterpolationFilters[direction]; } } diff --git a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolDecoder.cs index f35d43e1c..e2a872603 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolDecoder.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolDecoder.cs @@ -20,11 +20,6 @@ internal ref struct Av1SymbolDecoder /// private readonly Av1FrameEntropyContext context; - /// - /// The configuration providing temporary coefficient-context memory. - /// - private readonly Configuration configuration; - /// /// The range decoder over the current tile payload. /// @@ -59,7 +54,6 @@ internal ref struct Av1SymbolDecoder // The context owner controls reset and publication. Holding one reference here keeps the range decoder small // and prevents a second set of aliases from becoming a competing source of entropy state. this.context = context; - this.configuration = configuration; this.reader = new Av1SymbolReader(tileData, updateCdf); } @@ -345,15 +339,15 @@ internal ref struct Av1SymbolDecoder { ref Av1SymbolReader r = ref this.reader; Av1PredictionMode aboveMode = Av1PredictionMode.DC; - if (aboveModeInfo != null) + if (aboveModeInfo is not null) { - aboveMode = aboveModeInfo.YMode; + aboveMode = aboveModeInfo.Value.YMode; } Av1PredictionMode leftMode = Av1PredictionMode.DC; - if (leftModeInfo != null) + if (leftModeInfo is not null) { - leftMode = leftModeInfo.YMode; + leftMode = leftModeInfo.Value.YMode; } int aboveContext = IntraModeContext[(int)aboveMode]; @@ -971,6 +965,7 @@ internal ref struct Av1SymbolDecoder /// The transform descriptor updated with the decoded type and coded-block flag. /// The signed distance from the mode block to the right frame edge. /// The signed distance from the mode block to the bottom frame edge. + /// Reusable padded coefficient-context storage owned by the tile reader. /// The destination receiving the coefficient count followed by scan-ordered signed levels. /// The one-based end-of-block position, or zero for an empty transform block. public int ReadCoefficients( @@ -991,6 +986,7 @@ internal ref struct Av1SymbolDecoder ref Av1TransformInfo transformInfo, int modeBlocksToRightEdge, int modeBlocksToBottomEdge, + Av1LevelBuffer levels, Span coefficientBuffer) { Av1TransformSize adjustedTransformSize = transformSize.GetAdjusted(); @@ -1000,8 +996,9 @@ internal ref struct Av1SymbolDecoder Av1PlaneType planeType = (Av1PlaneType)Math.Min(plane, 1); int culLevel = 0; - // AV1 omits high-frequency coefficients beyond 32 samples on every 64-point transform dimension. - using Av1LevelBuffer levels = new(this.configuration, new Size(width, height)); + // AV1 omits high-frequency coefficients beyond 32 samples on every 64-point transform dimension. Reusing + // tile-owned storage avoids an allocator round trip for every transform block. + levels.Reset(new Size(width, height)); bool allZero = this.ReadTransformBlockSkip(transformSizeContext, transformBlockContext.SkipContext); int endOfBlock; diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1FrameDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1FrameDecoder.cs index 6a09e20f0..47aa19c57 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1FrameDecoder.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1FrameDecoder.cs @@ -160,7 +160,7 @@ internal sealed class Av1FrameDecoder : IAv1FrameDecoder, IDisposable /// Reconstructs every tile row in one tile column. /// /// The zero-based tile-column index. - /// SVT-AV1: decode_tile. + /// Follows libaom's single-threaded decode_tiles ordering. private void DecodeFrameTiles(int tileColumn) { ObuTileGroupHeader tileInfo = this.frameHeader.TilesInfo; @@ -187,7 +187,7 @@ internal sealed class Av1FrameDecoder : IAv1FrameDecoder, IDisposable /// The zero-based tile-column index. /// The frame-relative row in 4x4 mode-info units. /// The frame-relative superblock row. - /// SVT-AV1: decode_tile_row. + /// Corresponds to the superblock-row traversal in libaom's decode_tile. private void DecodeTileSuperblockRow(int tileRow, int tileColumn, int modeInfoRow, int superblockRow) { ObuTileGroupHeader tileInfo = this.frameHeader.TilesInfo; @@ -210,7 +210,7 @@ internal sealed class Av1FrameDecoder : IAv1FrameDecoder, IDisposable /// The superblock's top-left position in 4x4 mode-info units. /// The decoded syntax and block modes for the superblock. /// The tile that contains the superblock. - /// SVT-AV1: svt_aom_decode_super_block. + /// Corresponds to libaom's superblock decode boundary. public void DecodeSuperblock(Point modeInfoPosition, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo) { this.blockDecoder.UpdateSuperblock(superblockInfo); @@ -224,7 +224,7 @@ internal sealed class Av1FrameDecoder : IAv1FrameDecoder, IDisposable /// The superblock's frame-relative origin in 4x4 mode-info units. /// The superblock whose block modes are traversed. /// The tile boundary information used by intra prediction. - /// SVT-AV1: decode_partition. + /// Replays the depth-first block order produced by libaom's decode_partition. private void DecodePartition(Point modeInfoPosition, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo) { foreach (Av1BlockModeInfo modeInfo in superblockInfo.GetModeInfos()) diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs index cd7b68b39..0fec310ce 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs @@ -237,9 +237,9 @@ internal class Av1LoopFilterDecoder bool currentSkippedTransform = modeInfo.Skip && modeInfo.ReferenceFrames[0] > Av1ReferenceFrameType.Intra; bool previousSkippedTransform = previousModeInfo.Skip && previousModeInfo.ReferenceFrames[0] > Av1ReferenceFrameType.Intra; - // The mode-info map stores one object for every covered position, so object identity is the exact equivalent - // of libaom's current-versus-previous MB_MODE_INFO pointer comparison at a prediction-unit boundary. - bool isBlockEdge = !ReferenceEquals(modeInfo, previousModeInfo); + // Every covered 4x4 position carries the owning block's storage index. Comparing those indices is the value-type + // equivalent of libaom's current-versus-previous MB_MODE_INFO pointer comparison at a prediction-unit boundary. + bool isBlockEdge = modeInfo.ModeInfoIndex != previousModeInfo.ModeInfoIndex; bool applyFilter = isTransformEdge && (isBlockEdge || !currentSkippedTransform || !previousSkippedTransform); if (!applyFilter) { diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1DeQuantizationContext.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1DeQuantizationContext.cs index 408e66ef9..0f795b3b2 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1DeQuantizationContext.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1DeQuantizationContext.cs @@ -26,7 +26,7 @@ internal class Av1DeQuantizationContext /// /// The sequence header that supplies the coded bit depth. /// The frame header that supplies segmentation and quantization parameters. - /// SVT-AV1: svt_aom_setup_segmentation_dequant. + /// Corresponds to setup_segmentation_dequant in libaom. public Av1DeQuantizationContext(ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) { Av1BitDepth bitDepth = sequenceHeader.ColorConfig.BitDepth; @@ -40,11 +40,11 @@ internal class Av1DeQuantizationContext for (int plane = 0; plane < Av1Constants.MaxPlanes; plane++) { - int dc_delta_q = frameHeader.QuantizationParameters.DeltaQDc[plane]; - int ac_delta_q = frameHeader.QuantizationParameters.DeltaQAc[plane]; + int dcDeltaQ = frameHeader.QuantizationParameters.DeltaQDc[plane]; + int acDeltaQ = frameHeader.QuantizationParameters.DeltaQAc[plane]; - this.dcContent[segmentId][plane] = Av1QuantizationLookup.GetDcQuant(qindex, dc_delta_q, bitDepth); - this.acContent[segmentId][plane] = Av1QuantizationLookup.GetAcQuant(qindex, ac_delta_q, bitDepth); + this.dcContent[segmentId][plane] = Av1QuantizationLookup.GetDcQuant(qindex, dcDeltaQ, bitDepth); + this.acContent[segmentId][plane] = Av1QuantizationLookup.GetAcQuant(qindex, acDeltaQ, bitDepth); } } } diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1InverseQuantizer.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1InverseQuantizer.cs index 233117e07..97014e363 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1InverseQuantizer.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1InverseQuantizer.cs @@ -80,7 +80,7 @@ internal class Av1InverseQuantizer /// The transform dimensions and scale. /// The color plane whose quantizer and matrix are used. /// The number of coefficient levels consumed. - /// SVT-AV1: svt_aom_inverse_quantize. + /// Matches the coefficient dequantization arithmetic in libaom's read_coeffs_txb. public int InverseQuantize(Av1BlockModeInfo mode, Span level, Span qCoefficients, Av1TransformType transformType, Av1TransformSize transformSize, Av1Plane plane) { Guard.NotNull(this.deQuantsDeltaQ); @@ -159,7 +159,7 @@ internal class Av1InverseQuantizer /// The raster coefficient index into the inverse matrix. /// The inverse quantization matrix for the current level, plane, and transform size. /// The matrix-weighted dequantization value. - /// SVT-AV1: get_dqv. + /// Corresponds to get_dqv in libaom. private static int GetDeQuantizedValue(short dequant, int coefficientIndex, ReadOnlySpan iqMatrix) { // Matrix elements use fixed-point precision; adding half a unit produces nearest-integer rounding on shift. diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PalettePredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PalettePredictor.Operator.cs index ca500cbf3..f1cd3fd20 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PalettePredictor.Operator.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PalettePredictor.Operator.cs @@ -4,6 +4,7 @@ using System.Runtime.CompilerServices; using System.Runtime.InteropServices; using System.Runtime.Intrinsics; +using SixLabors.ImageSharp.Memory; namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; @@ -97,26 +98,24 @@ internal static class Av1PalettePredictor /// public static void Predict( ReadOnlySpan paletteColors, - ReadOnlySpan colorIndexMap, - int colorIndexMapStride, + Buffer2DRegion colorIndexMap, Span destination, int destinationStride, int width, int height) - => Predictor.Predict(paletteColors, colorIndexMap, colorIndexMapStride, destination, destinationStride, width, height); + => Predictor.Predict(paletteColors, colorIndexMap, destination, destinationStride, width, height); /// /// Reconstructs a high-bit-depth palette-predicted block. /// public static void Predict( ReadOnlySpan paletteColors, - ReadOnlySpan colorIndexMap, - int colorIndexMapStride, + Buffer2DRegion colorIndexMap, Span destination, int destinationStride, int width, int height) - => Predictor.Predict(paletteColors, colorIndexMap, colorIndexMapStride, destination, destinationStride, width, height); + => Predictor.Predict(paletteColors, colorIndexMap, destination, destinationStride, width, height); /// /// Maps decoded palette indices to reconstructed samples. @@ -186,14 +185,12 @@ internal static class Av1PalettePredictor /// public static void Predict( ReadOnlySpan paletteColors, - ReadOnlySpan colorIndexMap, - int colorIndexMapStride, + Buffer2DRegion colorIndexMap, Span destination, int destinationStride, int width, int height) { - ref byte mapBase = ref MemoryMarshal.GetReference(colorIndexMap); ref byte destinationBase = ref MemoryMarshal.GetReference(destination); // AV1 palettes contain at most eight colors. Repeating all eight entries in every 128-bit lane keeps native @@ -209,7 +206,7 @@ internal static class Av1PalettePredictor for (int row = 0; row < height; row++) { - ref byte mapRow = ref Unsafe.Add(ref mapBase, row * colorIndexMapStride); + ref byte mapRow = ref MemoryMarshal.GetReference(colorIndexMap.DangerousGetRowSpan(row)); ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); int column = 0; @@ -278,14 +275,12 @@ internal static class Av1PalettePredictor /// public static void Predict( ReadOnlySpan paletteColors, - ReadOnlySpan colorIndexMap, - int colorIndexMapStride, + Buffer2DRegion colorIndexMap, Span destination, int destinationStride, int width, int height) { - ref byte mapBase = ref MemoryMarshal.GetReference(colorIndexMap); ref short destinationBase = ref MemoryMarshal.GetReference(destination); InlineArray8 paletteStorage = default; paletteColors.CopyTo(paletteStorage); @@ -295,7 +290,7 @@ internal static class Av1PalettePredictor for (int row = 0; row < height; row++) { - ref byte mapRow = ref Unsafe.Add(ref mapBase, row * colorIndexMapStride); + ref byte mapRow = ref MemoryMarshal.GetReference(colorIndexMap.DangerousGetRowSpan(row)); ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); int column = 0; diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs index de7872cb6..775ebcff0 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs @@ -10,6 +10,7 @@ using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; @@ -462,22 +463,24 @@ internal class Av1PredictionDecoder if (usePalette) { ReadOnlySpan paletteColors = modeInfo.GetPaletteColors(plane); - ReadOnlySpan colorIndexMap = modeInfo.GetPaletteColorIndexMap(plane); - int paletteStride = partitionInfo.GetWidthInPixels(plane); - int mapOffset = ((blockModeInfoRowOffset << Av1Constants.ModeInfoSizeLog2) * paletteStride) + - (blockModeInfoColumnOffset << Av1Constants.ModeInfoSizeLog2); - - // Every transform reconstructs its own window of the block-level palette map. Keeping the map padded to - // the coded block dimensions lets edge transforms use the same addressing rule as interior transforms. + Buffer2DRegion colorIndexMap = modeInfo.GetPaletteColorIndexMap(plane); + Buffer2DRegion transformColorIndexMap = colorIndexMap.GetSubRegion( + blockModeInfoColumnOffset << Av1Constants.ModeInfoSizeLog2, + blockModeInfoRowOffset << Av1Constants.ModeInfoSizeLog2, + transformWidth, + transformHeight); + + // Every transform reconstructs its own window of the block-level palette map. The row-oriented region + // keeps this traversal valid when the frame-owned map spans multiple allocator memory groups. if (typeof(T) == typeof(byte)) { Span byteDestination = MemoryMarshal.Cast(pixelBuffer); - Av1PalettePredictor.Predict(paletteColors, colorIndexMap[mapOffset..], paletteStride, byteDestination, pixelBufferStride, transformWidth, transformHeight); + Av1PalettePredictor.Predict(paletteColors, transformColorIndexMap, byteDestination, pixelBufferStride, transformWidth, transformHeight); } else { Span highBitDepthDestination = MemoryMarshal.Cast(pixelBuffer); - Av1PalettePredictor.Predict(paletteColors, colorIndexMap[mapOffset..], paletteStride, highBitDepthDestination, pixelBufferStride, transformWidth, transformHeight); + Av1PalettePredictor.Predict(paletteColors, transformColorIndexMap, highBitDepthDestination, pixelBufferStride, transformWidth, transformHeight); } return; @@ -1983,8 +1986,8 @@ internal class Av1PredictionDecoder left = partitionInfo.LeftModeInfoForChroma; } - bool aboveIsSmooth = (above != null) && IsSmooth(above, plane); - bool leftIsSmooth = (left != null) && IsSmooth(left, plane); + bool aboveIsSmooth = above is not null && IsSmooth(above.Value, plane); + bool leftIsSmooth = left is not null && IsSmooth(left.Value, plane); return aboveIsSmooth || leftIsSmooth; } diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1BlockModeInfo.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1BlockModeInfo.cs index d7328702c..cdd03d8bc 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1BlockModeInfo.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1BlockModeInfo.cs @@ -1,17 +1,19 @@ // Copyright (c) Six Labors. // Licensed under the Six Labors Split License. +using System.Diagnostics.CodeAnalysis; using System.Runtime.CompilerServices; using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Memory; namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; /// /// Stores block-size, intra/inter prediction, transform, and palette decisions shared by AV1 block processing. /// -internal class Av1BlockModeInfo +internal struct Av1BlockModeInfo { /// /// Stores the primary and optional secondary reference-frame labels. @@ -56,12 +58,12 @@ internal class Av1BlockModeInfo /// /// Stores the luma palette color-index map. /// - private byte[] lumaPaletteColorIndexMap = []; + private Buffer2DRegion lumaPaletteColorIndexMap; /// /// Stores the shared chroma palette color-index map. /// - private byte[] chromaPaletteColorIndexMap = []; + private Buffer2DRegion chromaPaletteColorIndexMap; /// /// The directional prediction angle adjustment for luma. @@ -94,7 +96,7 @@ internal class Av1BlockModeInfo private int chromaTransformUnitCount; /// - /// Initializes a new instance of the class. + /// Initializes a new instance of the structure. /// /// The decoded block size. /// The block origin relative to its superblock in 4x4 mode-information units. @@ -114,6 +116,11 @@ internal class Av1BlockModeInfo /// public Av1BlockSize BlockSize { get; } + /// + /// Gets or sets the frame storage index shared by every mode-information position covered by this block. + /// + public int ModeInfoIndex { get; set; } + /// /// Gets or sets the for the luminance channel. /// @@ -127,11 +134,13 @@ internal class Av1BlockModeInfo /// block, for an inter-intra block, or the secondary inter-reference label /// for compound prediction. /// + [UnscopedRef] public Span ReferenceFrames => this.referenceFrames; /// /// Gets the decoded motion vectors corresponding to . /// + [UnscopedRef] public Span MotionVectors => this.motionVectors; /// @@ -141,6 +150,7 @@ internal class Av1BlockModeInfo /// Index zero is the vertical filter and index one is the horizontal filter, matching libaom's /// InterpFilters.y_filter and InterpFilters.x_filter layout. /// + [UnscopedRef] public Span InterpolationFilters => this.interpolationFilters; /// @@ -392,6 +402,7 @@ internal class Av1BlockModeInfo /// /// The color plane. /// The palette colors in prediction-index order. + [UnscopedRef] public ReadOnlySpan GetPaletteColors(Av1Plane plane) { if (plane == Av1Plane.Y) @@ -430,7 +441,7 @@ internal class Av1BlockModeInfo /// /// The color plane. /// The luma map for or the shared chroma map for either chroma plane. - public ReadOnlySpan GetPaletteColorIndexMap(Av1Plane plane) + public Buffer2DRegion GetPaletteColorIndexMap(Av1Plane plane) => plane == Av1Plane.Y ? this.lumaPaletteColorIndexMap : this.chromaPaletteColorIndexMap; /// @@ -438,7 +449,7 @@ internal class Av1BlockModeInfo /// /// The luma or shared chroma plane class. /// The row-major color-index map including coded-block edge padding. - public void SetPaletteColorIndexMap(Av1PlaneType planeType, byte[] colorIndexMap) + public void SetPaletteColorIndexMap(Av1PlaneType planeType, Buffer2DRegion colorIndexMap) { if (planeType == Av1PlaneType.Y) { diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.MotionField.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.MotionField.cs index ae7757dd7..457f3434b 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.MotionField.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.MotionField.cs @@ -486,19 +486,23 @@ internal partial class Av1FrameInfo } /// - /// Releases one owner and returns motion-field storage after the final owner is released. + /// Releases one owner and returns allocator-backed frame storage after the final owner is released. /// public void ReleaseOwner() { this.ownerCount--; if (this.ownerCount == 0) { - // Retained and temporal fields can each be frame-sized. Return both together only after the tile reader, - // every reference or presentation frame, and the decoder's inspectable result have released ownership. + // Frame-sized motion and palette storage remains addressable through retained mode information. Return + // all of it together only after tile, reference, presentation, and decoder-result owners are gone. this.retainedMotionField?.Dispose(); this.retainedMotionField = null; this.temporalMotionField?.Dispose(); this.temporalMotionField = null; + this.lumaPaletteColorIndexMap?.Dispose(); + this.lumaPaletteColorIndexMap = null; + this.chromaPaletteColorIndexMap?.Dispose(); + this.chromaPaletteColorIndexMap = null; } } diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.cs index 2484d1b15..46ff97db0 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.cs @@ -2,6 +2,7 @@ // Licensed under the Six Labors Split License. using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Memory; namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; @@ -55,6 +56,36 @@ internal partial class Av1FrameInfo : IDisposable /// private readonly int subsamplingFactor; + /// + /// The aligned luma palette-map width in samples. + /// + private readonly int lumaPaletteColorIndexMapWidth; + + /// + /// The aligned luma palette-map height in samples. + /// + private readonly int lumaPaletteColorIndexMapHeight; + + /// + /// The aligned chroma palette-map width in samples. + /// + private readonly int chromaPaletteColorIndexMapWidth; + + /// + /// The aligned chroma palette-map height in samples. + /// + private readonly int chromaPaletteColorIndexMapHeight; + + /// + /// Owns row-addressable luma palette indices for the frame. + /// + private Buffer2D? lumaPaletteColorIndexMap; + + /// + /// Owns row-addressable chroma palette indices for the frame. + /// + private Buffer2D? chromaPaletteColorIndexMap; + /// /// Stores one addressing view for each frame superblock. /// @@ -179,6 +210,10 @@ internal partial class Av1FrameInfo : IDisposable // Chroma capacity scales by two for each sampled axis: 4:4:4 => 0, 4:2:2 => 1, 4:2:0 => 2. this.subsamplingFactor = (subX && subY) ? 2 : (subX && !subY) ? 1 : (!subX && !subY) ? 0 : -1; Guard.IsFalse(this.subsamplingFactor == -1, nameof(this.subsamplingFactor), "Invalid combination of subsampling."); + this.lumaPaletteColorIndexMapWidth = superblockAlignedWidth; + this.lumaPaletteColorIndexMapHeight = superblockAlignedHeight; + this.chromaPaletteColorIndexMapWidth = superblockAlignedWidth >> (subX ? 1 : 0); + this.chromaPaletteColorIndexMapHeight = superblockAlignedHeight >> (subY ? 1 : 0); int lumaCoefficientCountPerSuperblock = this.modeInfoCountPerSuperblock * CoefficientCountPerModeInfo; int chromaCoefficientCountPerSuperblock = lumaCoefficientCountPerSuperblock >> this.subsamplingFactor; this.coefficientsY = new int[superblockCount * lumaCoefficientCountPerSuperblock]; @@ -203,6 +238,51 @@ internal partial class Av1FrameInfo : IDisposable /// public int SuperblockModeInfoSize => this.modeInfoSizePerSuperblock; + /// + /// Gets frame-owned row-addressable palette-map storage for one coding block. + /// + /// The decoder configuration providing frame storage. + /// The luma or shared chroma plane class. + /// The block bounds in plane samples. + /// The palette-map region assigned to the coding block. + public Buffer2DRegion GetPaletteColorIndexMap( + Configuration configuration, + Av1PlaneType planeType, + Rectangle bounds) + { + Buffer2D? buffer; + if (planeType == Av1PlaneType.Y) + { + buffer = this.lumaPaletteColorIndexMap; + if (buffer is null) + { + // A 2D allocation may contain multiple memory groups, but row alignment guarantees that every + // palette row remains contiguous for entropy decoding and SIMD reconstruction. + buffer = configuration.MemoryAllocator.Allocate2D( + this.lumaPaletteColorIndexMapWidth, + this.lumaPaletteColorIndexMapHeight); + + this.lumaPaletteColorIndexMap = buffer; + } + } + else + { + buffer = this.chromaPaletteColorIndexMap; + if (buffer is null) + { + buffer = configuration.MemoryAllocator.Allocate2D( + this.chromaPaletteColorIndexMapWidth, + this.chromaPaletteColorIndexMapHeight); + + this.chromaPaletteColorIndexMap = buffer; + } + } + + // Partition traversal assigns non-overlapping frame regions, so retaining a view records the complete + // identify-time syntax without copying block maps or allocating storage for each coding block. + return new Buffer2DRegion(buffer, bounds); + } + /// /// Initializes the active frame's contiguous segment map and applies whole-map inheritance when requested. /// @@ -559,6 +639,7 @@ internal partial class Av1FrameInfo : IDisposable public void UpdateModeInfo(Av1BlockModeInfo modeInfo, Av1SuperblockInfo superblockInfo) { Point modeInfoPosition = this.GetModeInfoPosition(superblockInfo.Position, modeInfo.PositionInSuperblock); + modeInfo.ModeInfoIndex = this.modeInfoMap.NextIndex; this.modeInfos[this.modeInfoMap.NextIndex] = modeInfo; this.UpdateRetainedMotionField(modeInfo, modeInfoPosition); this.modeInfoMap.Update(modeInfoPosition, modeInfo.BlockSize); diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameModeInfoMap.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameModeInfoMap.cs index 54d338316..5ac36b199 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameModeInfoMap.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameModeInfoMap.cs @@ -9,14 +9,14 @@ namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; internal partial class Av1FrameInfo { /// - /// Mapping of instances, from position to index into the . + /// Mapping of values, from position to index into the . /// public class Av1FrameModeInfoMap { /// /// Stores the mode-information index assigned to each aligned 4x4 frame location. /// - private readonly ushort[] offsets; + private readonly int[] offsets; /// /// The dimensions of in 4x4 mode-information units. @@ -31,7 +31,7 @@ internal partial class Av1FrameInfo { this.alignedModeInfoCount = modeInfoCount; this.NextIndex = 0; - this.offsets = new ushort[this.alignedModeInfoCount.Width * this.alignedModeInfoCount.Height]; + this.offsets = new int[this.alignedModeInfoCount.Width * this.alignedModeInfoCount.Height]; } /// @@ -70,7 +70,7 @@ internal partial class Av1FrameInfo // because later blocks query their above and left neighbors at cell granularity. for (int i = modeInfoLocation.Y; i < modeInfoLocation.Y + bh4; i++) { - Array.Fill(this.offsets, (ushort)this.NextIndex, (i * this.alignedModeInfoCount.Width) + modeInfoLocation.X, bw4); + Array.Fill(this.offsets, this.NextIndex, (i * this.alignedModeInfoCount.Width) + modeInfoLocation.X, bw4); } this.NextIndex++; diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LevelBuffer.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LevelBuffer.cs index 66e891e6c..cec5f849c 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LevelBuffer.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LevelBuffer.cs @@ -19,11 +19,14 @@ internal sealed class Av1LevelBuffer : IDisposable private IMemoryOwner? memory; /// - /// Initializes a new instance of the class for the maximum AV1 transform size. + /// Initializes a new instance of the class for the maximum entropy-coded + /// coefficient dimensions. /// /// The configuration providing the memory allocator. public Av1LevelBuffer(Configuration configuration) - : this(configuration, new Size(Av1Constants.MaxTransformSize, Av1Constants.MaxTransformSize)) + : this( + configuration, + new Size(Av1Constants.MaxTransformSize / 2, Av1Constants.MaxTransformSize / 2)) { } @@ -46,12 +49,12 @@ internal sealed class Av1LevelBuffer : IDisposable /// /// Gets the unpadded coefficient dimensions. /// - public Size Size { get; } + public Size Size { get; private set; } /// /// Gets the padded row stride in bytes. /// - public int Stride { get; } + public int Stride { get; private set; } /// /// Gets the coefficient level at the specified unpadded position. @@ -121,6 +124,22 @@ internal sealed class Av1LevelBuffer : IDisposable this.memory = null; } + /// + /// Selects new active coefficient dimensions and clears their padded context storage. + /// + /// The unpadded coefficient dimensions. + public void Reset(Size size) + { + ObjectDisposedException.ThrowIf(this.memory == null, this); + this.Size = size; + this.Stride = Av1Constants.TransformPadHorizontal + size.Width; + + // Tile parsing is sequential, so one maximum-sized rent can serve every transform. Clear only the active + // layout because stale neighboring levels would otherwise select the wrong coefficient distributions. + int totalHeight = Av1Constants.TransformPadTop + size.Height + Av1Constants.TransformPadBottom; + this.memory.Memory.Span[..(this.Stride * totalHeight)].Clear(); + } + /// /// Clears all coefficient levels and context padding. /// diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PartitionInfo.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PartitionInfo.cs index b71d6aa2f..12b1552c6 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PartitionInfo.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PartitionInfo.cs @@ -1,6 +1,7 @@ // Copyright (c) Six Labors. // Licensed under the Six Labors Split License. +using System.Diagnostics.CodeAnalysis; using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; @@ -12,6 +13,11 @@ namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; /// internal ref struct Av1PartitionInfo { + /// + /// The decoded mode information populated for this partition. + /// + private Av1BlockModeInfo modeInfo; + /// /// The luma block width in samples. /// @@ -41,7 +47,7 @@ internal ref struct Av1PartitionInfo /// The partition type that produced the block. public Av1PartitionInfo(Av1BlockModeInfo modeInfo, Av1SuperblockInfo superblockInfo, bool isChroma, Av1PartitionType partitionType) { - this.ModeInfo = modeInfo; + this.modeInfo = modeInfo; this.SuperblockInfo = superblockInfo; this.IsChroma = isChroma; this.Type = partitionType; @@ -50,7 +56,8 @@ internal ref struct Av1PartitionInfo /// /// Gets the decoded block mode information. /// - public Av1BlockModeInfo ModeInfo { get; } + [UnscopedRef] + public ref Av1BlockModeInfo ModeInfo => ref this.modeInfo; /// /// Gets the this partition resides inside. @@ -120,6 +127,7 @@ internal ref struct Av1PartitionInfo /// /// Gets the reference-frame types selected for the block. /// + [UnscopedRef] public Span ReferenceFrames => this.ModeInfo.ReferenceFrames; /// diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs index ae78c7936..af43ed72b 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs @@ -11,6 +11,7 @@ using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; @@ -95,6 +96,11 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// private readonly int[] coefficientIndex = []; + /// + /// Reusable padded coefficient-context storage for the sequential transform traversal. + /// + private readonly Av1LevelBuffer coefficientLevels; + /// /// Reusable storage for the eight spatial displacement-vector candidates permitted by AV1. /// @@ -225,6 +231,20 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable throw; } + try + { + this.coefficientLevels = new Av1LevelBuffer(configuration); + } + catch + { + // The coefficient scratch allocation follows both neighbor contexts. Unwind those successful rents when + // construction cannot publish an owning tile reader. + this.aboveNeighborContext.Dispose(); + this.leftNeighborContext.Dispose(); + this.FrameInfo.Dispose(); + throw; + } + if (referenceFrames is not null) { try @@ -237,6 +257,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable { this.aboveNeighborContext.Dispose(); this.leftNeighborContext.Dispose(); + this.coefficientLevels.Dispose(); this.FrameInfo.Dispose(); throw; } @@ -328,6 +349,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable { this.aboveNeighborContext.Dispose(); this.leftNeighborContext.Dispose(); + this.coefficientLevels.Dispose(); this.FrameInfo.Dispose(); } @@ -336,7 +358,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// /// The entropy-coded tile payload. /// The zero-based tile index in row-major order. - /// Corresponds to parse_tile in SVT-AV1. + /// Corresponds to decode_tile in libaom. public void ReadTile(Span tileData, int tileNum) { // AV1 tiles never inherit adaptation from another tile in the same frame. Reusing one graph is safe because @@ -661,6 +683,19 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable } Av1BlockSize subSize = partitionType.GetBlockSubSize(blockSize); + if (subSize == Av1BlockSize.Invalid) + { + throw new InvalidImageContentException($"The decoded AV1 partition type {partitionType} is invalid for block size {blockSize}."); + } + + ObuColorConfig colorConfig = this.SequenceHeader.ColorConfig; + if (subSize.GetSubsampled(colorConfig.SubSamplingX, colorConfig.SubSamplingY) == Av1BlockSize.Invalid) + { + // Luma partition syntax can describe a sub-8x8 shape that has no legal representation after chroma + // subsampling. Reject it before any block state is published, matching libaom's decode_partition boundary. + throw new InvalidImageContentException($"The decoded AV1 block size {subSize} is invalid for the sequence chroma subsampling."); + } + Av1BlockSize splitSize = Av1PartitionType.Split.GetBlockSubSize(blockSize); // Partition syntax is depth-first. The visit order here is also the order in which mode, @@ -815,7 +850,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable this.Residual(ref reader, ref partitionInfo, superblockInfo, tileInfo, blockSize); // Store the record only after all syntax has populated it, then map every covered 4x4 position. - this.FrameInfo.UpdateModeInfo(blockModeInfo, superblockInfo); + this.FrameInfo.UpdateModeInfo(partitionInfo.ModeInfo, superblockInfo); } /// @@ -823,7 +858,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// /// The skipped block and its frame position. /// The active tile boundaries. - /// Corresponds to reset_skip_context in SVT-AV1. + /// Implements AV1 section 5.11.37. private void ResetSkipContext(ref Av1PartitionInfo partitionInfo, Av1TileInfo tileInfo) { // Subsampled 4x4 luma blocks can share chroma ownership with an adjacent luma block. A skipped block that is @@ -852,7 +887,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// The containing superblock and coefficient storage. /// The active tile boundaries. /// The coding block size. - /// Implements AV1 section 5.11.34 and corresponds to parse_residual in SVT-AV1. + /// Implements AV1 section 5.11.34. private void Residual( ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo, @@ -1039,7 +1074,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// A value indicating whether the target plane is vertically subsampled. /// The decoded end-of-block coefficient position, or zero for an all-zero transform. /// - /// Implements AV1 section 5.11.35 using the traversal shape of the corresponding SVT-AV1 implementation. + /// Implements AV1 section 5.11.35. /// private int ParseTransformBlock( ref Av1SymbolDecoder reader, @@ -1120,7 +1155,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// The destination beginning at this transform's coefficient slot. /// The decoded end-of-block coefficient position, or zero for an all-zero transform. /// - /// Implements AV1 section 5.11.39 using the traversal shape of the corresponding SVT-AV1 implementation. + /// Implements AV1 section 5.11.39. /// private int ParseCoefficients( ref Av1SymbolDecoder reader, @@ -1169,6 +1204,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable ref transformInfo, partitionInfo.ModeBlockToRightEdge, partitionInfo.ModeBlockToBottomEdge, + this.coefficientLevels, coefficientBuffer); } @@ -1317,7 +1353,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable Av1TileInfo tileInfo, bool allowSelect) { - Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; if (this.FrameHeader.LosslessArray[modeInfo.SegmentId]) { return Av1TransformSize.Size4x4; @@ -1406,7 +1442,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// The current coding block. /// The containing superblock. /// The active tile boundaries. - /// Implements AV1 section 5.11.16 and corresponds to read_block_tx_size in SVT-AV1. + /// Implements AV1 section 5.11.16 and corresponds to read_tx_size in libaom. private void ReadBlockTransformSize( ref Av1SymbolDecoder reader, Point modeInfoLocation, @@ -1418,7 +1454,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable int block4x4Width = blockSize.Get4x4WideCount(); int block4x4Height = blockSize.Get4x4HighCount(); - Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; bool usesInterTransformSyntax = modeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Last || modeInfo.UseIntraBlockCopy; this.transformUnitCount[(int)Av1Plane.Y].AsSpan(0, 4).Clear(); @@ -1738,7 +1774,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// Implements AV1 section 5.11.49. private void ReadPaletteTokens(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) { - Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; if (modeInfo.GetPaletteSize(Av1PlaneType.Y) != 0) { GetPaletteMapDimensions( @@ -1750,14 +1786,24 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable out int rows, out int columns); - byte[] colorIndexMap = DecodePaletteColorMap( + Buffer2DRegion colorIndexMap = this.FrameInfo.GetPaletteColorIndexMap( + this.configuration, + Av1PlaneType.Y, + new Rectangle( + partitionInfo.ColumnIndex << Av1Constants.ModeInfoSizeLog2, + partitionInfo.RowIndex << Av1Constants.ModeInfoSizeLog2, + planeWidth, + planeHeight)); + + DecodePaletteColorMap( ref reader, modeInfo.GetPaletteSize(Av1PlaneType.Y), Av1PlaneType.Y, planeWidth, planeHeight, rows, - columns); + columns, + colorIndexMap); modeInfo.SetPaletteColorIndexMap(Av1PlaneType.Y, colorIndexMap); } @@ -1773,14 +1819,26 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable out int rows, out int columns); - byte[] colorIndexMap = DecodePaletteColorMap( + int subX = this.SequenceHeader.ColorConfig.SubSamplingX ? 1 : 0; + int subY = this.SequenceHeader.ColorConfig.SubSamplingY ? 1 : 0; + Buffer2DRegion colorIndexMap = this.FrameInfo.GetPaletteColorIndexMap( + this.configuration, + Av1PlaneType.Uv, + new Rectangle( + (partitionInfo.ColumnIndex << Av1Constants.ModeInfoSizeLog2) >> subX, + (partitionInfo.RowIndex << Av1Constants.ModeInfoSizeLog2) >> subY, + planeWidth, + planeHeight)); + + DecodePaletteColorMap( ref reader, modeInfo.GetPaletteSize(Av1PlaneType.Uv), Av1PlaneType.Uv, planeWidth, planeHeight, rows, - columns); + columns, + colorIndexMap); modeInfo.SetPaletteColorIndexMap(Av1PlaneType.Uv, colorIndexMap); } @@ -1814,7 +1872,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// Implements the prefix, intra, and translational inter branches of AV1 section 5.11.7. public void ReadInterFrameModeInfo(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo, Av1TileInfo tileInfo) { - Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; modeInfo.MotionVectors.Clear(); this.ReadInterSegmentId(ref reader, ref partitionInfo, beforeSkip: true); @@ -2292,7 +2350,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable ref Av1PartitionInfo partitionInfo, Av1PredictionMode yMode) { - Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; modeInfo.YMode = yMode; modeInfo.SetAngleDelta(Av1PlaneType.Y, IntraAngleInfo(ref reader, yMode, modeInfo.BlockSize)); @@ -2302,7 +2360,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable if (modeInfo.UvMode == Av1ChromaPredictionMode.ChromaFromLuma) { - ReadChromaFromLumaAlphas(ref reader, modeInfo); + ReadChromaFromLumaAlphas(ref reader, ref modeInfo); } modeInfo.SetAngleDelta( @@ -2390,7 +2448,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// Implements AV1 section 5.11.46. private void PaletteModeInfo(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) { - Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; Av1BlockSize blockSize = modeInfo.BlockSize; // The palette block-size context is the base-two block-area difference from an 8-by-8 block. @@ -2401,12 +2459,12 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable if (modeInfo.YMode == Av1PredictionMode.DC) { int neighborContext = 0; - if (partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.GetPaletteSize(Av1PlaneType.Y) != 0) + if (partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.Value.GetPaletteSize(Av1PlaneType.Y) != 0) { neighborContext++; } - if (partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.GetPaletteSize(Av1PlaneType.Y) != 0) + if (partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.Value.GetPaletteSize(Av1PlaneType.Y) != 0) { neighborContext++; } @@ -2595,10 +2653,12 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable : partitionInfo.AboveModeInfo; Av1BlockModeInfo? leftModeInfo = partitionInfo.LeftModeInfo; - int abovePaletteSize = aboveModeInfo?.GetPaletteSize(plane) ?? 0; - int leftPaletteSize = leftModeInfo?.GetPaletteSize(plane) ?? 0; - ReadOnlySpan aboveColors = aboveModeInfo is null ? [] : aboveModeInfo.GetPaletteColors(plane); - ReadOnlySpan leftColors = leftModeInfo is null ? [] : leftModeInfo.GetPaletteColors(plane); + Av1BlockModeInfo above = aboveModeInfo.GetValueOrDefault(); + Av1BlockModeInfo left = leftModeInfo.GetValueOrDefault(); + int abovePaletteSize = aboveModeInfo is null ? 0 : above.GetPaletteSize(plane); + int leftPaletteSize = leftModeInfo is null ? 0 : left.GetPaletteSize(plane); + ReadOnlySpan aboveColors = aboveModeInfo is null ? [] : above.GetPaletteColors(plane); + ReadOnlySpan leftColors = leftModeInfo is null ? [] : left.GetPaletteColors(plane); int aboveIndex = 0; int leftIndex = 0; int count = 0; @@ -2725,18 +2785,18 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// The padded plane-block height. /// The number of rows inside the coded image. /// The number of columns inside the coded image. - /// The decoded row-major color-index map. - private static byte[] DecodePaletteColorMap( + /// The row-addressable destination map. + private static void DecodePaletteColorMap( ref Av1SymbolDecoder reader, int paletteSize, Av1PlaneType planeType, int planeWidth, int planeHeight, int rows, - int columns) + int columns, + Buffer2DRegion colorIndexMap) { - byte[] colorIndexMap = new byte[planeWidth * planeHeight]; - colorIndexMap[0] = (byte)reader.ReadUniform(paletteSize); + colorIndexMap.DangerousGetRowSpan(0)[0] = (byte)reader.ReadUniform(paletteSize); Span colorOrder = stackalloc byte[Av1Constants.PaletteMaxSize]; for (int diagonal = 1; diagonal < rows + columns - 1; diagonal++) { @@ -2747,14 +2807,13 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable int row = diagonal - column; int colorContext = GetPaletteColorIndexContext( colorIndexMap, - planeWidth, row, column, paletteSize, colorOrder); int colorOrderIndex = reader.ReadPaletteColorIndex(paletteSize, colorContext, planeType); - colorIndexMap[(row * planeWidth) + column] = colorOrder[colorOrderIndex]; + colorIndexMap.DangerousGetRowSpan(row)[column] = colorOrder[colorOrderIndex]; } } @@ -2763,44 +2822,50 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable // Blocks clipped by the right image edge repeat their final coded column into the padded block area. for (int row = 0; row < rows; row++) { - int rowOffset = row * planeWidth; - colorIndexMap.AsSpan(rowOffset + columns, planeWidth - columns) - .Fill(colorIndexMap[rowOffset + columns - 1]); + Span colorIndexRow = colorIndexMap.DangerousGetRowSpan(row); + colorIndexRow.Slice(columns, planeWidth - columns) + .Fill(colorIndexRow[columns - 1]); } } // Blocks clipped by the bottom image edge repeat their final coded row for later transform reconstruction. - ReadOnlySpan finalRow = colorIndexMap.AsSpan((rows - 1) * planeWidth, planeWidth); + ReadOnlySpan finalRow = colorIndexMap.DangerousGetRowSpan(rows - 1); for (int row = rows; row < planeHeight; row++) { - finalRow.CopyTo(colorIndexMap.AsSpan(row * planeWidth, planeWidth)); + finalRow.CopyTo(colorIndexMap.DangerousGetRowSpan(row)); } - - return colorIndexMap; } /// /// Derives the palette color order and entropy context from the left, upper-left, and above indices. /// /// The partially decoded color-index map. - /// The map row stride. /// The current map row. /// The current map column. /// The number of palette colors. /// The destination color order for the current context. /// The color-index entropy context in the range from zero through four. private static int GetPaletteColorIndexContext( - ReadOnlySpan colorIndexMap, - int stride, + Buffer2DRegion colorIndexMap, int row, int column, int paletteSize, Span colorOrder) { Span neighborColors = stackalloc int[3]; - neighborColors[0] = column > 0 ? colorIndexMap[(row * stride) + column - 1] : -1; - neighborColors[1] = column > 0 && row > 0 ? colorIndexMap[((row - 1) * stride) + column - 1] : -1; - neighborColors[2] = row > 0 ? colorIndexMap[((row - 1) * stride) + column] : -1; + ReadOnlySpan currentRow = colorIndexMap.DangerousGetRowSpan(row); + neighborColors[0] = column > 0 ? currentRow[column - 1] : -1; + if (row > 0) + { + ReadOnlySpan aboveRow = colorIndexMap.DangerousGetRowSpan(row - 1); + neighborColors[1] = column > 0 ? aboveRow[column - 1] : -1; + neighborColors[2] = aboveRow[column]; + } + else + { + neighborColors[1] = -1; + neighborColors[2] = -1; + } Span scores = stackalloc int[Av1Constants.PaletteMaxSize]; ReadOnlySpan neighborWeights = [2, 1, 2]; @@ -2855,7 +2920,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// The tile symbol decoder. /// The block mode information to populate. /// Implements AV1 section 5.11.45. - private static void ReadChromaFromLumaAlphas(ref Av1SymbolDecoder reader, Av1BlockModeInfo modeInfo) + private static void ReadChromaFromLumaAlphas(ref Av1SymbolDecoder reader, ref Av1BlockModeInfo modeInfo) { int jointSignPlus1 = reader.ReadChromFromLumaSign() + 1; int index = 0; @@ -2932,7 +2997,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable public void ReadInterSegmentId(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo, bool beforeSkip) { ObuSegmentationParameters segmentationParameters = this.FrameHeader.SegmentationParameters; - Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; if (!segmentationParameters.Enabled) { @@ -3049,7 +3114,15 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable : prevUL == prevU && prevUL == prevL ? 2 : prevUL == prevU || prevUL == prevL || prevU == prevL ? 1 : 0; int lastActiveSegmentId = this.FrameHeader.SegmentationParameters.LastActiveSegmentId; - partitionInfo.ModeInfo.SegmentId = Av1SymbolContextHelper.NegativeDeinterleave(reader.ReadSegmentId(ctx), predictor, lastActiveSegmentId + 1); + int segmentId = Av1SymbolContextHelper.NegativeDeinterleave(reader.ReadSegmentId(ctx), predictor, lastActiveSegmentId + 1); + if (segmentId is < 0 || segmentId > lastActiveSegmentId) + { + // The coded alphabet always contains eight symbols, even when the frame activates fewer segments. + // Validate the reconstructed ID at the same corruption boundary as libaom's read_segment_id. + throw new InvalidImageContentException("The decoded AV1 segment identifier exceeds the active segment range."); + } + + partitionInfo.ModeInfo.SegmentId = segmentId; } } @@ -3151,8 +3224,8 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable } else { - int aboveSkip = partitionInfo.AboveModeInfo != null && partitionInfo.AboveModeInfo.Skip ? 1 : 0; - int leftSkip = partitionInfo.LeftModeInfo != null && partitionInfo.LeftModeInfo.Skip ? 1 : 0; + int aboveSkip = partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.Value.Skip ? 1 : 0; + int leftSkip = partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.Value.Skip ? 1 : 0; return reader.ReadSkip(aboveSkip + leftSkip); } } @@ -3165,7 +3238,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// when the block selects the frame's derived skip-mode reference pair. private bool ReadSkipMode(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) { - Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; ObuSegmentationParameters segmentationParameters = this.FrameHeader.SegmentationParameters; int segmentId = modeInfo.SegmentId; @@ -3180,8 +3253,8 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable return false; } - int aboveSkipMode = partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.SkipMode ? 1 : 0; - int leftSkipMode = partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.SkipMode ? 1 : 0; + int aboveSkipMode = partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.Value.SkipMode ? 1 : 0; + int leftSkipMode = partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.Value.SkipMode ? 1 : 0; return reader.ReadSkipMode(aboveSkipMode + leftSkipMode); } @@ -3220,7 +3293,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// The current coding block and its available neighbors. private void ReadReferenceFrames(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) { - Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; + ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo; Span references = modeInfo.ReferenceFrames; if (modeInfo.SkipMode) { @@ -3369,7 +3442,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// /// The tile symbol decoder. /// The current coding block and superblock quantizer storage. - /// Corresponds to read_delta_qindex in SVT-AV1. + /// Corresponds to read_delta_qindex in libaom. private void ReadDeltaQuantizerIndex(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) { if (!this.FrameHeader.DeltaQParameters.IsPresent || partitionInfo.ModeInfo.PositionInSuperblock != Point.Empty) @@ -3411,7 +3484,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable /// The active tile boundaries. /// The containing superblock. /// The partition entropy context. - /// Corresponds to partition_plane_context in SVT-AV1. + /// Corresponds to partition_plane_context in libaom. private int GetPartitionPlaneContext(Point location, Av1BlockSize blockSize, Av1TileInfo tileInfo, Av1SuperblockInfo superblockInfo) { // The five stored split bits begin at the 8x8 partition point, so normalize the block-size log to that bit index. diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1PalettePredictionBenchmarks.cs b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1PalettePredictionBenchmarks.cs index 5f06831c0..52e205d78 100644 --- a/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1PalettePredictionBenchmarks.cs +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1PalettePredictionBenchmarks.cs @@ -6,6 +6,7 @@ using BenchmarkDotNet.Columns; using BenchmarkDotNet.Configs; using BenchmarkDotNet.Jobs; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Memory; namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; @@ -46,7 +47,12 @@ public class Av1PalettePredictionBenchmarks /// /// The decoded color-index map for one maximum-size palette block. /// - private readonly byte[] colorIndexMap = new byte[BlockSize * BlockSize]; + private Buffer2D colorIndexMapBuffer; + + /// + /// The row-addressable view of . + /// + private Buffer2DRegion colorIndexMap; /// /// The frame-wide 8-bit reconstruction surface. @@ -64,15 +70,24 @@ public class Av1PalettePredictionBenchmarks [GlobalSetup] public void Setup() { + this.colorIndexMapBuffer = SixLabors.ImageSharp.Configuration.Default.MemoryAllocator.Allocate2D(BlockSize, BlockSize); + this.colorIndexMap = new Buffer2DRegion(this.colorIndexMapBuffer); for (int row = 0; row < BlockSize; row++) { + Span colorIndexRow = this.colorIndexMap.DangerousGetRowSpan(row); for (int column = 0; column < BlockSize; column++) { - this.colorIndexMap[(row * BlockSize) + column] = (byte)(((row * 5) + (column * 3)) & 7); + colorIndexRow[column] = (byte)(((row * 5) + (column * 3)) & 7); } } } + /// + /// Releases the row-addressable color-index map after the benchmark run. + /// + [GlobalCleanup] + public void Cleanup() => this.colorIndexMapBuffer?.Dispose(); + /// /// Measures frame-wide 8-bit palette reconstruction. /// @@ -85,7 +100,7 @@ public class Av1PalettePredictionBenchmarks { for (int column = 0; column < Width; column += BlockSize) { - Av1PalettePredictor.Predict(this.palette8, this.colorIndexMap, BlockSize, this.destination8.AsSpan((row * Width) + column), Width, BlockSize, BlockSize); + Av1PalettePredictor.Predict(this.palette8, this.colorIndexMap, this.destination8.AsSpan((row * Width) + column), Width, BlockSize, BlockSize); } } @@ -104,7 +119,7 @@ public class Av1PalettePredictionBenchmarks { for (int column = 0; column < Width; column += BlockSize) { - Av1PalettePredictor.Predict(this.palette12, this.colorIndexMap, BlockSize, this.destination12.AsSpan((row * Width) + column), Width, BlockSize, BlockSize); + Av1PalettePredictor.Predict(this.palette12, this.colorIndexMap, this.destination12.AsSpan((row * Width) + column), Width, BlockSize, BlockSize); } } diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CoefficientsEntropyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CoefficientsEntropyTests.cs index b996311e1..771ae5e0a 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CoefficientsEntropyTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CoefficientsEntropyTests.cs @@ -44,10 +44,27 @@ public class Av1CoefficientsEntropyTests using IMemoryOwner encoded = encoder.Exit(); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + using Av1LevelBuffer levels = new(Configuration.Default); decoder.ReadCoefficients( - modeInfo, new Point(0, 0), aboveContexts, leftContexts, - 0, 0, 0, 1, 1, transformBlockContext, transformSize, - false, true, transformType, ref transformInfo, 0, 0, actuals); + modeInfo, + new Point(0, 0), + aboveContexts, + leftContexts, + 0, + 0, + 0, + 1, + 1, + transformBlockContext, + transformSize, + false, + true, + transformType, + ref transformInfo, + 0, + 0, + levels, + actuals); // Assert Assert.Equal(endOfBlock, actuals[0]); @@ -96,11 +113,28 @@ public class Av1CoefficientsEntropyTests using IMemoryOwner encoded = encoder.Exit(); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + using Av1LevelBuffer levels = new(Configuration.Default); int plane = Math.Min((int)componentType, 1); decoder.ReadCoefficients( - modeInfo, new Point(0, 0), aboveContexts, leftContexts, - 0, 0, plane, 1, 1, transformBlockContext, transformSize, - false, true, transformType, ref transformInfo, 0, 0, actuals); + modeInfo, + new Point(0, 0), + aboveContexts, + leftContexts, + 0, + 0, + plane, + 1, + 1, + transformBlockContext, + transformSize, + false, + true, + transformType, + ref transformInfo, + 0, + 0, + levels, + actuals); // Assert Assert.Equal(endOfBlock, actuals[0]); @@ -153,11 +187,28 @@ public class Av1CoefficientsEntropyTests using IMemoryOwner encoded = encoder.Exit(); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); + using Av1LevelBuffer levels = new(Configuration.Default); int plane = Math.Min((int)componentType, 1); decoder.ReadCoefficients( - modeInfo, new Point(0, 0), aboveContexts, leftContexts, - 0, 0, plane, 1, 1, transformBlockContext, transformSize, - false, true, transformType, ref transformInfo, 0, 0, actuals); + modeInfo, + new Point(0, 0), + aboveContexts, + leftContexts, + 0, + 0, + plane, + 1, + 1, + transformBlockContext, + transformSize, + false, + true, + transformType, + ref transformInfo, + 0, + 0, + levels, + actuals); // Assert Assert.Equal(endOfBlock, actuals[0]); diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameIntraEntropyTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameIntraEntropyTests.cs index cf2baf76a..72524cd60 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameIntraEntropyTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameIntraEntropyTests.cs @@ -160,8 +160,8 @@ public class Av1InterFrameIntraEntropyTests bool leftIsInter, int expected) { - Av1BlockModeInfo above = hasAbove ? CreateModeInfo(aboveIsInter) : null; - Av1BlockModeInfo left = hasLeft ? CreateModeInfo(leftIsInter) : null; + Av1BlockModeInfo? above = hasAbove ? CreateModeInfo(aboveIsInter) : null; + Av1BlockModeInfo? left = hasLeft ? CreateModeInfo(leftIsInter) : null; int actual = Av1SymbolContextHelper.GetIntraInterContext(above, left); diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameModeInfoTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameModeInfoTests.cs index 9b926153a..11419a348 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameModeInfoTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameModeInfoTests.cs @@ -43,6 +43,7 @@ public class Av1InterFrameModeInfoTests Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); + modeInfo = partitionInfo.ModeInfo; Assert.False(modeInfo.SkipMode); Assert.False(modeInfo.Skip); @@ -89,7 +90,7 @@ public class Av1InterFrameModeInfoTests using IMemoryOwner encoded = writer.Exit(); Memory encodedMemory = encoded.Memory; - ReadInterFrameModeInfo(tileReader, encodedMemory, modeInfo, aboveModeInfo); + modeInfo = ReadInterFrameModeInfo(tileReader, encodedMemory, modeInfo, aboveModeInfo); Assert.True(modeInfo.SkipMode); Assert.True(modeInfo.Skip); @@ -139,6 +140,7 @@ public class Av1InterFrameModeInfoTests Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); + modeInfo = partitionInfo.ModeInfo; Assert.Equal(Av1InterpolationFilter.Smooth, modeInfo.InterpolationFilters[0]); Assert.Equal((Av1InterpolationFilter)expectedHorizontalFilter, modeInfo.InterpolationFilters[1]); @@ -173,6 +175,7 @@ public class Av1InterFrameModeInfoTests Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); + modeInfo = partitionInfo.ModeInfo; Assert.Equal(Av1InterpolationFilter.Regular, modeInfo.InterpolationFilters[0]); Assert.Equal(Av1InterpolationFilter.Regular, modeInfo.InterpolationFilters[1]); @@ -213,7 +216,7 @@ public class Av1InterFrameModeInfoTests using IMemoryOwner encoded = writer.Exit(); Memory encodedMemory = encoded.Memory; - ReadInterFrameModeInfo(tileReader, encodedMemory, modeInfo); + modeInfo = ReadInterFrameModeInfo(tileReader, encodedMemory, modeInfo); Assert.Equal((Av1ReferenceFrameType)expectedPrimary, modeInfo.ReferenceFrames[0]); Assert.Equal((Av1ReferenceFrameType)expectedSecondary, modeInfo.ReferenceFrames[1]); @@ -285,7 +288,7 @@ public class Av1InterFrameModeInfoTests writer.WriteSymbol((int)Av1InterpolationFilter.Sharp, Av1DefaultDistributions.SwitchableInterpolation[3]); using IMemoryOwner encoded = writer.Exit(); - ReadInterFrameModeInfo(tileReader, encoded.Memory, modeInfo); + modeInfo = ReadInterFrameModeInfo(tileReader, encoded.Memory, modeInfo); Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]); Assert.Equal(Av1ReferenceFrameType.Last2, modeInfo.ReferenceFrames[1]); @@ -311,7 +314,8 @@ public class Av1InterFrameModeInfoTests /// The range-coded block-prefix symbols. /// The current coding block. /// The available above block supplying skip-mode context. - private static void ReadInterFrameModeInfo( + /// The decoded block mode information. + private static Av1BlockModeInfo ReadInterFrameModeInfo( Av1TileReader tileReader, Memory encoded, Av1BlockModeInfo modeInfo, @@ -326,12 +330,14 @@ public class Av1InterFrameModeInfoTests Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Span, 0, updateCdf: true); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, tileReader.FrameHeader)); + return partitionInfo.ModeInfo; } /// /// Invokes the ref-struct mode parser without spatial neighbors. /// - private static void ReadInterFrameModeInfo( + /// The decoded block mode information. + private static Av1BlockModeInfo ReadInterFrameModeInfo( Av1TileReader tileReader, Memory encoded, Av1BlockModeInfo modeInfo) @@ -340,6 +346,7 @@ public class Av1InterFrameModeInfoTests Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Span, 0, updateCdf: true); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, tileReader.FrameHeader)); + return partitionInfo.ModeInfo; } /// diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InverseTransformTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InverseTransformTests.cs index 753454afc..82f9c12b0 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InverseTransformTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InverseTransformTests.cs @@ -43,10 +43,10 @@ public class Av1InverseTransformTests => FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertIdentityOperatorParity, TransformConfigurations); /// - /// Verifies the pinned-libaom widened operations at the twelve-bit inverse row-stage bounds. + /// Verifies the current-libaom widened operations at the twelve-bit inverse row-stage bounds. /// [Fact] - public void TwelveBitWideIntermediatesMatchPinnedLibaom() + public void TwelveBitWideIntermediatesMatchCurrentLibaom() => FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertTwelveBitWideIntermediateParity, TransformConfigurations); /// @@ -123,7 +123,7 @@ public class Av1InverseTransformTests cosBit, stageRange); - // These are the exact outputs of pinned libaom's signed Int64 terminal round. The first positive lane has an + // These are the exact outputs of current libaom's signed Int64 terminal round. The first positive lane has an // Int32 fixed-point sum of 2,147,482,471, so adding the 2,048 rounding bias in Int32 would wrap. Vector128 adstExpected0 = Vector128.Create(524_288, -524_288, 524_287, -524_287); Vector128 adstExpected1 = Vector128.Create(33_612, -33_612, 33_612, -33_612); @@ -167,7 +167,7 @@ public class Av1InverseTransformTests } /// - /// Verifies one identity operator against exact pinned-libaom widened fixed-point results. + /// Verifies one identity operator against exact current-libaom widened fixed-point results. /// /// The inverse identity operator. /// The identity-transform length. diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1LevelBufferTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1LevelBufferTests.cs index 6eabd6c7e..937d1bc67 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1LevelBufferTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1LevelBufferTests.cs @@ -20,7 +20,7 @@ public class Av1LevelBufferTests { // Arrange Size size = new(width, height); - Av1LevelBuffer levels = new(Configuration.Default, size); + using Av1LevelBuffer levels = new(Configuration.Default, size); for (byte i = 0; i < 4; i++) { levels.GetRow(i).Fill(i); @@ -43,7 +43,7 @@ public class Av1LevelBufferTests { // Arrange Size size = new(width, height); - Av1LevelBuffer levels = new(Configuration.Default, size); + using Av1LevelBuffer levels = new(Configuration.Default, size); for (byte i = 0; i < height; i++) { levels.GetRow(i).Fill(i); @@ -69,7 +69,7 @@ public class Av1LevelBufferTests { // Arrange Size size = new(width, height); - Av1LevelBuffer levels = new(Configuration.Default, size); + using Av1LevelBuffer levels = new(Configuration.Default, size); for (byte i = 0; i < height; i++) { levels.GetRow(i).Fill(i); diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionModeInfoTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionModeInfoTests.cs index b14ab0533..2ddb0a44a 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionModeInfoTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionModeInfoTests.cs @@ -47,6 +47,7 @@ public class Av1MotionModeInfoTests Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); + modeInfo = partitionInfo.ModeInfo; Assert.Equal(Av1MotionMode.SimpleTranslation, modeInfo.MotionMode); Assert.Equal(Av1InterpolationFilter.Sharp, modeInfo.InterpolationFilters[0]); @@ -103,7 +104,7 @@ public class Av1MotionModeInfoTests aboveModeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last; aboveModeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.None; - aboveModeInfo.InterpolationFilters.Fill(Av1InterpolationFilter.Regular); + aboveModeInfo.InterpolationFilters.Clear(); tileReader.FrameInfo.UpdateModeInfo(aboveModeInfo, superblockInfo); superblockInfo.BlockCount++; @@ -137,6 +138,7 @@ public class Av1MotionModeInfoTests Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); + modeInfo = partitionInfo.ModeInfo; Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]); Assert.Equal(Av1ReferenceFrameType.None, modeInfo.ReferenceFrames[1]); @@ -187,6 +189,7 @@ public class Av1MotionModeInfoTests Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); + modeInfo = partitionInfo.ModeInfo; Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]); Assert.Equal(Av1ReferenceFrameType.Intra, modeInfo.ReferenceFrames[1]); diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PalettePredictorTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PalettePredictorTests.cs index cfa55ce88..7cb37e6bb 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PalettePredictorTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PalettePredictorTests.cs @@ -3,6 +3,7 @@ using SixLabors.ImageSharp.Formats.Heif.Av1; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Memory; using SixLabors.ImageSharp.Tests.TestUtilities; namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; @@ -40,12 +41,19 @@ public class Av1PalettePredictorTests int mapStride = width + 5; int destinationStride = width + 9; byte[] colorIndexMap = CreateColorIndexMap(mapStride, height, width, paletteSize); + using Buffer2D colorIndexMapBuffer = Configuration.Default.MemoryAllocator.Allocate2D(mapStride, height); + for (int row = 0; row < height; row++) + { + colorIndexMap.AsSpan(row * mapStride, mapStride).CopyTo(colorIndexMapBuffer.DangerousGetRowSpan(row)); + } + + Buffer2DRegion colorIndexMapRegion = new(colorIndexMapBuffer); ushort[] bytePalette = CreatePalette(paletteSize, 8); byte[] expectedBytes = Enumerable.Repeat((byte)251, destinationStride * height).ToArray(); byte[] actualBytes = (byte[])expectedBytes.Clone(); ApplyReference(bytePalette, colorIndexMap, mapStride, expectedBytes, destinationStride, width, height); - Av1PalettePredictor.Predict(bytePalette, colorIndexMap, mapStride, actualBytes, destinationStride, width, height); + Av1PalettePredictor.Predict(bytePalette, colorIndexMapRegion, actualBytes, destinationStride, width, height); Assert.Equal(expectedBytes, actualBytes); foreach (int bitDepth in new[] { 10, 12 }) @@ -55,7 +63,7 @@ public class Av1PalettePredictorTests short[] actual = (short[])expected.Clone(); ApplyReference(palette, colorIndexMap, mapStride, expected, destinationStride, width, height); - Av1PalettePredictor.Predict(palette, colorIndexMap, mapStride, actual, destinationStride, width, height); + Av1PalettePredictor.Predict(palette, colorIndexMapRegion, actual, destinationStride, width, height); Assert.Equal(expected, actual); } } diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs index cb542a71c..625d8f521 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs @@ -461,7 +461,7 @@ public class Av1ReconstructionConformanceTests /// under normal SIMD dispatch and with hardware intrinsics disabled. /// [Fact] - public void DecodeWithActiveCdefMatchesPinnedLibaomReference() + public void DecodeWithActiveCdefMatchesCurrentLibaomReference() => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateActiveCdefFixtures, ReconstructionConfigurations); /// @@ -547,13 +547,70 @@ public class Av1ReconstructionConformanceTests } /// - /// Verifies decoded luma and chroma palette syntax and exact native samples against scalar libaom for an + /// Verifies decoded luma and chroma palette syntax and exact native samples against current official libaom for an /// independently encoded AV1 still-picture stream. /// [Fact] - public void DecodeWithPaletteMatchesPinnedLibaomReference() + public void DecodeWithPaletteMatchesCurrentLibaomReference() => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidatePaletteNativeFixture, PaletteConfigurations); + /// + /// Verifies exact palette reconstruction through segmented frame-owned map storage and tracked disposal. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodePaletteWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 1_024 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + byte[] payload = TestFile.Create(TestImages.Heif.Av1Palette8BitPayload).Bytes; + byte[] reference = TestFile.Create(TestImages.Heif.Av1Palette8BitReference).Bytes; + bool foundSegmentedLumaMap = false; + bool foundSegmentedChromaMap = false; + + using (Av1Decoder decoder = new(configuration)) + { + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _); + + Assert.Equal(RequiredPaletteCoverage, GetPaletteCoverage(decoder)); + AssertNativePlanesEqual(decoder, frameBuffer, reference); + Assert.NotNull(decoder.FrameHeader); + Assert.NotNull(decoder.FrameInfo); + int modeInfoWidth = Av1Math.DivideLog2Ceiling(decoder.FrameHeader.FrameSize.FrameWidth, Av1Constants.ModeInfoSizeLog2); + int modeInfoHeight = Av1Math.DivideLog2Ceiling(decoder.FrameHeader.FrameSize.FrameHeight, Av1Constants.ModeInfoSizeLog2); + for (int y = 0; y < modeInfoHeight && (!foundSegmentedLumaMap || !foundSegmentedChromaMap); y++) + { + for (int x = 0; x < modeInfoWidth && (!foundSegmentedLumaMap || !foundSegmentedChromaMap); x++) + { + Av1BlockModeInfo modeInfo = decoder.FrameInfo.GetModeInfoAt(new Point(x, y)); + if (!foundSegmentedLumaMap && modeInfo.GetPaletteSize(Av1PlaneType.Y) != 0) + { + Buffer2DRegion map = modeInfo.GetPaletteColorIndexMap(Av1Plane.Y); + foundSegmentedLumaMap = map.Buffer.MemoryGroup.Count > 1; + } + + if (!foundSegmentedChromaMap && modeInfo.GetPaletteSize(Av1PlaneType.Uv) != 0) + { + Buffer2DRegion map = modeInfo.GetPaletteColorIndexMap(Av1Plane.U); + foundSegmentedChromaMap = map.Buffer.MemoryGroup.Count > 1; + } + } + } + + Assert.True(foundSegmentedLumaMap); + Assert.True(foundSegmentedChromaMap); + } + + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + /// /// Verifies that a real palette frame whose tile entropy payload ends early is rejected instead of being decoded /// from the range decoder's implicit zero padding. @@ -607,13 +664,13 @@ public class Av1ReconstructionConformanceTests } /// - /// Verifies decoded luma and chroma palette syntax and exact presented pixels for an independently encoded AVIF - /// image across the available vector widths and the scalar fallback. + /// Verifies decoded luma and chroma palette syntax and exact presented pixels against the retained reference image + /// across the available vector widths and the scalar fallback. /// /// The AVIF input and matching reference-output naming context. [Theory] [WithFile(TestImages.Heif.Av1Palette8BitAvif, PixelTypes.Rgba32)] - public void DecodeWithPaletteMatchesPinnedLibavifPresentation(TestImageProvider provider) + public void DecodeWithPaletteMatchesRetainedPresentationReference(TestImageProvider provider) => FeatureTestRunner.RunWithHwIntrinsicsFeature( ValidatePresentedFixture, PresentationConfigurations, @@ -1145,10 +1202,10 @@ public class Av1ReconstructionConformanceTests /// /// Verifies every intra prediction mode and the fixture's seven transform types against the official - /// pinned-libaom all-intra conformance sequence and its exact native output. + /// current-libaom all-intra conformance sequence and its exact native output. /// [Fact] - public void DecodeOfficialAllIntraSequenceMatchesPinnedLibaomReference() => ValidateOfficialAllIntraFixture(); + public void DecodeOfficialAllIntraSequenceMatchesCurrentLibaomReference() => ValidateOfficialAllIntraFixture(); /// /// Decodes every all-intra IVF sample in one session, compares each frame exactly, and records the syntax @@ -1746,10 +1803,10 @@ public class Av1ReconstructionConformanceTests } /// - /// Verifies the official eight-bit quantizer boundaries against exact pinned-libaom native output. + /// Verifies the official eight-bit quantizer boundaries against exact current-libaom native output. /// [Fact] - public void DecodeOfficialEightBitQuantizerBoundarySequencesMatchPinnedLibaomReferences() + public void DecodeOfficialEightBitQuantizerBoundarySequencesMatchCurrentLibaomReferences() => FeatureTestRunner.RunWithHwIntrinsicsFeature( ValidateOfficialEightBitQuantizerBoundaryFixtures, ReconstructionConfigurations); @@ -1761,10 +1818,10 @@ public class Av1ReconstructionConformanceTests => ValidateOfficialEightBitQuantizerBoundaryFixturesWithConfiguration(Configuration.Default); /// - /// Verifies the official ten-bit quantizer boundaries against exact pinned-libaom native output. + /// Verifies the official ten-bit quantizer boundaries against exact current-libaom native output. /// [Fact] - public void DecodeOfficialTenBitQuantizerBoundarySequencesMatchPinnedLibaomReferences() + public void DecodeOfficialTenBitQuantizerBoundarySequencesMatchCurrentLibaomReferences() => FeatureTestRunner.RunWithHwIntrinsicsFeature( ValidateOfficialTenBitQuantizerBoundaryFixtures, ReconstructionConfigurations); @@ -1790,6 +1847,16 @@ public class Av1ReconstructionConformanceTests ValidateOfficialEightBitQuantizerBoundaryFixturesWithConfiguration(configuration); ValidateOfficialTenBitQuantizerBoundaryFixturesWithConfiguration(configuration); + // Each decoded frame owns one maximum-sized coefficient-context scratch rent. An allocation per transform + // would produce hundreds of identically typed smaller rents for these deliberately dense fixtures. + int coefficientScratchLength = + ((Av1Constants.MaxTransformSize / 2) + Av1Constants.TransformPadHorizontal) * + (Av1Constants.TransformPadTop + (Av1Constants.MaxTransformSize / 2) + Av1Constants.TransformPadBottom); + + int coefficientScratchAllocations = allocator.AllocationLog.Count( + allocation => allocation.ElementType == typeof(byte) && allocation.Length == coefficientScratchLength); + + Assert.Equal(4 * OfficialQuantizerFixtureFrameCount, coefficientScratchAllocations); Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); Assert.All( allocator.AllocationLog, @@ -4194,13 +4261,15 @@ public class Av1ReconstructionConformanceTests int modeInfoRow = (y << subsamplingY) >> Av1Constants.ModeInfoSizeLog2; Av1BlockModeInfo modeInfo = frameInfo.GetModeInfoAt(new Point(modeInfoColumn, modeInfoRow)); int blockColumn = modeInfoColumn; - while (blockColumn > 0 && ReferenceEquals(frameInfo.GetModeInfoAt(new Point(blockColumn - 1, modeInfoRow)), modeInfo)) + while (blockColumn > 0 && + frameInfo.GetModeInfoAt(new Point(blockColumn - 1, modeInfoRow)).ModeInfoIndex == modeInfo.ModeInfoIndex) { blockColumn--; } int blockRow = modeInfoRow; - while (blockRow > 0 && ReferenceEquals(frameInfo.GetModeInfoAt(new Point(modeInfoColumn, blockRow - 1)), modeInfo)) + while (blockRow > 0 && + frameInfo.GetModeInfoAt(new Point(modeInfoColumn, blockRow - 1)).ModeInfoIndex == modeInfo.ModeInfoIndex) { blockRow--; } diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceMotionVectorsTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceMotionVectorsTests.cs index fb7d134d0..b4853d5b6 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceMotionVectorsTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceMotionVectorsTests.cs @@ -245,20 +245,19 @@ public class Av1ReferenceMotionVectorsTests frameInfo.InitializeMotionField(Configuration.Default, sequenceHeader, frameHeader, referenceFrames); FillFrameWithIntraBlocks(frameInfo, sequenceHeader); Av1MotionVector direct = new(16, 24); - Av1BlockModeInfo candidate = AddModeInfo( + AddModeInfo( frameInfo, sequenceHeader, new Point(8, 4), Av1BlockSize.Block16x16, Av1ReferenceFrameType.Last, direct, - Av1PredictionMode.NearestMotionVector); + Av1PredictionMode.NearestMotionVector, + Av1ReferenceFrameType.Backward, + new Av1MotionVector(40, -24)); // The direct scan adds the first reference with its normative adjacent weight. Extension visits both entries: // it must ignore that duplicate and append only the sign-corrected backward-reference vector. - candidate.ReferenceFrames[1] = Av1ReferenceFrameType.Backward; - candidate.MotionVectors[1] = new Av1MotionVector(40, -24); - Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, new Point(8, 8)); Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, true, Av1PartitionType.None) @@ -361,31 +360,29 @@ public class Av1ReferenceMotionVectorsTests Av1MotionVector abovePrimary = new(8, 16); Av1MotionVector aboveSecondary = new(24, 32); - Av1BlockModeInfo above = AddModeInfo( + AddModeInfo( frameInfo, sequenceHeader, new Point(8, 4), Av1BlockSize.Block16x16, Av1ReferenceFrameType.Last, abovePrimary, - Av1PredictionMode.NewNewMotionVector); - - above.ReferenceFrames[1] = Av1ReferenceFrameType.Backward; - above.MotionVectors[1] = aboveSecondary; + Av1PredictionMode.NewNewMotionVector, + Av1ReferenceFrameType.Backward, + aboveSecondary); Av1MotionVector leftPrimary = new(40, 48); Av1MotionVector leftSecondary = new(56, 64); - Av1BlockModeInfo left = AddModeInfo( + AddModeInfo( frameInfo, sequenceHeader, new Point(4, 8), Av1BlockSize.Block16x16, Av1ReferenceFrameType.Last, leftPrimary, - Av1PredictionMode.NearestNearestMotionVector); - - left.ReferenceFrames[1] = Av1ReferenceFrameType.Backward; - left.MotionVectors[1] = leftSecondary; + Av1PredictionMode.NearestNearestMotionVector, + Av1ReferenceFrameType.Backward, + leftSecondary); Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, new Point(8, 8)); @@ -592,6 +589,8 @@ public class Av1ReferenceMotionVectorsTests /// The primary prediction reference. /// The primary motion vector. /// The decoded luma or inter prediction mode. + /// The optional secondary prediction reference. + /// The optional secondary motion vector. /// The mapped mode-information block. private static Av1BlockModeInfo AddModeInfo( Av1FrameInfo frameInfo, @@ -600,7 +599,9 @@ public class Av1ReferenceMotionVectorsTests Av1BlockSize blockSize, Av1ReferenceFrameType referenceFrame, Av1MotionVector motionVector, - Av1PredictionMode predictionMode) + Av1PredictionMode predictionMode, + Av1ReferenceFrameType secondaryReferenceFrame = Av1ReferenceFrameType.None, + Av1MotionVector secondaryMotionVector = default) { int superblockSize = sequenceHeader.SuperblockModeInfoSize; Point superblockPosition = new(position.X / superblockSize, position.Y / superblockSize); @@ -612,7 +613,9 @@ public class Av1ReferenceMotionVectorsTests }; modeInfo.ReferenceFrames[0] = referenceFrame; + modeInfo.ReferenceFrames[1] = secondaryReferenceFrame; modeInfo.MotionVectors[0] = motionVector; + modeInfo.MotionVectors[1] = secondaryMotionVector; frameInfo.UpdateModeInfo(modeInfo, superblockInfo); superblockInfo.BlockCount++; return modeInfo; diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceTransform.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceTransform.cs index 1de49d220..e7cf638ed 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceTransform.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceTransform.cs @@ -8,21 +8,11 @@ namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; internal class Av1ReferenceTransform { - /****************************************************************************** - * SVT file: test/ref/TxfmRef.cc - * - * Reference implementation for txfm, including : - * - reference_dct_1d - * - reference_adst_1d - * - reference_idtx_1d - * - reference_txfm_1d - * - reference_txfm_2d - * - fadst_ref - * - * Original authors: Cidana-Edmond, Cidana-Wenyao - * - ******************************************************************************/ - + /// + /// Gets the analytical amplification used by current libaom's forward-transform tests. + /// + /// The transform configuration. + /// The two-dimensional transform amplification. public static double GetScaleFactor(Av1Transform2dFlipConfiguration config) { int transformWidth = config.TransformSize.GetWidth(); @@ -42,8 +32,14 @@ internal class Av1ReferenceTransform } /// - /// SVT: reference_txfm_2d + /// Applies the analytical two-dimensional transform used by current libaom's + /// test/av1_txfm_test.cc. /// + /// The raster input samples. + /// The raster output coefficients. + /// The two-dimensional transform type. + /// The transform dimensions. + /// The configured two-dimensional amplification. public static void ReferenceTransformFunction2d(Span input, Span output, Av1TransformType transformType, Av1TransformSize transformSize, double scaleFactor) { // Get transform type and size of each dimension. @@ -99,7 +95,7 @@ internal class Av1ReferenceTransform } } - private static void Adst4Reference(Span input, Span output) + private static void Adst4Reference(ReadOnlySpan input, Span output) { // 16384 * sqrt(2) * sin(kPi/9) * 2 / 3 const long sinPi19 = 5283; @@ -146,7 +142,7 @@ internal class Av1ReferenceTransform output[3] = Av1Math.RoundShift(s3, 14); } - private static void ReferenceIdentity1d(Span input, Span output, int size) + private static void ReferenceIdentity1d(ReadOnlySpan input, Span output, int size) { const double sqrt2 = 1.4142135623730950488016887242097f; double scale = 0; @@ -178,7 +174,7 @@ internal class Av1ReferenceTransform } } - private static void ReferenceDct1d(Span input, Span output, int size) + private static void ReferenceDct1d(ReadOnlySpan input, Span output, int size) { const double kInvSqrt2 = 0.707106781186547524400844362104f; for (int k = 0; k < size; ++k) @@ -196,7 +192,7 @@ internal class Av1ReferenceTransform } } - private static void ReferenceAdst1d(Span input, Span output, int size) + private static void ReferenceAdst1d(ReadOnlySpan input, Span output, int size) { if (size == 4) { @@ -227,7 +223,7 @@ internal class Av1ReferenceTransform } } - internal static void ReferenceTransform1d(Av1TransformType1d type, Span input, Span output, int size) + internal static void ReferenceTransform1d(Av1TransformType1d type, ReadOnlySpan input, Span output, int size) { switch (type) { diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TemporalSegmentationTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TemporalSegmentationTests.cs index ad7f021d4..e3237ede2 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TemporalSegmentationTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TemporalSegmentationTests.cs @@ -61,6 +61,32 @@ public class Av1TemporalSegmentationTests Assert.NotEqual(16384U, destination.SegmentIdPredicted[2][0]); } + /// + /// Verifies that a spatial segment symbol cannot select an identifier above the frame's last active segment. + /// + [Fact] + public void SpatialSegmentIdOutsideActiveRangeIsRejected() + { + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(64, 64); + ObuFrameHeader frameHeader = CreateFrameHeader(16, 16, segmentationUpdateMap: 1, segmentationTemporalUpdate: 0); + frameHeader.SegmentationParameters.LastActiveSegmentId = 0; + using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader); + using Av1SymbolWriter writer = new(Configuration.Default, 1, updateCdf: true); + writer.WriteSymbol(Av1Constants.MaxSegmentCount - 1, Av1DefaultDistributions.SegmentId[0]); + using IMemoryOwner encoded = writer.Exit(); + + Assert.Throws( + () => + { + Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty); + Av1SuperblockInfo superblockInfo = new(tileReader.FrameInfo, Point.Empty); + Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None); + Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); + + tileReader.ReadInterSegmentId(ref decoder, ref partitionInfo, beforeSkip: false); + }); + } + /// /// Verifies that only neighboring blocks which selected temporal prediction contribute to the binary CDF context. /// @@ -82,8 +108,8 @@ public class Av1TemporalSegmentationTests bool leftPredicted, int expected) { - Av1BlockModeInfo aboveModeInfo = hasAbove ? CreateModeInfo(abovePredicted) : null; - Av1BlockModeInfo leftModeInfo = hasLeft ? CreateModeInfo(leftPredicted) : null; + Av1BlockModeInfo? aboveModeInfo = hasAbove ? CreateModeInfo(abovePredicted) : null; + Av1BlockModeInfo? leftModeInfo = hasLeft ? CreateModeInfo(leftPredicted) : null; int actual = Av1SymbolContextHelper.GetSegmentIdPredictedContext(aboveModeInfo, leftModeInfo); @@ -146,6 +172,7 @@ public class Av1TemporalSegmentationTests Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); tileReader.ReadInterSegmentId(ref decoder, ref partitionInfo, beforeSkip: segmentIdPrecedesSkip); + modeInfo = partitionInfo.ModeInfo; Assert.True(modeInfo.SegmentIdPredicted); Assert.Equal(2, modeInfo.SegmentId); @@ -194,6 +221,7 @@ public class Av1TemporalSegmentationTests Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); tileReader.ReadInterSegmentId(ref decoder, ref partitionInfo, beforeSkip: false); + modeInfo = partitionInfo.ModeInfo; Assert.False(modeInfo.SegmentIdPredicted); Assert.Equal(3, modeInfo.SegmentId); diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TilingTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TilingTests.cs index 372058be1..3f9221aa4 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TilingTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TilingTests.cs @@ -1,7 +1,9 @@ // Copyright (c) Six Labors. // Licensed under the Six Labors Split License. +using System.Buffers; using SixLabors.ImageSharp.Formats.Heif.Av1; +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; @@ -15,6 +17,23 @@ namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; [Trait("Format", "Avif")] public class Av1TilingTests { + /// + /// Verifies that frame mode-information indices do not wrap at the unsigned 16-bit boundary. + /// + [Fact] + public void ModeInfoMapSupportsMoreThanUShortMaxBlocks() + { + const int blockCount = ushort.MaxValue + 2; + Av1FrameInfo.Av1FrameModeInfoMap map = new(new Size(blockCount, 1)); + for (int index = 0; index < blockCount; index++) + { + map.Update(new Point(index, 0), Av1BlockSize.Block4x4); + } + + Assert.Equal(blockCount, map.NextIndex); + Assert.Equal(blockCount - 1, map[new Point(blockCount - 1, 0)]); + } + /// /// Verifies the decoded block geometry and prediction modes against libaom inspection output for a real AVIF image item. /// @@ -75,6 +94,59 @@ public class Av1TilingTests Assert.True(image.Frames.RootFrame.PixelBuffer.DangerousGetSingleSpan().ContainsAnyExcept(default(Rgba32))); } + /// + /// Verifies that partition syntax cannot produce a luma block with no valid 4:2:0 chroma representation. + /// + [Fact] + public void RejectsPartitionThatCannotRepresentSubsampledChroma() + { + ObuSequenceHeader sequenceHeader = new() + { + MaxFrameWidth = 64, + MaxFrameHeight = 64, + Use128x128Superblock = false, + ColorConfig = new ObuColorConfig + { + BitDepth = Av1BitDepth.EightBit, + SubSamplingX = true, + SubSamplingY = true + } + }; + ObuTileGroupHeader tileInfo = new() + { + TileColumnCount = 1, + TileRowCount = 1 + }; + tileInfo.TileColumnStartModeInfo[1] = sequenceHeader.SuperblockModeInfoSize; + tileInfo.TileRowStartModeInfo[1] = sequenceHeader.SuperblockModeInfoSize; + ObuFrameHeader frameHeader = new() + { + ModeInfoColumnCount = sequenceHeader.SuperblockModeInfoSize, + ModeInfoRowCount = sequenceHeader.SuperblockModeInfoSize, + ModeInfoStride = sequenceHeader.SuperblockModeInfoSize, + TilesInfo = tileInfo, + DisableCdfUpdate = true, + DisableFrameEndUpdateCdf = true + }; + + using Av1SymbolWriter writer = new(Configuration.Default, 1, updateCdf: false); + Av1Distribution[] partitionTypes = Av1DefaultDistributions.PartitionTypes; + Av1BlockSize blockSize = sequenceHeader.SuperblockSize; + while (blockSize > Av1BlockSize.Block8x8) + { + int blockSizeLog = blockSize.Get4x4WidthLog2() - Av1BlockSize.Block8x8.Get4x4WidthLog2(); + int context = blockSizeLog * Av1Constants.PartitionProbabilitySet; + writer.WriteSymbol((int)Av1PartitionType.Split, partitionTypes[context]); + blockSize = Av1PartitionType.Split.GetBlockSubSize(blockSize); + } + + writer.WriteSymbol((int)Av1PartitionType.Horizontal, partitionTypes[0]); + using IMemoryOwner encoded = writer.Exit(); + using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader); + + Assert.Throws(() => tileReader.ReadTile(encoded.GetSpan(), 0)); + } + [Theory] [InlineData(TestImages.Heif.XnConvert, 0x010E, 0x03CC, 18, 16)] [InlineData(TestImages.Heif.Orange4x4, 0x010E, 0x001d, 21, 1)] @@ -190,8 +262,7 @@ public class Av1TilingTests Span modeInfos = superblockInfo.GetModeInfos(); Assert.Equal(superblockInfo.BlockCount, modeInfos.Length); - Assert.DoesNotContain(modeInfos.ToArray(), modeInfo => modeInfo is null); - Assert.Same(modeInfos[0], tileReader.FrameInfo.GetModeInfo(superblockPosition)); + Assert.Equal(modeInfos[0].ModeInfoIndex, tileReader.FrameInfo.GetModeInfo(superblockPosition).ModeInfoIndex); foreach (Av1BlockModeInfo modeInfo in modeInfos) { @@ -203,7 +274,9 @@ public class Av1TilingTests { for (int x = 0; x < modeInfo.BlockSize.Get4x4WideCount(); x++) { - Assert.Same(modeInfo, tileReader.FrameInfo.GetModeInfoAt(new Point(modeInfoPosition.X + x, modeInfoPosition.Y + y))); + Assert.Equal( + modeInfo.ModeInfoIndex, + tileReader.FrameInfo.GetModeInfoAt(new Point(modeInfoPosition.X + x, modeInfoPosition.Y + y)).ModeInfoIndex); } } } diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeWithPaletteMatchesPinnedLibavifPresentation_Rgba32_libavif-palette-draw-points-8b.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeWithPaletteMatchesRetainedPresentationReference_Rgba32_libavif-palette-draw-points-8b.png similarity index 100% rename from tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeWithPaletteMatchesPinnedLibavifPresentation_Rgba32_libavif-palette-draw-points-8b.png rename to tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeWithPaletteMatchesRetainedPresentationReference_Rgba32_libavif-palette-draw-points-8b.png diff --git a/tests/Images/Input/Heif/Av1/Conformance/README.md b/tests/Images/Input/Heif/Av1/Conformance/README.md index ecdc3f6d1..c8ae9c305 100644 --- a/tests/Images/Input/Heif/Av1/Conformance/README.md +++ b/tests/Images/Input/Heif/Av1/Conformance/README.md @@ -102,7 +102,33 @@ The film-grain Y4M SHA-256 is `A1B553BE140F48ABDDB2A6D39917AB714BA03AC7FFD6359EA The retained `quantizer-00` and `quantizer-63` streams are the minimum- and maximum-quantizer boundaries from libaom's official eight- and ten-bit test matrices. Their SHA-1 values are `C2E1EC9936B95254187A359E94AA32A9F3DAD1B7`, `2A8AA33513D8E01AE9410C4BF5FE1E471B775482`, `9BBE8499796AA588FF02E313FB0D4349940D2FEA`, and `8B6EB3FFF2E0DB7EAC775B08C745250CA591E2D9`, exactly matching `test/test-data.sha1` at pinned libaom commit `03087864cf4bea6abb0d28f95cf7843511413d8f`. Their SHA-256 values, in the same order, are `6382DBD2BEFBBC93D4EA283586F4FB43FEA5F1C52400E3D2C5281A46B1104C00`, `0E4EC80680F7AF8DE9621B016E0F2D7C0858B2951DEBC173DDA50C6A051547D3`, `FE6053CE4EE20A1C0EC6F7FE35DB097E92AD25D8A3505598BD89162C74D7944F`, and `39759AB77483E1D11049DC38B5F5262158FD9C3CBC9D1F82A02462FC5DF30E0C`. -The native references were generated with the pinned generic `aomdec --threads=1` build. Their SHA-256 values are `D499028E0606DB70CD56A72F151E04F36C09F300A448CCCD8430DD920D3589C5`, `4CC9892B3EE3399B293E31014B9F566C21E0C7A4765FC5F444528769C33E6D67`, `78373C28F401EB95D3E563D146622ED6C714ED96661E5E57C539CE71D7BED599`, and `A9DF86F671B8CF01EFC130660556412D4EBAF31A81D6F26FBDAEB0A7E839D8EA`. Each reference's two raw-frame MD5 values also match the corresponding official `.ivf.md5` file exactly. The tests compare every native sample under normal and scalar `FeatureTestRunner` dispatch and run all four sequences through a 2,560-byte row-aligned constrained tracked allocator. +The native references were originally generated with the historical generic `aomdec --threads=1` build. On 2026-08-31 current official libaom `main` at observed revision `441c439b9916474cac15d2822af47a9ad70674a8` reproduced all four references byte for byte. Their SHA-256 values are `D499028E0606DB70CD56A72F151E04F36C09F300A448CCCD8430DD920D3589C5`, `4CC9892B3EE3399B293E31014B9F566C21E0C7A4765FC5F444528769C33E6D67`, `78373C28F401EB95D3E563D146622ED6C714ED96661E5E57C539CE71D7BED599`, and `A9DF86F671B8CF01EFC130660556412D4EBAF31A81D6F26FBDAEB0A7E839D8EA`. Each reference's two raw-frame MD5 values also match the corresponding official `.ivf.md5` file exactly. The tests compare every native sample under normal and scalar `FeatureTestRunner` dispatch and run all four sequences through a 2,560-byte row-aligned constrained tracked allocator. + +## Palette reconstruction fixture + +The 42-byte `libaom-palette-draw-points-8b-444.bit` payload has SHA-256 +`F412A9E7F19D1C009D0329B993BB503D74CCDA58505C54BAE8FB3C16142181DC`. On 2026-08-31 +current official libaom `main` at observed revision `441c439b9916474cac15d2822af47a9ad70674a8` +decoded it with one thread, row threading disabled, raw output, and eight-bit output depth. The resulting +1,089-byte YUV444 output matches the retained native reference exactly at SHA-256 +`E05F7C0DF06ECCF0E43869D1D7B03DAA1D635ACD26A766F8940899BE18D53251`. + +The production tests require active luma and chroma palette syntax, compare every native sample under +`FeatureTestRunner`, and compare the final AVIF presentation through the established reference-output API. +The retained PNG has SHA-256 +`1148EBF6AA4B0F2D069D5E9B9605F6FB2A315E525F18016CDCAE23EFDD81DA84`. A 1 KiB +constrained tracked allocator forces both frame-owned palette map surfaces across multiple memory groups; +the test verifies exact reconstruction and exactly one return for every recorded allocation. + +## Official all-intra fixture + +On 2026-08-31 current official libaom `main` at observed revision +`441c439b9916474cac15d2822af47a9ad70674a8` reproduced the retained 39-frame all-intra Y4M byte for +byte. The IVF SHA-256 is `5FCD265FD9F9BDD0D3179340B4C4532F1422CA5E5D97741C7481B84CB5DC122F`; +the native reference SHA-256 is +`1211EBEFBC9CCEF9ED19BE4CCE3F807D69FFFE338E95CCA1B5F4CA8023482175`. The production test +decodes all 39 frames in one decoder session, compares every native sample exactly, and requires coverage +of every intra prediction mode and all seven transform types selected by the fixture. ## Official frame-size corner fixtures