Browse Source

Complete AV1 core reconstruction checkpoint

pull/2633/head
James Jackson-South 3 days ago
parent
commit
57a3f6668e
  1. 47
      HEIF_IMPLEMENTATION_PLAN.md
  2. 106
      src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolContextHelper.cs
  3. 21
      src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolDecoder.cs
  4. 8
      src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1FrameDecoder.cs
  5. 6
      src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs
  6. 10
      src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1DeQuantizationContext.cs
  7. 4
      src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1InverseQuantizer.cs
  8. 23
      src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PalettePredictor.Operator.cs
  9. 25
      src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs
  10. 23
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1BlockModeInfo.cs
  11. 10
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.MotionField.cs
  12. 81
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.cs
  13. 8
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameModeInfoMap.cs
  14. 27
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LevelBuffer.cs
  15. 12
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PartitionInfo.cs
  16. 183
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs
  17. 23
      tests/ImageSharp.Benchmarks/Codecs/Heif/Av1PalettePredictionBenchmarks.cs
  18. 69
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CoefficientsEntropyTests.cs
  19. 4
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameIntraEntropyTests.cs
  20. 17
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameModeInfoTests.cs
  21. 8
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InverseTransformTests.cs
  22. 6
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1LevelBufferTests.cs
  23. 5
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionModeInfoTests.cs
  24. 12
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PalettePredictorTests.cs
  25. 97
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs
  26. 35
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceMotionVectorsTests.cs
  27. 38
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceTransform.cs
  28. 32
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TemporalSegmentationTests.cs
  29. 79
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TilingTests.cs
  30. 0
      tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeWithPaletteMatchesRetainedPresentationReference_Rgba32_libavif-palette-draw-points-8b.png
  31. 28
      tests/Images/Input/Heif/Av1/Conformance/README.md

47
HEIF_IMPLEMENTATION_PLAN.md

@ -559,7 +559,7 @@ For every item:
Previously verified algorithm checkpoints remain valuable evidence, but the final decoder gate requires a fresh current-tree run after the inter and cleanup corrections. Previously verified algorithm checkpoints remain valuable evidence, but the final decoder gate requires a fresh current-tree run after the inter and cleanup corrections.
- [x] Bounded OBU framing, sequence headers, frame headers, tile groups, alignment, and trailing-bit parsing have been re-audited and verified against current libaom `main`. - [x] Bounded OBU framing, sequence headers, frame headers, tile groups, alignment, and trailing-bit parsing have been re-audited and verified against current libaom `main`.
- [~] Partition traversal, mode information, segmentation, delta quantization, transform-size selection, coefficient decoding, inverse quantization, and inverse transforms have historical checkpoint evidence against an obsolete pinned tree. Re-audit the current libaom `main` implementation before restoring verified status. - [x] Partition traversal, mode information, segmentation, delta quantization, transform-size selection, coefficient decoding, inverse quantization, and inverse transforms have been re-audited and verified against current libaom `main`.
- [x] Intra prediction covers directional, DC, smooth, Paeth, chroma-from-luma, filter-intra, and palette families with the established operator architecture. - [x] Intra prediction covers directional, DC, smooth, Paeth, chroma-from-luma, filter-intra, and palette families with the established operator architecture.
- [x] Intra-block copy has exact native reconstruction and feature-isolated SIMD evidence. - [x] Intra-block copy has exact native reconstruction and feature-isolated SIMD evidence.
- [x] Lossless inverse transform, loop filtering, CDEF, super-resolution, restoration, and film grain have focused checkpoint evidence. - [x] Lossless inverse transform, loop filtering, CDEF, super-resolution, restoration, and film grain have focused checkpoint evidence.
@ -604,6 +604,51 @@ Verified bounded-OBU checkpoint evidence on 2026-08-31:
`diff=lfs`, and `.gitattributes` was not edited. `diff=lfs`, and `.gitattributes` was not edited.
- [x] Release source builds pass for net10.0 and net11.0 with zero warnings and zero errors. Roslynk reports - [x] Release source builds pass for net10.0 and net11.0 with zero warnings and zero errors. Roslynk reports
zero compiler errors, and scoped production and test analyzer verification reports no changes. zero compiler errors, and scoped production and test analyzer verification reports no changes.
- [x] The completed checkpoint was committed as `243524c2c0b52a49d8d161fab806ab092cabe47c` with author
and committer `James Jackson-South <james_south@hotmail.com>`.
Verified partition, mode, segmentation, quantization, and transform checkpoint evidence on 2026-08-31:
- [x] Audited partition traversal and chroma representability against `read_partition` and the subsampled
plane-size rejection in current libaom `av1/decoder/decodeframe.c`; spatial segment-ID decoding and
corruption handling against `read_segment_id` in `av1/decoder/decodemv.c`; delta-Q syntax, resolution,
arithmetic, and clamping against `read_delta_qindex` and `read_delta_q_params` in the same file.
- [x] Audited selected and variable transform-size traversal against `read_tx_size`, `read_tx_size_vartx`,
and transform-block traversal in `av1/decoder/decodeframe.c`; coefficient syntax and arithmetic against
`av1_read_coeffs_txb` in `av1/decoder/decodetxb.c`; inverse quantization and transform application against
current `av1/decoder/decodeframe.c`, `av1/common/idct.c`, and the current libaom transform test oracle.
The observed clean `HEAD` and `origin/main` revision was
`441c439b9916474cac15d2822af47a9ad70674a8`; this is verification evidence, not a pin.
- [x] Partition decoding now rejects an invalid partition subsize and a block size that cannot represent the
current subsampled chroma plane. Spatial segmentation rejects decoded IDs above the active segment range.
Focused tests exercise both current-libaom corruption boundaries through the production tile reader.
- [x] Coefficient entropy decoding uses one allocator-owned maximum-size `Av1LevelBuffer` per tile reader.
Each transform resets and clears only its active padded geometry, so no transform creates an allocation.
Allocation tracking over all eight minimum- and maximum-quantizer frames proves exactly one coefficient
scratch allocation per frame and exactly-once return after decoder disposal.
- [x] Palette index maps are allocator-backed frame surfaces addressed row by row through `Buffer2DRegion`.
The wavefront context, stable neighbor ordering, right/bottom padding, transform offsets, and prediction
were audited against current libaom `av1/decoder/detokenize.c`, `av1/common/entropymode.c`,
`av1/decoder/decodeframe.c`, and `av1/common/reconintra.c`. A 1 KiB constrained allocator forces both
luma and chroma maps across multiple memory groups without copies or per-block allocations and proves
exactly-once disposal.
- [x] `Av1BlockModeInfo` is value storage, removing the managed object allocation formerly created for every
decoded coding block. Explicit `ModeInfoIndex` values preserve libaom's mode-info identity semantics at
prediction-unit loop-filter edges, and the frame map now uses integer offsets so more than 65,535 decoded
blocks cannot wrap its lookup identity.
- [x] Current official libaom reproduced the 39-frame all-intra reference and all four 8/10-bit minimum- and
maximum-quantizer references byte for byte. The production tests compare every native sample exactly,
cover every intra mode and seven selected transform types, execute SIMD and scalar paths through
`FeatureTestRunner`, and exercise the quantizer sequences under constrained tracked allocation.
- [x] Current official libaom decoded the 42-byte palette payload into the retained 1,089-byte YUV444
reference at SHA-256 `E05F7C0DF06ECCF0E43869D1D7B03DAA1D635ACD26A766F8940899BE18D53251`.
The exact native test requires luma and chroma palette syntax. The established reference-output test uses
the unchanged presentation PNG at SHA-256
`1148EBF6AA4B0F2D069D5E9B9605F6FB2A315E525F18016CDCAE23EFDD81DA84`, whose renamed path still
resolves to `diff=lfs`; `.gitattributes` was not edited.
- [x] The exact final AV1 namespace passes 8,732 of 8,732 cases on net10.0 and 8,732 of 8,732 cases on
net11.0, with zero failures or skips. Release source builds pass for net10.0 and net11.0 with zero warnings
and zero errors. Roslynk reports zero compiler errors, and scoped analyzer verification reports no changes.
Decoder exit gate: Decoder exit gate:

106
src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolContextHelper.cs

@ -200,17 +200,19 @@ internal static class Av1SymbolContextHelper
int aboveContext = 0; int aboveContext = 0;
if (above is not null) if (above is not null)
{ {
aboveContext = above.ReferenceFrames[1] > Av1ReferenceFrameType.Intra Av1BlockModeInfo aboveModeInfo = above.Value;
? above.CompoundGroupIndex ? 1 : 0 aboveContext = aboveModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra
: above.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 3 : 0; ? aboveModeInfo.CompoundGroupIndex ? 1 : 0
: aboveModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 3 : 0;
} }
int leftContext = 0; int leftContext = 0;
if (left is not null) if (left is not null)
{ {
leftContext = left.ReferenceFrames[1] > Av1ReferenceFrameType.Intra Av1BlockModeInfo leftModeInfo = left.Value;
? left.CompoundGroupIndex ? 1 : 0 leftContext = leftModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra
: left.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 3 : 0; ? leftModeInfo.CompoundGroupIndex ? 1 : 0
: leftModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 3 : 0;
} }
return Math.Min(5, aboveContext + leftContext); return Math.Min(5, aboveContext + leftContext);
@ -244,17 +246,19 @@ internal static class Av1SymbolContextHelper
int aboveContext = 0; int aboveContext = 0;
if (above is not null) if (above is not null)
{ {
aboveContext = above.ReferenceFrames[1] > Av1ReferenceFrameType.Intra Av1BlockModeInfo aboveModeInfo = above.Value;
? above.CompoundIndex ? 1 : 0 aboveContext = aboveModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra
: above.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 1 : 0; ? aboveModeInfo.CompoundIndex ? 1 : 0
: aboveModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 1 : 0;
} }
int leftContext = 0; int leftContext = 0;
if (left is not null) if (left is not null)
{ {
leftContext = left.ReferenceFrames[1] > Av1ReferenceFrameType.Intra Av1BlockModeInfo leftModeInfo = left.Value;
? left.CompoundIndex ? 1 : 0 leftContext = leftModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra
: left.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 1 : 0; ? leftModeInfo.CompoundIndex ? 1 : 0
: leftModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 1 : 0;
} }
return aboveContext + leftContext + (forwardDistance == backwardDistance ? 3 : 0); return aboveContext + leftContext + (forwardDistance == backwardDistance ? 3 : 0);
@ -703,8 +707,10 @@ internal static class Av1SymbolContextHelper
{ {
if (above is not null && left is not null) if (above is not null && left is not null)
{ {
bool aboveIsIntra = above.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; Av1BlockModeInfo aboveModeInfo = above.Value;
bool leftIsIntra = left.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; Av1BlockModeInfo leftModeInfo = left.Value;
bool aboveIsIntra = aboveModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra;
bool leftIsIntra = leftModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra;
// AV1 reserves context three for two intra neighbors, context one for a mixed pair, and context zero for // AV1 reserves context three for two intra neighbors, context one for a mixed pair, and context zero for
// two inter neighbors. These values directly index intra_inter_cdf and are not probability ranks. // two inter neighbors. These values directly index intra_inter_cdf and are not probability ranks.
@ -720,12 +726,12 @@ internal static class Av1SymbolContextHelper
// context zero, matching the unavailable-neighbor behavior in libaom's av1_get_intra_inter_context. // context zero, matching the unavailable-neighbor behavior in libaom's av1_get_intra_inter_context.
if (above is not null) if (above is not null)
{ {
return above.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra ? 2 : 0; return above.Value.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra ? 2 : 0;
} }
if (left is not null) if (left is not null)
{ {
return left.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra ? 2 : 0; return left.Value.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra ? 2 : 0;
} }
return 0; return 0;
@ -743,29 +749,31 @@ internal static class Av1SymbolContextHelper
// their forward/backward direction, while intra neighbors take the same branch as a non-forward reference. // their forward/backward direction, while intra neighbors take the same branch as a non-forward reference.
if (above is not null && left is not null) if (above is not null && left is not null)
{ {
bool aboveIsCompound = above.ReferenceFrames[1] > Av1ReferenceFrameType.Intra; Av1BlockModeInfo aboveModeInfo = above.Value;
bool leftIsCompound = left.ReferenceFrames[1] > Av1ReferenceFrameType.Intra; Av1BlockModeInfo leftModeInfo = left.Value;
bool aboveIsCompound = aboveModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra;
bool leftIsCompound = leftModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra;
if (!aboveIsCompound && !leftIsCompound) if (!aboveIsCompound && !leftIsCompound)
{ {
bool aboveIsBackward = above.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; bool aboveIsBackward = aboveModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward;
bool leftIsBackward = left.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; bool leftIsBackward = leftModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward;
return aboveIsBackward == leftIsBackward ? 0 : 1; return aboveIsBackward == leftIsBackward ? 0 : 1;
} }
if (!aboveIsCompound) if (!aboveIsCompound)
{ {
bool aboveIsBackward = above.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; bool aboveIsBackward = aboveModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward;
bool aboveIsIntra = above.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; bool aboveIsIntra = aboveModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra;
return 2 + (aboveIsBackward || aboveIsIntra ? 1 : 0); return 2 + (aboveIsBackward || aboveIsIntra ? 1 : 0);
} }
if (!leftIsCompound) if (!leftIsCompound)
{ {
bool leftIsBackward = left.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward; bool leftIsBackward = leftModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward;
bool leftIsIntra = left.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra; bool leftIsIntra = leftModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra;
return 2 + (leftIsBackward || leftIsIntra ? 1 : 0); return 2 + (leftIsBackward || leftIsIntra ? 1 : 0);
} }
@ -777,14 +785,15 @@ internal static class Av1SymbolContextHelper
if (neighbor is not null) if (neighbor is not null)
{ {
bool isCompound = neighbor.ReferenceFrames[1] > Av1ReferenceFrameType.Intra; Av1BlockModeInfo neighborModeInfo = neighbor.Value;
bool isCompound = neighborModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra;
if (isCompound) if (isCompound)
{ {
return 3; return 3;
} }
return neighbor.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward ? 1 : 0; return neighborModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward ? 1 : 0;
} }
// With no spatial votes, AV1 uses the neutral single-versus-compound context rather than context zero. // With no spatial votes, AV1 uses the neutral single-versus-compound context rather than context zero.
@ -801,8 +810,10 @@ internal static class Av1SymbolContextHelper
{ {
if (above is not null && left is not null) if (above is not null && left is not null)
{ {
bool aboveIntra = !IsInterBlock(above); Av1BlockModeInfo aboveModeInfo = above.Value;
bool leftIntra = !IsInterBlock(left); Av1BlockModeInfo leftModeInfo = left.Value;
bool aboveIntra = !IsInterBlock(aboveModeInfo);
bool leftIntra = !IsInterBlock(leftModeInfo);
if (aboveIntra && leftIntra) if (aboveIntra && leftIntra)
{ {
return 2; return 2;
@ -810,14 +821,14 @@ internal static class Av1SymbolContextHelper
if (aboveIntra || leftIntra) if (aboveIntra || leftIntra)
{ {
Av1BlockModeInfo inter = aboveIntra ? left : above; Av1BlockModeInfo inter = aboveIntra ? leftModeInfo : aboveModeInfo;
return HasCompoundReference(inter) ? 1 + (2 * (HasUnidirectionalCompoundReferences(inter) ? 1 : 0)) : 2; return HasCompoundReference(inter) ? 1 + (2 * (HasUnidirectionalCompoundReferences(inter) ? 1 : 0)) : 2;
} }
bool aboveSingle = !HasCompoundReference(above); bool aboveSingle = !HasCompoundReference(aboveModeInfo);
bool leftSingle = !HasCompoundReference(left); bool leftSingle = !HasCompoundReference(leftModeInfo);
Av1ReferenceFrameType abovePrimary = above.ReferenceFrames[0]; Av1ReferenceFrameType abovePrimary = aboveModeInfo.ReferenceFrames[0];
Av1ReferenceFrameType leftPrimary = left.ReferenceFrames[0]; Av1ReferenceFrameType leftPrimary = leftModeInfo.ReferenceFrames[0];
if (aboveSingle && leftSingle) if (aboveSingle && leftSingle)
{ {
return 1 + (2 * (IsBackwardReference(abovePrimary) == IsBackwardReference(leftPrimary) ? 1 : 0)); return 1 + (2 * (IsBackwardReference(abovePrimary) == IsBackwardReference(leftPrimary) ? 1 : 0));
@ -825,7 +836,7 @@ internal static class Av1SymbolContextHelper
if (aboveSingle || leftSingle) if (aboveSingle || leftSingle)
{ {
Av1BlockModeInfo compound = aboveSingle ? left : above; Av1BlockModeInfo compound = aboveSingle ? leftModeInfo : aboveModeInfo;
if (!HasUnidirectionalCompoundReferences(compound)) if (!HasUnidirectionalCompoundReferences(compound))
{ {
return 1; return 1;
@ -834,8 +845,8 @@ internal static class Av1SymbolContextHelper
return 3 + (IsBackwardReference(abovePrimary) == IsBackwardReference(leftPrimary) ? 1 : 0); return 3 + (IsBackwardReference(abovePrimary) == IsBackwardReference(leftPrimary) ? 1 : 0);
} }
bool aboveUnidirectional = HasUnidirectionalCompoundReferences(above); bool aboveUnidirectional = HasUnidirectionalCompoundReferences(aboveModeInfo);
bool leftUnidirectional = HasUnidirectionalCompoundReferences(left); bool leftUnidirectional = HasUnidirectionalCompoundReferences(leftModeInfo);
if (!aboveUnidirectional && !leftUnidirectional) if (!aboveUnidirectional && !leftUnidirectional)
{ {
return 0; return 0;
@ -850,12 +861,18 @@ internal static class Av1SymbolContextHelper
} }
Av1BlockModeInfo? edge = above ?? left; Av1BlockModeInfo? edge = above ?? left;
if (edge is null || !IsInterBlock(edge) || !HasCompoundReference(edge)) if (edge is null)
{ {
return 2; return 2;
} }
return HasUnidirectionalCompoundReferences(edge) ? 4 : 0; Av1BlockModeInfo edgeModeInfo = edge.Value;
if (!IsInterBlock(edgeModeInfo) || !HasCompoundReference(edgeModeInfo))
{
return 2;
}
return HasUnidirectionalCompoundReferences(edgeModeInfo) ? 4 : 0;
} }
/// <summary> /// <summary>
@ -984,12 +1001,12 @@ internal static class Av1SymbolContextHelper
if (above is not null) if (above is not null)
{ {
AddNeighborReferenceCounts(above, referenceCounts); AddNeighborReferenceCounts(above.Value, referenceCounts);
} }
if (left is not null) if (left is not null)
{ {
AddNeighborReferenceCounts(left, referenceCounts); AddNeighborReferenceCounts(left.Value, referenceCounts);
} }
} }
@ -1144,8 +1161,8 @@ internal static class Av1SymbolContextHelper
/// <returns>The context in the inclusive range zero through two.</returns> /// <returns>The context in the inclusive range zero through two.</returns>
public static int GetSegmentIdPredictedContext(Av1BlockModeInfo? aboveModeInfo, Av1BlockModeInfo? leftModeInfo) public static int GetSegmentIdPredictedContext(Av1BlockModeInfo? aboveModeInfo, Av1BlockModeInfo? leftModeInfo)
{ {
int abovePredicted = aboveModeInfo is not null && aboveModeInfo.SegmentIdPredicted ? 1 : 0; int abovePredicted = aboveModeInfo is not null && aboveModeInfo.Value.SegmentIdPredicted ? 1 : 0;
int leftPredicted = leftModeInfo is not null && leftModeInfo.SegmentIdPredicted ? 1 : 0; int leftPredicted = leftModeInfo is not null && leftModeInfo.Value.SegmentIdPredicted ? 1 : 0;
return abovePredicted + leftPredicted; return abovePredicted + leftPredicted;
} }
@ -1306,7 +1323,8 @@ internal static class Av1SymbolContextHelper
return SwitchableInterpolationFilterCount; return SwitchableInterpolationFilterCount;
} }
ReadOnlySpan<Av1ReferenceFrameType> referenceFrames = modeInfo.ReferenceFrames; Av1BlockModeInfo neighborModeInfo = modeInfo.Value;
ReadOnlySpan<Av1ReferenceFrameType> referenceFrames = neighborModeInfo.ReferenceFrames;
// A compound neighbor contributes when either of its references matches the current primary reference. // A compound neighbor contributes when either of its references matches the current primary reference.
if (referenceFrames[0] != referenceFrame && referenceFrames[1] != referenceFrame) if (referenceFrames[0] != referenceFrame && referenceFrames[1] != referenceFrame)
@ -1314,6 +1332,6 @@ internal static class Av1SymbolContextHelper
return SwitchableInterpolationFilterCount; return SwitchableInterpolationFilterCount;
} }
return (int)modeInfo.InterpolationFilters[direction]; return (int)neighborModeInfo.InterpolationFilters[direction];
} }
} }

21
src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolDecoder.cs

@ -20,11 +20,6 @@ internal ref struct Av1SymbolDecoder
/// </summary> /// </summary>
private readonly Av1FrameEntropyContext context; private readonly Av1FrameEntropyContext context;
/// <summary>
/// The configuration providing temporary coefficient-context memory.
/// </summary>
private readonly Configuration configuration;
/// <summary> /// <summary>
/// The range decoder over the current tile payload. /// The range decoder over the current tile payload.
/// </summary> /// </summary>
@ -59,7 +54,6 @@ internal ref struct Av1SymbolDecoder
// The context owner controls reset and publication. Holding one reference here keeps the range decoder small // The context owner controls reset and publication. Holding one reference here keeps the range decoder small
// and prevents a second set of aliases from becoming a competing source of entropy state. // and prevents a second set of aliases from becoming a competing source of entropy state.
this.context = context; this.context = context;
this.configuration = configuration;
this.reader = new Av1SymbolReader(tileData, updateCdf); this.reader = new Av1SymbolReader(tileData, updateCdf);
} }
@ -345,15 +339,15 @@ internal ref struct Av1SymbolDecoder
{ {
ref Av1SymbolReader r = ref this.reader; ref Av1SymbolReader r = ref this.reader;
Av1PredictionMode aboveMode = Av1PredictionMode.DC; Av1PredictionMode aboveMode = Av1PredictionMode.DC;
if (aboveModeInfo != null) if (aboveModeInfo is not null)
{ {
aboveMode = aboveModeInfo.YMode; aboveMode = aboveModeInfo.Value.YMode;
} }
Av1PredictionMode leftMode = Av1PredictionMode.DC; Av1PredictionMode leftMode = Av1PredictionMode.DC;
if (leftModeInfo != null) if (leftModeInfo is not null)
{ {
leftMode = leftModeInfo.YMode; leftMode = leftModeInfo.Value.YMode;
} }
int aboveContext = IntraModeContext[(int)aboveMode]; int aboveContext = IntraModeContext[(int)aboveMode];
@ -971,6 +965,7 @@ internal ref struct Av1SymbolDecoder
/// <param name="transformInfo">The transform descriptor updated with the decoded type and coded-block flag.</param> /// <param name="transformInfo">The transform descriptor updated with the decoded type and coded-block flag.</param>
/// <param name="modeBlocksToRightEdge">The signed distance from the mode block to the right frame edge.</param> /// <param name="modeBlocksToRightEdge">The signed distance from the mode block to the right frame edge.</param>
/// <param name="modeBlocksToBottomEdge">The signed distance from the mode block to the bottom frame edge.</param> /// <param name="modeBlocksToBottomEdge">The signed distance from the mode block to the bottom frame edge.</param>
/// <param name="levels">Reusable padded coefficient-context storage owned by the tile reader.</param>
/// <param name="coefficientBuffer">The destination receiving the coefficient count followed by scan-ordered signed levels.</param> /// <param name="coefficientBuffer">The destination receiving the coefficient count followed by scan-ordered signed levels.</param>
/// <returns>The one-based end-of-block position, or zero for an empty transform block.</returns> /// <returns>The one-based end-of-block position, or zero for an empty transform block.</returns>
public int ReadCoefficients( public int ReadCoefficients(
@ -991,6 +986,7 @@ internal ref struct Av1SymbolDecoder
ref Av1TransformInfo transformInfo, ref Av1TransformInfo transformInfo,
int modeBlocksToRightEdge, int modeBlocksToRightEdge,
int modeBlocksToBottomEdge, int modeBlocksToBottomEdge,
Av1LevelBuffer levels,
Span<int> coefficientBuffer) Span<int> coefficientBuffer)
{ {
Av1TransformSize adjustedTransformSize = transformSize.GetAdjusted(); Av1TransformSize adjustedTransformSize = transformSize.GetAdjusted();
@ -1000,8 +996,9 @@ internal ref struct Av1SymbolDecoder
Av1PlaneType planeType = (Av1PlaneType)Math.Min(plane, 1); Av1PlaneType planeType = (Av1PlaneType)Math.Min(plane, 1);
int culLevel = 0; int culLevel = 0;
// AV1 omits high-frequency coefficients beyond 32 samples on every 64-point transform dimension. // AV1 omits high-frequency coefficients beyond 32 samples on every 64-point transform dimension. Reusing
using Av1LevelBuffer levels = new(this.configuration, new Size(width, height)); // tile-owned storage avoids an allocator round trip for every transform block.
levels.Reset(new Size(width, height));
bool allZero = this.ReadTransformBlockSkip(transformSizeContext, transformBlockContext.SkipContext); bool allZero = this.ReadTransformBlockSkip(transformSizeContext, transformBlockContext.SkipContext);
int endOfBlock; int endOfBlock;

8
src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1FrameDecoder.cs

@ -160,7 +160,7 @@ internal sealed class Av1FrameDecoder : IAv1FrameDecoder, IDisposable
/// Reconstructs every tile row in one tile column. /// Reconstructs every tile row in one tile column.
/// </summary> /// </summary>
/// <param name="tileColumn">The zero-based tile-column index.</param> /// <param name="tileColumn">The zero-based tile-column index.</param>
/// <remarks>SVT-AV1: <c>decode_tile</c>.</remarks> /// <remarks>Follows libaom's single-threaded <c>decode_tiles</c> ordering.</remarks>
private void DecodeFrameTiles(int tileColumn) private void DecodeFrameTiles(int tileColumn)
{ {
ObuTileGroupHeader tileInfo = this.frameHeader.TilesInfo; ObuTileGroupHeader tileInfo = this.frameHeader.TilesInfo;
@ -187,7 +187,7 @@ internal sealed class Av1FrameDecoder : IAv1FrameDecoder, IDisposable
/// <param name="tileColumn">The zero-based tile-column index.</param> /// <param name="tileColumn">The zero-based tile-column index.</param>
/// <param name="modeInfoRow">The frame-relative row in 4x4 mode-info units.</param> /// <param name="modeInfoRow">The frame-relative row in 4x4 mode-info units.</param>
/// <param name="superblockRow">The frame-relative superblock row.</param> /// <param name="superblockRow">The frame-relative superblock row.</param>
/// <remarks>SVT-AV1: <c>decode_tile_row</c>.</remarks> /// <remarks>Corresponds to the superblock-row traversal in libaom's <c>decode_tile</c>.</remarks>
private void DecodeTileSuperblockRow(int tileRow, int tileColumn, int modeInfoRow, int superblockRow) private void DecodeTileSuperblockRow(int tileRow, int tileColumn, int modeInfoRow, int superblockRow)
{ {
ObuTileGroupHeader tileInfo = this.frameHeader.TilesInfo; ObuTileGroupHeader tileInfo = this.frameHeader.TilesInfo;
@ -210,7 +210,7 @@ internal sealed class Av1FrameDecoder : IAv1FrameDecoder, IDisposable
/// <param name="modeInfoPosition">The superblock's top-left position in 4x4 mode-info units.</param> /// <param name="modeInfoPosition">The superblock's top-left position in 4x4 mode-info units.</param>
/// <param name="superblockInfo">The decoded syntax and block modes for the superblock.</param> /// <param name="superblockInfo">The decoded syntax and block modes for the superblock.</param>
/// <param name="tileInfo">The tile that contains the superblock.</param> /// <param name="tileInfo">The tile that contains the superblock.</param>
/// <remarks>SVT-AV1: <c>svt_aom_decode_super_block</c>.</remarks> /// <remarks>Corresponds to libaom's superblock decode boundary.</remarks>
public void DecodeSuperblock(Point modeInfoPosition, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo) public void DecodeSuperblock(Point modeInfoPosition, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo)
{ {
this.blockDecoder.UpdateSuperblock(superblockInfo); this.blockDecoder.UpdateSuperblock(superblockInfo);
@ -224,7 +224,7 @@ internal sealed class Av1FrameDecoder : IAv1FrameDecoder, IDisposable
/// <param name="modeInfoPosition">The superblock's frame-relative origin in 4x4 mode-info units.</param> /// <param name="modeInfoPosition">The superblock's frame-relative origin in 4x4 mode-info units.</param>
/// <param name="superblockInfo">The superblock whose block modes are traversed.</param> /// <param name="superblockInfo">The superblock whose block modes are traversed.</param>
/// <param name="tileInfo">The tile boundary information used by intra prediction.</param> /// <param name="tileInfo">The tile boundary information used by intra prediction.</param>
/// <remarks>SVT-AV1: <c>decode_partition</c>.</remarks> /// <remarks>Replays the depth-first block order produced by libaom's <c>decode_partition</c>.</remarks>
private void DecodePartition(Point modeInfoPosition, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo) private void DecodePartition(Point modeInfoPosition, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo)
{ {
foreach (Av1BlockModeInfo modeInfo in superblockInfo.GetModeInfos()) foreach (Av1BlockModeInfo modeInfo in superblockInfo.GetModeInfos())

6
src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs

@ -237,9 +237,9 @@ internal class Av1LoopFilterDecoder
bool currentSkippedTransform = modeInfo.Skip && modeInfo.ReferenceFrames[0] > Av1ReferenceFrameType.Intra; bool currentSkippedTransform = modeInfo.Skip && modeInfo.ReferenceFrames[0] > Av1ReferenceFrameType.Intra;
bool previousSkippedTransform = previousModeInfo.Skip && previousModeInfo.ReferenceFrames[0] > Av1ReferenceFrameType.Intra; bool previousSkippedTransform = previousModeInfo.Skip && previousModeInfo.ReferenceFrames[0] > Av1ReferenceFrameType.Intra;
// The mode-info map stores one object for every covered position, so object identity is the exact equivalent // Every covered 4x4 position carries the owning block's storage index. Comparing those indices is the value-type
// of libaom's current-versus-previous MB_MODE_INFO pointer comparison at a prediction-unit boundary. // equivalent of libaom's current-versus-previous MB_MODE_INFO pointer comparison at a prediction-unit boundary.
bool isBlockEdge = !ReferenceEquals(modeInfo, previousModeInfo); bool isBlockEdge = modeInfo.ModeInfoIndex != previousModeInfo.ModeInfoIndex;
bool applyFilter = isTransformEdge && (isBlockEdge || !currentSkippedTransform || !previousSkippedTransform); bool applyFilter = isTransformEdge && (isBlockEdge || !currentSkippedTransform || !previousSkippedTransform);
if (!applyFilter) if (!applyFilter)
{ {

10
src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1DeQuantizationContext.cs

@ -26,7 +26,7 @@ internal class Av1DeQuantizationContext
/// </summary> /// </summary>
/// <param name="sequenceHeader">The sequence header that supplies the coded bit depth.</param> /// <param name="sequenceHeader">The sequence header that supplies the coded bit depth.</param>
/// <param name="frameHeader">The frame header that supplies segmentation and quantization parameters.</param> /// <param name="frameHeader">The frame header that supplies segmentation and quantization parameters.</param>
/// <remarks>SVT-AV1: <c>svt_aom_setup_segmentation_dequant</c>.</remarks> /// <remarks>Corresponds to <c>setup_segmentation_dequant</c> in libaom.</remarks>
public Av1DeQuantizationContext(ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader) public Av1DeQuantizationContext(ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader)
{ {
Av1BitDepth bitDepth = sequenceHeader.ColorConfig.BitDepth; Av1BitDepth bitDepth = sequenceHeader.ColorConfig.BitDepth;
@ -40,11 +40,11 @@ internal class Av1DeQuantizationContext
for (int plane = 0; plane < Av1Constants.MaxPlanes; plane++) for (int plane = 0; plane < Av1Constants.MaxPlanes; plane++)
{ {
int dc_delta_q = frameHeader.QuantizationParameters.DeltaQDc[plane]; int dcDeltaQ = frameHeader.QuantizationParameters.DeltaQDc[plane];
int ac_delta_q = frameHeader.QuantizationParameters.DeltaQAc[plane]; int acDeltaQ = frameHeader.QuantizationParameters.DeltaQAc[plane];
this.dcContent[segmentId][plane] = Av1QuantizationLookup.GetDcQuant(qindex, dc_delta_q, bitDepth); this.dcContent[segmentId][plane] = Av1QuantizationLookup.GetDcQuant(qindex, dcDeltaQ, bitDepth);
this.acContent[segmentId][plane] = Av1QuantizationLookup.GetAcQuant(qindex, ac_delta_q, bitDepth); this.acContent[segmentId][plane] = Av1QuantizationLookup.GetAcQuant(qindex, acDeltaQ, bitDepth);
} }
} }
} }

4
src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1InverseQuantizer.cs

@ -80,7 +80,7 @@ internal class Av1InverseQuantizer
/// <param name="transformSize">The transform dimensions and scale.</param> /// <param name="transformSize">The transform dimensions and scale.</param>
/// <param name="plane">The color plane whose quantizer and matrix are used.</param> /// <param name="plane">The color plane whose quantizer and matrix are used.</param>
/// <returns>The number of coefficient levels consumed.</returns> /// <returns>The number of coefficient levels consumed.</returns>
/// <remarks>SVT-AV1: <c>svt_aom_inverse_quantize</c>.</remarks> /// <remarks>Matches the coefficient dequantization arithmetic in libaom's <c>read_coeffs_txb</c>.</remarks>
public int InverseQuantize(Av1BlockModeInfo mode, Span<int> level, Span<int> qCoefficients, Av1TransformType transformType, Av1TransformSize transformSize, Av1Plane plane) public int InverseQuantize(Av1BlockModeInfo mode, Span<int> level, Span<int> qCoefficients, Av1TransformType transformType, Av1TransformSize transformSize, Av1Plane plane)
{ {
Guard.NotNull(this.deQuantsDeltaQ); Guard.NotNull(this.deQuantsDeltaQ);
@ -159,7 +159,7 @@ internal class Av1InverseQuantizer
/// <param name="coefficientIndex">The raster coefficient index into the inverse matrix.</param> /// <param name="coefficientIndex">The raster coefficient index into the inverse matrix.</param>
/// <param name="iqMatrix">The inverse quantization matrix for the current level, plane, and transform size.</param> /// <param name="iqMatrix">The inverse quantization matrix for the current level, plane, and transform size.</param>
/// <returns>The matrix-weighted dequantization value.</returns> /// <returns>The matrix-weighted dequantization value.</returns>
/// <remarks>SVT-AV1: <c>get_dqv</c>.</remarks> /// <remarks>Corresponds to <c>get_dqv</c> in libaom.</remarks>
private static int GetDeQuantizedValue(short dequant, int coefficientIndex, ReadOnlySpan<int> iqMatrix) private static int GetDeQuantizedValue(short dequant, int coefficientIndex, ReadOnlySpan<int> iqMatrix)
{ {
// Matrix elements use fixed-point precision; adding half a unit produces nearest-integer rounding on shift. // Matrix elements use fixed-point precision; adding half a unit produces nearest-integer rounding on shift.

23
src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PalettePredictor.Operator.cs

@ -4,6 +4,7 @@
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using System.Runtime.InteropServices; using System.Runtime.InteropServices;
using System.Runtime.Intrinsics; using System.Runtime.Intrinsics;
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
@ -97,26 +98,24 @@ internal static class Av1PalettePredictor
/// </summary> /// </summary>
public static void Predict( public static void Predict(
ReadOnlySpan<ushort> paletteColors, ReadOnlySpan<ushort> paletteColors,
ReadOnlySpan<byte> colorIndexMap, Buffer2DRegion<byte> colorIndexMap,
int colorIndexMapStride,
Span<byte> destination, Span<byte> destination,
int destinationStride, int destinationStride,
int width, int width,
int height) int height)
=> Predictor<PaletteOperator>.Predict(paletteColors, colorIndexMap, colorIndexMapStride, destination, destinationStride, width, height); => Predictor<PaletteOperator>.Predict(paletteColors, colorIndexMap, destination, destinationStride, width, height);
/// <summary> /// <summary>
/// Reconstructs a high-bit-depth palette-predicted block. /// Reconstructs a high-bit-depth palette-predicted block.
/// </summary> /// </summary>
public static void Predict( public static void Predict(
ReadOnlySpan<ushort> paletteColors, ReadOnlySpan<ushort> paletteColors,
ReadOnlySpan<byte> colorIndexMap, Buffer2DRegion<byte> colorIndexMap,
int colorIndexMapStride,
Span<short> destination, Span<short> destination,
int destinationStride, int destinationStride,
int width, int width,
int height) int height)
=> Predictor<PaletteOperator>.Predict(paletteColors, colorIndexMap, colorIndexMapStride, destination, destinationStride, width, height); => Predictor<PaletteOperator>.Predict(paletteColors, colorIndexMap, destination, destinationStride, width, height);
/// <summary> /// <summary>
/// Maps decoded palette indices to reconstructed samples. /// Maps decoded palette indices to reconstructed samples.
@ -186,14 +185,12 @@ internal static class Av1PalettePredictor
/// </summary> /// </summary>
public static void Predict( public static void Predict(
ReadOnlySpan<ushort> paletteColors, ReadOnlySpan<ushort> paletteColors,
ReadOnlySpan<byte> colorIndexMap, Buffer2DRegion<byte> colorIndexMap,
int colorIndexMapStride,
Span<byte> destination, Span<byte> destination,
int destinationStride, int destinationStride,
int width, int width,
int height) int height)
{ {
ref byte mapBase = ref MemoryMarshal.GetReference(colorIndexMap);
ref byte destinationBase = ref MemoryMarshal.GetReference(destination); ref byte destinationBase = ref MemoryMarshal.GetReference(destination);
// AV1 palettes contain at most eight colors. Repeating all eight entries in every 128-bit lane keeps native // AV1 palettes contain at most eight colors. Repeating all eight entries in every 128-bit lane keeps native
@ -209,7 +206,7 @@ internal static class Av1PalettePredictor
for (int row = 0; row < height; row++) for (int row = 0; row < height; row++)
{ {
ref byte mapRow = ref Unsafe.Add(ref mapBase, row * colorIndexMapStride); ref byte mapRow = ref MemoryMarshal.GetReference(colorIndexMap.DangerousGetRowSpan(row));
ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride);
int column = 0; int column = 0;
@ -278,14 +275,12 @@ internal static class Av1PalettePredictor
/// </summary> /// </summary>
public static void Predict( public static void Predict(
ReadOnlySpan<ushort> paletteColors, ReadOnlySpan<ushort> paletteColors,
ReadOnlySpan<byte> colorIndexMap, Buffer2DRegion<byte> colorIndexMap,
int colorIndexMapStride,
Span<short> destination, Span<short> destination,
int destinationStride, int destinationStride,
int width, int width,
int height) int height)
{ {
ref byte mapBase = ref MemoryMarshal.GetReference(colorIndexMap);
ref short destinationBase = ref MemoryMarshal.GetReference(destination); ref short destinationBase = ref MemoryMarshal.GetReference(destination);
InlineArray8<ushort> paletteStorage = default; InlineArray8<ushort> paletteStorage = default;
paletteColors.CopyTo(paletteStorage); paletteColors.CopyTo(paletteStorage);
@ -295,7 +290,7 @@ internal static class Av1PalettePredictor
for (int row = 0; row < height; row++) for (int row = 0; row < height; row++)
{ {
ref byte mapRow = ref Unsafe.Add(ref mapBase, row * colorIndexMapStride); ref byte mapRow = ref MemoryMarshal.GetReference(colorIndexMap.DangerousGetRowSpan(row));
ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride);
int column = 0; int column = 0;

25
src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs

@ -10,6 +10,7 @@ using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma;
using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; using SixLabors.ImageSharp.Formats.Heif.Av1.Transform;
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
@ -462,22 +463,24 @@ internal class Av1PredictionDecoder
if (usePalette) if (usePalette)
{ {
ReadOnlySpan<ushort> paletteColors = modeInfo.GetPaletteColors(plane); ReadOnlySpan<ushort> paletteColors = modeInfo.GetPaletteColors(plane);
ReadOnlySpan<byte> colorIndexMap = modeInfo.GetPaletteColorIndexMap(plane); Buffer2DRegion<byte> colorIndexMap = modeInfo.GetPaletteColorIndexMap(plane);
int paletteStride = partitionInfo.GetWidthInPixels(plane); Buffer2DRegion<byte> transformColorIndexMap = colorIndexMap.GetSubRegion(
int mapOffset = ((blockModeInfoRowOffset << Av1Constants.ModeInfoSizeLog2) * paletteStride) + blockModeInfoColumnOffset << Av1Constants.ModeInfoSizeLog2,
(blockModeInfoColumnOffset << Av1Constants.ModeInfoSizeLog2); blockModeInfoRowOffset << Av1Constants.ModeInfoSizeLog2,
transformWidth,
// Every transform reconstructs its own window of the block-level palette map. Keeping the map padded to transformHeight);
// the coded block dimensions lets edge transforms use the same addressing rule as interior transforms.
// Every transform reconstructs its own window of the block-level palette map. The row-oriented region
// keeps this traversal valid when the frame-owned map spans multiple allocator memory groups.
if (typeof(T) == typeof(byte)) if (typeof(T) == typeof(byte))
{ {
Span<byte> byteDestination = MemoryMarshal.Cast<T, byte>(pixelBuffer); Span<byte> byteDestination = MemoryMarshal.Cast<T, byte>(pixelBuffer);
Av1PalettePredictor.Predict(paletteColors, colorIndexMap[mapOffset..], paletteStride, byteDestination, pixelBufferStride, transformWidth, transformHeight); Av1PalettePredictor.Predict(paletteColors, transformColorIndexMap, byteDestination, pixelBufferStride, transformWidth, transformHeight);
} }
else else
{ {
Span<short> highBitDepthDestination = MemoryMarshal.Cast<T, short>(pixelBuffer); Span<short> highBitDepthDestination = MemoryMarshal.Cast<T, short>(pixelBuffer);
Av1PalettePredictor.Predict(paletteColors, colorIndexMap[mapOffset..], paletteStride, highBitDepthDestination, pixelBufferStride, transformWidth, transformHeight); Av1PalettePredictor.Predict(paletteColors, transformColorIndexMap, highBitDepthDestination, pixelBufferStride, transformWidth, transformHeight);
} }
return; return;
@ -1983,8 +1986,8 @@ internal class Av1PredictionDecoder
left = partitionInfo.LeftModeInfoForChroma; left = partitionInfo.LeftModeInfoForChroma;
} }
bool aboveIsSmooth = (above != null) && IsSmooth(above, plane); bool aboveIsSmooth = above is not null && IsSmooth(above.Value, plane);
bool leftIsSmooth = (left != null) && IsSmooth(left, plane); bool leftIsSmooth = left is not null && IsSmooth(left.Value, plane);
return aboveIsSmooth || leftIsSmooth; return aboveIsSmooth || leftIsSmooth;
} }

23
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1BlockModeInfo.cs

@ -1,17 +1,19 @@
// Copyright (c) Six Labors. // Copyright (c) Six Labors.
// Licensed under the Six Labors Split License. // Licensed under the Six Labors Split License.
using System.Diagnostics.CodeAnalysis;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; using SixLabors.ImageSharp.Formats.Heif.Av1.Motion;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter;
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
/// <summary> /// <summary>
/// Stores block-size, intra/inter prediction, transform, and palette decisions shared by AV1 block processing. /// Stores block-size, intra/inter prediction, transform, and palette decisions shared by AV1 block processing.
/// </summary> /// </summary>
internal class Av1BlockModeInfo internal struct Av1BlockModeInfo
{ {
/// <summary> /// <summary>
/// Stores the primary and optional secondary reference-frame labels. /// Stores the primary and optional secondary reference-frame labels.
@ -56,12 +58,12 @@ internal class Av1BlockModeInfo
/// <summary> /// <summary>
/// Stores the luma palette color-index map. /// Stores the luma palette color-index map.
/// </summary> /// </summary>
private byte[] lumaPaletteColorIndexMap = []; private Buffer2DRegion<byte> lumaPaletteColorIndexMap;
/// <summary> /// <summary>
/// Stores the shared chroma palette color-index map. /// Stores the shared chroma palette color-index map.
/// </summary> /// </summary>
private byte[] chromaPaletteColorIndexMap = []; private Buffer2DRegion<byte> chromaPaletteColorIndexMap;
/// <summary> /// <summary>
/// The directional prediction angle adjustment for luma. /// The directional prediction angle adjustment for luma.
@ -94,7 +96,7 @@ internal class Av1BlockModeInfo
private int chromaTransformUnitCount; private int chromaTransformUnitCount;
/// <summary> /// <summary>
/// Initializes a new instance of the <see cref="Av1BlockModeInfo"/> class. /// Initializes a new instance of the <see cref="Av1BlockModeInfo"/> structure.
/// </summary> /// </summary>
/// <param name="blockSize">The decoded block size.</param> /// <param name="blockSize">The decoded block size.</param>
/// <param name="positionInSuperblock">The block origin relative to its superblock in 4x4 mode-information units.</param> /// <param name="positionInSuperblock">The block origin relative to its superblock in 4x4 mode-information units.</param>
@ -114,6 +116,11 @@ internal class Av1BlockModeInfo
/// </summary> /// </summary>
public Av1BlockSize BlockSize { get; } public Av1BlockSize BlockSize { get; }
/// <summary>
/// Gets or sets the frame storage index shared by every mode-information position covered by this block.
/// </summary>
public int ModeInfoIndex { get; set; }
/// <summary> /// <summary>
/// Gets or sets the <see cref="Av1PredictionMode"/> for the luminance channel. /// Gets or sets the <see cref="Av1PredictionMode"/> for the luminance channel.
/// </summary> /// </summary>
@ -127,11 +134,13 @@ internal class Av1BlockModeInfo
/// block, <see cref="Av1ReferenceFrameType.Intra"/> for an inter-intra block, or the secondary inter-reference label /// block, <see cref="Av1ReferenceFrameType.Intra"/> for an inter-intra block, or the secondary inter-reference label
/// for compound prediction. /// for compound prediction.
/// </remarks> /// </remarks>
[UnscopedRef]
public Span<Av1ReferenceFrameType> ReferenceFrames => this.referenceFrames; public Span<Av1ReferenceFrameType> ReferenceFrames => this.referenceFrames;
/// <summary> /// <summary>
/// Gets the decoded motion vectors corresponding to <see cref="ReferenceFrames"/>. /// Gets the decoded motion vectors corresponding to <see cref="ReferenceFrames"/>.
/// </summary> /// </summary>
[UnscopedRef]
public Span<Av1MotionVector> MotionVectors => this.motionVectors; public Span<Av1MotionVector> MotionVectors => this.motionVectors;
/// <summary> /// <summary>
@ -141,6 +150,7 @@ internal class Av1BlockModeInfo
/// Index zero is the vertical filter and index one is the horizontal filter, matching libaom's /// Index zero is the vertical filter and index one is the horizontal filter, matching libaom's
/// <c>InterpFilters.y_filter</c> and <c>InterpFilters.x_filter</c> layout. /// <c>InterpFilters.y_filter</c> and <c>InterpFilters.x_filter</c> layout.
/// </remarks> /// </remarks>
[UnscopedRef]
public Span<Av1InterpolationFilter> InterpolationFilters => this.interpolationFilters; public Span<Av1InterpolationFilter> InterpolationFilters => this.interpolationFilters;
/// <summary> /// <summary>
@ -392,6 +402,7 @@ internal class Av1BlockModeInfo
/// </summary> /// </summary>
/// <param name="plane">The color plane.</param> /// <param name="plane">The color plane.</param>
/// <returns>The palette colors in prediction-index order.</returns> /// <returns>The palette colors in prediction-index order.</returns>
[UnscopedRef]
public ReadOnlySpan<ushort> GetPaletteColors(Av1Plane plane) public ReadOnlySpan<ushort> GetPaletteColors(Av1Plane plane)
{ {
if (plane == Av1Plane.Y) if (plane == Av1Plane.Y)
@ -430,7 +441,7 @@ internal class Av1BlockModeInfo
/// </summary> /// </summary>
/// <param name="plane">The color plane.</param> /// <param name="plane">The color plane.</param>
/// <returns>The luma map for <see cref="Av1Plane.Y"/> or the shared chroma map for either chroma plane.</returns> /// <returns>The luma map for <see cref="Av1Plane.Y"/> or the shared chroma map for either chroma plane.</returns>
public ReadOnlySpan<byte> GetPaletteColorIndexMap(Av1Plane plane) public Buffer2DRegion<byte> GetPaletteColorIndexMap(Av1Plane plane)
=> plane == Av1Plane.Y ? this.lumaPaletteColorIndexMap : this.chromaPaletteColorIndexMap; => plane == Av1Plane.Y ? this.lumaPaletteColorIndexMap : this.chromaPaletteColorIndexMap;
/// <summary> /// <summary>
@ -438,7 +449,7 @@ internal class Av1BlockModeInfo
/// </summary> /// </summary>
/// <param name="planeType">The luma or shared chroma plane class.</param> /// <param name="planeType">The luma or shared chroma plane class.</param>
/// <param name="colorIndexMap">The row-major color-index map including coded-block edge padding.</param> /// <param name="colorIndexMap">The row-major color-index map including coded-block edge padding.</param>
public void SetPaletteColorIndexMap(Av1PlaneType planeType, byte[] colorIndexMap) public void SetPaletteColorIndexMap(Av1PlaneType planeType, Buffer2DRegion<byte> colorIndexMap)
{ {
if (planeType == Av1PlaneType.Y) if (planeType == Av1PlaneType.Y)
{ {

10
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.MotionField.cs

@ -486,19 +486,23 @@ internal partial class Av1FrameInfo
} }
/// <summary> /// <summary>
/// Releases one owner and returns motion-field storage after the final owner is released. /// Releases one owner and returns allocator-backed frame storage after the final owner is released.
/// </summary> /// </summary>
public void ReleaseOwner() public void ReleaseOwner()
{ {
this.ownerCount--; this.ownerCount--;
if (this.ownerCount == 0) if (this.ownerCount == 0)
{ {
// Retained and temporal fields can each be frame-sized. Return both together only after the tile reader, // Frame-sized motion and palette storage remains addressable through retained mode information. Return
// every reference or presentation frame, and the decoder's inspectable result have released ownership. // all of it together only after tile, reference, presentation, and decoder-result owners are gone.
this.retainedMotionField?.Dispose(); this.retainedMotionField?.Dispose();
this.retainedMotionField = null; this.retainedMotionField = null;
this.temporalMotionField?.Dispose(); this.temporalMotionField?.Dispose();
this.temporalMotionField = null; this.temporalMotionField = null;
this.lumaPaletteColorIndexMap?.Dispose();
this.lumaPaletteColorIndexMap = null;
this.chromaPaletteColorIndexMap?.Dispose();
this.chromaPaletteColorIndexMap = null;
} }
} }

81
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.cs

@ -2,6 +2,7 @@
// Licensed under the Six Labors Split License. // Licensed under the Six Labors Split License.
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
@ -55,6 +56,36 @@ internal partial class Av1FrameInfo : IDisposable
/// </summary> /// </summary>
private readonly int subsamplingFactor; private readonly int subsamplingFactor;
/// <summary>
/// The aligned luma palette-map width in samples.
/// </summary>
private readonly int lumaPaletteColorIndexMapWidth;
/// <summary>
/// The aligned luma palette-map height in samples.
/// </summary>
private readonly int lumaPaletteColorIndexMapHeight;
/// <summary>
/// The aligned chroma palette-map width in samples.
/// </summary>
private readonly int chromaPaletteColorIndexMapWidth;
/// <summary>
/// The aligned chroma palette-map height in samples.
/// </summary>
private readonly int chromaPaletteColorIndexMapHeight;
/// <summary>
/// Owns row-addressable luma palette indices for the frame.
/// </summary>
private Buffer2D<byte>? lumaPaletteColorIndexMap;
/// <summary>
/// Owns row-addressable chroma palette indices for the frame.
/// </summary>
private Buffer2D<byte>? chromaPaletteColorIndexMap;
/// <summary> /// <summary>
/// Stores one addressing view for each frame superblock. /// Stores one addressing view for each frame superblock.
/// </summary> /// </summary>
@ -179,6 +210,10 @@ internal partial class Av1FrameInfo : IDisposable
// Chroma capacity scales by two for each sampled axis: 4:4:4 => 0, 4:2:2 => 1, 4:2:0 => 2. // Chroma capacity scales by two for each sampled axis: 4:4:4 => 0, 4:2:2 => 1, 4:2:0 => 2.
this.subsamplingFactor = (subX && subY) ? 2 : (subX && !subY) ? 1 : (!subX && !subY) ? 0 : -1; this.subsamplingFactor = (subX && subY) ? 2 : (subX && !subY) ? 1 : (!subX && !subY) ? 0 : -1;
Guard.IsFalse(this.subsamplingFactor == -1, nameof(this.subsamplingFactor), "Invalid combination of subsampling."); Guard.IsFalse(this.subsamplingFactor == -1, nameof(this.subsamplingFactor), "Invalid combination of subsampling.");
this.lumaPaletteColorIndexMapWidth = superblockAlignedWidth;
this.lumaPaletteColorIndexMapHeight = superblockAlignedHeight;
this.chromaPaletteColorIndexMapWidth = superblockAlignedWidth >> (subX ? 1 : 0);
this.chromaPaletteColorIndexMapHeight = superblockAlignedHeight >> (subY ? 1 : 0);
int lumaCoefficientCountPerSuperblock = this.modeInfoCountPerSuperblock * CoefficientCountPerModeInfo; int lumaCoefficientCountPerSuperblock = this.modeInfoCountPerSuperblock * CoefficientCountPerModeInfo;
int chromaCoefficientCountPerSuperblock = lumaCoefficientCountPerSuperblock >> this.subsamplingFactor; int chromaCoefficientCountPerSuperblock = lumaCoefficientCountPerSuperblock >> this.subsamplingFactor;
this.coefficientsY = new int[superblockCount * lumaCoefficientCountPerSuperblock]; this.coefficientsY = new int[superblockCount * lumaCoefficientCountPerSuperblock];
@ -203,6 +238,51 @@ internal partial class Av1FrameInfo : IDisposable
/// </summary> /// </summary>
public int SuperblockModeInfoSize => this.modeInfoSizePerSuperblock; public int SuperblockModeInfoSize => this.modeInfoSizePerSuperblock;
/// <summary>
/// Gets frame-owned row-addressable palette-map storage for one coding block.
/// </summary>
/// <param name="configuration">The decoder configuration providing frame storage.</param>
/// <param name="planeType">The luma or shared chroma plane class.</param>
/// <param name="bounds">The block bounds in plane samples.</param>
/// <returns>The palette-map region assigned to the coding block.</returns>
public Buffer2DRegion<byte> GetPaletteColorIndexMap(
Configuration configuration,
Av1PlaneType planeType,
Rectangle bounds)
{
Buffer2D<byte>? buffer;
if (planeType == Av1PlaneType.Y)
{
buffer = this.lumaPaletteColorIndexMap;
if (buffer is null)
{
// A 2D allocation may contain multiple memory groups, but row alignment guarantees that every
// palette row remains contiguous for entropy decoding and SIMD reconstruction.
buffer = configuration.MemoryAllocator.Allocate2D<byte>(
this.lumaPaletteColorIndexMapWidth,
this.lumaPaletteColorIndexMapHeight);
this.lumaPaletteColorIndexMap = buffer;
}
}
else
{
buffer = this.chromaPaletteColorIndexMap;
if (buffer is null)
{
buffer = configuration.MemoryAllocator.Allocate2D<byte>(
this.chromaPaletteColorIndexMapWidth,
this.chromaPaletteColorIndexMapHeight);
this.chromaPaletteColorIndexMap = buffer;
}
}
// Partition traversal assigns non-overlapping frame regions, so retaining a view records the complete
// identify-time syntax without copying block maps or allocating storage for each coding block.
return new Buffer2DRegion<byte>(buffer, bounds);
}
/// <summary> /// <summary>
/// Initializes the active frame's contiguous segment map and applies whole-map inheritance when requested. /// Initializes the active frame's contiguous segment map and applies whole-map inheritance when requested.
/// </summary> /// </summary>
@ -559,6 +639,7 @@ internal partial class Av1FrameInfo : IDisposable
public void UpdateModeInfo(Av1BlockModeInfo modeInfo, Av1SuperblockInfo superblockInfo) public void UpdateModeInfo(Av1BlockModeInfo modeInfo, Av1SuperblockInfo superblockInfo)
{ {
Point modeInfoPosition = this.GetModeInfoPosition(superblockInfo.Position, modeInfo.PositionInSuperblock); Point modeInfoPosition = this.GetModeInfoPosition(superblockInfo.Position, modeInfo.PositionInSuperblock);
modeInfo.ModeInfoIndex = this.modeInfoMap.NextIndex;
this.modeInfos[this.modeInfoMap.NextIndex] = modeInfo; this.modeInfos[this.modeInfoMap.NextIndex] = modeInfo;
this.UpdateRetainedMotionField(modeInfo, modeInfoPosition); this.UpdateRetainedMotionField(modeInfo, modeInfoPosition);
this.modeInfoMap.Update(modeInfoPosition, modeInfo.BlockSize); this.modeInfoMap.Update(modeInfoPosition, modeInfo.BlockSize);

8
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameModeInfoMap.cs

@ -9,14 +9,14 @@ namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
internal partial class Av1FrameInfo internal partial class Av1FrameInfo
{ {
/// <summary> /// <summary>
/// Mapping of <see cref="Av1BlockModeInfo"/> instances, from position to index into the <see cref="Av1FrameInfo"/>. /// Mapping of <see cref="Av1BlockModeInfo"/> values, from position to index into the <see cref="Av1FrameInfo"/>.
/// </summary> /// </summary>
public class Av1FrameModeInfoMap public class Av1FrameModeInfoMap
{ {
/// <summary> /// <summary>
/// Stores the mode-information index assigned to each aligned 4x4 frame location. /// Stores the mode-information index assigned to each aligned 4x4 frame location.
/// </summary> /// </summary>
private readonly ushort[] offsets; private readonly int[] offsets;
/// <summary> /// <summary>
/// The dimensions of <see cref="offsets"/> in 4x4 mode-information units. /// The dimensions of <see cref="offsets"/> in 4x4 mode-information units.
@ -31,7 +31,7 @@ internal partial class Av1FrameInfo
{ {
this.alignedModeInfoCount = modeInfoCount; this.alignedModeInfoCount = modeInfoCount;
this.NextIndex = 0; this.NextIndex = 0;
this.offsets = new ushort[this.alignedModeInfoCount.Width * this.alignedModeInfoCount.Height]; this.offsets = new int[this.alignedModeInfoCount.Width * this.alignedModeInfoCount.Height];
} }
/// <summary> /// <summary>
@ -70,7 +70,7 @@ internal partial class Av1FrameInfo
// because later blocks query their above and left neighbors at cell granularity. // because later blocks query their above and left neighbors at cell granularity.
for (int i = modeInfoLocation.Y; i < modeInfoLocation.Y + bh4; i++) for (int i = modeInfoLocation.Y; i < modeInfoLocation.Y + bh4; i++)
{ {
Array.Fill(this.offsets, (ushort)this.NextIndex, (i * this.alignedModeInfoCount.Width) + modeInfoLocation.X, bw4); Array.Fill(this.offsets, this.NextIndex, (i * this.alignedModeInfoCount.Width) + modeInfoLocation.X, bw4);
} }
this.NextIndex++; this.NextIndex++;

27
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LevelBuffer.cs

@ -19,11 +19,14 @@ internal sealed class Av1LevelBuffer : IDisposable
private IMemoryOwner<byte>? memory; private IMemoryOwner<byte>? memory;
/// <summary> /// <summary>
/// Initializes a new instance of the <see cref="Av1LevelBuffer"/> class for the maximum AV1 transform size. /// Initializes a new instance of the <see cref="Av1LevelBuffer"/> class for the maximum entropy-coded
/// coefficient dimensions.
/// </summary> /// </summary>
/// <param name="configuration">The configuration providing the memory allocator.</param> /// <param name="configuration">The configuration providing the memory allocator.</param>
public Av1LevelBuffer(Configuration configuration) public Av1LevelBuffer(Configuration configuration)
: this(configuration, new Size(Av1Constants.MaxTransformSize, Av1Constants.MaxTransformSize)) : this(
configuration,
new Size(Av1Constants.MaxTransformSize / 2, Av1Constants.MaxTransformSize / 2))
{ {
} }
@ -46,12 +49,12 @@ internal sealed class Av1LevelBuffer : IDisposable
/// <summary> /// <summary>
/// Gets the unpadded coefficient dimensions. /// Gets the unpadded coefficient dimensions.
/// </summary> /// </summary>
public Size Size { get; } public Size Size { get; private set; }
/// <summary> /// <summary>
/// Gets the padded row stride in bytes. /// Gets the padded row stride in bytes.
/// </summary> /// </summary>
public int Stride { get; } public int Stride { get; private set; }
/// <summary> /// <summary>
/// Gets the coefficient level at the specified unpadded position. /// Gets the coefficient level at the specified unpadded position.
@ -121,6 +124,22 @@ internal sealed class Av1LevelBuffer : IDisposable
this.memory = null; this.memory = null;
} }
/// <summary>
/// Selects new active coefficient dimensions and clears their padded context storage.
/// </summary>
/// <param name="size">The unpadded coefficient dimensions.</param>
public void Reset(Size size)
{
ObjectDisposedException.ThrowIf(this.memory == null, this);
this.Size = size;
this.Stride = Av1Constants.TransformPadHorizontal + size.Width;
// Tile parsing is sequential, so one maximum-sized rent can serve every transform. Clear only the active
// layout because stale neighboring levels would otherwise select the wrong coefficient distributions.
int totalHeight = Av1Constants.TransformPadTop + size.Height + Av1Constants.TransformPadBottom;
this.memory.Memory.Span[..(this.Stride * totalHeight)].Clear();
}
/// <summary> /// <summary>
/// Clears all coefficient levels and context padding. /// Clears all coefficient levels and context padding.
/// </summary> /// </summary>

12
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PartitionInfo.cs

@ -1,6 +1,7 @@
// Copyright (c) Six Labors. // Copyright (c) Six Labors.
// Licensed under the Six Labors Split License. // Licensed under the Six Labors Split License.
using System.Diagnostics.CodeAnalysis;
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma;
using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; using SixLabors.ImageSharp.Formats.Heif.Av1.Transform;
@ -12,6 +13,11 @@ namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
/// </summary> /// </summary>
internal ref struct Av1PartitionInfo internal ref struct Av1PartitionInfo
{ {
/// <summary>
/// The decoded mode information populated for this partition.
/// </summary>
private Av1BlockModeInfo modeInfo;
/// <summary> /// <summary>
/// The luma block width in samples. /// The luma block width in samples.
/// </summary> /// </summary>
@ -41,7 +47,7 @@ internal ref struct Av1PartitionInfo
/// <param name="partitionType">The partition type that produced the block.</param> /// <param name="partitionType">The partition type that produced the block.</param>
public Av1PartitionInfo(Av1BlockModeInfo modeInfo, Av1SuperblockInfo superblockInfo, bool isChroma, Av1PartitionType partitionType) public Av1PartitionInfo(Av1BlockModeInfo modeInfo, Av1SuperblockInfo superblockInfo, bool isChroma, Av1PartitionType partitionType)
{ {
this.ModeInfo = modeInfo; this.modeInfo = modeInfo;
this.SuperblockInfo = superblockInfo; this.SuperblockInfo = superblockInfo;
this.IsChroma = isChroma; this.IsChroma = isChroma;
this.Type = partitionType; this.Type = partitionType;
@ -50,7 +56,8 @@ internal ref struct Av1PartitionInfo
/// <summary> /// <summary>
/// Gets the decoded block mode information. /// Gets the decoded block mode information.
/// </summary> /// </summary>
public Av1BlockModeInfo ModeInfo { get; } [UnscopedRef]
public ref Av1BlockModeInfo ModeInfo => ref this.modeInfo;
/// <summary> /// <summary>
/// Gets the <see cref="Av1SuperblockInfo"/> this partition resides inside. /// Gets the <see cref="Av1SuperblockInfo"/> this partition resides inside.
@ -120,6 +127,7 @@ internal ref struct Av1PartitionInfo
/// <summary> /// <summary>
/// Gets the reference-frame types selected for the block. /// Gets the reference-frame types selected for the block.
/// </summary> /// </summary>
[UnscopedRef]
public Span<Av1ReferenceFrameType> ReferenceFrames => this.ModeInfo.ReferenceFrames; public Span<Av1ReferenceFrameType> ReferenceFrames => this.ModeInfo.ReferenceFrames;
/// <summary> /// <summary>

183
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs

@ -11,6 +11,7 @@ using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter;
using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames; using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames;
using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; using SixLabors.ImageSharp.Formats.Heif.Av1.Transform;
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
@ -95,6 +96,11 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// </summary> /// </summary>
private readonly int[] coefficientIndex = []; private readonly int[] coefficientIndex = [];
/// <summary>
/// Reusable padded coefficient-context storage for the sequential transform traversal.
/// </summary>
private readonly Av1LevelBuffer coefficientLevels;
/// <summary> /// <summary>
/// Reusable storage for the eight spatial displacement-vector candidates permitted by AV1. /// Reusable storage for the eight spatial displacement-vector candidates permitted by AV1.
/// </summary> /// </summary>
@ -225,6 +231,20 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
throw; throw;
} }
try
{
this.coefficientLevels = new Av1LevelBuffer(configuration);
}
catch
{
// The coefficient scratch allocation follows both neighbor contexts. Unwind those successful rents when
// construction cannot publish an owning tile reader.
this.aboveNeighborContext.Dispose();
this.leftNeighborContext.Dispose();
this.FrameInfo.Dispose();
throw;
}
if (referenceFrames is not null) if (referenceFrames is not null)
{ {
try try
@ -237,6 +257,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
{ {
this.aboveNeighborContext.Dispose(); this.aboveNeighborContext.Dispose();
this.leftNeighborContext.Dispose(); this.leftNeighborContext.Dispose();
this.coefficientLevels.Dispose();
this.FrameInfo.Dispose(); this.FrameInfo.Dispose();
throw; throw;
} }
@ -328,6 +349,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
{ {
this.aboveNeighborContext.Dispose(); this.aboveNeighborContext.Dispose();
this.leftNeighborContext.Dispose(); this.leftNeighborContext.Dispose();
this.coefficientLevels.Dispose();
this.FrameInfo.Dispose(); this.FrameInfo.Dispose();
} }
@ -336,7 +358,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// </summary> /// </summary>
/// <param name="tileData">The entropy-coded tile payload.</param> /// <param name="tileData">The entropy-coded tile payload.</param>
/// <param name="tileNum">The zero-based tile index in row-major order.</param> /// <param name="tileNum">The zero-based tile index in row-major order.</param>
/// <remarks>Corresponds to <c>parse_tile</c> in SVT-AV1.</remarks> /// <remarks>Corresponds to <c>decode_tile</c> in libaom.</remarks>
public void ReadTile(Span<byte> tileData, int tileNum) public void ReadTile(Span<byte> tileData, int tileNum)
{ {
// AV1 tiles never inherit adaptation from another tile in the same frame. Reusing one graph is safe because // AV1 tiles never inherit adaptation from another tile in the same frame. Reusing one graph is safe because
@ -661,6 +683,19 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
} }
Av1BlockSize subSize = partitionType.GetBlockSubSize(blockSize); Av1BlockSize subSize = partitionType.GetBlockSubSize(blockSize);
if (subSize == Av1BlockSize.Invalid)
{
throw new InvalidImageContentException($"The decoded AV1 partition type {partitionType} is invalid for block size {blockSize}.");
}
ObuColorConfig colorConfig = this.SequenceHeader.ColorConfig;
if (subSize.GetSubsampled(colorConfig.SubSamplingX, colorConfig.SubSamplingY) == Av1BlockSize.Invalid)
{
// Luma partition syntax can describe a sub-8x8 shape that has no legal representation after chroma
// subsampling. Reject it before any block state is published, matching libaom's decode_partition boundary.
throw new InvalidImageContentException($"The decoded AV1 block size {subSize} is invalid for the sequence chroma subsampling.");
}
Av1BlockSize splitSize = Av1PartitionType.Split.GetBlockSubSize(blockSize); Av1BlockSize splitSize = Av1PartitionType.Split.GetBlockSubSize(blockSize);
// Partition syntax is depth-first. The visit order here is also the order in which mode, // Partition syntax is depth-first. The visit order here is also the order in which mode,
@ -815,7 +850,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
this.Residual(ref reader, ref partitionInfo, superblockInfo, tileInfo, blockSize); this.Residual(ref reader, ref partitionInfo, superblockInfo, tileInfo, blockSize);
// Store the record only after all syntax has populated it, then map every covered 4x4 position. // Store the record only after all syntax has populated it, then map every covered 4x4 position.
this.FrameInfo.UpdateModeInfo(blockModeInfo, superblockInfo); this.FrameInfo.UpdateModeInfo(partitionInfo.ModeInfo, superblockInfo);
} }
/// <summary> /// <summary>
@ -823,7 +858,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// </summary> /// </summary>
/// <param name="partitionInfo">The skipped block and its frame position.</param> /// <param name="partitionInfo">The skipped block and its frame position.</param>
/// <param name="tileInfo">The active tile boundaries.</param> /// <param name="tileInfo">The active tile boundaries.</param>
/// <remarks>Corresponds to <c>reset_skip_context</c> in SVT-AV1.</remarks> /// <remarks>Implements AV1 section 5.11.37.</remarks>
private void ResetSkipContext(ref Av1PartitionInfo partitionInfo, Av1TileInfo tileInfo) private void ResetSkipContext(ref Av1PartitionInfo partitionInfo, Av1TileInfo tileInfo)
{ {
// Subsampled 4x4 luma blocks can share chroma ownership with an adjacent luma block. A skipped block that is // Subsampled 4x4 luma blocks can share chroma ownership with an adjacent luma block. A skipped block that is
@ -852,7 +887,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="superblockInfo">The containing superblock and coefficient storage.</param> /// <param name="superblockInfo">The containing superblock and coefficient storage.</param>
/// <param name="tileInfo">The active tile boundaries.</param> /// <param name="tileInfo">The active tile boundaries.</param>
/// <param name="blockSize">The coding block size.</param> /// <param name="blockSize">The coding block size.</param>
/// <remarks>Implements AV1 section 5.11.34 and corresponds to <c>parse_residual</c> in SVT-AV1.</remarks> /// <remarks>Implements AV1 section 5.11.34.</remarks>
private void Residual( private void Residual(
ref Av1SymbolDecoder reader, ref Av1SymbolDecoder reader,
ref Av1PartitionInfo partitionInfo, ref Av1PartitionInfo partitionInfo,
@ -1039,7 +1074,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="subY">A value indicating whether the target plane is vertically subsampled.</param> /// <param name="subY">A value indicating whether the target plane is vertically subsampled.</param>
/// <returns>The decoded end-of-block coefficient position, or zero for an all-zero transform.</returns> /// <returns>The decoded end-of-block coefficient position, or zero for an all-zero transform.</returns>
/// <remarks> /// <remarks>
/// Implements AV1 section 5.11.35 using the traversal shape of the corresponding SVT-AV1 implementation. /// Implements AV1 section 5.11.35.
/// </remarks> /// </remarks>
private int ParseTransformBlock( private int ParseTransformBlock(
ref Av1SymbolDecoder reader, ref Av1SymbolDecoder reader,
@ -1120,7 +1155,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="coefficientBuffer">The destination beginning at this transform's coefficient slot.</param> /// <param name="coefficientBuffer">The destination beginning at this transform's coefficient slot.</param>
/// <returns>The decoded end-of-block coefficient position, or zero for an all-zero transform.</returns> /// <returns>The decoded end-of-block coefficient position, or zero for an all-zero transform.</returns>
/// <remarks> /// <remarks>
/// Implements AV1 section 5.11.39 using the traversal shape of the corresponding SVT-AV1 implementation. /// Implements AV1 section 5.11.39.
/// </remarks> /// </remarks>
private int ParseCoefficients( private int ParseCoefficients(
ref Av1SymbolDecoder reader, ref Av1SymbolDecoder reader,
@ -1169,6 +1204,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
ref transformInfo, ref transformInfo,
partitionInfo.ModeBlockToRightEdge, partitionInfo.ModeBlockToRightEdge,
partitionInfo.ModeBlockToBottomEdge, partitionInfo.ModeBlockToBottomEdge,
this.coefficientLevels,
coefficientBuffer); coefficientBuffer);
} }
@ -1317,7 +1353,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
Av1TileInfo tileInfo, Av1TileInfo tileInfo,
bool allowSelect) bool allowSelect)
{ {
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
if (this.FrameHeader.LosslessArray[modeInfo.SegmentId]) if (this.FrameHeader.LosslessArray[modeInfo.SegmentId])
{ {
return Av1TransformSize.Size4x4; return Av1TransformSize.Size4x4;
@ -1406,7 +1442,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="partitionInfo">The current coding block.</param> /// <param name="partitionInfo">The current coding block.</param>
/// <param name="superblockInfo">The containing superblock.</param> /// <param name="superblockInfo">The containing superblock.</param>
/// <param name="tileInfo">The active tile boundaries.</param> /// <param name="tileInfo">The active tile boundaries.</param>
/// <remarks>Implements AV1 section 5.11.16 and corresponds to <c>read_block_tx_size</c> in SVT-AV1.</remarks> /// <remarks>Implements AV1 section 5.11.16 and corresponds to <c>read_tx_size</c> in libaom.</remarks>
private void ReadBlockTransformSize( private void ReadBlockTransformSize(
ref Av1SymbolDecoder reader, ref Av1SymbolDecoder reader,
Point modeInfoLocation, Point modeInfoLocation,
@ -1418,7 +1454,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
int block4x4Width = blockSize.Get4x4WideCount(); int block4x4Width = blockSize.Get4x4WideCount();
int block4x4Height = blockSize.Get4x4HighCount(); int block4x4Height = blockSize.Get4x4HighCount();
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
bool usesInterTransformSyntax = modeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Last || modeInfo.UseIntraBlockCopy; bool usesInterTransformSyntax = modeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Last || modeInfo.UseIntraBlockCopy;
this.transformUnitCount[(int)Av1Plane.Y].AsSpan(0, 4).Clear(); this.transformUnitCount[(int)Av1Plane.Y].AsSpan(0, 4).Clear();
@ -1738,7 +1774,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <remarks>Implements AV1 section 5.11.49.</remarks> /// <remarks>Implements AV1 section 5.11.49.</remarks>
private void ReadPaletteTokens(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) private void ReadPaletteTokens(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo)
{ {
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
if (modeInfo.GetPaletteSize(Av1PlaneType.Y) != 0) if (modeInfo.GetPaletteSize(Av1PlaneType.Y) != 0)
{ {
GetPaletteMapDimensions( GetPaletteMapDimensions(
@ -1750,14 +1786,24 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
out int rows, out int rows,
out int columns); out int columns);
byte[] colorIndexMap = DecodePaletteColorMap( Buffer2DRegion<byte> colorIndexMap = this.FrameInfo.GetPaletteColorIndexMap(
this.configuration,
Av1PlaneType.Y,
new Rectangle(
partitionInfo.ColumnIndex << Av1Constants.ModeInfoSizeLog2,
partitionInfo.RowIndex << Av1Constants.ModeInfoSizeLog2,
planeWidth,
planeHeight));
DecodePaletteColorMap(
ref reader, ref reader,
modeInfo.GetPaletteSize(Av1PlaneType.Y), modeInfo.GetPaletteSize(Av1PlaneType.Y),
Av1PlaneType.Y, Av1PlaneType.Y,
planeWidth, planeWidth,
planeHeight, planeHeight,
rows, rows,
columns); columns,
colorIndexMap);
modeInfo.SetPaletteColorIndexMap(Av1PlaneType.Y, colorIndexMap); modeInfo.SetPaletteColorIndexMap(Av1PlaneType.Y, colorIndexMap);
} }
@ -1773,14 +1819,26 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
out int rows, out int rows,
out int columns); out int columns);
byte[] colorIndexMap = DecodePaletteColorMap( int subX = this.SequenceHeader.ColorConfig.SubSamplingX ? 1 : 0;
int subY = this.SequenceHeader.ColorConfig.SubSamplingY ? 1 : 0;
Buffer2DRegion<byte> colorIndexMap = this.FrameInfo.GetPaletteColorIndexMap(
this.configuration,
Av1PlaneType.Uv,
new Rectangle(
(partitionInfo.ColumnIndex << Av1Constants.ModeInfoSizeLog2) >> subX,
(partitionInfo.RowIndex << Av1Constants.ModeInfoSizeLog2) >> subY,
planeWidth,
planeHeight));
DecodePaletteColorMap(
ref reader, ref reader,
modeInfo.GetPaletteSize(Av1PlaneType.Uv), modeInfo.GetPaletteSize(Av1PlaneType.Uv),
Av1PlaneType.Uv, Av1PlaneType.Uv,
planeWidth, planeWidth,
planeHeight, planeHeight,
rows, rows,
columns); columns,
colorIndexMap);
modeInfo.SetPaletteColorIndexMap(Av1PlaneType.Uv, colorIndexMap); modeInfo.SetPaletteColorIndexMap(Av1PlaneType.Uv, colorIndexMap);
} }
@ -1814,7 +1872,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <remarks>Implements the prefix, intra, and translational inter branches of AV1 section 5.11.7.</remarks> /// <remarks>Implements the prefix, intra, and translational inter branches of AV1 section 5.11.7.</remarks>
public void ReadInterFrameModeInfo(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo, Av1TileInfo tileInfo) public void ReadInterFrameModeInfo(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo, Av1TileInfo tileInfo)
{ {
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
modeInfo.MotionVectors.Clear(); modeInfo.MotionVectors.Clear();
this.ReadInterSegmentId(ref reader, ref partitionInfo, beforeSkip: true); this.ReadInterSegmentId(ref reader, ref partitionInfo, beforeSkip: true);
@ -2292,7 +2350,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
ref Av1PartitionInfo partitionInfo, ref Av1PartitionInfo partitionInfo,
Av1PredictionMode yMode) Av1PredictionMode yMode)
{ {
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
modeInfo.YMode = yMode; modeInfo.YMode = yMode;
modeInfo.SetAngleDelta(Av1PlaneType.Y, IntraAngleInfo(ref reader, yMode, modeInfo.BlockSize)); modeInfo.SetAngleDelta(Av1PlaneType.Y, IntraAngleInfo(ref reader, yMode, modeInfo.BlockSize));
@ -2302,7 +2360,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
if (modeInfo.UvMode == Av1ChromaPredictionMode.ChromaFromLuma) if (modeInfo.UvMode == Av1ChromaPredictionMode.ChromaFromLuma)
{ {
ReadChromaFromLumaAlphas(ref reader, modeInfo); ReadChromaFromLumaAlphas(ref reader, ref modeInfo);
} }
modeInfo.SetAngleDelta( modeInfo.SetAngleDelta(
@ -2390,7 +2448,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <remarks>Implements AV1 section 5.11.46.</remarks> /// <remarks>Implements AV1 section 5.11.46.</remarks>
private void PaletteModeInfo(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) private void PaletteModeInfo(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo)
{ {
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
Av1BlockSize blockSize = modeInfo.BlockSize; Av1BlockSize blockSize = modeInfo.BlockSize;
// The palette block-size context is the base-two block-area difference from an 8-by-8 block. // The palette block-size context is the base-two block-area difference from an 8-by-8 block.
@ -2401,12 +2459,12 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
if (modeInfo.YMode == Av1PredictionMode.DC) if (modeInfo.YMode == Av1PredictionMode.DC)
{ {
int neighborContext = 0; int neighborContext = 0;
if (partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.GetPaletteSize(Av1PlaneType.Y) != 0) if (partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.Value.GetPaletteSize(Av1PlaneType.Y) != 0)
{ {
neighborContext++; neighborContext++;
} }
if (partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.GetPaletteSize(Av1PlaneType.Y) != 0) if (partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.Value.GetPaletteSize(Av1PlaneType.Y) != 0)
{ {
neighborContext++; neighborContext++;
} }
@ -2595,10 +2653,12 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
: partitionInfo.AboveModeInfo; : partitionInfo.AboveModeInfo;
Av1BlockModeInfo? leftModeInfo = partitionInfo.LeftModeInfo; Av1BlockModeInfo? leftModeInfo = partitionInfo.LeftModeInfo;
int abovePaletteSize = aboveModeInfo?.GetPaletteSize(plane) ?? 0; Av1BlockModeInfo above = aboveModeInfo.GetValueOrDefault();
int leftPaletteSize = leftModeInfo?.GetPaletteSize(plane) ?? 0; Av1BlockModeInfo left = leftModeInfo.GetValueOrDefault();
ReadOnlySpan<ushort> aboveColors = aboveModeInfo is null ? [] : aboveModeInfo.GetPaletteColors(plane); int abovePaletteSize = aboveModeInfo is null ? 0 : above.GetPaletteSize(plane);
ReadOnlySpan<ushort> leftColors = leftModeInfo is null ? [] : leftModeInfo.GetPaletteColors(plane); int leftPaletteSize = leftModeInfo is null ? 0 : left.GetPaletteSize(plane);
ReadOnlySpan<ushort> aboveColors = aboveModeInfo is null ? [] : above.GetPaletteColors(plane);
ReadOnlySpan<ushort> leftColors = leftModeInfo is null ? [] : left.GetPaletteColors(plane);
int aboveIndex = 0; int aboveIndex = 0;
int leftIndex = 0; int leftIndex = 0;
int count = 0; int count = 0;
@ -2725,18 +2785,18 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="planeHeight">The padded plane-block height.</param> /// <param name="planeHeight">The padded plane-block height.</param>
/// <param name="rows">The number of rows inside the coded image.</param> /// <param name="rows">The number of rows inside the coded image.</param>
/// <param name="columns">The number of columns inside the coded image.</param> /// <param name="columns">The number of columns inside the coded image.</param>
/// <returns>The decoded row-major color-index map.</returns> /// <param name="colorIndexMap">The row-addressable destination map.</param>
private static byte[] DecodePaletteColorMap( private static void DecodePaletteColorMap(
ref Av1SymbolDecoder reader, ref Av1SymbolDecoder reader,
int paletteSize, int paletteSize,
Av1PlaneType planeType, Av1PlaneType planeType,
int planeWidth, int planeWidth,
int planeHeight, int planeHeight,
int rows, int rows,
int columns) int columns,
Buffer2DRegion<byte> colorIndexMap)
{ {
byte[] colorIndexMap = new byte[planeWidth * planeHeight]; colorIndexMap.DangerousGetRowSpan(0)[0] = (byte)reader.ReadUniform(paletteSize);
colorIndexMap[0] = (byte)reader.ReadUniform(paletteSize);
Span<byte> colorOrder = stackalloc byte[Av1Constants.PaletteMaxSize]; Span<byte> colorOrder = stackalloc byte[Av1Constants.PaletteMaxSize];
for (int diagonal = 1; diagonal < rows + columns - 1; diagonal++) for (int diagonal = 1; diagonal < rows + columns - 1; diagonal++)
{ {
@ -2747,14 +2807,13 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
int row = diagonal - column; int row = diagonal - column;
int colorContext = GetPaletteColorIndexContext( int colorContext = GetPaletteColorIndexContext(
colorIndexMap, colorIndexMap,
planeWidth,
row, row,
column, column,
paletteSize, paletteSize,
colorOrder); colorOrder);
int colorOrderIndex = reader.ReadPaletteColorIndex(paletteSize, colorContext, planeType); int colorOrderIndex = reader.ReadPaletteColorIndex(paletteSize, colorContext, planeType);
colorIndexMap[(row * planeWidth) + column] = colorOrder[colorOrderIndex]; colorIndexMap.DangerousGetRowSpan(row)[column] = colorOrder[colorOrderIndex];
} }
} }
@ -2763,44 +2822,50 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
// Blocks clipped by the right image edge repeat their final coded column into the padded block area. // Blocks clipped by the right image edge repeat their final coded column into the padded block area.
for (int row = 0; row < rows; row++) for (int row = 0; row < rows; row++)
{ {
int rowOffset = row * planeWidth; Span<byte> colorIndexRow = colorIndexMap.DangerousGetRowSpan(row);
colorIndexMap.AsSpan(rowOffset + columns, planeWidth - columns) colorIndexRow.Slice(columns, planeWidth - columns)
.Fill(colorIndexMap[rowOffset + columns - 1]); .Fill(colorIndexRow[columns - 1]);
} }
} }
// Blocks clipped by the bottom image edge repeat their final coded row for later transform reconstruction. // Blocks clipped by the bottom image edge repeat their final coded row for later transform reconstruction.
ReadOnlySpan<byte> finalRow = colorIndexMap.AsSpan((rows - 1) * planeWidth, planeWidth); ReadOnlySpan<byte> finalRow = colorIndexMap.DangerousGetRowSpan(rows - 1);
for (int row = rows; row < planeHeight; row++) for (int row = rows; row < planeHeight; row++)
{ {
finalRow.CopyTo(colorIndexMap.AsSpan(row * planeWidth, planeWidth)); finalRow.CopyTo(colorIndexMap.DangerousGetRowSpan(row));
} }
return colorIndexMap;
} }
/// <summary> /// <summary>
/// Derives the palette color order and entropy context from the left, upper-left, and above indices. /// Derives the palette color order and entropy context from the left, upper-left, and above indices.
/// </summary> /// </summary>
/// <param name="colorIndexMap">The partially decoded color-index map.</param> /// <param name="colorIndexMap">The partially decoded color-index map.</param>
/// <param name="stride">The map row stride.</param>
/// <param name="row">The current map row.</param> /// <param name="row">The current map row.</param>
/// <param name="column">The current map column.</param> /// <param name="column">The current map column.</param>
/// <param name="paletteSize">The number of palette colors.</param> /// <param name="paletteSize">The number of palette colors.</param>
/// <param name="colorOrder">The destination color order for the current context.</param> /// <param name="colorOrder">The destination color order for the current context.</param>
/// <returns>The color-index entropy context in the range from zero through four.</returns> /// <returns>The color-index entropy context in the range from zero through four.</returns>
private static int GetPaletteColorIndexContext( private static int GetPaletteColorIndexContext(
ReadOnlySpan<byte> colorIndexMap, Buffer2DRegion<byte> colorIndexMap,
int stride,
int row, int row,
int column, int column,
int paletteSize, int paletteSize,
Span<byte> colorOrder) Span<byte> colorOrder)
{ {
Span<int> neighborColors = stackalloc int[3]; Span<int> neighborColors = stackalloc int[3];
neighborColors[0] = column > 0 ? colorIndexMap[(row * stride) + column - 1] : -1; ReadOnlySpan<byte> currentRow = colorIndexMap.DangerousGetRowSpan(row);
neighborColors[1] = column > 0 && row > 0 ? colorIndexMap[((row - 1) * stride) + column - 1] : -1; neighborColors[0] = column > 0 ? currentRow[column - 1] : -1;
neighborColors[2] = row > 0 ? colorIndexMap[((row - 1) * stride) + column] : -1; if (row > 0)
{
ReadOnlySpan<byte> aboveRow = colorIndexMap.DangerousGetRowSpan(row - 1);
neighborColors[1] = column > 0 ? aboveRow[column - 1] : -1;
neighborColors[2] = aboveRow[column];
}
else
{
neighborColors[1] = -1;
neighborColors[2] = -1;
}
Span<int> scores = stackalloc int[Av1Constants.PaletteMaxSize]; Span<int> scores = stackalloc int[Av1Constants.PaletteMaxSize];
ReadOnlySpan<int> neighborWeights = [2, 1, 2]; ReadOnlySpan<int> neighborWeights = [2, 1, 2];
@ -2855,7 +2920,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="reader">The tile symbol decoder.</param> /// <param name="reader">The tile symbol decoder.</param>
/// <param name="modeInfo">The block mode information to populate.</param> /// <param name="modeInfo">The block mode information to populate.</param>
/// <remarks>Implements AV1 section 5.11.45.</remarks> /// <remarks>Implements AV1 section 5.11.45.</remarks>
private static void ReadChromaFromLumaAlphas(ref Av1SymbolDecoder reader, Av1BlockModeInfo modeInfo) private static void ReadChromaFromLumaAlphas(ref Av1SymbolDecoder reader, ref Av1BlockModeInfo modeInfo)
{ {
int jointSignPlus1 = reader.ReadChromFromLumaSign() + 1; int jointSignPlus1 = reader.ReadChromFromLumaSign() + 1;
int index = 0; int index = 0;
@ -2932,7 +2997,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
public void ReadInterSegmentId(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo, bool beforeSkip) public void ReadInterSegmentId(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo, bool beforeSkip)
{ {
ObuSegmentationParameters segmentationParameters = this.FrameHeader.SegmentationParameters; ObuSegmentationParameters segmentationParameters = this.FrameHeader.SegmentationParameters;
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
if (!segmentationParameters.Enabled) if (!segmentationParameters.Enabled)
{ {
@ -3049,7 +3114,15 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
: prevUL == prevU && prevUL == prevL ? 2 : prevUL == prevU && prevUL == prevL ? 2
: prevUL == prevU || prevUL == prevL || prevU == prevL ? 1 : 0; : prevUL == prevU || prevUL == prevL || prevU == prevL ? 1 : 0;
int lastActiveSegmentId = this.FrameHeader.SegmentationParameters.LastActiveSegmentId; int lastActiveSegmentId = this.FrameHeader.SegmentationParameters.LastActiveSegmentId;
partitionInfo.ModeInfo.SegmentId = Av1SymbolContextHelper.NegativeDeinterleave(reader.ReadSegmentId(ctx), predictor, lastActiveSegmentId + 1); int segmentId = Av1SymbolContextHelper.NegativeDeinterleave(reader.ReadSegmentId(ctx), predictor, lastActiveSegmentId + 1);
if (segmentId is < 0 || segmentId > lastActiveSegmentId)
{
// The coded alphabet always contains eight symbols, even when the frame activates fewer segments.
// Validate the reconstructed ID at the same corruption boundary as libaom's read_segment_id.
throw new InvalidImageContentException("The decoded AV1 segment identifier exceeds the active segment range.");
}
partitionInfo.ModeInfo.SegmentId = segmentId;
} }
} }
@ -3151,8 +3224,8 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
} }
else else
{ {
int aboveSkip = partitionInfo.AboveModeInfo != null && partitionInfo.AboveModeInfo.Skip ? 1 : 0; int aboveSkip = partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.Value.Skip ? 1 : 0;
int leftSkip = partitionInfo.LeftModeInfo != null && partitionInfo.LeftModeInfo.Skip ? 1 : 0; int leftSkip = partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.Value.Skip ? 1 : 0;
return reader.ReadSkip(aboveSkip + leftSkip); return reader.ReadSkip(aboveSkip + leftSkip);
} }
} }
@ -3165,7 +3238,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <returns><see langword="true"/> when the block selects the frame's derived skip-mode reference pair.</returns> /// <returns><see langword="true"/> when the block selects the frame's derived skip-mode reference pair.</returns>
private bool ReadSkipMode(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) private bool ReadSkipMode(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo)
{ {
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
ObuSegmentationParameters segmentationParameters = this.FrameHeader.SegmentationParameters; ObuSegmentationParameters segmentationParameters = this.FrameHeader.SegmentationParameters;
int segmentId = modeInfo.SegmentId; int segmentId = modeInfo.SegmentId;
@ -3180,8 +3253,8 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
return false; return false;
} }
int aboveSkipMode = partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.SkipMode ? 1 : 0; int aboveSkipMode = partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.Value.SkipMode ? 1 : 0;
int leftSkipMode = partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.SkipMode ? 1 : 0; int leftSkipMode = partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.Value.SkipMode ? 1 : 0;
return reader.ReadSkipMode(aboveSkipMode + leftSkipMode); return reader.ReadSkipMode(aboveSkipMode + leftSkipMode);
} }
@ -3220,7 +3293,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="partitionInfo">The current coding block and its available neighbors.</param> /// <param name="partitionInfo">The current coding block and its available neighbors.</param>
private void ReadReferenceFrames(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) private void ReadReferenceFrames(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo)
{ {
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo; ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
Span<Av1ReferenceFrameType> references = modeInfo.ReferenceFrames; Span<Av1ReferenceFrameType> references = modeInfo.ReferenceFrames;
if (modeInfo.SkipMode) if (modeInfo.SkipMode)
{ {
@ -3369,7 +3442,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// </summary> /// </summary>
/// <param name="reader">The tile symbol decoder.</param> /// <param name="reader">The tile symbol decoder.</param>
/// <param name="partitionInfo">The current coding block and superblock quantizer storage.</param> /// <param name="partitionInfo">The current coding block and superblock quantizer storage.</param>
/// <remarks>Corresponds to <c>read_delta_qindex</c> in SVT-AV1.</remarks> /// <remarks>Corresponds to <c>read_delta_qindex</c> in libaom.</remarks>
private void ReadDeltaQuantizerIndex(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo) private void ReadDeltaQuantizerIndex(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo)
{ {
if (!this.FrameHeader.DeltaQParameters.IsPresent || partitionInfo.ModeInfo.PositionInSuperblock != Point.Empty) if (!this.FrameHeader.DeltaQParameters.IsPresent || partitionInfo.ModeInfo.PositionInSuperblock != Point.Empty)
@ -3411,7 +3484,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="tileInfo">The active tile boundaries.</param> /// <param name="tileInfo">The active tile boundaries.</param>
/// <param name="superblockInfo">The containing superblock.</param> /// <param name="superblockInfo">The containing superblock.</param>
/// <returns>The partition entropy context.</returns> /// <returns>The partition entropy context.</returns>
/// <remarks>Corresponds to <c>partition_plane_context</c> in SVT-AV1.</remarks> /// <remarks>Corresponds to <c>partition_plane_context</c> in libaom.</remarks>
private int GetPartitionPlaneContext(Point location, Av1BlockSize blockSize, Av1TileInfo tileInfo, Av1SuperblockInfo superblockInfo) private int GetPartitionPlaneContext(Point location, Av1BlockSize blockSize, Av1TileInfo tileInfo, Av1SuperblockInfo superblockInfo)
{ {
// The five stored split bits begin at the 8x8 partition point, so normalize the block-size log to that bit index. // The five stored split bits begin at the 8x8 partition point, so normalize the block-size log to that bit index.

23
tests/ImageSharp.Benchmarks/Codecs/Heif/Av1PalettePredictionBenchmarks.cs

@ -6,6 +6,7 @@ using BenchmarkDotNet.Columns;
using BenchmarkDotNet.Configs; using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif;
@ -46,7 +47,12 @@ public class Av1PalettePredictionBenchmarks
/// <summary> /// <summary>
/// The decoded color-index map for one maximum-size palette block. /// The decoded color-index map for one maximum-size palette block.
/// </summary> /// </summary>
private readonly byte[] colorIndexMap = new byte[BlockSize * BlockSize]; private Buffer2D<byte> colorIndexMapBuffer;
/// <summary>
/// The row-addressable view of <see cref="colorIndexMapBuffer"/>.
/// </summary>
private Buffer2DRegion<byte> colorIndexMap;
/// <summary> /// <summary>
/// The frame-wide 8-bit reconstruction surface. /// The frame-wide 8-bit reconstruction surface.
@ -64,15 +70,24 @@ public class Av1PalettePredictionBenchmarks
[GlobalSetup] [GlobalSetup]
public void Setup() public void Setup()
{ {
this.colorIndexMapBuffer = SixLabors.ImageSharp.Configuration.Default.MemoryAllocator.Allocate2D<byte>(BlockSize, BlockSize);
this.colorIndexMap = new Buffer2DRegion<byte>(this.colorIndexMapBuffer);
for (int row = 0; row < BlockSize; row++) for (int row = 0; row < BlockSize; row++)
{ {
Span<byte> colorIndexRow = this.colorIndexMap.DangerousGetRowSpan(row);
for (int column = 0; column < BlockSize; column++) for (int column = 0; column < BlockSize; column++)
{ {
this.colorIndexMap[(row * BlockSize) + column] = (byte)(((row * 5) + (column * 3)) & 7); colorIndexRow[column] = (byte)(((row * 5) + (column * 3)) & 7);
} }
} }
} }
/// <summary>
/// Releases the row-addressable color-index map after the benchmark run.
/// </summary>
[GlobalCleanup]
public void Cleanup() => this.colorIndexMapBuffer?.Dispose();
/// <summary> /// <summary>
/// Measures frame-wide 8-bit palette reconstruction. /// Measures frame-wide 8-bit palette reconstruction.
/// </summary> /// </summary>
@ -85,7 +100,7 @@ public class Av1PalettePredictionBenchmarks
{ {
for (int column = 0; column < Width; column += BlockSize) for (int column = 0; column < Width; column += BlockSize)
{ {
Av1PalettePredictor.Predict(this.palette8, this.colorIndexMap, BlockSize, this.destination8.AsSpan((row * Width) + column), Width, BlockSize, BlockSize); Av1PalettePredictor.Predict(this.palette8, this.colorIndexMap, this.destination8.AsSpan((row * Width) + column), Width, BlockSize, BlockSize);
} }
} }
@ -104,7 +119,7 @@ public class Av1PalettePredictionBenchmarks
{ {
for (int column = 0; column < Width; column += BlockSize) for (int column = 0; column < Width; column += BlockSize)
{ {
Av1PalettePredictor.Predict(this.palette12, this.colorIndexMap, BlockSize, this.destination12.AsSpan((row * Width) + column), Width, BlockSize, BlockSize); Av1PalettePredictor.Predict(this.palette12, this.colorIndexMap, this.destination12.AsSpan((row * Width) + column), Width, BlockSize, BlockSize);
} }
} }

69
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CoefficientsEntropyTests.cs

@ -44,10 +44,27 @@ public class Av1CoefficientsEntropyTests
using IMemoryOwner<byte> encoded = encoder.Exit(); using IMemoryOwner<byte> encoded = encoder.Exit();
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex);
using Av1LevelBuffer levels = new(Configuration.Default);
decoder.ReadCoefficients( decoder.ReadCoefficients(
modeInfo, new Point(0, 0), aboveContexts, leftContexts, modeInfo,
0, 0, 0, 1, 1, transformBlockContext, transformSize, new Point(0, 0),
false, true, transformType, ref transformInfo, 0, 0, actuals); aboveContexts,
leftContexts,
0,
0,
0,
1,
1,
transformBlockContext,
transformSize,
false,
true,
transformType,
ref transformInfo,
0,
0,
levels,
actuals);
// Assert // Assert
Assert.Equal(endOfBlock, actuals[0]); Assert.Equal(endOfBlock, actuals[0]);
@ -96,11 +113,28 @@ public class Av1CoefficientsEntropyTests
using IMemoryOwner<byte> encoded = encoder.Exit(); using IMemoryOwner<byte> encoded = encoder.Exit();
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex);
using Av1LevelBuffer levels = new(Configuration.Default);
int plane = Math.Min((int)componentType, 1); int plane = Math.Min((int)componentType, 1);
decoder.ReadCoefficients( decoder.ReadCoefficients(
modeInfo, new Point(0, 0), aboveContexts, leftContexts, modeInfo,
0, 0, plane, 1, 1, transformBlockContext, transformSize, new Point(0, 0),
false, true, transformType, ref transformInfo, 0, 0, actuals); aboveContexts,
leftContexts,
0,
0,
plane,
1,
1,
transformBlockContext,
transformSize,
false,
true,
transformType,
ref transformInfo,
0,
0,
levels,
actuals);
// Assert // Assert
Assert.Equal(endOfBlock, actuals[0]); Assert.Equal(endOfBlock, actuals[0]);
@ -153,11 +187,28 @@ public class Av1CoefficientsEntropyTests
using IMemoryOwner<byte> encoded = encoder.Exit(); using IMemoryOwner<byte> encoded = encoder.Exit();
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex);
using Av1LevelBuffer levels = new(Configuration.Default);
int plane = Math.Min((int)componentType, 1); int plane = Math.Min((int)componentType, 1);
decoder.ReadCoefficients( decoder.ReadCoefficients(
modeInfo, new Point(0, 0), aboveContexts, leftContexts, modeInfo,
0, 0, plane, 1, 1, transformBlockContext, transformSize, new Point(0, 0),
false, true, transformType, ref transformInfo, 0, 0, actuals); aboveContexts,
leftContexts,
0,
0,
plane,
1,
1,
transformBlockContext,
transformSize,
false,
true,
transformType,
ref transformInfo,
0,
0,
levels,
actuals);
// Assert // Assert
Assert.Equal(endOfBlock, actuals[0]); Assert.Equal(endOfBlock, actuals[0]);

4
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameIntraEntropyTests.cs

@ -160,8 +160,8 @@ public class Av1InterFrameIntraEntropyTests
bool leftIsInter, bool leftIsInter,
int expected) int expected)
{ {
Av1BlockModeInfo above = hasAbove ? CreateModeInfo(aboveIsInter) : null; Av1BlockModeInfo? above = hasAbove ? CreateModeInfo(aboveIsInter) : null;
Av1BlockModeInfo left = hasLeft ? CreateModeInfo(leftIsInter) : null; Av1BlockModeInfo? left = hasLeft ? CreateModeInfo(leftIsInter) : null;
int actual = Av1SymbolContextHelper.GetIntraInterContext(above, left); int actual = Av1SymbolContextHelper.GetIntraInterContext(above, left);

17
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameModeInfoTests.cs

@ -43,6 +43,7 @@ public class Av1InterFrameModeInfoTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader));
modeInfo = partitionInfo.ModeInfo;
Assert.False(modeInfo.SkipMode); Assert.False(modeInfo.SkipMode);
Assert.False(modeInfo.Skip); Assert.False(modeInfo.Skip);
@ -89,7 +90,7 @@ public class Av1InterFrameModeInfoTests
using IMemoryOwner<byte> encoded = writer.Exit(); using IMemoryOwner<byte> encoded = writer.Exit();
Memory<byte> encodedMemory = encoded.Memory; Memory<byte> encodedMemory = encoded.Memory;
ReadInterFrameModeInfo(tileReader, encodedMemory, modeInfo, aboveModeInfo); modeInfo = ReadInterFrameModeInfo(tileReader, encodedMemory, modeInfo, aboveModeInfo);
Assert.True(modeInfo.SkipMode); Assert.True(modeInfo.SkipMode);
Assert.True(modeInfo.Skip); Assert.True(modeInfo.Skip);
@ -139,6 +140,7 @@ public class Av1InterFrameModeInfoTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader));
modeInfo = partitionInfo.ModeInfo;
Assert.Equal(Av1InterpolationFilter.Smooth, modeInfo.InterpolationFilters[0]); Assert.Equal(Av1InterpolationFilter.Smooth, modeInfo.InterpolationFilters[0]);
Assert.Equal((Av1InterpolationFilter)expectedHorizontalFilter, modeInfo.InterpolationFilters[1]); Assert.Equal((Av1InterpolationFilter)expectedHorizontalFilter, modeInfo.InterpolationFilters[1]);
@ -173,6 +175,7 @@ public class Av1InterFrameModeInfoTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader));
modeInfo = partitionInfo.ModeInfo;
Assert.Equal(Av1InterpolationFilter.Regular, modeInfo.InterpolationFilters[0]); Assert.Equal(Av1InterpolationFilter.Regular, modeInfo.InterpolationFilters[0]);
Assert.Equal(Av1InterpolationFilter.Regular, modeInfo.InterpolationFilters[1]); Assert.Equal(Av1InterpolationFilter.Regular, modeInfo.InterpolationFilters[1]);
@ -213,7 +216,7 @@ public class Av1InterFrameModeInfoTests
using IMemoryOwner<byte> encoded = writer.Exit(); using IMemoryOwner<byte> encoded = writer.Exit();
Memory<byte> encodedMemory = encoded.Memory; Memory<byte> encodedMemory = encoded.Memory;
ReadInterFrameModeInfo(tileReader, encodedMemory, modeInfo); modeInfo = ReadInterFrameModeInfo(tileReader, encodedMemory, modeInfo);
Assert.Equal((Av1ReferenceFrameType)expectedPrimary, modeInfo.ReferenceFrames[0]); Assert.Equal((Av1ReferenceFrameType)expectedPrimary, modeInfo.ReferenceFrames[0]);
Assert.Equal((Av1ReferenceFrameType)expectedSecondary, modeInfo.ReferenceFrames[1]); Assert.Equal((Av1ReferenceFrameType)expectedSecondary, modeInfo.ReferenceFrames[1]);
@ -285,7 +288,7 @@ public class Av1InterFrameModeInfoTests
writer.WriteSymbol((int)Av1InterpolationFilter.Sharp, Av1DefaultDistributions.SwitchableInterpolation[3]); writer.WriteSymbol((int)Av1InterpolationFilter.Sharp, Av1DefaultDistributions.SwitchableInterpolation[3]);
using IMemoryOwner<byte> encoded = writer.Exit(); using IMemoryOwner<byte> encoded = writer.Exit();
ReadInterFrameModeInfo(tileReader, encoded.Memory, modeInfo); modeInfo = ReadInterFrameModeInfo(tileReader, encoded.Memory, modeInfo);
Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]); Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]);
Assert.Equal(Av1ReferenceFrameType.Last2, modeInfo.ReferenceFrames[1]); Assert.Equal(Av1ReferenceFrameType.Last2, modeInfo.ReferenceFrames[1]);
@ -311,7 +314,8 @@ public class Av1InterFrameModeInfoTests
/// <param name="encoded">The range-coded block-prefix symbols.</param> /// <param name="encoded">The range-coded block-prefix symbols.</param>
/// <param name="modeInfo">The current coding block.</param> /// <param name="modeInfo">The current coding block.</param>
/// <param name="aboveModeInfo">The available above block supplying skip-mode context.</param> /// <param name="aboveModeInfo">The available above block supplying skip-mode context.</param>
private static void ReadInterFrameModeInfo( /// <returns>The decoded block mode information.</returns>
private static Av1BlockModeInfo ReadInterFrameModeInfo(
Av1TileReader tileReader, Av1TileReader tileReader,
Memory<byte> encoded, Memory<byte> encoded,
Av1BlockModeInfo modeInfo, Av1BlockModeInfo modeInfo,
@ -326,12 +330,14 @@ public class Av1InterFrameModeInfoTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Span, 0, updateCdf: true); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Span, 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, tileReader.FrameHeader)); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, tileReader.FrameHeader));
return partitionInfo.ModeInfo;
} }
/// <summary> /// <summary>
/// Invokes the ref-struct mode parser without spatial neighbors. /// Invokes the ref-struct mode parser without spatial neighbors.
/// </summary> /// </summary>
private static void ReadInterFrameModeInfo( /// <returns>The decoded block mode information.</returns>
private static Av1BlockModeInfo ReadInterFrameModeInfo(
Av1TileReader tileReader, Av1TileReader tileReader,
Memory<byte> encoded, Memory<byte> encoded,
Av1BlockModeInfo modeInfo) Av1BlockModeInfo modeInfo)
@ -340,6 +346,7 @@ public class Av1InterFrameModeInfoTests
Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None); Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None);
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Span, 0, updateCdf: true); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Span, 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, tileReader.FrameHeader)); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, tileReader.FrameHeader));
return partitionInfo.ModeInfo;
} }
/// <summary> /// <summary>

8
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InverseTransformTests.cs

@ -43,10 +43,10 @@ public class Av1InverseTransformTests
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertIdentityOperatorParity, TransformConfigurations); => FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertIdentityOperatorParity, TransformConfigurations);
/// <summary> /// <summary>
/// Verifies the pinned-libaom widened operations at the twelve-bit inverse row-stage bounds. /// Verifies the current-libaom widened operations at the twelve-bit inverse row-stage bounds.
/// </summary> /// </summary>
[Fact] [Fact]
public void TwelveBitWideIntermediatesMatchPinnedLibaom() public void TwelveBitWideIntermediatesMatchCurrentLibaom()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertTwelveBitWideIntermediateParity, TransformConfigurations); => FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertTwelveBitWideIntermediateParity, TransformConfigurations);
/// <summary> /// <summary>
@ -123,7 +123,7 @@ public class Av1InverseTransformTests
cosBit, cosBit,
stageRange); stageRange);
// These are the exact outputs of pinned libaom's signed Int64 terminal round. The first positive lane has an // These are the exact outputs of current libaom's signed Int64 terminal round. The first positive lane has an
// Int32 fixed-point sum of 2,147,482,471, so adding the 2,048 rounding bias in Int32 would wrap. // Int32 fixed-point sum of 2,147,482,471, so adding the 2,048 rounding bias in Int32 would wrap.
Vector128<int> adstExpected0 = Vector128.Create(524_288, -524_288, 524_287, -524_287); Vector128<int> adstExpected0 = Vector128.Create(524_288, -524_288, 524_287, -524_287);
Vector128<int> adstExpected1 = Vector128.Create(33_612, -33_612, 33_612, -33_612); Vector128<int> adstExpected1 = Vector128.Create(33_612, -33_612, 33_612, -33_612);
@ -167,7 +167,7 @@ public class Av1InverseTransformTests
} }
/// <summary> /// <summary>
/// Verifies one identity operator against exact pinned-libaom widened fixed-point results. /// Verifies one identity operator against exact current-libaom widened fixed-point results.
/// </summary> /// </summary>
/// <typeparam name="TOperator">The inverse identity operator.</typeparam> /// <typeparam name="TOperator">The inverse identity operator.</typeparam>
/// <param name="length">The identity-transform length.</param> /// <param name="length">The identity-transform length.</param>

6
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1LevelBufferTests.cs

@ -20,7 +20,7 @@ public class Av1LevelBufferTests
{ {
// Arrange // Arrange
Size size = new(width, height); Size size = new(width, height);
Av1LevelBuffer levels = new(Configuration.Default, size); using Av1LevelBuffer levels = new(Configuration.Default, size);
for (byte i = 0; i < 4; i++) for (byte i = 0; i < 4; i++)
{ {
levels.GetRow(i).Fill(i); levels.GetRow(i).Fill(i);
@ -43,7 +43,7 @@ public class Av1LevelBufferTests
{ {
// Arrange // Arrange
Size size = new(width, height); Size size = new(width, height);
Av1LevelBuffer levels = new(Configuration.Default, size); using Av1LevelBuffer levels = new(Configuration.Default, size);
for (byte i = 0; i < height; i++) for (byte i = 0; i < height; i++)
{ {
levels.GetRow(i).Fill(i); levels.GetRow(i).Fill(i);
@ -69,7 +69,7 @@ public class Av1LevelBufferTests
{ {
// Arrange // Arrange
Size size = new(width, height); Size size = new(width, height);
Av1LevelBuffer levels = new(Configuration.Default, size); using Av1LevelBuffer levels = new(Configuration.Default, size);
for (byte i = 0; i < height; i++) for (byte i = 0; i < height; i++)
{ {
levels.GetRow(i).Fill(i); levels.GetRow(i).Fill(i);

5
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionModeInfoTests.cs

@ -47,6 +47,7 @@ public class Av1MotionModeInfoTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader));
modeInfo = partitionInfo.ModeInfo;
Assert.Equal(Av1MotionMode.SimpleTranslation, modeInfo.MotionMode); Assert.Equal(Av1MotionMode.SimpleTranslation, modeInfo.MotionMode);
Assert.Equal(Av1InterpolationFilter.Sharp, modeInfo.InterpolationFilters[0]); Assert.Equal(Av1InterpolationFilter.Sharp, modeInfo.InterpolationFilters[0]);
@ -103,7 +104,7 @@ public class Av1MotionModeInfoTests
aboveModeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last; aboveModeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last;
aboveModeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.None; aboveModeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.None;
aboveModeInfo.InterpolationFilters.Fill(Av1InterpolationFilter.Regular); aboveModeInfo.InterpolationFilters.Clear();
tileReader.FrameInfo.UpdateModeInfo(aboveModeInfo, superblockInfo); tileReader.FrameInfo.UpdateModeInfo(aboveModeInfo, superblockInfo);
superblockInfo.BlockCount++; superblockInfo.BlockCount++;
@ -137,6 +138,7 @@ public class Av1MotionModeInfoTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader));
modeInfo = partitionInfo.ModeInfo;
Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]); Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]);
Assert.Equal(Av1ReferenceFrameType.None, modeInfo.ReferenceFrames[1]); Assert.Equal(Av1ReferenceFrameType.None, modeInfo.ReferenceFrames[1]);
@ -187,6 +189,7 @@ public class Av1MotionModeInfoTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader)); tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader));
modeInfo = partitionInfo.ModeInfo;
Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]); Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]);
Assert.Equal(Av1ReferenceFrameType.Intra, modeInfo.ReferenceFrames[1]); Assert.Equal(Av1ReferenceFrameType.Intra, modeInfo.ReferenceFrames[1]);

12
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PalettePredictorTests.cs

@ -3,6 +3,7 @@
using SixLabors.ImageSharp.Formats.Heif.Av1; using SixLabors.ImageSharp.Formats.Heif.Av1;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
using SixLabors.ImageSharp.Memory;
using SixLabors.ImageSharp.Tests.TestUtilities; using SixLabors.ImageSharp.Tests.TestUtilities;
namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1;
@ -40,12 +41,19 @@ public class Av1PalettePredictorTests
int mapStride = width + 5; int mapStride = width + 5;
int destinationStride = width + 9; int destinationStride = width + 9;
byte[] colorIndexMap = CreateColorIndexMap(mapStride, height, width, paletteSize); byte[] colorIndexMap = CreateColorIndexMap(mapStride, height, width, paletteSize);
using Buffer2D<byte> colorIndexMapBuffer = Configuration.Default.MemoryAllocator.Allocate2D<byte>(mapStride, height);
for (int row = 0; row < height; row++)
{
colorIndexMap.AsSpan(row * mapStride, mapStride).CopyTo(colorIndexMapBuffer.DangerousGetRowSpan(row));
}
Buffer2DRegion<byte> colorIndexMapRegion = new(colorIndexMapBuffer);
ushort[] bytePalette = CreatePalette(paletteSize, 8); ushort[] bytePalette = CreatePalette(paletteSize, 8);
byte[] expectedBytes = Enumerable.Repeat((byte)251, destinationStride * height).ToArray(); byte[] expectedBytes = Enumerable.Repeat((byte)251, destinationStride * height).ToArray();
byte[] actualBytes = (byte[])expectedBytes.Clone(); byte[] actualBytes = (byte[])expectedBytes.Clone();
ApplyReference(bytePalette, colorIndexMap, mapStride, expectedBytes, destinationStride, width, height); ApplyReference(bytePalette, colorIndexMap, mapStride, expectedBytes, destinationStride, width, height);
Av1PalettePredictor.Predict(bytePalette, colorIndexMap, mapStride, actualBytes, destinationStride, width, height); Av1PalettePredictor.Predict(bytePalette, colorIndexMapRegion, actualBytes, destinationStride, width, height);
Assert.Equal(expectedBytes, actualBytes); Assert.Equal(expectedBytes, actualBytes);
foreach (int bitDepth in new[] { 10, 12 }) foreach (int bitDepth in new[] { 10, 12 })
@ -55,7 +63,7 @@ public class Av1PalettePredictorTests
short[] actual = (short[])expected.Clone(); short[] actual = (short[])expected.Clone();
ApplyReference(palette, colorIndexMap, mapStride, expected, destinationStride, width, height); ApplyReference(palette, colorIndexMap, mapStride, expected, destinationStride, width, height);
Av1PalettePredictor.Predict(palette, colorIndexMap, mapStride, actual, destinationStride, width, height); Av1PalettePredictor.Predict(palette, colorIndexMapRegion, actual, destinationStride, width, height);
Assert.Equal(expected, actual); Assert.Equal(expected, actual);
} }
} }

97
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs

@ -461,7 +461,7 @@ public class Av1ReconstructionConformanceTests
/// under normal SIMD dispatch and with hardware intrinsics disabled. /// under normal SIMD dispatch and with hardware intrinsics disabled.
/// </summary> /// </summary>
[Fact] [Fact]
public void DecodeWithActiveCdefMatchesPinnedLibaomReference() public void DecodeWithActiveCdefMatchesCurrentLibaomReference()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateActiveCdefFixtures, ReconstructionConfigurations); => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateActiveCdefFixtures, ReconstructionConfigurations);
/// <summary> /// <summary>
@ -547,13 +547,70 @@ public class Av1ReconstructionConformanceTests
} }
/// <summary> /// <summary>
/// Verifies decoded luma and chroma palette syntax and exact native samples against scalar libaom for an /// Verifies decoded luma and chroma palette syntax and exact native samples against current official libaom for an
/// independently encoded AV1 still-picture stream. /// independently encoded AV1 still-picture stream.
/// </summary> /// </summary>
[Fact] [Fact]
public void DecodeWithPaletteMatchesPinnedLibaomReference() public void DecodeWithPaletteMatchesCurrentLibaomReference()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidatePaletteNativeFixture, PaletteConfigurations); => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidatePaletteNativeFixture, PaletteConfigurations);
/// <summary>
/// Verifies exact palette reconstruction through segmented frame-owned map storage and tracked disposal.
/// </summary>
[Fact]
[ValidateDisposedMemoryAllocations]
public void DecodePaletteWithConstrainedAllocator()
{
TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 1_024 };
allocator.EnableNonThreadSafeLogging();
Configuration configuration = Configuration.Default.Clone();
configuration.MemoryAllocator = allocator;
byte[] payload = TestFile.Create(TestImages.Heif.Av1Palette8BitPayload).Bytes;
byte[] reference = TestFile.Create(TestImages.Heif.Av1Palette8BitReference).Bytes;
bool foundSegmentedLumaMap = false;
bool foundSegmentedChromaMap = false;
using (Av1Decoder decoder = new(configuration))
{
using Av1FrameBuffer<byte> frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _);
Assert.Equal(RequiredPaletteCoverage, GetPaletteCoverage(decoder));
AssertNativePlanesEqual(decoder, frameBuffer, reference);
Assert.NotNull(decoder.FrameHeader);
Assert.NotNull(decoder.FrameInfo);
int modeInfoWidth = Av1Math.DivideLog2Ceiling(decoder.FrameHeader.FrameSize.FrameWidth, Av1Constants.ModeInfoSizeLog2);
int modeInfoHeight = Av1Math.DivideLog2Ceiling(decoder.FrameHeader.FrameSize.FrameHeight, Av1Constants.ModeInfoSizeLog2);
for (int y = 0; y < modeInfoHeight && (!foundSegmentedLumaMap || !foundSegmentedChromaMap); y++)
{
for (int x = 0; x < modeInfoWidth && (!foundSegmentedLumaMap || !foundSegmentedChromaMap); x++)
{
Av1BlockModeInfo modeInfo = decoder.FrameInfo.GetModeInfoAt(new Point(x, y));
if (!foundSegmentedLumaMap && modeInfo.GetPaletteSize(Av1PlaneType.Y) != 0)
{
Buffer2DRegion<byte> map = modeInfo.GetPaletteColorIndexMap(Av1Plane.Y);
foundSegmentedLumaMap = map.Buffer.MemoryGroup.Count > 1;
}
if (!foundSegmentedChromaMap && modeInfo.GetPaletteSize(Av1PlaneType.Uv) != 0)
{
Buffer2DRegion<byte> map = modeInfo.GetPaletteColorIndexMap(Av1Plane.U);
foundSegmentedChromaMap = map.Buffer.MemoryGroup.Count > 1;
}
}
}
Assert.True(foundSegmentedLumaMap);
Assert.True(foundSegmentedChromaMap);
}
Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count);
Assert.All(
allocator.AllocationLog,
allocation => Assert.Single(
allocator.ReturnLog,
returned => returned.AllocationId == allocation.AllocationId));
}
/// <summary> /// <summary>
/// Verifies that a real palette frame whose tile entropy payload ends early is rejected instead of being decoded /// Verifies that a real palette frame whose tile entropy payload ends early is rejected instead of being decoded
/// from the range decoder's implicit zero padding. /// from the range decoder's implicit zero padding.
@ -607,13 +664,13 @@ public class Av1ReconstructionConformanceTests
} }
/// <summary> /// <summary>
/// Verifies decoded luma and chroma palette syntax and exact presented pixels for an independently encoded AVIF /// Verifies decoded luma and chroma palette syntax and exact presented pixels against the retained reference image
/// image across the available vector widths and the scalar fallback. /// across the available vector widths and the scalar fallback.
/// </summary> /// </summary>
/// <param name="provider">The AVIF input and matching reference-output naming context.</param> /// <param name="provider">The AVIF input and matching reference-output naming context.</param>
[Theory] [Theory]
[WithFile(TestImages.Heif.Av1Palette8BitAvif, PixelTypes.Rgba32)] [WithFile(TestImages.Heif.Av1Palette8BitAvif, PixelTypes.Rgba32)]
public void DecodeWithPaletteMatchesPinnedLibavifPresentation(TestImageProvider<Rgba32> provider) public void DecodeWithPaletteMatchesRetainedPresentationReference(TestImageProvider<Rgba32> provider)
=> FeatureTestRunner.RunWithHwIntrinsicsFeature( => FeatureTestRunner.RunWithHwIntrinsicsFeature(
ValidatePresentedFixture, ValidatePresentedFixture,
PresentationConfigurations, PresentationConfigurations,
@ -1145,10 +1202,10 @@ public class Av1ReconstructionConformanceTests
/// <summary> /// <summary>
/// Verifies every intra prediction mode and the fixture's seven transform types against the official /// Verifies every intra prediction mode and the fixture's seven transform types against the official
/// pinned-libaom all-intra conformance sequence and its exact native output. /// current-libaom all-intra conformance sequence and its exact native output.
/// </summary> /// </summary>
[Fact] [Fact]
public void DecodeOfficialAllIntraSequenceMatchesPinnedLibaomReference() => ValidateOfficialAllIntraFixture(); public void DecodeOfficialAllIntraSequenceMatchesCurrentLibaomReference() => ValidateOfficialAllIntraFixture();
/// <summary> /// <summary>
/// Decodes every all-intra IVF sample in one session, compares each frame exactly, and records the syntax /// Decodes every all-intra IVF sample in one session, compares each frame exactly, and records the syntax
@ -1746,10 +1803,10 @@ public class Av1ReconstructionConformanceTests
} }
/// <summary> /// <summary>
/// Verifies the official eight-bit quantizer boundaries against exact pinned-libaom native output. /// Verifies the official eight-bit quantizer boundaries against exact current-libaom native output.
/// </summary> /// </summary>
[Fact] [Fact]
public void DecodeOfficialEightBitQuantizerBoundarySequencesMatchPinnedLibaomReferences() public void DecodeOfficialEightBitQuantizerBoundarySequencesMatchCurrentLibaomReferences()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature( => FeatureTestRunner.RunWithHwIntrinsicsFeature(
ValidateOfficialEightBitQuantizerBoundaryFixtures, ValidateOfficialEightBitQuantizerBoundaryFixtures,
ReconstructionConfigurations); ReconstructionConfigurations);
@ -1761,10 +1818,10 @@ public class Av1ReconstructionConformanceTests
=> ValidateOfficialEightBitQuantizerBoundaryFixturesWithConfiguration(Configuration.Default); => ValidateOfficialEightBitQuantizerBoundaryFixturesWithConfiguration(Configuration.Default);
/// <summary> /// <summary>
/// Verifies the official ten-bit quantizer boundaries against exact pinned-libaom native output. /// Verifies the official ten-bit quantizer boundaries against exact current-libaom native output.
/// </summary> /// </summary>
[Fact] [Fact]
public void DecodeOfficialTenBitQuantizerBoundarySequencesMatchPinnedLibaomReferences() public void DecodeOfficialTenBitQuantizerBoundarySequencesMatchCurrentLibaomReferences()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature( => FeatureTestRunner.RunWithHwIntrinsicsFeature(
ValidateOfficialTenBitQuantizerBoundaryFixtures, ValidateOfficialTenBitQuantizerBoundaryFixtures,
ReconstructionConfigurations); ReconstructionConfigurations);
@ -1790,6 +1847,16 @@ public class Av1ReconstructionConformanceTests
ValidateOfficialEightBitQuantizerBoundaryFixturesWithConfiguration(configuration); ValidateOfficialEightBitQuantizerBoundaryFixturesWithConfiguration(configuration);
ValidateOfficialTenBitQuantizerBoundaryFixturesWithConfiguration(configuration); ValidateOfficialTenBitQuantizerBoundaryFixturesWithConfiguration(configuration);
// Each decoded frame owns one maximum-sized coefficient-context scratch rent. An allocation per transform
// would produce hundreds of identically typed smaller rents for these deliberately dense fixtures.
int coefficientScratchLength =
((Av1Constants.MaxTransformSize / 2) + Av1Constants.TransformPadHorizontal) *
(Av1Constants.TransformPadTop + (Av1Constants.MaxTransformSize / 2) + Av1Constants.TransformPadBottom);
int coefficientScratchAllocations = allocator.AllocationLog.Count(
allocation => allocation.ElementType == typeof(byte) && allocation.Length == coefficientScratchLength);
Assert.Equal(4 * OfficialQuantizerFixtureFrameCount, coefficientScratchAllocations);
Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count);
Assert.All( Assert.All(
allocator.AllocationLog, allocator.AllocationLog,
@ -4194,13 +4261,15 @@ public class Av1ReconstructionConformanceTests
int modeInfoRow = (y << subsamplingY) >> Av1Constants.ModeInfoSizeLog2; int modeInfoRow = (y << subsamplingY) >> Av1Constants.ModeInfoSizeLog2;
Av1BlockModeInfo modeInfo = frameInfo.GetModeInfoAt(new Point(modeInfoColumn, modeInfoRow)); Av1BlockModeInfo modeInfo = frameInfo.GetModeInfoAt(new Point(modeInfoColumn, modeInfoRow));
int blockColumn = modeInfoColumn; int blockColumn = modeInfoColumn;
while (blockColumn > 0 && ReferenceEquals(frameInfo.GetModeInfoAt(new Point(blockColumn - 1, modeInfoRow)), modeInfo)) while (blockColumn > 0 &&
frameInfo.GetModeInfoAt(new Point(blockColumn - 1, modeInfoRow)).ModeInfoIndex == modeInfo.ModeInfoIndex)
{ {
blockColumn--; blockColumn--;
} }
int blockRow = modeInfoRow; int blockRow = modeInfoRow;
while (blockRow > 0 && ReferenceEquals(frameInfo.GetModeInfoAt(new Point(modeInfoColumn, blockRow - 1)), modeInfo)) while (blockRow > 0 &&
frameInfo.GetModeInfoAt(new Point(modeInfoColumn, blockRow - 1)).ModeInfoIndex == modeInfo.ModeInfoIndex)
{ {
blockRow--; blockRow--;
} }

35
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceMotionVectorsTests.cs

@ -245,20 +245,19 @@ public class Av1ReferenceMotionVectorsTests
frameInfo.InitializeMotionField(Configuration.Default, sequenceHeader, frameHeader, referenceFrames); frameInfo.InitializeMotionField(Configuration.Default, sequenceHeader, frameHeader, referenceFrames);
FillFrameWithIntraBlocks(frameInfo, sequenceHeader); FillFrameWithIntraBlocks(frameInfo, sequenceHeader);
Av1MotionVector direct = new(16, 24); Av1MotionVector direct = new(16, 24);
Av1BlockModeInfo candidate = AddModeInfo( AddModeInfo(
frameInfo, frameInfo,
sequenceHeader, sequenceHeader,
new Point(8, 4), new Point(8, 4),
Av1BlockSize.Block16x16, Av1BlockSize.Block16x16,
Av1ReferenceFrameType.Last, Av1ReferenceFrameType.Last,
direct, direct,
Av1PredictionMode.NearestMotionVector); Av1PredictionMode.NearestMotionVector,
Av1ReferenceFrameType.Backward,
new Av1MotionVector(40, -24));
// The direct scan adds the first reference with its normative adjacent weight. Extension visits both entries: // The direct scan adds the first reference with its normative adjacent weight. Extension visits both entries:
// it must ignore that duplicate and append only the sign-corrected backward-reference vector. // it must ignore that duplicate and append only the sign-corrected backward-reference vector.
candidate.ReferenceFrames[1] = Av1ReferenceFrameType.Backward;
candidate.MotionVectors[1] = new Av1MotionVector(40, -24);
Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty);
Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, new Point(8, 8)); Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, new Point(8, 8));
Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, true, Av1PartitionType.None) Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, true, Av1PartitionType.None)
@ -361,31 +360,29 @@ public class Av1ReferenceMotionVectorsTests
Av1MotionVector abovePrimary = new(8, 16); Av1MotionVector abovePrimary = new(8, 16);
Av1MotionVector aboveSecondary = new(24, 32); Av1MotionVector aboveSecondary = new(24, 32);
Av1BlockModeInfo above = AddModeInfo( AddModeInfo(
frameInfo, frameInfo,
sequenceHeader, sequenceHeader,
new Point(8, 4), new Point(8, 4),
Av1BlockSize.Block16x16, Av1BlockSize.Block16x16,
Av1ReferenceFrameType.Last, Av1ReferenceFrameType.Last,
abovePrimary, abovePrimary,
Av1PredictionMode.NewNewMotionVector); Av1PredictionMode.NewNewMotionVector,
Av1ReferenceFrameType.Backward,
above.ReferenceFrames[1] = Av1ReferenceFrameType.Backward; aboveSecondary);
above.MotionVectors[1] = aboveSecondary;
Av1MotionVector leftPrimary = new(40, 48); Av1MotionVector leftPrimary = new(40, 48);
Av1MotionVector leftSecondary = new(56, 64); Av1MotionVector leftSecondary = new(56, 64);
Av1BlockModeInfo left = AddModeInfo( AddModeInfo(
frameInfo, frameInfo,
sequenceHeader, sequenceHeader,
new Point(4, 8), new Point(4, 8),
Av1BlockSize.Block16x16, Av1BlockSize.Block16x16,
Av1ReferenceFrameType.Last, Av1ReferenceFrameType.Last,
leftPrimary, leftPrimary,
Av1PredictionMode.NearestNearestMotionVector); Av1PredictionMode.NearestNearestMotionVector,
Av1ReferenceFrameType.Backward,
left.ReferenceFrames[1] = Av1ReferenceFrameType.Backward; leftSecondary);
left.MotionVectors[1] = leftSecondary;
Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty);
Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, new Point(8, 8)); Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, new Point(8, 8));
@ -592,6 +589,8 @@ public class Av1ReferenceMotionVectorsTests
/// <param name="referenceFrame">The primary prediction reference.</param> /// <param name="referenceFrame">The primary prediction reference.</param>
/// <param name="motionVector">The primary motion vector.</param> /// <param name="motionVector">The primary motion vector.</param>
/// <param name="predictionMode">The decoded luma or inter prediction mode.</param> /// <param name="predictionMode">The decoded luma or inter prediction mode.</param>
/// <param name="secondaryReferenceFrame">The optional secondary prediction reference.</param>
/// <param name="secondaryMotionVector">The optional secondary motion vector.</param>
/// <returns>The mapped mode-information block.</returns> /// <returns>The mapped mode-information block.</returns>
private static Av1BlockModeInfo AddModeInfo( private static Av1BlockModeInfo AddModeInfo(
Av1FrameInfo frameInfo, Av1FrameInfo frameInfo,
@ -600,7 +599,9 @@ public class Av1ReferenceMotionVectorsTests
Av1BlockSize blockSize, Av1BlockSize blockSize,
Av1ReferenceFrameType referenceFrame, Av1ReferenceFrameType referenceFrame,
Av1MotionVector motionVector, Av1MotionVector motionVector,
Av1PredictionMode predictionMode) Av1PredictionMode predictionMode,
Av1ReferenceFrameType secondaryReferenceFrame = Av1ReferenceFrameType.None,
Av1MotionVector secondaryMotionVector = default)
{ {
int superblockSize = sequenceHeader.SuperblockModeInfoSize; int superblockSize = sequenceHeader.SuperblockModeInfoSize;
Point superblockPosition = new(position.X / superblockSize, position.Y / superblockSize); Point superblockPosition = new(position.X / superblockSize, position.Y / superblockSize);
@ -612,7 +613,9 @@ public class Av1ReferenceMotionVectorsTests
}; };
modeInfo.ReferenceFrames[0] = referenceFrame; modeInfo.ReferenceFrames[0] = referenceFrame;
modeInfo.ReferenceFrames[1] = secondaryReferenceFrame;
modeInfo.MotionVectors[0] = motionVector; modeInfo.MotionVectors[0] = motionVector;
modeInfo.MotionVectors[1] = secondaryMotionVector;
frameInfo.UpdateModeInfo(modeInfo, superblockInfo); frameInfo.UpdateModeInfo(modeInfo, superblockInfo);
superblockInfo.BlockCount++; superblockInfo.BlockCount++;
return modeInfo; return modeInfo;

38
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceTransform.cs

@ -8,21 +8,11 @@ namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1;
internal class Av1ReferenceTransform internal class Av1ReferenceTransform
{ {
/****************************************************************************** /// <summary>
* SVT file: test/ref/TxfmRef.cc /// Gets the analytical amplification used by current libaom's forward-transform tests.
* /// </summary>
* Reference implementation for txfm, including : /// <param name="config">The transform configuration.</param>
* - reference_dct_1d /// <returns>The two-dimensional transform amplification.</returns>
* - reference_adst_1d
* - reference_idtx_1d
* - reference_txfm_1d
* - reference_txfm_2d
* - fadst_ref
*
* Original authors: Cidana-Edmond, Cidana-Wenyao
*
******************************************************************************/
public static double GetScaleFactor(Av1Transform2dFlipConfiguration config) public static double GetScaleFactor(Av1Transform2dFlipConfiguration config)
{ {
int transformWidth = config.TransformSize.GetWidth(); int transformWidth = config.TransformSize.GetWidth();
@ -42,8 +32,14 @@ internal class Av1ReferenceTransform
} }
/// <summary> /// <summary>
/// SVT: reference_txfm_2d /// Applies the analytical two-dimensional transform used by current libaom's
/// <c>test/av1_txfm_test.cc</c>.
/// </summary> /// </summary>
/// <param name="input">The raster input samples.</param>
/// <param name="output">The raster output coefficients.</param>
/// <param name="transformType">The two-dimensional transform type.</param>
/// <param name="transformSize">The transform dimensions.</param>
/// <param name="scaleFactor">The configured two-dimensional amplification.</param>
public static void ReferenceTransformFunction2d(Span<double> input, Span<double> output, Av1TransformType transformType, Av1TransformSize transformSize, double scaleFactor) public static void ReferenceTransformFunction2d(Span<double> input, Span<double> output, Av1TransformType transformType, Av1TransformSize transformSize, double scaleFactor)
{ {
// Get transform type and size of each dimension. // Get transform type and size of each dimension.
@ -99,7 +95,7 @@ internal class Av1ReferenceTransform
} }
} }
private static void Adst4Reference(Span<int> input, Span<int> output) private static void Adst4Reference(ReadOnlySpan<int> input, Span<int> output)
{ {
// 16384 * sqrt(2) * sin(kPi/9) * 2 / 3 // 16384 * sqrt(2) * sin(kPi/9) * 2 / 3
const long sinPi19 = 5283; const long sinPi19 = 5283;
@ -146,7 +142,7 @@ internal class Av1ReferenceTransform
output[3] = Av1Math.RoundShift(s3, 14); output[3] = Av1Math.RoundShift(s3, 14);
} }
private static void ReferenceIdentity1d(Span<double> input, Span<double> output, int size) private static void ReferenceIdentity1d(ReadOnlySpan<double> input, Span<double> output, int size)
{ {
const double sqrt2 = 1.4142135623730950488016887242097f; const double sqrt2 = 1.4142135623730950488016887242097f;
double scale = 0; double scale = 0;
@ -178,7 +174,7 @@ internal class Av1ReferenceTransform
} }
} }
private static void ReferenceDct1d(Span<double> input, Span<double> output, int size) private static void ReferenceDct1d(ReadOnlySpan<double> input, Span<double> output, int size)
{ {
const double kInvSqrt2 = 0.707106781186547524400844362104f; const double kInvSqrt2 = 0.707106781186547524400844362104f;
for (int k = 0; k < size; ++k) for (int k = 0; k < size; ++k)
@ -196,7 +192,7 @@ internal class Av1ReferenceTransform
} }
} }
private static void ReferenceAdst1d(Span<double> input, Span<double> output, int size) private static void ReferenceAdst1d(ReadOnlySpan<double> input, Span<double> output, int size)
{ {
if (size == 4) if (size == 4)
{ {
@ -227,7 +223,7 @@ internal class Av1ReferenceTransform
} }
} }
internal static void ReferenceTransform1d(Av1TransformType1d type, Span<double> input, Span<double> output, int size) internal static void ReferenceTransform1d(Av1TransformType1d type, ReadOnlySpan<double> input, Span<double> output, int size)
{ {
switch (type) switch (type)
{ {

32
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TemporalSegmentationTests.cs

@ -61,6 +61,32 @@ public class Av1TemporalSegmentationTests
Assert.NotEqual(16384U, destination.SegmentIdPredicted[2][0]); Assert.NotEqual(16384U, destination.SegmentIdPredicted[2][0]);
} }
/// <summary>
/// Verifies that a spatial segment symbol cannot select an identifier above the frame's last active segment.
/// </summary>
[Fact]
public void SpatialSegmentIdOutsideActiveRangeIsRejected()
{
ObuSequenceHeader sequenceHeader = CreateSequenceHeader(64, 64);
ObuFrameHeader frameHeader = CreateFrameHeader(16, 16, segmentationUpdateMap: 1, segmentationTemporalUpdate: 0);
frameHeader.SegmentationParameters.LastActiveSegmentId = 0;
using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader);
using Av1SymbolWriter writer = new(Configuration.Default, 1, updateCdf: true);
writer.WriteSymbol(Av1Constants.MaxSegmentCount - 1, Av1DefaultDistributions.SegmentId[0]);
using IMemoryOwner<byte> encoded = writer.Exit();
Assert.Throws<InvalidImageContentException>(
() =>
{
Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty);
Av1SuperblockInfo superblockInfo = new(tileReader.FrameInfo, Point.Empty);
Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None);
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true);
tileReader.ReadInterSegmentId(ref decoder, ref partitionInfo, beforeSkip: false);
});
}
/// <summary> /// <summary>
/// Verifies that only neighboring blocks which selected temporal prediction contribute to the binary CDF context. /// Verifies that only neighboring blocks which selected temporal prediction contribute to the binary CDF context.
/// </summary> /// </summary>
@ -82,8 +108,8 @@ public class Av1TemporalSegmentationTests
bool leftPredicted, bool leftPredicted,
int expected) int expected)
{ {
Av1BlockModeInfo aboveModeInfo = hasAbove ? CreateModeInfo(abovePredicted) : null; Av1BlockModeInfo? aboveModeInfo = hasAbove ? CreateModeInfo(abovePredicted) : null;
Av1BlockModeInfo leftModeInfo = hasLeft ? CreateModeInfo(leftPredicted) : null; Av1BlockModeInfo? leftModeInfo = hasLeft ? CreateModeInfo(leftPredicted) : null;
int actual = Av1SymbolContextHelper.GetSegmentIdPredictedContext(aboveModeInfo, leftModeInfo); int actual = Av1SymbolContextHelper.GetSegmentIdPredictedContext(aboveModeInfo, leftModeInfo);
@ -146,6 +172,7 @@ public class Av1TemporalSegmentationTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true);
tileReader.ReadInterSegmentId(ref decoder, ref partitionInfo, beforeSkip: segmentIdPrecedesSkip); tileReader.ReadInterSegmentId(ref decoder, ref partitionInfo, beforeSkip: segmentIdPrecedesSkip);
modeInfo = partitionInfo.ModeInfo;
Assert.True(modeInfo.SegmentIdPredicted); Assert.True(modeInfo.SegmentIdPredicted);
Assert.Equal(2, modeInfo.SegmentId); Assert.Equal(2, modeInfo.SegmentId);
@ -194,6 +221,7 @@ public class Av1TemporalSegmentationTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true); Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true);
tileReader.ReadInterSegmentId(ref decoder, ref partitionInfo, beforeSkip: false); tileReader.ReadInterSegmentId(ref decoder, ref partitionInfo, beforeSkip: false);
modeInfo = partitionInfo.ModeInfo;
Assert.False(modeInfo.SegmentIdPredicted); Assert.False(modeInfo.SegmentIdPredicted);
Assert.Equal(3, modeInfo.SegmentId); Assert.Equal(3, modeInfo.SegmentId);

79
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TilingTests.cs

@ -1,7 +1,9 @@
// Copyright (c) Six Labors. // Copyright (c) Six Labors.
// Licensed under the Six Labors Split License. // Licensed under the Six Labors Split License.
using System.Buffers;
using SixLabors.ImageSharp.Formats.Heif.Av1; using SixLabors.ImageSharp.Formats.Heif.Av1;
using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy;
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
@ -15,6 +17,23 @@ namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1;
[Trait("Format", "Avif")] [Trait("Format", "Avif")]
public class Av1TilingTests public class Av1TilingTests
{ {
/// <summary>
/// Verifies that frame mode-information indices do not wrap at the unsigned 16-bit boundary.
/// </summary>
[Fact]
public void ModeInfoMapSupportsMoreThanUShortMaxBlocks()
{
const int blockCount = ushort.MaxValue + 2;
Av1FrameInfo.Av1FrameModeInfoMap map = new(new Size(blockCount, 1));
for (int index = 0; index < blockCount; index++)
{
map.Update(new Point(index, 0), Av1BlockSize.Block4x4);
}
Assert.Equal(blockCount, map.NextIndex);
Assert.Equal(blockCount - 1, map[new Point(blockCount - 1, 0)]);
}
/// <summary> /// <summary>
/// Verifies the decoded block geometry and prediction modes against libaom inspection output for a real AVIF image item. /// Verifies the decoded block geometry and prediction modes against libaom inspection output for a real AVIF image item.
/// </summary> /// </summary>
@ -75,6 +94,59 @@ public class Av1TilingTests
Assert.True(image.Frames.RootFrame.PixelBuffer.DangerousGetSingleSpan().ContainsAnyExcept(default(Rgba32))); Assert.True(image.Frames.RootFrame.PixelBuffer.DangerousGetSingleSpan().ContainsAnyExcept(default(Rgba32)));
} }
/// <summary>
/// Verifies that partition syntax cannot produce a luma block with no valid 4:2:0 chroma representation.
/// </summary>
[Fact]
public void RejectsPartitionThatCannotRepresentSubsampledChroma()
{
ObuSequenceHeader sequenceHeader = new()
{
MaxFrameWidth = 64,
MaxFrameHeight = 64,
Use128x128Superblock = false,
ColorConfig = new ObuColorConfig
{
BitDepth = Av1BitDepth.EightBit,
SubSamplingX = true,
SubSamplingY = true
}
};
ObuTileGroupHeader tileInfo = new()
{
TileColumnCount = 1,
TileRowCount = 1
};
tileInfo.TileColumnStartModeInfo[1] = sequenceHeader.SuperblockModeInfoSize;
tileInfo.TileRowStartModeInfo[1] = sequenceHeader.SuperblockModeInfoSize;
ObuFrameHeader frameHeader = new()
{
ModeInfoColumnCount = sequenceHeader.SuperblockModeInfoSize,
ModeInfoRowCount = sequenceHeader.SuperblockModeInfoSize,
ModeInfoStride = sequenceHeader.SuperblockModeInfoSize,
TilesInfo = tileInfo,
DisableCdfUpdate = true,
DisableFrameEndUpdateCdf = true
};
using Av1SymbolWriter writer = new(Configuration.Default, 1, updateCdf: false);
Av1Distribution[] partitionTypes = Av1DefaultDistributions.PartitionTypes;
Av1BlockSize blockSize = sequenceHeader.SuperblockSize;
while (blockSize > Av1BlockSize.Block8x8)
{
int blockSizeLog = blockSize.Get4x4WidthLog2() - Av1BlockSize.Block8x8.Get4x4WidthLog2();
int context = blockSizeLog * Av1Constants.PartitionProbabilitySet;
writer.WriteSymbol((int)Av1PartitionType.Split, partitionTypes[context]);
blockSize = Av1PartitionType.Split.GetBlockSubSize(blockSize);
}
writer.WriteSymbol((int)Av1PartitionType.Horizontal, partitionTypes[0]);
using IMemoryOwner<byte> encoded = writer.Exit();
using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader);
Assert.Throws<InvalidImageContentException>(() => tileReader.ReadTile(encoded.GetSpan(), 0));
}
[Theory] [Theory]
[InlineData(TestImages.Heif.XnConvert, 0x010E, 0x03CC, 18, 16)] [InlineData(TestImages.Heif.XnConvert, 0x010E, 0x03CC, 18, 16)]
[InlineData(TestImages.Heif.Orange4x4, 0x010E, 0x001d, 21, 1)] [InlineData(TestImages.Heif.Orange4x4, 0x010E, 0x001d, 21, 1)]
@ -190,8 +262,7 @@ public class Av1TilingTests
Span<Av1BlockModeInfo> modeInfos = superblockInfo.GetModeInfos(); Span<Av1BlockModeInfo> modeInfos = superblockInfo.GetModeInfos();
Assert.Equal(superblockInfo.BlockCount, modeInfos.Length); Assert.Equal(superblockInfo.BlockCount, modeInfos.Length);
Assert.DoesNotContain(modeInfos.ToArray(), modeInfo => modeInfo is null); Assert.Equal(modeInfos[0].ModeInfoIndex, tileReader.FrameInfo.GetModeInfo(superblockPosition).ModeInfoIndex);
Assert.Same(modeInfos[0], tileReader.FrameInfo.GetModeInfo(superblockPosition));
foreach (Av1BlockModeInfo modeInfo in modeInfos) foreach (Av1BlockModeInfo modeInfo in modeInfos)
{ {
@ -203,7 +274,9 @@ public class Av1TilingTests
{ {
for (int x = 0; x < modeInfo.BlockSize.Get4x4WideCount(); x++) for (int x = 0; x < modeInfo.BlockSize.Get4x4WideCount(); x++)
{ {
Assert.Same(modeInfo, tileReader.FrameInfo.GetModeInfoAt(new Point(modeInfoPosition.X + x, modeInfoPosition.Y + y))); Assert.Equal(
modeInfo.ModeInfoIndex,
tileReader.FrameInfo.GetModeInfoAt(new Point(modeInfoPosition.X + x, modeInfoPosition.Y + y)).ModeInfoIndex);
} }
} }
} }

0
tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeWithPaletteMatchesPinnedLibavifPresentation_Rgba32_libavif-palette-draw-points-8b.png → tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeWithPaletteMatchesRetainedPresentationReference_Rgba32_libavif-palette-draw-points-8b.png

28
tests/Images/Input/Heif/Av1/Conformance/README.md

@ -102,7 +102,33 @@ The film-grain Y4M SHA-256 is `A1B553BE140F48ABDDB2A6D39917AB714BA03AC7FFD6359EA
The retained `quantizer-00` and `quantizer-63` streams are the minimum- and maximum-quantizer boundaries from libaom's official eight- and ten-bit test matrices. Their SHA-1 values are `C2E1EC9936B95254187A359E94AA32A9F3DAD1B7`, `2A8AA33513D8E01AE9410C4BF5FE1E471B775482`, `9BBE8499796AA588FF02E313FB0D4349940D2FEA`, and `8B6EB3FFF2E0DB7EAC775B08C745250CA591E2D9`, exactly matching `test/test-data.sha1` at pinned libaom commit `03087864cf4bea6abb0d28f95cf7843511413d8f`. Their SHA-256 values, in the same order, are `6382DBD2BEFBBC93D4EA283586F4FB43FEA5F1C52400E3D2C5281A46B1104C00`, `0E4EC80680F7AF8DE9621B016E0F2D7C0858B2951DEBC173DDA50C6A051547D3`, `FE6053CE4EE20A1C0EC6F7FE35DB097E92AD25D8A3505598BD89162C74D7944F`, and `39759AB77483E1D11049DC38B5F5262158FD9C3CBC9D1F82A02462FC5DF30E0C`. The retained `quantizer-00` and `quantizer-63` streams are the minimum- and maximum-quantizer boundaries from libaom's official eight- and ten-bit test matrices. Their SHA-1 values are `C2E1EC9936B95254187A359E94AA32A9F3DAD1B7`, `2A8AA33513D8E01AE9410C4BF5FE1E471B775482`, `9BBE8499796AA588FF02E313FB0D4349940D2FEA`, and `8B6EB3FFF2E0DB7EAC775B08C745250CA591E2D9`, exactly matching `test/test-data.sha1` at pinned libaom commit `03087864cf4bea6abb0d28f95cf7843511413d8f`. Their SHA-256 values, in the same order, are `6382DBD2BEFBBC93D4EA283586F4FB43FEA5F1C52400E3D2C5281A46B1104C00`, `0E4EC80680F7AF8DE9621B016E0F2D7C0858B2951DEBC173DDA50C6A051547D3`, `FE6053CE4EE20A1C0EC6F7FE35DB097E92AD25D8A3505598BD89162C74D7944F`, and `39759AB77483E1D11049DC38B5F5262158FD9C3CBC9D1F82A02462FC5DF30E0C`.
The native references were generated with the pinned generic `aomdec --threads=1` build. Their SHA-256 values are `D499028E0606DB70CD56A72F151E04F36C09F300A448CCCD8430DD920D3589C5`, `4CC9892B3EE3399B293E31014B9F566C21E0C7A4765FC5F444528769C33E6D67`, `78373C28F401EB95D3E563D146622ED6C714ED96661E5E57C539CE71D7BED599`, and `A9DF86F671B8CF01EFC130660556412D4EBAF31A81D6F26FBDAEB0A7E839D8EA`. Each reference's two raw-frame MD5 values also match the corresponding official `.ivf.md5` file exactly. The tests compare every native sample under normal and scalar `FeatureTestRunner` dispatch and run all four sequences through a 2,560-byte row-aligned constrained tracked allocator. The native references were originally generated with the historical generic `aomdec --threads=1` build. On 2026-08-31 current official libaom `main` at observed revision `441c439b9916474cac15d2822af47a9ad70674a8` reproduced all four references byte for byte. Their SHA-256 values are `D499028E0606DB70CD56A72F151E04F36C09F300A448CCCD8430DD920D3589C5`, `4CC9892B3EE3399B293E31014B9F566C21E0C7A4765FC5F444528769C33E6D67`, `78373C28F401EB95D3E563D146622ED6C714ED96661E5E57C539CE71D7BED599`, and `A9DF86F671B8CF01EFC130660556412D4EBAF31A81D6F26FBDAEB0A7E839D8EA`. Each reference's two raw-frame MD5 values also match the corresponding official `.ivf.md5` file exactly. The tests compare every native sample under normal and scalar `FeatureTestRunner` dispatch and run all four sequences through a 2,560-byte row-aligned constrained tracked allocator.
## Palette reconstruction fixture
The 42-byte `libaom-palette-draw-points-8b-444.bit` payload has SHA-256
`F412A9E7F19D1C009D0329B993BB503D74CCDA58505C54BAE8FB3C16142181DC`. On 2026-08-31
current official libaom `main` at observed revision `441c439b9916474cac15d2822af47a9ad70674a8`
decoded it with one thread, row threading disabled, raw output, and eight-bit output depth. The resulting
1,089-byte YUV444 output matches the retained native reference exactly at SHA-256
`E05F7C0DF06ECCF0E43869D1D7B03DAA1D635ACD26A766F8940899BE18D53251`.
The production tests require active luma and chroma palette syntax, compare every native sample under
`FeatureTestRunner`, and compare the final AVIF presentation through the established reference-output API.
The retained PNG has SHA-256
`1148EBF6AA4B0F2D069D5E9B9605F6FB2A315E525F18016CDCAE23EFDD81DA84`. A 1 KiB
constrained tracked allocator forces both frame-owned palette map surfaces across multiple memory groups;
the test verifies exact reconstruction and exactly one return for every recorded allocation.
## Official all-intra fixture
On 2026-08-31 current official libaom `main` at observed revision
`441c439b9916474cac15d2822af47a9ad70674a8` reproduced the retained 39-frame all-intra Y4M byte for
byte. The IVF SHA-256 is `5FCD265FD9F9BDD0D3179340B4C4532F1422CA5E5D97741C7481B84CB5DC122F`;
the native reference SHA-256 is
`1211EBEFBC9CCEF9ED19BE4CCE3F807D69FFFE338E95CCA1B5F4CA8023482175`. The production test
decodes all 39 frames in one decoder session, compares every native sample exactly, and requires coverage
of every intra prediction mode and all seven transform types selected by the fixture.
## Official frame-size corner fixtures ## Official frame-size corner fixtures

Loading…
Cancel
Save