Browse Source

Complete AV1 core reconstruction checkpoint

pull/2633/head
James Jackson-South 3 days ago
parent
commit
57a3f6668e
  1. 47
      HEIF_IMPLEMENTATION_PLAN.md
  2. 106
      src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolContextHelper.cs
  3. 21
      src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolDecoder.cs
  4. 8
      src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1FrameDecoder.cs
  5. 6
      src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs
  6. 10
      src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1DeQuantizationContext.cs
  7. 4
      src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1InverseQuantizer.cs
  8. 23
      src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PalettePredictor.Operator.cs
  9. 25
      src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs
  10. 23
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1BlockModeInfo.cs
  11. 10
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.MotionField.cs
  12. 81
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.cs
  13. 8
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameModeInfoMap.cs
  14. 27
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LevelBuffer.cs
  15. 12
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PartitionInfo.cs
  16. 183
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs
  17. 23
      tests/ImageSharp.Benchmarks/Codecs/Heif/Av1PalettePredictionBenchmarks.cs
  18. 69
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CoefficientsEntropyTests.cs
  19. 4
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameIntraEntropyTests.cs
  20. 17
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameModeInfoTests.cs
  21. 8
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InverseTransformTests.cs
  22. 6
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1LevelBufferTests.cs
  23. 5
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionModeInfoTests.cs
  24. 12
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PalettePredictorTests.cs
  25. 97
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs
  26. 35
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceMotionVectorsTests.cs
  27. 38
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceTransform.cs
  28. 32
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TemporalSegmentationTests.cs
  29. 79
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TilingTests.cs
  30. 0
      tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeWithPaletteMatchesRetainedPresentationReference_Rgba32_libavif-palette-draw-points-8b.png
  31. 28
      tests/Images/Input/Heif/Av1/Conformance/README.md

47
HEIF_IMPLEMENTATION_PLAN.md

@ -559,7 +559,7 @@ For every item:
Previously verified algorithm checkpoints remain valuable evidence, but the final decoder gate requires a fresh current-tree run after the inter and cleanup corrections.
- [x] Bounded OBU framing, sequence headers, frame headers, tile groups, alignment, and trailing-bit parsing have been re-audited and verified against current libaom `main`.
- [~] Partition traversal, mode information, segmentation, delta quantization, transform-size selection, coefficient decoding, inverse quantization, and inverse transforms have historical checkpoint evidence against an obsolete pinned tree. Re-audit the current libaom `main` implementation before restoring verified status.
- [x] Partition traversal, mode information, segmentation, delta quantization, transform-size selection, coefficient decoding, inverse quantization, and inverse transforms have been re-audited and verified against current libaom `main`.
- [x] Intra prediction covers directional, DC, smooth, Paeth, chroma-from-luma, filter-intra, and palette families with the established operator architecture.
- [x] Intra-block copy has exact native reconstruction and feature-isolated SIMD evidence.
- [x] Lossless inverse transform, loop filtering, CDEF, super-resolution, restoration, and film grain have focused checkpoint evidence.
@ -604,6 +604,51 @@ Verified bounded-OBU checkpoint evidence on 2026-08-31:
`diff=lfs`, and `.gitattributes` was not edited.
- [x] Release source builds pass for net10.0 and net11.0 with zero warnings and zero errors. Roslynk reports
zero compiler errors, and scoped production and test analyzer verification reports no changes.
- [x] The completed checkpoint was committed as `243524c2c0b52a49d8d161fab806ab092cabe47c` with author
and committer `James Jackson-South <james_south@hotmail.com>`.
Verified partition, mode, segmentation, quantization, and transform checkpoint evidence on 2026-08-31:
- [x] Audited partition traversal and chroma representability against `read_partition` and the subsampled
plane-size rejection in current libaom `av1/decoder/decodeframe.c`; spatial segment-ID decoding and
corruption handling against `read_segment_id` in `av1/decoder/decodemv.c`; delta-Q syntax, resolution,
arithmetic, and clamping against `read_delta_qindex` and `read_delta_q_params` in the same file.
- [x] Audited selected and variable transform-size traversal against `read_tx_size`, `read_tx_size_vartx`,
and transform-block traversal in `av1/decoder/decodeframe.c`; coefficient syntax and arithmetic against
`av1_read_coeffs_txb` in `av1/decoder/decodetxb.c`; inverse quantization and transform application against
current `av1/decoder/decodeframe.c`, `av1/common/idct.c`, and the current libaom transform test oracle.
The observed clean `HEAD` and `origin/main` revision was
`441c439b9916474cac15d2822af47a9ad70674a8`; this is verification evidence, not a pin.
- [x] Partition decoding now rejects an invalid partition subsize and a block size that cannot represent the
current subsampled chroma plane. Spatial segmentation rejects decoded IDs above the active segment range.
Focused tests exercise both current-libaom corruption boundaries through the production tile reader.
- [x] Coefficient entropy decoding uses one allocator-owned maximum-size `Av1LevelBuffer` per tile reader.
Each transform resets and clears only its active padded geometry, so no transform creates an allocation.
Allocation tracking over all eight minimum- and maximum-quantizer frames proves exactly one coefficient
scratch allocation per frame and exactly-once return after decoder disposal.
- [x] Palette index maps are allocator-backed frame surfaces addressed row by row through `Buffer2DRegion`.
The wavefront context, stable neighbor ordering, right/bottom padding, transform offsets, and prediction
were audited against current libaom `av1/decoder/detokenize.c`, `av1/common/entropymode.c`,
`av1/decoder/decodeframe.c`, and `av1/common/reconintra.c`. A 1 KiB constrained allocator forces both
luma and chroma maps across multiple memory groups without copies or per-block allocations and proves
exactly-once disposal.
- [x] `Av1BlockModeInfo` is value storage, removing the managed object allocation formerly created for every
decoded coding block. Explicit `ModeInfoIndex` values preserve libaom's mode-info identity semantics at
prediction-unit loop-filter edges, and the frame map now uses integer offsets so more than 65,535 decoded
blocks cannot wrap its lookup identity.
- [x] Current official libaom reproduced the 39-frame all-intra reference and all four 8/10-bit minimum- and
maximum-quantizer references byte for byte. The production tests compare every native sample exactly,
cover every intra mode and seven selected transform types, execute SIMD and scalar paths through
`FeatureTestRunner`, and exercise the quantizer sequences under constrained tracked allocation.
- [x] Current official libaom decoded the 42-byte palette payload into the retained 1,089-byte YUV444
reference at SHA-256 `E05F7C0DF06ECCF0E43869D1D7B03DAA1D635ACD26A766F8940899BE18D53251`.
The exact native test requires luma and chroma palette syntax. The established reference-output test uses
the unchanged presentation PNG at SHA-256
`1148EBF6AA4B0F2D069D5E9B9605F6FB2A315E525F18016CDCAE23EFDD81DA84`, whose renamed path still
resolves to `diff=lfs`; `.gitattributes` was not edited.
- [x] The exact final AV1 namespace passes 8,732 of 8,732 cases on net10.0 and 8,732 of 8,732 cases on
net11.0, with zero failures or skips. Release source builds pass for net10.0 and net11.0 with zero warnings
and zero errors. Roslynk reports zero compiler errors, and scoped analyzer verification reports no changes.
Decoder exit gate:

106
src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolContextHelper.cs

@ -200,17 +200,19 @@ internal static class Av1SymbolContextHelper
int aboveContext = 0;
if (above is not null)
{
aboveContext = above.ReferenceFrames[1] > Av1ReferenceFrameType.Intra
? above.CompoundGroupIndex ? 1 : 0
: above.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 3 : 0;
Av1BlockModeInfo aboveModeInfo = above.Value;
aboveContext = aboveModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra
? aboveModeInfo.CompoundGroupIndex ? 1 : 0
: aboveModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 3 : 0;
}
int leftContext = 0;
if (left is not null)
{
leftContext = left.ReferenceFrames[1] > Av1ReferenceFrameType.Intra
? left.CompoundGroupIndex ? 1 : 0
: left.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 3 : 0;
Av1BlockModeInfo leftModeInfo = left.Value;
leftContext = leftModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra
? leftModeInfo.CompoundGroupIndex ? 1 : 0
: leftModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 3 : 0;
}
return Math.Min(5, aboveContext + leftContext);
@ -244,17 +246,19 @@ internal static class Av1SymbolContextHelper
int aboveContext = 0;
if (above is not null)
{
aboveContext = above.ReferenceFrames[1] > Av1ReferenceFrameType.Intra
? above.CompoundIndex ? 1 : 0
: above.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 1 : 0;
Av1BlockModeInfo aboveModeInfo = above.Value;
aboveContext = aboveModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra
? aboveModeInfo.CompoundIndex ? 1 : 0
: aboveModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 1 : 0;
}
int leftContext = 0;
if (left is not null)
{
leftContext = left.ReferenceFrames[1] > Av1ReferenceFrameType.Intra
? left.CompoundIndex ? 1 : 0
: left.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 1 : 0;
Av1BlockModeInfo leftModeInfo = left.Value;
leftContext = leftModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra
? leftModeInfo.CompoundIndex ? 1 : 0
: leftModeInfo.ReferenceFrames[0] == Av1ReferenceFrameType.Alternate ? 1 : 0;
}
return aboveContext + leftContext + (forwardDistance == backwardDistance ? 3 : 0);
@ -703,8 +707,10 @@ internal static class Av1SymbolContextHelper
{
if (above is not null && left is not null)
{
bool aboveIsIntra = above.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra;
bool leftIsIntra = left.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra;
Av1BlockModeInfo aboveModeInfo = above.Value;
Av1BlockModeInfo leftModeInfo = left.Value;
bool aboveIsIntra = aboveModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra;
bool leftIsIntra = leftModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra;
// AV1 reserves context three for two intra neighbors, context one for a mixed pair, and context zero for
// two inter neighbors. These values directly index intra_inter_cdf and are not probability ranks.
@ -720,12 +726,12 @@ internal static class Av1SymbolContextHelper
// context zero, matching the unavailable-neighbor behavior in libaom's av1_get_intra_inter_context.
if (above is not null)
{
return above.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra ? 2 : 0;
return above.Value.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra ? 2 : 0;
}
if (left is not null)
{
return left.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra ? 2 : 0;
return left.Value.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra ? 2 : 0;
}
return 0;
@ -743,29 +749,31 @@ internal static class Av1SymbolContextHelper
// their forward/backward direction, while intra neighbors take the same branch as a non-forward reference.
if (above is not null && left is not null)
{
bool aboveIsCompound = above.ReferenceFrames[1] > Av1ReferenceFrameType.Intra;
bool leftIsCompound = left.ReferenceFrames[1] > Av1ReferenceFrameType.Intra;
Av1BlockModeInfo aboveModeInfo = above.Value;
Av1BlockModeInfo leftModeInfo = left.Value;
bool aboveIsCompound = aboveModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra;
bool leftIsCompound = leftModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra;
if (!aboveIsCompound && !leftIsCompound)
{
bool aboveIsBackward = above.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward;
bool leftIsBackward = left.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward;
bool aboveIsBackward = aboveModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward;
bool leftIsBackward = leftModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward;
return aboveIsBackward == leftIsBackward ? 0 : 1;
}
if (!aboveIsCompound)
{
bool aboveIsBackward = above.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward;
bool aboveIsIntra = above.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra;
bool aboveIsBackward = aboveModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward;
bool aboveIsIntra = aboveModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra;
return 2 + (aboveIsBackward || aboveIsIntra ? 1 : 0);
}
if (!leftIsCompound)
{
bool leftIsBackward = left.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward;
bool leftIsIntra = left.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra;
bool leftIsBackward = leftModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward;
bool leftIsIntra = leftModeInfo.ReferenceFrames[0] <= Av1ReferenceFrameType.Intra;
return 2 + (leftIsBackward || leftIsIntra ? 1 : 0);
}
@ -777,14 +785,15 @@ internal static class Av1SymbolContextHelper
if (neighbor is not null)
{
bool isCompound = neighbor.ReferenceFrames[1] > Av1ReferenceFrameType.Intra;
Av1BlockModeInfo neighborModeInfo = neighbor.Value;
bool isCompound = neighborModeInfo.ReferenceFrames[1] > Av1ReferenceFrameType.Intra;
if (isCompound)
{
return 3;
}
return neighbor.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward ? 1 : 0;
return neighborModeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Backward ? 1 : 0;
}
// With no spatial votes, AV1 uses the neutral single-versus-compound context rather than context zero.
@ -801,8 +810,10 @@ internal static class Av1SymbolContextHelper
{
if (above is not null && left is not null)
{
bool aboveIntra = !IsInterBlock(above);
bool leftIntra = !IsInterBlock(left);
Av1BlockModeInfo aboveModeInfo = above.Value;
Av1BlockModeInfo leftModeInfo = left.Value;
bool aboveIntra = !IsInterBlock(aboveModeInfo);
bool leftIntra = !IsInterBlock(leftModeInfo);
if (aboveIntra && leftIntra)
{
return 2;
@ -810,14 +821,14 @@ internal static class Av1SymbolContextHelper
if (aboveIntra || leftIntra)
{
Av1BlockModeInfo inter = aboveIntra ? left : above;
Av1BlockModeInfo inter = aboveIntra ? leftModeInfo : aboveModeInfo;
return HasCompoundReference(inter) ? 1 + (2 * (HasUnidirectionalCompoundReferences(inter) ? 1 : 0)) : 2;
}
bool aboveSingle = !HasCompoundReference(above);
bool leftSingle = !HasCompoundReference(left);
Av1ReferenceFrameType abovePrimary = above.ReferenceFrames[0];
Av1ReferenceFrameType leftPrimary = left.ReferenceFrames[0];
bool aboveSingle = !HasCompoundReference(aboveModeInfo);
bool leftSingle = !HasCompoundReference(leftModeInfo);
Av1ReferenceFrameType abovePrimary = aboveModeInfo.ReferenceFrames[0];
Av1ReferenceFrameType leftPrimary = leftModeInfo.ReferenceFrames[0];
if (aboveSingle && leftSingle)
{
return 1 + (2 * (IsBackwardReference(abovePrimary) == IsBackwardReference(leftPrimary) ? 1 : 0));
@ -825,7 +836,7 @@ internal static class Av1SymbolContextHelper
if (aboveSingle || leftSingle)
{
Av1BlockModeInfo compound = aboveSingle ? left : above;
Av1BlockModeInfo compound = aboveSingle ? leftModeInfo : aboveModeInfo;
if (!HasUnidirectionalCompoundReferences(compound))
{
return 1;
@ -834,8 +845,8 @@ internal static class Av1SymbolContextHelper
return 3 + (IsBackwardReference(abovePrimary) == IsBackwardReference(leftPrimary) ? 1 : 0);
}
bool aboveUnidirectional = HasUnidirectionalCompoundReferences(above);
bool leftUnidirectional = HasUnidirectionalCompoundReferences(left);
bool aboveUnidirectional = HasUnidirectionalCompoundReferences(aboveModeInfo);
bool leftUnidirectional = HasUnidirectionalCompoundReferences(leftModeInfo);
if (!aboveUnidirectional && !leftUnidirectional)
{
return 0;
@ -850,12 +861,18 @@ internal static class Av1SymbolContextHelper
}
Av1BlockModeInfo? edge = above ?? left;
if (edge is null || !IsInterBlock(edge) || !HasCompoundReference(edge))
if (edge is null)
{
return 2;
}
return HasUnidirectionalCompoundReferences(edge) ? 4 : 0;
Av1BlockModeInfo edgeModeInfo = edge.Value;
if (!IsInterBlock(edgeModeInfo) || !HasCompoundReference(edgeModeInfo))
{
return 2;
}
return HasUnidirectionalCompoundReferences(edgeModeInfo) ? 4 : 0;
}
/// <summary>
@ -984,12 +1001,12 @@ internal static class Av1SymbolContextHelper
if (above is not null)
{
AddNeighborReferenceCounts(above, referenceCounts);
AddNeighborReferenceCounts(above.Value, referenceCounts);
}
if (left is not null)
{
AddNeighborReferenceCounts(left, referenceCounts);
AddNeighborReferenceCounts(left.Value, referenceCounts);
}
}
@ -1144,8 +1161,8 @@ internal static class Av1SymbolContextHelper
/// <returns>The context in the inclusive range zero through two.</returns>
public static int GetSegmentIdPredictedContext(Av1BlockModeInfo? aboveModeInfo, Av1BlockModeInfo? leftModeInfo)
{
int abovePredicted = aboveModeInfo is not null && aboveModeInfo.SegmentIdPredicted ? 1 : 0;
int leftPredicted = leftModeInfo is not null && leftModeInfo.SegmentIdPredicted ? 1 : 0;
int abovePredicted = aboveModeInfo is not null && aboveModeInfo.Value.SegmentIdPredicted ? 1 : 0;
int leftPredicted = leftModeInfo is not null && leftModeInfo.Value.SegmentIdPredicted ? 1 : 0;
return abovePredicted + leftPredicted;
}
@ -1306,7 +1323,8 @@ internal static class Av1SymbolContextHelper
return SwitchableInterpolationFilterCount;
}
ReadOnlySpan<Av1ReferenceFrameType> referenceFrames = modeInfo.ReferenceFrames;
Av1BlockModeInfo neighborModeInfo = modeInfo.Value;
ReadOnlySpan<Av1ReferenceFrameType> referenceFrames = neighborModeInfo.ReferenceFrames;
// A compound neighbor contributes when either of its references matches the current primary reference.
if (referenceFrames[0] != referenceFrame && referenceFrames[1] != referenceFrame)
@ -1314,6 +1332,6 @@ internal static class Av1SymbolContextHelper
return SwitchableInterpolationFilterCount;
}
return (int)modeInfo.InterpolationFilters[direction];
return (int)neighborModeInfo.InterpolationFilters[direction];
}
}

21
src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolDecoder.cs

@ -20,11 +20,6 @@ internal ref struct Av1SymbolDecoder
/// </summary>
private readonly Av1FrameEntropyContext context;
/// <summary>
/// The configuration providing temporary coefficient-context memory.
/// </summary>
private readonly Configuration configuration;
/// <summary>
/// The range decoder over the current tile payload.
/// </summary>
@ -59,7 +54,6 @@ internal ref struct Av1SymbolDecoder
// The context owner controls reset and publication. Holding one reference here keeps the range decoder small
// and prevents a second set of aliases from becoming a competing source of entropy state.
this.context = context;
this.configuration = configuration;
this.reader = new Av1SymbolReader(tileData, updateCdf);
}
@ -345,15 +339,15 @@ internal ref struct Av1SymbolDecoder
{
ref Av1SymbolReader r = ref this.reader;
Av1PredictionMode aboveMode = Av1PredictionMode.DC;
if (aboveModeInfo != null)
if (aboveModeInfo is not null)
{
aboveMode = aboveModeInfo.YMode;
aboveMode = aboveModeInfo.Value.YMode;
}
Av1PredictionMode leftMode = Av1PredictionMode.DC;
if (leftModeInfo != null)
if (leftModeInfo is not null)
{
leftMode = leftModeInfo.YMode;
leftMode = leftModeInfo.Value.YMode;
}
int aboveContext = IntraModeContext[(int)aboveMode];
@ -971,6 +965,7 @@ internal ref struct Av1SymbolDecoder
/// <param name="transformInfo">The transform descriptor updated with the decoded type and coded-block flag.</param>
/// <param name="modeBlocksToRightEdge">The signed distance from the mode block to the right frame edge.</param>
/// <param name="modeBlocksToBottomEdge">The signed distance from the mode block to the bottom frame edge.</param>
/// <param name="levels">Reusable padded coefficient-context storage owned by the tile reader.</param>
/// <param name="coefficientBuffer">The destination receiving the coefficient count followed by scan-ordered signed levels.</param>
/// <returns>The one-based end-of-block position, or zero for an empty transform block.</returns>
public int ReadCoefficients(
@ -991,6 +986,7 @@ internal ref struct Av1SymbolDecoder
ref Av1TransformInfo transformInfo,
int modeBlocksToRightEdge,
int modeBlocksToBottomEdge,
Av1LevelBuffer levels,
Span<int> coefficientBuffer)
{
Av1TransformSize adjustedTransformSize = transformSize.GetAdjusted();
@ -1000,8 +996,9 @@ internal ref struct Av1SymbolDecoder
Av1PlaneType planeType = (Av1PlaneType)Math.Min(plane, 1);
int culLevel = 0;
// AV1 omits high-frequency coefficients beyond 32 samples on every 64-point transform dimension.
using Av1LevelBuffer levels = new(this.configuration, new Size(width, height));
// AV1 omits high-frequency coefficients beyond 32 samples on every 64-point transform dimension. Reusing
// tile-owned storage avoids an allocator round trip for every transform block.
levels.Reset(new Size(width, height));
bool allZero = this.ReadTransformBlockSkip(transformSizeContext, transformBlockContext.SkipContext);
int endOfBlock;

8
src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1FrameDecoder.cs

@ -160,7 +160,7 @@ internal sealed class Av1FrameDecoder : IAv1FrameDecoder, IDisposable
/// Reconstructs every tile row in one tile column.
/// </summary>
/// <param name="tileColumn">The zero-based tile-column index.</param>
/// <remarks>SVT-AV1: <c>decode_tile</c>.</remarks>
/// <remarks>Follows libaom's single-threaded <c>decode_tiles</c> ordering.</remarks>
private void DecodeFrameTiles(int tileColumn)
{
ObuTileGroupHeader tileInfo = this.frameHeader.TilesInfo;
@ -187,7 +187,7 @@ internal sealed class Av1FrameDecoder : IAv1FrameDecoder, IDisposable
/// <param name="tileColumn">The zero-based tile-column index.</param>
/// <param name="modeInfoRow">The frame-relative row in 4x4 mode-info units.</param>
/// <param name="superblockRow">The frame-relative superblock row.</param>
/// <remarks>SVT-AV1: <c>decode_tile_row</c>.</remarks>
/// <remarks>Corresponds to the superblock-row traversal in libaom's <c>decode_tile</c>.</remarks>
private void DecodeTileSuperblockRow(int tileRow, int tileColumn, int modeInfoRow, int superblockRow)
{
ObuTileGroupHeader tileInfo = this.frameHeader.TilesInfo;
@ -210,7 +210,7 @@ internal sealed class Av1FrameDecoder : IAv1FrameDecoder, IDisposable
/// <param name="modeInfoPosition">The superblock's top-left position in 4x4 mode-info units.</param>
/// <param name="superblockInfo">The decoded syntax and block modes for the superblock.</param>
/// <param name="tileInfo">The tile that contains the superblock.</param>
/// <remarks>SVT-AV1: <c>svt_aom_decode_super_block</c>.</remarks>
/// <remarks>Corresponds to libaom's superblock decode boundary.</remarks>
public void DecodeSuperblock(Point modeInfoPosition, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo)
{
this.blockDecoder.UpdateSuperblock(superblockInfo);
@ -224,7 +224,7 @@ internal sealed class Av1FrameDecoder : IAv1FrameDecoder, IDisposable
/// <param name="modeInfoPosition">The superblock's frame-relative origin in 4x4 mode-info units.</param>
/// <param name="superblockInfo">The superblock whose block modes are traversed.</param>
/// <param name="tileInfo">The tile boundary information used by intra prediction.</param>
/// <remarks>SVT-AV1: <c>decode_partition</c>.</remarks>
/// <remarks>Replays the depth-first block order produced by libaom's <c>decode_partition</c>.</remarks>
private void DecodePartition(Point modeInfoPosition, Av1SuperblockInfo superblockInfo, Av1TileInfo tileInfo)
{
foreach (Av1BlockModeInfo modeInfo in superblockInfo.GetModeInfos())

6
src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs

@ -237,9 +237,9 @@ internal class Av1LoopFilterDecoder
bool currentSkippedTransform = modeInfo.Skip && modeInfo.ReferenceFrames[0] > Av1ReferenceFrameType.Intra;
bool previousSkippedTransform = previousModeInfo.Skip && previousModeInfo.ReferenceFrames[0] > Av1ReferenceFrameType.Intra;
// The mode-info map stores one object for every covered position, so object identity is the exact equivalent
// of libaom's current-versus-previous MB_MODE_INFO pointer comparison at a prediction-unit boundary.
bool isBlockEdge = !ReferenceEquals(modeInfo, previousModeInfo);
// Every covered 4x4 position carries the owning block's storage index. Comparing those indices is the value-type
// equivalent of libaom's current-versus-previous MB_MODE_INFO pointer comparison at a prediction-unit boundary.
bool isBlockEdge = modeInfo.ModeInfoIndex != previousModeInfo.ModeInfoIndex;
bool applyFilter = isTransformEdge && (isBlockEdge || !currentSkippedTransform || !previousSkippedTransform);
if (!applyFilter)
{

10
src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1DeQuantizationContext.cs

@ -26,7 +26,7 @@ internal class Av1DeQuantizationContext
/// </summary>
/// <param name="sequenceHeader">The sequence header that supplies the coded bit depth.</param>
/// <param name="frameHeader">The frame header that supplies segmentation and quantization parameters.</param>
/// <remarks>SVT-AV1: <c>svt_aom_setup_segmentation_dequant</c>.</remarks>
/// <remarks>Corresponds to <c>setup_segmentation_dequant</c> in libaom.</remarks>
public Av1DeQuantizationContext(ObuSequenceHeader sequenceHeader, ObuFrameHeader frameHeader)
{
Av1BitDepth bitDepth = sequenceHeader.ColorConfig.BitDepth;
@ -40,11 +40,11 @@ internal class Av1DeQuantizationContext
for (int plane = 0; plane < Av1Constants.MaxPlanes; plane++)
{
int dc_delta_q = frameHeader.QuantizationParameters.DeltaQDc[plane];
int ac_delta_q = frameHeader.QuantizationParameters.DeltaQAc[plane];
int dcDeltaQ = frameHeader.QuantizationParameters.DeltaQDc[plane];
int acDeltaQ = frameHeader.QuantizationParameters.DeltaQAc[plane];
this.dcContent[segmentId][plane] = Av1QuantizationLookup.GetDcQuant(qindex, dc_delta_q, bitDepth);
this.acContent[segmentId][plane] = Av1QuantizationLookup.GetAcQuant(qindex, ac_delta_q, bitDepth);
this.dcContent[segmentId][plane] = Av1QuantizationLookup.GetDcQuant(qindex, dcDeltaQ, bitDepth);
this.acContent[segmentId][plane] = Av1QuantizationLookup.GetAcQuant(qindex, acDeltaQ, bitDepth);
}
}
}

4
src/ImageSharp/Formats/Heif/Av1/Pipeline/Quantizers/Av1InverseQuantizer.cs

@ -80,7 +80,7 @@ internal class Av1InverseQuantizer
/// <param name="transformSize">The transform dimensions and scale.</param>
/// <param name="plane">The color plane whose quantizer and matrix are used.</param>
/// <returns>The number of coefficient levels consumed.</returns>
/// <remarks>SVT-AV1: <c>svt_aom_inverse_quantize</c>.</remarks>
/// <remarks>Matches the coefficient dequantization arithmetic in libaom's <c>read_coeffs_txb</c>.</remarks>
public int InverseQuantize(Av1BlockModeInfo mode, Span<int> level, Span<int> qCoefficients, Av1TransformType transformType, Av1TransformSize transformSize, Av1Plane plane)
{
Guard.NotNull(this.deQuantsDeltaQ);
@ -159,7 +159,7 @@ internal class Av1InverseQuantizer
/// <param name="coefficientIndex">The raster coefficient index into the inverse matrix.</param>
/// <param name="iqMatrix">The inverse quantization matrix for the current level, plane, and transform size.</param>
/// <returns>The matrix-weighted dequantization value.</returns>
/// <remarks>SVT-AV1: <c>get_dqv</c>.</remarks>
/// <remarks>Corresponds to <c>get_dqv</c> in libaom.</remarks>
private static int GetDeQuantizedValue(short dequant, int coefficientIndex, ReadOnlySpan<int> iqMatrix)
{
// Matrix elements use fixed-point precision; adding half a unit produces nearest-integer rounding on shift.

23
src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PalettePredictor.Operator.cs

@ -4,6 +4,7 @@
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Runtime.Intrinsics;
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
@ -97,26 +98,24 @@ internal static class Av1PalettePredictor
/// </summary>
public static void Predict(
ReadOnlySpan<ushort> paletteColors,
ReadOnlySpan<byte> colorIndexMap,
int colorIndexMapStride,
Buffer2DRegion<byte> colorIndexMap,
Span<byte> destination,
int destinationStride,
int width,
int height)
=> Predictor<PaletteOperator>.Predict(paletteColors, colorIndexMap, colorIndexMapStride, destination, destinationStride, width, height);
=> Predictor<PaletteOperator>.Predict(paletteColors, colorIndexMap, destination, destinationStride, width, height);
/// <summary>
/// Reconstructs a high-bit-depth palette-predicted block.
/// </summary>
public static void Predict(
ReadOnlySpan<ushort> paletteColors,
ReadOnlySpan<byte> colorIndexMap,
int colorIndexMapStride,
Buffer2DRegion<byte> colorIndexMap,
Span<short> destination,
int destinationStride,
int width,
int height)
=> Predictor<PaletteOperator>.Predict(paletteColors, colorIndexMap, colorIndexMapStride, destination, destinationStride, width, height);
=> Predictor<PaletteOperator>.Predict(paletteColors, colorIndexMap, destination, destinationStride, width, height);
/// <summary>
/// Maps decoded palette indices to reconstructed samples.
@ -186,14 +185,12 @@ internal static class Av1PalettePredictor
/// </summary>
public static void Predict(
ReadOnlySpan<ushort> paletteColors,
ReadOnlySpan<byte> colorIndexMap,
int colorIndexMapStride,
Buffer2DRegion<byte> colorIndexMap,
Span<byte> destination,
int destinationStride,
int width,
int height)
{
ref byte mapBase = ref MemoryMarshal.GetReference(colorIndexMap);
ref byte destinationBase = ref MemoryMarshal.GetReference(destination);
// AV1 palettes contain at most eight colors. Repeating all eight entries in every 128-bit lane keeps native
@ -209,7 +206,7 @@ internal static class Av1PalettePredictor
for (int row = 0; row < height; row++)
{
ref byte mapRow = ref Unsafe.Add(ref mapBase, row * colorIndexMapStride);
ref byte mapRow = ref MemoryMarshal.GetReference(colorIndexMap.DangerousGetRowSpan(row));
ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride);
int column = 0;
@ -278,14 +275,12 @@ internal static class Av1PalettePredictor
/// </summary>
public static void Predict(
ReadOnlySpan<ushort> paletteColors,
ReadOnlySpan<byte> colorIndexMap,
int colorIndexMapStride,
Buffer2DRegion<byte> colorIndexMap,
Span<short> destination,
int destinationStride,
int width,
int height)
{
ref byte mapBase = ref MemoryMarshal.GetReference(colorIndexMap);
ref short destinationBase = ref MemoryMarshal.GetReference(destination);
InlineArray8<ushort> paletteStorage = default;
paletteColors.CopyTo(paletteStorage);
@ -295,7 +290,7 @@ internal static class Av1PalettePredictor
for (int row = 0; row < height; row++)
{
ref byte mapRow = ref Unsafe.Add(ref mapBase, row * colorIndexMapStride);
ref byte mapRow = ref MemoryMarshal.GetReference(colorIndexMap.DangerousGetRowSpan(row));
ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride);
int column = 0;

25
src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs

@ -10,6 +10,7 @@ using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma;
using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
using SixLabors.ImageSharp.Formats.Heif.Av1.Transform;
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
@ -462,22 +463,24 @@ internal class Av1PredictionDecoder
if (usePalette)
{
ReadOnlySpan<ushort> paletteColors = modeInfo.GetPaletteColors(plane);
ReadOnlySpan<byte> colorIndexMap = modeInfo.GetPaletteColorIndexMap(plane);
int paletteStride = partitionInfo.GetWidthInPixels(plane);
int mapOffset = ((blockModeInfoRowOffset << Av1Constants.ModeInfoSizeLog2) * paletteStride) +
(blockModeInfoColumnOffset << Av1Constants.ModeInfoSizeLog2);
// Every transform reconstructs its own window of the block-level palette map. Keeping the map padded to
// the coded block dimensions lets edge transforms use the same addressing rule as interior transforms.
Buffer2DRegion<byte> colorIndexMap = modeInfo.GetPaletteColorIndexMap(plane);
Buffer2DRegion<byte> transformColorIndexMap = colorIndexMap.GetSubRegion(
blockModeInfoColumnOffset << Av1Constants.ModeInfoSizeLog2,
blockModeInfoRowOffset << Av1Constants.ModeInfoSizeLog2,
transformWidth,
transformHeight);
// Every transform reconstructs its own window of the block-level palette map. The row-oriented region
// keeps this traversal valid when the frame-owned map spans multiple allocator memory groups.
if (typeof(T) == typeof(byte))
{
Span<byte> byteDestination = MemoryMarshal.Cast<T, byte>(pixelBuffer);
Av1PalettePredictor.Predict(paletteColors, colorIndexMap[mapOffset..], paletteStride, byteDestination, pixelBufferStride, transformWidth, transformHeight);
Av1PalettePredictor.Predict(paletteColors, transformColorIndexMap, byteDestination, pixelBufferStride, transformWidth, transformHeight);
}
else
{
Span<short> highBitDepthDestination = MemoryMarshal.Cast<T, short>(pixelBuffer);
Av1PalettePredictor.Predict(paletteColors, colorIndexMap[mapOffset..], paletteStride, highBitDepthDestination, pixelBufferStride, transformWidth, transformHeight);
Av1PalettePredictor.Predict(paletteColors, transformColorIndexMap, highBitDepthDestination, pixelBufferStride, transformWidth, transformHeight);
}
return;
@ -1983,8 +1986,8 @@ internal class Av1PredictionDecoder
left = partitionInfo.LeftModeInfoForChroma;
}
bool aboveIsSmooth = (above != null) && IsSmooth(above, plane);
bool leftIsSmooth = (left != null) && IsSmooth(left, plane);
bool aboveIsSmooth = above is not null && IsSmooth(above.Value, plane);
bool leftIsSmooth = left is not null && IsSmooth(left.Value, plane);
return aboveIsSmooth || leftIsSmooth;
}

23
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1BlockModeInfo.cs

@ -1,17 +1,19 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using System.Diagnostics.CodeAnalysis;
using System.Runtime.CompilerServices;
using SixLabors.ImageSharp.Formats.Heif.Av1.Motion;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter;
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
/// <summary>
/// Stores block-size, intra/inter prediction, transform, and palette decisions shared by AV1 block processing.
/// </summary>
internal class Av1BlockModeInfo
internal struct Av1BlockModeInfo
{
/// <summary>
/// Stores the primary and optional secondary reference-frame labels.
@ -56,12 +58,12 @@ internal class Av1BlockModeInfo
/// <summary>
/// Stores the luma palette color-index map.
/// </summary>
private byte[] lumaPaletteColorIndexMap = [];
private Buffer2DRegion<byte> lumaPaletteColorIndexMap;
/// <summary>
/// Stores the shared chroma palette color-index map.
/// </summary>
private byte[] chromaPaletteColorIndexMap = [];
private Buffer2DRegion<byte> chromaPaletteColorIndexMap;
/// <summary>
/// The directional prediction angle adjustment for luma.
@ -94,7 +96,7 @@ internal class Av1BlockModeInfo
private int chromaTransformUnitCount;
/// <summary>
/// Initializes a new instance of the <see cref="Av1BlockModeInfo"/> class.
/// Initializes a new instance of the <see cref="Av1BlockModeInfo"/> structure.
/// </summary>
/// <param name="blockSize">The decoded block size.</param>
/// <param name="positionInSuperblock">The block origin relative to its superblock in 4x4 mode-information units.</param>
@ -114,6 +116,11 @@ internal class Av1BlockModeInfo
/// </summary>
public Av1BlockSize BlockSize { get; }
/// <summary>
/// Gets or sets the frame storage index shared by every mode-information position covered by this block.
/// </summary>
public int ModeInfoIndex { get; set; }
/// <summary>
/// Gets or sets the <see cref="Av1PredictionMode"/> for the luminance channel.
/// </summary>
@ -127,11 +134,13 @@ internal class Av1BlockModeInfo
/// block, <see cref="Av1ReferenceFrameType.Intra"/> for an inter-intra block, or the secondary inter-reference label
/// for compound prediction.
/// </remarks>
[UnscopedRef]
public Span<Av1ReferenceFrameType> ReferenceFrames => this.referenceFrames;
/// <summary>
/// Gets the decoded motion vectors corresponding to <see cref="ReferenceFrames"/>.
/// </summary>
[UnscopedRef]
public Span<Av1MotionVector> MotionVectors => this.motionVectors;
/// <summary>
@ -141,6 +150,7 @@ internal class Av1BlockModeInfo
/// Index zero is the vertical filter and index one is the horizontal filter, matching libaom's
/// <c>InterpFilters.y_filter</c> and <c>InterpFilters.x_filter</c> layout.
/// </remarks>
[UnscopedRef]
public Span<Av1InterpolationFilter> InterpolationFilters => this.interpolationFilters;
/// <summary>
@ -392,6 +402,7 @@ internal class Av1BlockModeInfo
/// </summary>
/// <param name="plane">The color plane.</param>
/// <returns>The palette colors in prediction-index order.</returns>
[UnscopedRef]
public ReadOnlySpan<ushort> GetPaletteColors(Av1Plane plane)
{
if (plane == Av1Plane.Y)
@ -430,7 +441,7 @@ internal class Av1BlockModeInfo
/// </summary>
/// <param name="plane">The color plane.</param>
/// <returns>The luma map for <see cref="Av1Plane.Y"/> or the shared chroma map for either chroma plane.</returns>
public ReadOnlySpan<byte> GetPaletteColorIndexMap(Av1Plane plane)
public Buffer2DRegion<byte> GetPaletteColorIndexMap(Av1Plane plane)
=> plane == Av1Plane.Y ? this.lumaPaletteColorIndexMap : this.chromaPaletteColorIndexMap;
/// <summary>
@ -438,7 +449,7 @@ internal class Av1BlockModeInfo
/// </summary>
/// <param name="planeType">The luma or shared chroma plane class.</param>
/// <param name="colorIndexMap">The row-major color-index map including coded-block edge padding.</param>
public void SetPaletteColorIndexMap(Av1PlaneType planeType, byte[] colorIndexMap)
public void SetPaletteColorIndexMap(Av1PlaneType planeType, Buffer2DRegion<byte> colorIndexMap)
{
if (planeType == Av1PlaneType.Y)
{

10
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.MotionField.cs

@ -486,19 +486,23 @@ internal partial class Av1FrameInfo
}
/// <summary>
/// Releases one owner and returns motion-field storage after the final owner is released.
/// Releases one owner and returns allocator-backed frame storage after the final owner is released.
/// </summary>
public void ReleaseOwner()
{
this.ownerCount--;
if (this.ownerCount == 0)
{
// Retained and temporal fields can each be frame-sized. Return both together only after the tile reader,
// every reference or presentation frame, and the decoder's inspectable result have released ownership.
// Frame-sized motion and palette storage remains addressable through retained mode information. Return
// all of it together only after tile, reference, presentation, and decoder-result owners are gone.
this.retainedMotionField?.Dispose();
this.retainedMotionField = null;
this.temporalMotionField?.Dispose();
this.temporalMotionField = null;
this.lumaPaletteColorIndexMap?.Dispose();
this.lumaPaletteColorIndexMap = null;
this.chromaPaletteColorIndexMap?.Dispose();
this.chromaPaletteColorIndexMap = null;
}
}

81
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameInfo.cs

@ -2,6 +2,7 @@
// Licensed under the Six Labors Split License.
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
@ -55,6 +56,36 @@ internal partial class Av1FrameInfo : IDisposable
/// </summary>
private readonly int subsamplingFactor;
/// <summary>
/// The aligned luma palette-map width in samples.
/// </summary>
private readonly int lumaPaletteColorIndexMapWidth;
/// <summary>
/// The aligned luma palette-map height in samples.
/// </summary>
private readonly int lumaPaletteColorIndexMapHeight;
/// <summary>
/// The aligned chroma palette-map width in samples.
/// </summary>
private readonly int chromaPaletteColorIndexMapWidth;
/// <summary>
/// The aligned chroma palette-map height in samples.
/// </summary>
private readonly int chromaPaletteColorIndexMapHeight;
/// <summary>
/// Owns row-addressable luma palette indices for the frame.
/// </summary>
private Buffer2D<byte>? lumaPaletteColorIndexMap;
/// <summary>
/// Owns row-addressable chroma palette indices for the frame.
/// </summary>
private Buffer2D<byte>? chromaPaletteColorIndexMap;
/// <summary>
/// Stores one addressing view for each frame superblock.
/// </summary>
@ -179,6 +210,10 @@ internal partial class Av1FrameInfo : IDisposable
// Chroma capacity scales by two for each sampled axis: 4:4:4 => 0, 4:2:2 => 1, 4:2:0 => 2.
this.subsamplingFactor = (subX && subY) ? 2 : (subX && !subY) ? 1 : (!subX && !subY) ? 0 : -1;
Guard.IsFalse(this.subsamplingFactor == -1, nameof(this.subsamplingFactor), "Invalid combination of subsampling.");
this.lumaPaletteColorIndexMapWidth = superblockAlignedWidth;
this.lumaPaletteColorIndexMapHeight = superblockAlignedHeight;
this.chromaPaletteColorIndexMapWidth = superblockAlignedWidth >> (subX ? 1 : 0);
this.chromaPaletteColorIndexMapHeight = superblockAlignedHeight >> (subY ? 1 : 0);
int lumaCoefficientCountPerSuperblock = this.modeInfoCountPerSuperblock * CoefficientCountPerModeInfo;
int chromaCoefficientCountPerSuperblock = lumaCoefficientCountPerSuperblock >> this.subsamplingFactor;
this.coefficientsY = new int[superblockCount * lumaCoefficientCountPerSuperblock];
@ -203,6 +238,51 @@ internal partial class Av1FrameInfo : IDisposable
/// </summary>
public int SuperblockModeInfoSize => this.modeInfoSizePerSuperblock;
/// <summary>
/// Gets frame-owned row-addressable palette-map storage for one coding block.
/// </summary>
/// <param name="configuration">The decoder configuration providing frame storage.</param>
/// <param name="planeType">The luma or shared chroma plane class.</param>
/// <param name="bounds">The block bounds in plane samples.</param>
/// <returns>The palette-map region assigned to the coding block.</returns>
public Buffer2DRegion<byte> GetPaletteColorIndexMap(
Configuration configuration,
Av1PlaneType planeType,
Rectangle bounds)
{
Buffer2D<byte>? buffer;
if (planeType == Av1PlaneType.Y)
{
buffer = this.lumaPaletteColorIndexMap;
if (buffer is null)
{
// A 2D allocation may contain multiple memory groups, but row alignment guarantees that every
// palette row remains contiguous for entropy decoding and SIMD reconstruction.
buffer = configuration.MemoryAllocator.Allocate2D<byte>(
this.lumaPaletteColorIndexMapWidth,
this.lumaPaletteColorIndexMapHeight);
this.lumaPaletteColorIndexMap = buffer;
}
}
else
{
buffer = this.chromaPaletteColorIndexMap;
if (buffer is null)
{
buffer = configuration.MemoryAllocator.Allocate2D<byte>(
this.chromaPaletteColorIndexMapWidth,
this.chromaPaletteColorIndexMapHeight);
this.chromaPaletteColorIndexMap = buffer;
}
}
// Partition traversal assigns non-overlapping frame regions, so retaining a view records the complete
// identify-time syntax without copying block maps or allocating storage for each coding block.
return new Buffer2DRegion<byte>(buffer, bounds);
}
/// <summary>
/// Initializes the active frame's contiguous segment map and applies whole-map inheritance when requested.
/// </summary>
@ -559,6 +639,7 @@ internal partial class Av1FrameInfo : IDisposable
public void UpdateModeInfo(Av1BlockModeInfo modeInfo, Av1SuperblockInfo superblockInfo)
{
Point modeInfoPosition = this.GetModeInfoPosition(superblockInfo.Position, modeInfo.PositionInSuperblock);
modeInfo.ModeInfoIndex = this.modeInfoMap.NextIndex;
this.modeInfos[this.modeInfoMap.NextIndex] = modeInfo;
this.UpdateRetainedMotionField(modeInfo, modeInfoPosition);
this.modeInfoMap.Update(modeInfoPosition, modeInfo.BlockSize);

8
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1FrameModeInfoMap.cs

@ -9,14 +9,14 @@ namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
internal partial class Av1FrameInfo
{
/// <summary>
/// Mapping of <see cref="Av1BlockModeInfo"/> instances, from position to index into the <see cref="Av1FrameInfo"/>.
/// Mapping of <see cref="Av1BlockModeInfo"/> values, from position to index into the <see cref="Av1FrameInfo"/>.
/// </summary>
public class Av1FrameModeInfoMap
{
/// <summary>
/// Stores the mode-information index assigned to each aligned 4x4 frame location.
/// </summary>
private readonly ushort[] offsets;
private readonly int[] offsets;
/// <summary>
/// The dimensions of <see cref="offsets"/> in 4x4 mode-information units.
@ -31,7 +31,7 @@ internal partial class Av1FrameInfo
{
this.alignedModeInfoCount = modeInfoCount;
this.NextIndex = 0;
this.offsets = new ushort[this.alignedModeInfoCount.Width * this.alignedModeInfoCount.Height];
this.offsets = new int[this.alignedModeInfoCount.Width * this.alignedModeInfoCount.Height];
}
/// <summary>
@ -70,7 +70,7 @@ internal partial class Av1FrameInfo
// because later blocks query their above and left neighbors at cell granularity.
for (int i = modeInfoLocation.Y; i < modeInfoLocation.Y + bh4; i++)
{
Array.Fill(this.offsets, (ushort)this.NextIndex, (i * this.alignedModeInfoCount.Width) + modeInfoLocation.X, bw4);
Array.Fill(this.offsets, this.NextIndex, (i * this.alignedModeInfoCount.Width) + modeInfoLocation.X, bw4);
}
this.NextIndex++;

27
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1LevelBuffer.cs

@ -19,11 +19,14 @@ internal sealed class Av1LevelBuffer : IDisposable
private IMemoryOwner<byte>? memory;
/// <summary>
/// Initializes a new instance of the <see cref="Av1LevelBuffer"/> class for the maximum AV1 transform size.
/// Initializes a new instance of the <see cref="Av1LevelBuffer"/> class for the maximum entropy-coded
/// coefficient dimensions.
/// </summary>
/// <param name="configuration">The configuration providing the memory allocator.</param>
public Av1LevelBuffer(Configuration configuration)
: this(configuration, new Size(Av1Constants.MaxTransformSize, Av1Constants.MaxTransformSize))
: this(
configuration,
new Size(Av1Constants.MaxTransformSize / 2, Av1Constants.MaxTransformSize / 2))
{
}
@ -46,12 +49,12 @@ internal sealed class Av1LevelBuffer : IDisposable
/// <summary>
/// Gets the unpadded coefficient dimensions.
/// </summary>
public Size Size { get; }
public Size Size { get; private set; }
/// <summary>
/// Gets the padded row stride in bytes.
/// </summary>
public int Stride { get; }
public int Stride { get; private set; }
/// <summary>
/// Gets the coefficient level at the specified unpadded position.
@ -121,6 +124,22 @@ internal sealed class Av1LevelBuffer : IDisposable
this.memory = null;
}
/// <summary>
/// Selects new active coefficient dimensions and clears their padded context storage.
/// </summary>
/// <param name="size">The unpadded coefficient dimensions.</param>
public void Reset(Size size)
{
ObjectDisposedException.ThrowIf(this.memory == null, this);
this.Size = size;
this.Stride = Av1Constants.TransformPadHorizontal + size.Width;
// Tile parsing is sequential, so one maximum-sized rent can serve every transform. Clear only the active
// layout because stale neighboring levels would otherwise select the wrong coefficient distributions.
int totalHeight = Av1Constants.TransformPadTop + size.Height + Av1Constants.TransformPadBottom;
this.memory.Memory.Span[..(this.Stride * totalHeight)].Clear();
}
/// <summary>
/// Clears all coefficient levels and context padding.
/// </summary>

12
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PartitionInfo.cs

@ -1,6 +1,7 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using System.Diagnostics.CodeAnalysis;
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma;
using SixLabors.ImageSharp.Formats.Heif.Av1.Transform;
@ -12,6 +13,11 @@ namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
/// </summary>
internal ref struct Av1PartitionInfo
{
/// <summary>
/// The decoded mode information populated for this partition.
/// </summary>
private Av1BlockModeInfo modeInfo;
/// <summary>
/// The luma block width in samples.
/// </summary>
@ -41,7 +47,7 @@ internal ref struct Av1PartitionInfo
/// <param name="partitionType">The partition type that produced the block.</param>
public Av1PartitionInfo(Av1BlockModeInfo modeInfo, Av1SuperblockInfo superblockInfo, bool isChroma, Av1PartitionType partitionType)
{
this.ModeInfo = modeInfo;
this.modeInfo = modeInfo;
this.SuperblockInfo = superblockInfo;
this.IsChroma = isChroma;
this.Type = partitionType;
@ -50,7 +56,8 @@ internal ref struct Av1PartitionInfo
/// <summary>
/// Gets the decoded block mode information.
/// </summary>
public Av1BlockModeInfo ModeInfo { get; }
[UnscopedRef]
public ref Av1BlockModeInfo ModeInfo => ref this.modeInfo;
/// <summary>
/// Gets the <see cref="Av1SuperblockInfo"/> this partition resides inside.
@ -120,6 +127,7 @@ internal ref struct Av1PartitionInfo
/// <summary>
/// Gets the reference-frame types selected for the block.
/// </summary>
[UnscopedRef]
public Span<Av1ReferenceFrameType> ReferenceFrames => this.ModeInfo.ReferenceFrames;
/// <summary>

183
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs

@ -11,6 +11,7 @@ using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter;
using SixLabors.ImageSharp.Formats.Heif.Av1.ReferenceFrames;
using SixLabors.ImageSharp.Formats.Heif.Av1.Transform;
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
@ -95,6 +96,11 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// </summary>
private readonly int[] coefficientIndex = [];
/// <summary>
/// Reusable padded coefficient-context storage for the sequential transform traversal.
/// </summary>
private readonly Av1LevelBuffer coefficientLevels;
/// <summary>
/// Reusable storage for the eight spatial displacement-vector candidates permitted by AV1.
/// </summary>
@ -225,6 +231,20 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
throw;
}
try
{
this.coefficientLevels = new Av1LevelBuffer(configuration);
}
catch
{
// The coefficient scratch allocation follows both neighbor contexts. Unwind those successful rents when
// construction cannot publish an owning tile reader.
this.aboveNeighborContext.Dispose();
this.leftNeighborContext.Dispose();
this.FrameInfo.Dispose();
throw;
}
if (referenceFrames is not null)
{
try
@ -237,6 +257,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
{
this.aboveNeighborContext.Dispose();
this.leftNeighborContext.Dispose();
this.coefficientLevels.Dispose();
this.FrameInfo.Dispose();
throw;
}
@ -328,6 +349,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
{
this.aboveNeighborContext.Dispose();
this.leftNeighborContext.Dispose();
this.coefficientLevels.Dispose();
this.FrameInfo.Dispose();
}
@ -336,7 +358,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// </summary>
/// <param name="tileData">The entropy-coded tile payload.</param>
/// <param name="tileNum">The zero-based tile index in row-major order.</param>
/// <remarks>Corresponds to <c>parse_tile</c> in SVT-AV1.</remarks>
/// <remarks>Corresponds to <c>decode_tile</c> in libaom.</remarks>
public void ReadTile(Span<byte> tileData, int tileNum)
{
// AV1 tiles never inherit adaptation from another tile in the same frame. Reusing one graph is safe because
@ -661,6 +683,19 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
}
Av1BlockSize subSize = partitionType.GetBlockSubSize(blockSize);
if (subSize == Av1BlockSize.Invalid)
{
throw new InvalidImageContentException($"The decoded AV1 partition type {partitionType} is invalid for block size {blockSize}.");
}
ObuColorConfig colorConfig = this.SequenceHeader.ColorConfig;
if (subSize.GetSubsampled(colorConfig.SubSamplingX, colorConfig.SubSamplingY) == Av1BlockSize.Invalid)
{
// Luma partition syntax can describe a sub-8x8 shape that has no legal representation after chroma
// subsampling. Reject it before any block state is published, matching libaom's decode_partition boundary.
throw new InvalidImageContentException($"The decoded AV1 block size {subSize} is invalid for the sequence chroma subsampling.");
}
Av1BlockSize splitSize = Av1PartitionType.Split.GetBlockSubSize(blockSize);
// Partition syntax is depth-first. The visit order here is also the order in which mode,
@ -815,7 +850,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
this.Residual(ref reader, ref partitionInfo, superblockInfo, tileInfo, blockSize);
// Store the record only after all syntax has populated it, then map every covered 4x4 position.
this.FrameInfo.UpdateModeInfo(blockModeInfo, superblockInfo);
this.FrameInfo.UpdateModeInfo(partitionInfo.ModeInfo, superblockInfo);
}
/// <summary>
@ -823,7 +858,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// </summary>
/// <param name="partitionInfo">The skipped block and its frame position.</param>
/// <param name="tileInfo">The active tile boundaries.</param>
/// <remarks>Corresponds to <c>reset_skip_context</c> in SVT-AV1.</remarks>
/// <remarks>Implements AV1 section 5.11.37.</remarks>
private void ResetSkipContext(ref Av1PartitionInfo partitionInfo, Av1TileInfo tileInfo)
{
// Subsampled 4x4 luma blocks can share chroma ownership with an adjacent luma block. A skipped block that is
@ -852,7 +887,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="superblockInfo">The containing superblock and coefficient storage.</param>
/// <param name="tileInfo">The active tile boundaries.</param>
/// <param name="blockSize">The coding block size.</param>
/// <remarks>Implements AV1 section 5.11.34 and corresponds to <c>parse_residual</c> in SVT-AV1.</remarks>
/// <remarks>Implements AV1 section 5.11.34.</remarks>
private void Residual(
ref Av1SymbolDecoder reader,
ref Av1PartitionInfo partitionInfo,
@ -1039,7 +1074,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="subY">A value indicating whether the target plane is vertically subsampled.</param>
/// <returns>The decoded end-of-block coefficient position, or zero for an all-zero transform.</returns>
/// <remarks>
/// Implements AV1 section 5.11.35 using the traversal shape of the corresponding SVT-AV1 implementation.
/// Implements AV1 section 5.11.35.
/// </remarks>
private int ParseTransformBlock(
ref Av1SymbolDecoder reader,
@ -1120,7 +1155,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="coefficientBuffer">The destination beginning at this transform's coefficient slot.</param>
/// <returns>The decoded end-of-block coefficient position, or zero for an all-zero transform.</returns>
/// <remarks>
/// Implements AV1 section 5.11.39 using the traversal shape of the corresponding SVT-AV1 implementation.
/// Implements AV1 section 5.11.39.
/// </remarks>
private int ParseCoefficients(
ref Av1SymbolDecoder reader,
@ -1169,6 +1204,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
ref transformInfo,
partitionInfo.ModeBlockToRightEdge,
partitionInfo.ModeBlockToBottomEdge,
this.coefficientLevels,
coefficientBuffer);
}
@ -1317,7 +1353,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
Av1TileInfo tileInfo,
bool allowSelect)
{
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo;
ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
if (this.FrameHeader.LosslessArray[modeInfo.SegmentId])
{
return Av1TransformSize.Size4x4;
@ -1406,7 +1442,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="partitionInfo">The current coding block.</param>
/// <param name="superblockInfo">The containing superblock.</param>
/// <param name="tileInfo">The active tile boundaries.</param>
/// <remarks>Implements AV1 section 5.11.16 and corresponds to <c>read_block_tx_size</c> in SVT-AV1.</remarks>
/// <remarks>Implements AV1 section 5.11.16 and corresponds to <c>read_tx_size</c> in libaom.</remarks>
private void ReadBlockTransformSize(
ref Av1SymbolDecoder reader,
Point modeInfoLocation,
@ -1418,7 +1454,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
int block4x4Width = blockSize.Get4x4WideCount();
int block4x4Height = blockSize.Get4x4HighCount();
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo;
ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
bool usesInterTransformSyntax = modeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Last || modeInfo.UseIntraBlockCopy;
this.transformUnitCount[(int)Av1Plane.Y].AsSpan(0, 4).Clear();
@ -1738,7 +1774,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <remarks>Implements AV1 section 5.11.49.</remarks>
private void ReadPaletteTokens(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo)
{
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo;
ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
if (modeInfo.GetPaletteSize(Av1PlaneType.Y) != 0)
{
GetPaletteMapDimensions(
@ -1750,14 +1786,24 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
out int rows,
out int columns);
byte[] colorIndexMap = DecodePaletteColorMap(
Buffer2DRegion<byte> colorIndexMap = this.FrameInfo.GetPaletteColorIndexMap(
this.configuration,
Av1PlaneType.Y,
new Rectangle(
partitionInfo.ColumnIndex << Av1Constants.ModeInfoSizeLog2,
partitionInfo.RowIndex << Av1Constants.ModeInfoSizeLog2,
planeWidth,
planeHeight));
DecodePaletteColorMap(
ref reader,
modeInfo.GetPaletteSize(Av1PlaneType.Y),
Av1PlaneType.Y,
planeWidth,
planeHeight,
rows,
columns);
columns,
colorIndexMap);
modeInfo.SetPaletteColorIndexMap(Av1PlaneType.Y, colorIndexMap);
}
@ -1773,14 +1819,26 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
out int rows,
out int columns);
byte[] colorIndexMap = DecodePaletteColorMap(
int subX = this.SequenceHeader.ColorConfig.SubSamplingX ? 1 : 0;
int subY = this.SequenceHeader.ColorConfig.SubSamplingY ? 1 : 0;
Buffer2DRegion<byte> colorIndexMap = this.FrameInfo.GetPaletteColorIndexMap(
this.configuration,
Av1PlaneType.Uv,
new Rectangle(
(partitionInfo.ColumnIndex << Av1Constants.ModeInfoSizeLog2) >> subX,
(partitionInfo.RowIndex << Av1Constants.ModeInfoSizeLog2) >> subY,
planeWidth,
planeHeight));
DecodePaletteColorMap(
ref reader,
modeInfo.GetPaletteSize(Av1PlaneType.Uv),
Av1PlaneType.Uv,
planeWidth,
planeHeight,
rows,
columns);
columns,
colorIndexMap);
modeInfo.SetPaletteColorIndexMap(Av1PlaneType.Uv, colorIndexMap);
}
@ -1814,7 +1872,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <remarks>Implements the prefix, intra, and translational inter branches of AV1 section 5.11.7.</remarks>
public void ReadInterFrameModeInfo(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo, Av1TileInfo tileInfo)
{
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo;
ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
modeInfo.MotionVectors.Clear();
this.ReadInterSegmentId(ref reader, ref partitionInfo, beforeSkip: true);
@ -2292,7 +2350,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
ref Av1PartitionInfo partitionInfo,
Av1PredictionMode yMode)
{
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo;
ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
modeInfo.YMode = yMode;
modeInfo.SetAngleDelta(Av1PlaneType.Y, IntraAngleInfo(ref reader, yMode, modeInfo.BlockSize));
@ -2302,7 +2360,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
if (modeInfo.UvMode == Av1ChromaPredictionMode.ChromaFromLuma)
{
ReadChromaFromLumaAlphas(ref reader, modeInfo);
ReadChromaFromLumaAlphas(ref reader, ref modeInfo);
}
modeInfo.SetAngleDelta(
@ -2390,7 +2448,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <remarks>Implements AV1 section 5.11.46.</remarks>
private void PaletteModeInfo(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo)
{
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo;
ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
Av1BlockSize blockSize = modeInfo.BlockSize;
// The palette block-size context is the base-two block-area difference from an 8-by-8 block.
@ -2401,12 +2459,12 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
if (modeInfo.YMode == Av1PredictionMode.DC)
{
int neighborContext = 0;
if (partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.GetPaletteSize(Av1PlaneType.Y) != 0)
if (partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.Value.GetPaletteSize(Av1PlaneType.Y) != 0)
{
neighborContext++;
}
if (partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.GetPaletteSize(Av1PlaneType.Y) != 0)
if (partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.Value.GetPaletteSize(Av1PlaneType.Y) != 0)
{
neighborContext++;
}
@ -2595,10 +2653,12 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
: partitionInfo.AboveModeInfo;
Av1BlockModeInfo? leftModeInfo = partitionInfo.LeftModeInfo;
int abovePaletteSize = aboveModeInfo?.GetPaletteSize(plane) ?? 0;
int leftPaletteSize = leftModeInfo?.GetPaletteSize(plane) ?? 0;
ReadOnlySpan<ushort> aboveColors = aboveModeInfo is null ? [] : aboveModeInfo.GetPaletteColors(plane);
ReadOnlySpan<ushort> leftColors = leftModeInfo is null ? [] : leftModeInfo.GetPaletteColors(plane);
Av1BlockModeInfo above = aboveModeInfo.GetValueOrDefault();
Av1BlockModeInfo left = leftModeInfo.GetValueOrDefault();
int abovePaletteSize = aboveModeInfo is null ? 0 : above.GetPaletteSize(plane);
int leftPaletteSize = leftModeInfo is null ? 0 : left.GetPaletteSize(plane);
ReadOnlySpan<ushort> aboveColors = aboveModeInfo is null ? [] : above.GetPaletteColors(plane);
ReadOnlySpan<ushort> leftColors = leftModeInfo is null ? [] : left.GetPaletteColors(plane);
int aboveIndex = 0;
int leftIndex = 0;
int count = 0;
@ -2725,18 +2785,18 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="planeHeight">The padded plane-block height.</param>
/// <param name="rows">The number of rows inside the coded image.</param>
/// <param name="columns">The number of columns inside the coded image.</param>
/// <returns>The decoded row-major color-index map.</returns>
private static byte[] DecodePaletteColorMap(
/// <param name="colorIndexMap">The row-addressable destination map.</param>
private static void DecodePaletteColorMap(
ref Av1SymbolDecoder reader,
int paletteSize,
Av1PlaneType planeType,
int planeWidth,
int planeHeight,
int rows,
int columns)
int columns,
Buffer2DRegion<byte> colorIndexMap)
{
byte[] colorIndexMap = new byte[planeWidth * planeHeight];
colorIndexMap[0] = (byte)reader.ReadUniform(paletteSize);
colorIndexMap.DangerousGetRowSpan(0)[0] = (byte)reader.ReadUniform(paletteSize);
Span<byte> colorOrder = stackalloc byte[Av1Constants.PaletteMaxSize];
for (int diagonal = 1; diagonal < rows + columns - 1; diagonal++)
{
@ -2747,14 +2807,13 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
int row = diagonal - column;
int colorContext = GetPaletteColorIndexContext(
colorIndexMap,
planeWidth,
row,
column,
paletteSize,
colorOrder);
int colorOrderIndex = reader.ReadPaletteColorIndex(paletteSize, colorContext, planeType);
colorIndexMap[(row * planeWidth) + column] = colorOrder[colorOrderIndex];
colorIndexMap.DangerousGetRowSpan(row)[column] = colorOrder[colorOrderIndex];
}
}
@ -2763,44 +2822,50 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
// Blocks clipped by the right image edge repeat their final coded column into the padded block area.
for (int row = 0; row < rows; row++)
{
int rowOffset = row * planeWidth;
colorIndexMap.AsSpan(rowOffset + columns, planeWidth - columns)
.Fill(colorIndexMap[rowOffset + columns - 1]);
Span<byte> colorIndexRow = colorIndexMap.DangerousGetRowSpan(row);
colorIndexRow.Slice(columns, planeWidth - columns)
.Fill(colorIndexRow[columns - 1]);
}
}
// Blocks clipped by the bottom image edge repeat their final coded row for later transform reconstruction.
ReadOnlySpan<byte> finalRow = colorIndexMap.AsSpan((rows - 1) * planeWidth, planeWidth);
ReadOnlySpan<byte> finalRow = colorIndexMap.DangerousGetRowSpan(rows - 1);
for (int row = rows; row < planeHeight; row++)
{
finalRow.CopyTo(colorIndexMap.AsSpan(row * planeWidth, planeWidth));
finalRow.CopyTo(colorIndexMap.DangerousGetRowSpan(row));
}
return colorIndexMap;
}
/// <summary>
/// Derives the palette color order and entropy context from the left, upper-left, and above indices.
/// </summary>
/// <param name="colorIndexMap">The partially decoded color-index map.</param>
/// <param name="stride">The map row stride.</param>
/// <param name="row">The current map row.</param>
/// <param name="column">The current map column.</param>
/// <param name="paletteSize">The number of palette colors.</param>
/// <param name="colorOrder">The destination color order for the current context.</param>
/// <returns>The color-index entropy context in the range from zero through four.</returns>
private static int GetPaletteColorIndexContext(
ReadOnlySpan<byte> colorIndexMap,
int stride,
Buffer2DRegion<byte> colorIndexMap,
int row,
int column,
int paletteSize,
Span<byte> colorOrder)
{
Span<int> neighborColors = stackalloc int[3];
neighborColors[0] = column > 0 ? colorIndexMap[(row * stride) + column - 1] : -1;
neighborColors[1] = column > 0 && row > 0 ? colorIndexMap[((row - 1) * stride) + column - 1] : -1;
neighborColors[2] = row > 0 ? colorIndexMap[((row - 1) * stride) + column] : -1;
ReadOnlySpan<byte> currentRow = colorIndexMap.DangerousGetRowSpan(row);
neighborColors[0] = column > 0 ? currentRow[column - 1] : -1;
if (row > 0)
{
ReadOnlySpan<byte> aboveRow = colorIndexMap.DangerousGetRowSpan(row - 1);
neighborColors[1] = column > 0 ? aboveRow[column - 1] : -1;
neighborColors[2] = aboveRow[column];
}
else
{
neighborColors[1] = -1;
neighborColors[2] = -1;
}
Span<int> scores = stackalloc int[Av1Constants.PaletteMaxSize];
ReadOnlySpan<int> neighborWeights = [2, 1, 2];
@ -2855,7 +2920,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="reader">The tile symbol decoder.</param>
/// <param name="modeInfo">The block mode information to populate.</param>
/// <remarks>Implements AV1 section 5.11.45.</remarks>
private static void ReadChromaFromLumaAlphas(ref Av1SymbolDecoder reader, Av1BlockModeInfo modeInfo)
private static void ReadChromaFromLumaAlphas(ref Av1SymbolDecoder reader, ref Av1BlockModeInfo modeInfo)
{
int jointSignPlus1 = reader.ReadChromFromLumaSign() + 1;
int index = 0;
@ -2932,7 +2997,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
public void ReadInterSegmentId(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo, bool beforeSkip)
{
ObuSegmentationParameters segmentationParameters = this.FrameHeader.SegmentationParameters;
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo;
ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
if (!segmentationParameters.Enabled)
{
@ -3049,7 +3114,15 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
: prevUL == prevU && prevUL == prevL ? 2
: prevUL == prevU || prevUL == prevL || prevU == prevL ? 1 : 0;
int lastActiveSegmentId = this.FrameHeader.SegmentationParameters.LastActiveSegmentId;
partitionInfo.ModeInfo.SegmentId = Av1SymbolContextHelper.NegativeDeinterleave(reader.ReadSegmentId(ctx), predictor, lastActiveSegmentId + 1);
int segmentId = Av1SymbolContextHelper.NegativeDeinterleave(reader.ReadSegmentId(ctx), predictor, lastActiveSegmentId + 1);
if (segmentId is < 0 || segmentId > lastActiveSegmentId)
{
// The coded alphabet always contains eight symbols, even when the frame activates fewer segments.
// Validate the reconstructed ID at the same corruption boundary as libaom's read_segment_id.
throw new InvalidImageContentException("The decoded AV1 segment identifier exceeds the active segment range.");
}
partitionInfo.ModeInfo.SegmentId = segmentId;
}
}
@ -3151,8 +3224,8 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
}
else
{
int aboveSkip = partitionInfo.AboveModeInfo != null && partitionInfo.AboveModeInfo.Skip ? 1 : 0;
int leftSkip = partitionInfo.LeftModeInfo != null && partitionInfo.LeftModeInfo.Skip ? 1 : 0;
int aboveSkip = partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.Value.Skip ? 1 : 0;
int leftSkip = partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.Value.Skip ? 1 : 0;
return reader.ReadSkip(aboveSkip + leftSkip);
}
}
@ -3165,7 +3238,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <returns><see langword="true"/> when the block selects the frame's derived skip-mode reference pair.</returns>
private bool ReadSkipMode(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo)
{
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo;
ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
ObuSegmentationParameters segmentationParameters = this.FrameHeader.SegmentationParameters;
int segmentId = modeInfo.SegmentId;
@ -3180,8 +3253,8 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
return false;
}
int aboveSkipMode = partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.SkipMode ? 1 : 0;
int leftSkipMode = partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.SkipMode ? 1 : 0;
int aboveSkipMode = partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.Value.SkipMode ? 1 : 0;
int leftSkipMode = partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.Value.SkipMode ? 1 : 0;
return reader.ReadSkipMode(aboveSkipMode + leftSkipMode);
}
@ -3220,7 +3293,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="partitionInfo">The current coding block and its available neighbors.</param>
private void ReadReferenceFrames(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo)
{
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo;
ref Av1BlockModeInfo modeInfo = ref partitionInfo.ModeInfo;
Span<Av1ReferenceFrameType> references = modeInfo.ReferenceFrames;
if (modeInfo.SkipMode)
{
@ -3369,7 +3442,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// </summary>
/// <param name="reader">The tile symbol decoder.</param>
/// <param name="partitionInfo">The current coding block and superblock quantizer storage.</param>
/// <remarks>Corresponds to <c>read_delta_qindex</c> in SVT-AV1.</remarks>
/// <remarks>Corresponds to <c>read_delta_qindex</c> in libaom.</remarks>
private void ReadDeltaQuantizerIndex(ref Av1SymbolDecoder reader, ref Av1PartitionInfo partitionInfo)
{
if (!this.FrameHeader.DeltaQParameters.IsPresent || partitionInfo.ModeInfo.PositionInSuperblock != Point.Empty)
@ -3411,7 +3484,7 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable
/// <param name="tileInfo">The active tile boundaries.</param>
/// <param name="superblockInfo">The containing superblock.</param>
/// <returns>The partition entropy context.</returns>
/// <remarks>Corresponds to <c>partition_plane_context</c> in SVT-AV1.</remarks>
/// <remarks>Corresponds to <c>partition_plane_context</c> in libaom.</remarks>
private int GetPartitionPlaneContext(Point location, Av1BlockSize blockSize, Av1TileInfo tileInfo, Av1SuperblockInfo superblockInfo)
{
// The five stored split bits begin at the 8x8 partition point, so normalize the block-size log to that bit index.

23
tests/ImageSharp.Benchmarks/Codecs/Heif/Av1PalettePredictionBenchmarks.cs

@ -6,6 +6,7 @@ using BenchmarkDotNet.Columns;
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Jobs;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif;
@ -46,7 +47,12 @@ public class Av1PalettePredictionBenchmarks
/// <summary>
/// The decoded color-index map for one maximum-size palette block.
/// </summary>
private readonly byte[] colorIndexMap = new byte[BlockSize * BlockSize];
private Buffer2D<byte> colorIndexMapBuffer;
/// <summary>
/// The row-addressable view of <see cref="colorIndexMapBuffer"/>.
/// </summary>
private Buffer2DRegion<byte> colorIndexMap;
/// <summary>
/// The frame-wide 8-bit reconstruction surface.
@ -64,15 +70,24 @@ public class Av1PalettePredictionBenchmarks
[GlobalSetup]
public void Setup()
{
this.colorIndexMapBuffer = SixLabors.ImageSharp.Configuration.Default.MemoryAllocator.Allocate2D<byte>(BlockSize, BlockSize);
this.colorIndexMap = new Buffer2DRegion<byte>(this.colorIndexMapBuffer);
for (int row = 0; row < BlockSize; row++)
{
Span<byte> colorIndexRow = this.colorIndexMap.DangerousGetRowSpan(row);
for (int column = 0; column < BlockSize; column++)
{
this.colorIndexMap[(row * BlockSize) + column] = (byte)(((row * 5) + (column * 3)) & 7);
colorIndexRow[column] = (byte)(((row * 5) + (column * 3)) & 7);
}
}
}
/// <summary>
/// Releases the row-addressable color-index map after the benchmark run.
/// </summary>
[GlobalCleanup]
public void Cleanup() => this.colorIndexMapBuffer?.Dispose();
/// <summary>
/// Measures frame-wide 8-bit palette reconstruction.
/// </summary>
@ -85,7 +100,7 @@ public class Av1PalettePredictionBenchmarks
{
for (int column = 0; column < Width; column += BlockSize)
{
Av1PalettePredictor.Predict(this.palette8, this.colorIndexMap, BlockSize, this.destination8.AsSpan((row * Width) + column), Width, BlockSize, BlockSize);
Av1PalettePredictor.Predict(this.palette8, this.colorIndexMap, this.destination8.AsSpan((row * Width) + column), Width, BlockSize, BlockSize);
}
}
@ -104,7 +119,7 @@ public class Av1PalettePredictionBenchmarks
{
for (int column = 0; column < Width; column += BlockSize)
{
Av1PalettePredictor.Predict(this.palette12, this.colorIndexMap, BlockSize, this.destination12.AsSpan((row * Width) + column), Width, BlockSize, BlockSize);
Av1PalettePredictor.Predict(this.palette12, this.colorIndexMap, this.destination12.AsSpan((row * Width) + column), Width, BlockSize, BlockSize);
}
}

69
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CoefficientsEntropyTests.cs

@ -44,10 +44,27 @@ public class Av1CoefficientsEntropyTests
using IMemoryOwner<byte> encoded = encoder.Exit();
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex);
using Av1LevelBuffer levels = new(Configuration.Default);
decoder.ReadCoefficients(
modeInfo, new Point(0, 0), aboveContexts, leftContexts,
0, 0, 0, 1, 1, transformBlockContext, transformSize,
false, true, transformType, ref transformInfo, 0, 0, actuals);
modeInfo,
new Point(0, 0),
aboveContexts,
leftContexts,
0,
0,
0,
1,
1,
transformBlockContext,
transformSize,
false,
true,
transformType,
ref transformInfo,
0,
0,
levels,
actuals);
// Assert
Assert.Equal(endOfBlock, actuals[0]);
@ -96,11 +113,28 @@ public class Av1CoefficientsEntropyTests
using IMemoryOwner<byte> encoded = encoder.Exit();
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex);
using Av1LevelBuffer levels = new(Configuration.Default);
int plane = Math.Min((int)componentType, 1);
decoder.ReadCoefficients(
modeInfo, new Point(0, 0), aboveContexts, leftContexts,
0, 0, plane, 1, 1, transformBlockContext, transformSize,
false, true, transformType, ref transformInfo, 0, 0, actuals);
modeInfo,
new Point(0, 0),
aboveContexts,
leftContexts,
0,
0,
plane,
1,
1,
transformBlockContext,
transformSize,
false,
true,
transformType,
ref transformInfo,
0,
0,
levels,
actuals);
// Assert
Assert.Equal(endOfBlock, actuals[0]);
@ -153,11 +187,28 @@ public class Av1CoefficientsEntropyTests
using IMemoryOwner<byte> encoded = encoder.Exit();
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), BaseQIndex);
using Av1LevelBuffer levels = new(Configuration.Default);
int plane = Math.Min((int)componentType, 1);
decoder.ReadCoefficients(
modeInfo, new Point(0, 0), aboveContexts, leftContexts,
0, 0, plane, 1, 1, transformBlockContext, transformSize,
false, true, transformType, ref transformInfo, 0, 0, actuals);
modeInfo,
new Point(0, 0),
aboveContexts,
leftContexts,
0,
0,
plane,
1,
1,
transformBlockContext,
transformSize,
false,
true,
transformType,
ref transformInfo,
0,
0,
levels,
actuals);
// Assert
Assert.Equal(endOfBlock, actuals[0]);

4
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameIntraEntropyTests.cs

@ -160,8 +160,8 @@ public class Av1InterFrameIntraEntropyTests
bool leftIsInter,
int expected)
{
Av1BlockModeInfo above = hasAbove ? CreateModeInfo(aboveIsInter) : null;
Av1BlockModeInfo left = hasLeft ? CreateModeInfo(leftIsInter) : null;
Av1BlockModeInfo? above = hasAbove ? CreateModeInfo(aboveIsInter) : null;
Av1BlockModeInfo? left = hasLeft ? CreateModeInfo(leftIsInter) : null;
int actual = Av1SymbolContextHelper.GetIntraInterContext(above, left);

17
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InterFrameModeInfoTests.cs

@ -43,6 +43,7 @@ public class Av1InterFrameModeInfoTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader));
modeInfo = partitionInfo.ModeInfo;
Assert.False(modeInfo.SkipMode);
Assert.False(modeInfo.Skip);
@ -89,7 +90,7 @@ public class Av1InterFrameModeInfoTests
using IMemoryOwner<byte> encoded = writer.Exit();
Memory<byte> encodedMemory = encoded.Memory;
ReadInterFrameModeInfo(tileReader, encodedMemory, modeInfo, aboveModeInfo);
modeInfo = ReadInterFrameModeInfo(tileReader, encodedMemory, modeInfo, aboveModeInfo);
Assert.True(modeInfo.SkipMode);
Assert.True(modeInfo.Skip);
@ -139,6 +140,7 @@ public class Av1InterFrameModeInfoTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader));
modeInfo = partitionInfo.ModeInfo;
Assert.Equal(Av1InterpolationFilter.Smooth, modeInfo.InterpolationFilters[0]);
Assert.Equal((Av1InterpolationFilter)expectedHorizontalFilter, modeInfo.InterpolationFilters[1]);
@ -173,6 +175,7 @@ public class Av1InterFrameModeInfoTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader));
modeInfo = partitionInfo.ModeInfo;
Assert.Equal(Av1InterpolationFilter.Regular, modeInfo.InterpolationFilters[0]);
Assert.Equal(Av1InterpolationFilter.Regular, modeInfo.InterpolationFilters[1]);
@ -213,7 +216,7 @@ public class Av1InterFrameModeInfoTests
using IMemoryOwner<byte> encoded = writer.Exit();
Memory<byte> encodedMemory = encoded.Memory;
ReadInterFrameModeInfo(tileReader, encodedMemory, modeInfo);
modeInfo = ReadInterFrameModeInfo(tileReader, encodedMemory, modeInfo);
Assert.Equal((Av1ReferenceFrameType)expectedPrimary, modeInfo.ReferenceFrames[0]);
Assert.Equal((Av1ReferenceFrameType)expectedSecondary, modeInfo.ReferenceFrames[1]);
@ -285,7 +288,7 @@ public class Av1InterFrameModeInfoTests
writer.WriteSymbol((int)Av1InterpolationFilter.Sharp, Av1DefaultDistributions.SwitchableInterpolation[3]);
using IMemoryOwner<byte> encoded = writer.Exit();
ReadInterFrameModeInfo(tileReader, encoded.Memory, modeInfo);
modeInfo = ReadInterFrameModeInfo(tileReader, encoded.Memory, modeInfo);
Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]);
Assert.Equal(Av1ReferenceFrameType.Last2, modeInfo.ReferenceFrames[1]);
@ -311,7 +314,8 @@ public class Av1InterFrameModeInfoTests
/// <param name="encoded">The range-coded block-prefix symbols.</param>
/// <param name="modeInfo">The current coding block.</param>
/// <param name="aboveModeInfo">The available above block supplying skip-mode context.</param>
private static void ReadInterFrameModeInfo(
/// <returns>The decoded block mode information.</returns>
private static Av1BlockModeInfo ReadInterFrameModeInfo(
Av1TileReader tileReader,
Memory<byte> encoded,
Av1BlockModeInfo modeInfo,
@ -326,12 +330,14 @@ public class Av1InterFrameModeInfoTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Span, 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, tileReader.FrameHeader));
return partitionInfo.ModeInfo;
}
/// <summary>
/// Invokes the ref-struct mode parser without spatial neighbors.
/// </summary>
private static void ReadInterFrameModeInfo(
/// <returns>The decoded block mode information.</returns>
private static Av1BlockModeInfo ReadInterFrameModeInfo(
Av1TileReader tileReader,
Memory<byte> encoded,
Av1BlockModeInfo modeInfo)
@ -340,6 +346,7 @@ public class Av1InterFrameModeInfoTests
Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None);
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Span, 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, tileReader.FrameHeader));
return partitionInfo.ModeInfo;
}
/// <summary>

8
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1InverseTransformTests.cs

@ -43,10 +43,10 @@ public class Av1InverseTransformTests
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertIdentityOperatorParity, TransformConfigurations);
/// <summary>
/// Verifies the pinned-libaom widened operations at the twelve-bit inverse row-stage bounds.
/// Verifies the current-libaom widened operations at the twelve-bit inverse row-stage bounds.
/// </summary>
[Fact]
public void TwelveBitWideIntermediatesMatchPinnedLibaom()
public void TwelveBitWideIntermediatesMatchCurrentLibaom()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(AssertTwelveBitWideIntermediateParity, TransformConfigurations);
/// <summary>
@ -123,7 +123,7 @@ public class Av1InverseTransformTests
cosBit,
stageRange);
// These are the exact outputs of pinned libaom's signed Int64 terminal round. The first positive lane has an
// These are the exact outputs of current libaom's signed Int64 terminal round. The first positive lane has an
// Int32 fixed-point sum of 2,147,482,471, so adding the 2,048 rounding bias in Int32 would wrap.
Vector128<int> adstExpected0 = Vector128.Create(524_288, -524_288, 524_287, -524_287);
Vector128<int> adstExpected1 = Vector128.Create(33_612, -33_612, 33_612, -33_612);
@ -167,7 +167,7 @@ public class Av1InverseTransformTests
}
/// <summary>
/// Verifies one identity operator against exact pinned-libaom widened fixed-point results.
/// Verifies one identity operator against exact current-libaom widened fixed-point results.
/// </summary>
/// <typeparam name="TOperator">The inverse identity operator.</typeparam>
/// <param name="length">The identity-transform length.</param>

6
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1LevelBufferTests.cs

@ -20,7 +20,7 @@ public class Av1LevelBufferTests
{
// Arrange
Size size = new(width, height);
Av1LevelBuffer levels = new(Configuration.Default, size);
using Av1LevelBuffer levels = new(Configuration.Default, size);
for (byte i = 0; i < 4; i++)
{
levels.GetRow(i).Fill(i);
@ -43,7 +43,7 @@ public class Av1LevelBufferTests
{
// Arrange
Size size = new(width, height);
Av1LevelBuffer levels = new(Configuration.Default, size);
using Av1LevelBuffer levels = new(Configuration.Default, size);
for (byte i = 0; i < height; i++)
{
levels.GetRow(i).Fill(i);
@ -69,7 +69,7 @@ public class Av1LevelBufferTests
{
// Arrange
Size size = new(width, height);
Av1LevelBuffer levels = new(Configuration.Default, size);
using Av1LevelBuffer levels = new(Configuration.Default, size);
for (byte i = 0; i < height; i++)
{
levels.GetRow(i).Fill(i);

5
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1MotionModeInfoTests.cs

@ -47,6 +47,7 @@ public class Av1MotionModeInfoTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader));
modeInfo = partitionInfo.ModeInfo;
Assert.Equal(Av1MotionMode.SimpleTranslation, modeInfo.MotionMode);
Assert.Equal(Av1InterpolationFilter.Sharp, modeInfo.InterpolationFilters[0]);
@ -103,7 +104,7 @@ public class Av1MotionModeInfoTests
aboveModeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last;
aboveModeInfo.ReferenceFrames[1] = Av1ReferenceFrameType.None;
aboveModeInfo.InterpolationFilters.Fill(Av1InterpolationFilter.Regular);
aboveModeInfo.InterpolationFilters.Clear();
tileReader.FrameInfo.UpdateModeInfo(aboveModeInfo, superblockInfo);
superblockInfo.BlockCount++;
@ -137,6 +138,7 @@ public class Av1MotionModeInfoTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader));
modeInfo = partitionInfo.ModeInfo;
Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]);
Assert.Equal(Av1ReferenceFrameType.None, modeInfo.ReferenceFrames[1]);
@ -187,6 +189,7 @@ public class Av1MotionModeInfoTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.Memory.Span, 0, updateCdf: true);
tileReader.ReadInterFrameModeInfo(ref decoder, ref partitionInfo, new Av1TileInfo(0, 0, frameHeader));
modeInfo = partitionInfo.ModeInfo;
Assert.Equal(Av1ReferenceFrameType.Last, modeInfo.ReferenceFrames[0]);
Assert.Equal(Av1ReferenceFrameType.Intra, modeInfo.ReferenceFrames[1]);

12
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PalettePredictorTests.cs

@ -3,6 +3,7 @@
using SixLabors.ImageSharp.Formats.Heif.Av1;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
using SixLabors.ImageSharp.Memory;
using SixLabors.ImageSharp.Tests.TestUtilities;
namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1;
@ -40,12 +41,19 @@ public class Av1PalettePredictorTests
int mapStride = width + 5;
int destinationStride = width + 9;
byte[] colorIndexMap = CreateColorIndexMap(mapStride, height, width, paletteSize);
using Buffer2D<byte> colorIndexMapBuffer = Configuration.Default.MemoryAllocator.Allocate2D<byte>(mapStride, height);
for (int row = 0; row < height; row++)
{
colorIndexMap.AsSpan(row * mapStride, mapStride).CopyTo(colorIndexMapBuffer.DangerousGetRowSpan(row));
}
Buffer2DRegion<byte> colorIndexMapRegion = new(colorIndexMapBuffer);
ushort[] bytePalette = CreatePalette(paletteSize, 8);
byte[] expectedBytes = Enumerable.Repeat((byte)251, destinationStride * height).ToArray();
byte[] actualBytes = (byte[])expectedBytes.Clone();
ApplyReference(bytePalette, colorIndexMap, mapStride, expectedBytes, destinationStride, width, height);
Av1PalettePredictor.Predict(bytePalette, colorIndexMap, mapStride, actualBytes, destinationStride, width, height);
Av1PalettePredictor.Predict(bytePalette, colorIndexMapRegion, actualBytes, destinationStride, width, height);
Assert.Equal(expectedBytes, actualBytes);
foreach (int bitDepth in new[] { 10, 12 })
@ -55,7 +63,7 @@ public class Av1PalettePredictorTests
short[] actual = (short[])expected.Clone();
ApplyReference(palette, colorIndexMap, mapStride, expected, destinationStride, width, height);
Av1PalettePredictor.Predict(palette, colorIndexMap, mapStride, actual, destinationStride, width, height);
Av1PalettePredictor.Predict(palette, colorIndexMapRegion, actual, destinationStride, width, height);
Assert.Equal(expected, actual);
}
}

97
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs

@ -461,7 +461,7 @@ public class Av1ReconstructionConformanceTests
/// under normal SIMD dispatch and with hardware intrinsics disabled.
/// </summary>
[Fact]
public void DecodeWithActiveCdefMatchesPinnedLibaomReference()
public void DecodeWithActiveCdefMatchesCurrentLibaomReference()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateActiveCdefFixtures, ReconstructionConfigurations);
/// <summary>
@ -547,13 +547,70 @@ public class Av1ReconstructionConformanceTests
}
/// <summary>
/// Verifies decoded luma and chroma palette syntax and exact native samples against scalar libaom for an
/// Verifies decoded luma and chroma palette syntax and exact native samples against current official libaom for an
/// independently encoded AV1 still-picture stream.
/// </summary>
[Fact]
public void DecodeWithPaletteMatchesPinnedLibaomReference()
public void DecodeWithPaletteMatchesCurrentLibaomReference()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidatePaletteNativeFixture, PaletteConfigurations);
/// <summary>
/// Verifies exact palette reconstruction through segmented frame-owned map storage and tracked disposal.
/// </summary>
[Fact]
[ValidateDisposedMemoryAllocations]
public void DecodePaletteWithConstrainedAllocator()
{
TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 1_024 };
allocator.EnableNonThreadSafeLogging();
Configuration configuration = Configuration.Default.Clone();
configuration.MemoryAllocator = allocator;
byte[] payload = TestFile.Create(TestImages.Heif.Av1Palette8BitPayload).Bytes;
byte[] reference = TestFile.Create(TestImages.Heif.Av1Palette8BitReference).Bytes;
bool foundSegmentedLumaMap = false;
bool foundSegmentedChromaMap = false;
using (Av1Decoder decoder = new(configuration))
{
using Av1FrameBuffer<byte> frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _);
Assert.Equal(RequiredPaletteCoverage, GetPaletteCoverage(decoder));
AssertNativePlanesEqual(decoder, frameBuffer, reference);
Assert.NotNull(decoder.FrameHeader);
Assert.NotNull(decoder.FrameInfo);
int modeInfoWidth = Av1Math.DivideLog2Ceiling(decoder.FrameHeader.FrameSize.FrameWidth, Av1Constants.ModeInfoSizeLog2);
int modeInfoHeight = Av1Math.DivideLog2Ceiling(decoder.FrameHeader.FrameSize.FrameHeight, Av1Constants.ModeInfoSizeLog2);
for (int y = 0; y < modeInfoHeight && (!foundSegmentedLumaMap || !foundSegmentedChromaMap); y++)
{
for (int x = 0; x < modeInfoWidth && (!foundSegmentedLumaMap || !foundSegmentedChromaMap); x++)
{
Av1BlockModeInfo modeInfo = decoder.FrameInfo.GetModeInfoAt(new Point(x, y));
if (!foundSegmentedLumaMap && modeInfo.GetPaletteSize(Av1PlaneType.Y) != 0)
{
Buffer2DRegion<byte> map = modeInfo.GetPaletteColorIndexMap(Av1Plane.Y);
foundSegmentedLumaMap = map.Buffer.MemoryGroup.Count > 1;
}
if (!foundSegmentedChromaMap && modeInfo.GetPaletteSize(Av1PlaneType.Uv) != 0)
{
Buffer2DRegion<byte> map = modeInfo.GetPaletteColorIndexMap(Av1Plane.U);
foundSegmentedChromaMap = map.Buffer.MemoryGroup.Count > 1;
}
}
}
Assert.True(foundSegmentedLumaMap);
Assert.True(foundSegmentedChromaMap);
}
Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count);
Assert.All(
allocator.AllocationLog,
allocation => Assert.Single(
allocator.ReturnLog,
returned => returned.AllocationId == allocation.AllocationId));
}
/// <summary>
/// Verifies that a real palette frame whose tile entropy payload ends early is rejected instead of being decoded
/// from the range decoder's implicit zero padding.
@ -607,13 +664,13 @@ public class Av1ReconstructionConformanceTests
}
/// <summary>
/// Verifies decoded luma and chroma palette syntax and exact presented pixels for an independently encoded AVIF
/// image across the available vector widths and the scalar fallback.
/// Verifies decoded luma and chroma palette syntax and exact presented pixels against the retained reference image
/// across the available vector widths and the scalar fallback.
/// </summary>
/// <param name="provider">The AVIF input and matching reference-output naming context.</param>
[Theory]
[WithFile(TestImages.Heif.Av1Palette8BitAvif, PixelTypes.Rgba32)]
public void DecodeWithPaletteMatchesPinnedLibavifPresentation(TestImageProvider<Rgba32> provider)
public void DecodeWithPaletteMatchesRetainedPresentationReference(TestImageProvider<Rgba32> provider)
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(
ValidatePresentedFixture,
PresentationConfigurations,
@ -1145,10 +1202,10 @@ public class Av1ReconstructionConformanceTests
/// <summary>
/// Verifies every intra prediction mode and the fixture's seven transform types against the official
/// pinned-libaom all-intra conformance sequence and its exact native output.
/// current-libaom all-intra conformance sequence and its exact native output.
/// </summary>
[Fact]
public void DecodeOfficialAllIntraSequenceMatchesPinnedLibaomReference() => ValidateOfficialAllIntraFixture();
public void DecodeOfficialAllIntraSequenceMatchesCurrentLibaomReference() => ValidateOfficialAllIntraFixture();
/// <summary>
/// Decodes every all-intra IVF sample in one session, compares each frame exactly, and records the syntax
@ -1746,10 +1803,10 @@ public class Av1ReconstructionConformanceTests
}
/// <summary>
/// Verifies the official eight-bit quantizer boundaries against exact pinned-libaom native output.
/// Verifies the official eight-bit quantizer boundaries against exact current-libaom native output.
/// </summary>
[Fact]
public void DecodeOfficialEightBitQuantizerBoundarySequencesMatchPinnedLibaomReferences()
public void DecodeOfficialEightBitQuantizerBoundarySequencesMatchCurrentLibaomReferences()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(
ValidateOfficialEightBitQuantizerBoundaryFixtures,
ReconstructionConfigurations);
@ -1761,10 +1818,10 @@ public class Av1ReconstructionConformanceTests
=> ValidateOfficialEightBitQuantizerBoundaryFixturesWithConfiguration(Configuration.Default);
/// <summary>
/// Verifies the official ten-bit quantizer boundaries against exact pinned-libaom native output.
/// Verifies the official ten-bit quantizer boundaries against exact current-libaom native output.
/// </summary>
[Fact]
public void DecodeOfficialTenBitQuantizerBoundarySequencesMatchPinnedLibaomReferences()
public void DecodeOfficialTenBitQuantizerBoundarySequencesMatchCurrentLibaomReferences()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(
ValidateOfficialTenBitQuantizerBoundaryFixtures,
ReconstructionConfigurations);
@ -1790,6 +1847,16 @@ public class Av1ReconstructionConformanceTests
ValidateOfficialEightBitQuantizerBoundaryFixturesWithConfiguration(configuration);
ValidateOfficialTenBitQuantizerBoundaryFixturesWithConfiguration(configuration);
// Each decoded frame owns one maximum-sized coefficient-context scratch rent. An allocation per transform
// would produce hundreds of identically typed smaller rents for these deliberately dense fixtures.
int coefficientScratchLength =
((Av1Constants.MaxTransformSize / 2) + Av1Constants.TransformPadHorizontal) *
(Av1Constants.TransformPadTop + (Av1Constants.MaxTransformSize / 2) + Av1Constants.TransformPadBottom);
int coefficientScratchAllocations = allocator.AllocationLog.Count(
allocation => allocation.ElementType == typeof(byte) && allocation.Length == coefficientScratchLength);
Assert.Equal(4 * OfficialQuantizerFixtureFrameCount, coefficientScratchAllocations);
Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count);
Assert.All(
allocator.AllocationLog,
@ -4194,13 +4261,15 @@ public class Av1ReconstructionConformanceTests
int modeInfoRow = (y << subsamplingY) >> Av1Constants.ModeInfoSizeLog2;
Av1BlockModeInfo modeInfo = frameInfo.GetModeInfoAt(new Point(modeInfoColumn, modeInfoRow));
int blockColumn = modeInfoColumn;
while (blockColumn > 0 && ReferenceEquals(frameInfo.GetModeInfoAt(new Point(blockColumn - 1, modeInfoRow)), modeInfo))
while (blockColumn > 0 &&
frameInfo.GetModeInfoAt(new Point(blockColumn - 1, modeInfoRow)).ModeInfoIndex == modeInfo.ModeInfoIndex)
{
blockColumn--;
}
int blockRow = modeInfoRow;
while (blockRow > 0 && ReferenceEquals(frameInfo.GetModeInfoAt(new Point(modeInfoColumn, blockRow - 1)), modeInfo))
while (blockRow > 0 &&
frameInfo.GetModeInfoAt(new Point(modeInfoColumn, blockRow - 1)).ModeInfoIndex == modeInfo.ModeInfoIndex)
{
blockRow--;
}

35
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceMotionVectorsTests.cs

@ -245,20 +245,19 @@ public class Av1ReferenceMotionVectorsTests
frameInfo.InitializeMotionField(Configuration.Default, sequenceHeader, frameHeader, referenceFrames);
FillFrameWithIntraBlocks(frameInfo, sequenceHeader);
Av1MotionVector direct = new(16, 24);
Av1BlockModeInfo candidate = AddModeInfo(
AddModeInfo(
frameInfo,
sequenceHeader,
new Point(8, 4),
Av1BlockSize.Block16x16,
Av1ReferenceFrameType.Last,
direct,
Av1PredictionMode.NearestMotionVector);
Av1PredictionMode.NearestMotionVector,
Av1ReferenceFrameType.Backward,
new Av1MotionVector(40, -24));
// The direct scan adds the first reference with its normative adjacent weight. Extension visits both entries:
// it must ignore that duplicate and append only the sign-corrected backward-reference vector.
candidate.ReferenceFrames[1] = Av1ReferenceFrameType.Backward;
candidate.MotionVectors[1] = new Av1MotionVector(40, -24);
Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty);
Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, new Point(8, 8));
Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, true, Av1PartitionType.None)
@ -361,31 +360,29 @@ public class Av1ReferenceMotionVectorsTests
Av1MotionVector abovePrimary = new(8, 16);
Av1MotionVector aboveSecondary = new(24, 32);
Av1BlockModeInfo above = AddModeInfo(
AddModeInfo(
frameInfo,
sequenceHeader,
new Point(8, 4),
Av1BlockSize.Block16x16,
Av1ReferenceFrameType.Last,
abovePrimary,
Av1PredictionMode.NewNewMotionVector);
above.ReferenceFrames[1] = Av1ReferenceFrameType.Backward;
above.MotionVectors[1] = aboveSecondary;
Av1PredictionMode.NewNewMotionVector,
Av1ReferenceFrameType.Backward,
aboveSecondary);
Av1MotionVector leftPrimary = new(40, 48);
Av1MotionVector leftSecondary = new(56, 64);
Av1BlockModeInfo left = AddModeInfo(
AddModeInfo(
frameInfo,
sequenceHeader,
new Point(4, 8),
Av1BlockSize.Block16x16,
Av1ReferenceFrameType.Last,
leftPrimary,
Av1PredictionMode.NearestNearestMotionVector);
left.ReferenceFrames[1] = Av1ReferenceFrameType.Backward;
left.MotionVectors[1] = leftSecondary;
Av1PredictionMode.NearestNearestMotionVector,
Av1ReferenceFrameType.Backward,
leftSecondary);
Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty);
Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block16x16, new Point(8, 8));
@ -592,6 +589,8 @@ public class Av1ReferenceMotionVectorsTests
/// <param name="referenceFrame">The primary prediction reference.</param>
/// <param name="motionVector">The primary motion vector.</param>
/// <param name="predictionMode">The decoded luma or inter prediction mode.</param>
/// <param name="secondaryReferenceFrame">The optional secondary prediction reference.</param>
/// <param name="secondaryMotionVector">The optional secondary motion vector.</param>
/// <returns>The mapped mode-information block.</returns>
private static Av1BlockModeInfo AddModeInfo(
Av1FrameInfo frameInfo,
@ -600,7 +599,9 @@ public class Av1ReferenceMotionVectorsTests
Av1BlockSize blockSize,
Av1ReferenceFrameType referenceFrame,
Av1MotionVector motionVector,
Av1PredictionMode predictionMode)
Av1PredictionMode predictionMode,
Av1ReferenceFrameType secondaryReferenceFrame = Av1ReferenceFrameType.None,
Av1MotionVector secondaryMotionVector = default)
{
int superblockSize = sequenceHeader.SuperblockModeInfoSize;
Point superblockPosition = new(position.X / superblockSize, position.Y / superblockSize);
@ -612,7 +613,9 @@ public class Av1ReferenceMotionVectorsTests
};
modeInfo.ReferenceFrames[0] = referenceFrame;
modeInfo.ReferenceFrames[1] = secondaryReferenceFrame;
modeInfo.MotionVectors[0] = motionVector;
modeInfo.MotionVectors[1] = secondaryMotionVector;
frameInfo.UpdateModeInfo(modeInfo, superblockInfo);
superblockInfo.BlockCount++;
return modeInfo;

38
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceTransform.cs

@ -8,21 +8,11 @@ namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1;
internal class Av1ReferenceTransform
{
/******************************************************************************
* SVT file: test/ref/TxfmRef.cc
*
* Reference implementation for txfm, including :
* - reference_dct_1d
* - reference_adst_1d
* - reference_idtx_1d
* - reference_txfm_1d
* - reference_txfm_2d
* - fadst_ref
*
* Original authors: Cidana-Edmond, Cidana-Wenyao
*
******************************************************************************/
/// <summary>
/// Gets the analytical amplification used by current libaom's forward-transform tests.
/// </summary>
/// <param name="config">The transform configuration.</param>
/// <returns>The two-dimensional transform amplification.</returns>
public static double GetScaleFactor(Av1Transform2dFlipConfiguration config)
{
int transformWidth = config.TransformSize.GetWidth();
@ -42,8 +32,14 @@ internal class Av1ReferenceTransform
}
/// <summary>
/// SVT: reference_txfm_2d
/// Applies the analytical two-dimensional transform used by current libaom's
/// <c>test/av1_txfm_test.cc</c>.
/// </summary>
/// <param name="input">The raster input samples.</param>
/// <param name="output">The raster output coefficients.</param>
/// <param name="transformType">The two-dimensional transform type.</param>
/// <param name="transformSize">The transform dimensions.</param>
/// <param name="scaleFactor">The configured two-dimensional amplification.</param>
public static void ReferenceTransformFunction2d(Span<double> input, Span<double> output, Av1TransformType transformType, Av1TransformSize transformSize, double scaleFactor)
{
// Get transform type and size of each dimension.
@ -99,7 +95,7 @@ internal class Av1ReferenceTransform
}
}
private static void Adst4Reference(Span<int> input, Span<int> output)
private static void Adst4Reference(ReadOnlySpan<int> input, Span<int> output)
{
// 16384 * sqrt(2) * sin(kPi/9) * 2 / 3
const long sinPi19 = 5283;
@ -146,7 +142,7 @@ internal class Av1ReferenceTransform
output[3] = Av1Math.RoundShift(s3, 14);
}
private static void ReferenceIdentity1d(Span<double> input, Span<double> output, int size)
private static void ReferenceIdentity1d(ReadOnlySpan<double> input, Span<double> output, int size)
{
const double sqrt2 = 1.4142135623730950488016887242097f;
double scale = 0;
@ -178,7 +174,7 @@ internal class Av1ReferenceTransform
}
}
private static void ReferenceDct1d(Span<double> input, Span<double> output, int size)
private static void ReferenceDct1d(ReadOnlySpan<double> input, Span<double> output, int size)
{
const double kInvSqrt2 = 0.707106781186547524400844362104f;
for (int k = 0; k < size; ++k)
@ -196,7 +192,7 @@ internal class Av1ReferenceTransform
}
}
private static void ReferenceAdst1d(Span<double> input, Span<double> output, int size)
private static void ReferenceAdst1d(ReadOnlySpan<double> input, Span<double> output, int size)
{
if (size == 4)
{
@ -227,7 +223,7 @@ internal class Av1ReferenceTransform
}
}
internal static void ReferenceTransform1d(Av1TransformType1d type, Span<double> input, Span<double> output, int size)
internal static void ReferenceTransform1d(Av1TransformType1d type, ReadOnlySpan<double> input, Span<double> output, int size)
{
switch (type)
{

32
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TemporalSegmentationTests.cs

@ -61,6 +61,32 @@ public class Av1TemporalSegmentationTests
Assert.NotEqual(16384U, destination.SegmentIdPredicted[2][0]);
}
/// <summary>
/// Verifies that a spatial segment symbol cannot select an identifier above the frame's last active segment.
/// </summary>
[Fact]
public void SpatialSegmentIdOutsideActiveRangeIsRejected()
{
ObuSequenceHeader sequenceHeader = CreateSequenceHeader(64, 64);
ObuFrameHeader frameHeader = CreateFrameHeader(16, 16, segmentationUpdateMap: 1, segmentationTemporalUpdate: 0);
frameHeader.SegmentationParameters.LastActiveSegmentId = 0;
using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader);
using Av1SymbolWriter writer = new(Configuration.Default, 1, updateCdf: true);
writer.WriteSymbol(Av1Constants.MaxSegmentCount - 1, Av1DefaultDistributions.SegmentId[0]);
using IMemoryOwner<byte> encoded = writer.Exit();
Assert.Throws<InvalidImageContentException>(
() =>
{
Av1BlockModeInfo modeInfo = new(Av1BlockSize.Block8x8, Point.Empty);
Av1SuperblockInfo superblockInfo = new(tileReader.FrameInfo, Point.Empty);
Av1PartitionInfo partitionInfo = new(modeInfo, superblockInfo, false, Av1PartitionType.None);
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true);
tileReader.ReadInterSegmentId(ref decoder, ref partitionInfo, beforeSkip: false);
});
}
/// <summary>
/// Verifies that only neighboring blocks which selected temporal prediction contribute to the binary CDF context.
/// </summary>
@ -82,8 +108,8 @@ public class Av1TemporalSegmentationTests
bool leftPredicted,
int expected)
{
Av1BlockModeInfo aboveModeInfo = hasAbove ? CreateModeInfo(abovePredicted) : null;
Av1BlockModeInfo leftModeInfo = hasLeft ? CreateModeInfo(leftPredicted) : null;
Av1BlockModeInfo? aboveModeInfo = hasAbove ? CreateModeInfo(abovePredicted) : null;
Av1BlockModeInfo? leftModeInfo = hasLeft ? CreateModeInfo(leftPredicted) : null;
int actual = Av1SymbolContextHelper.GetSegmentIdPredictedContext(aboveModeInfo, leftModeInfo);
@ -146,6 +172,7 @@ public class Av1TemporalSegmentationTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true);
tileReader.ReadInterSegmentId(ref decoder, ref partitionInfo, beforeSkip: segmentIdPrecedesSkip);
modeInfo = partitionInfo.ModeInfo;
Assert.True(modeInfo.SegmentIdPredicted);
Assert.Equal(2, modeInfo.SegmentId);
@ -194,6 +221,7 @@ public class Av1TemporalSegmentationTests
Av1SymbolDecoder decoder = new(Configuration.Default, encoded.GetSpan(), 0, updateCdf: true);
tileReader.ReadInterSegmentId(ref decoder, ref partitionInfo, beforeSkip: false);
modeInfo = partitionInfo.ModeInfo;
Assert.False(modeInfo.SegmentIdPredicted);
Assert.Equal(3, modeInfo.SegmentId);

79
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1TilingTests.cs

@ -1,7 +1,9 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using System.Buffers;
using SixLabors.ImageSharp.Formats.Heif.Av1;
using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy;
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
@ -15,6 +17,23 @@ namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1;
[Trait("Format", "Avif")]
public class Av1TilingTests
{
/// <summary>
/// Verifies that frame mode-information indices do not wrap at the unsigned 16-bit boundary.
/// </summary>
[Fact]
public void ModeInfoMapSupportsMoreThanUShortMaxBlocks()
{
const int blockCount = ushort.MaxValue + 2;
Av1FrameInfo.Av1FrameModeInfoMap map = new(new Size(blockCount, 1));
for (int index = 0; index < blockCount; index++)
{
map.Update(new Point(index, 0), Av1BlockSize.Block4x4);
}
Assert.Equal(blockCount, map.NextIndex);
Assert.Equal(blockCount - 1, map[new Point(blockCount - 1, 0)]);
}
/// <summary>
/// Verifies the decoded block geometry and prediction modes against libaom inspection output for a real AVIF image item.
/// </summary>
@ -75,6 +94,59 @@ public class Av1TilingTests
Assert.True(image.Frames.RootFrame.PixelBuffer.DangerousGetSingleSpan().ContainsAnyExcept(default(Rgba32)));
}
/// <summary>
/// Verifies that partition syntax cannot produce a luma block with no valid 4:2:0 chroma representation.
/// </summary>
[Fact]
public void RejectsPartitionThatCannotRepresentSubsampledChroma()
{
ObuSequenceHeader sequenceHeader = new()
{
MaxFrameWidth = 64,
MaxFrameHeight = 64,
Use128x128Superblock = false,
ColorConfig = new ObuColorConfig
{
BitDepth = Av1BitDepth.EightBit,
SubSamplingX = true,
SubSamplingY = true
}
};
ObuTileGroupHeader tileInfo = new()
{
TileColumnCount = 1,
TileRowCount = 1
};
tileInfo.TileColumnStartModeInfo[1] = sequenceHeader.SuperblockModeInfoSize;
tileInfo.TileRowStartModeInfo[1] = sequenceHeader.SuperblockModeInfoSize;
ObuFrameHeader frameHeader = new()
{
ModeInfoColumnCount = sequenceHeader.SuperblockModeInfoSize,
ModeInfoRowCount = sequenceHeader.SuperblockModeInfoSize,
ModeInfoStride = sequenceHeader.SuperblockModeInfoSize,
TilesInfo = tileInfo,
DisableCdfUpdate = true,
DisableFrameEndUpdateCdf = true
};
using Av1SymbolWriter writer = new(Configuration.Default, 1, updateCdf: false);
Av1Distribution[] partitionTypes = Av1DefaultDistributions.PartitionTypes;
Av1BlockSize blockSize = sequenceHeader.SuperblockSize;
while (blockSize > Av1BlockSize.Block8x8)
{
int blockSizeLog = blockSize.Get4x4WidthLog2() - Av1BlockSize.Block8x8.Get4x4WidthLog2();
int context = blockSizeLog * Av1Constants.PartitionProbabilitySet;
writer.WriteSymbol((int)Av1PartitionType.Split, partitionTypes[context]);
blockSize = Av1PartitionType.Split.GetBlockSubSize(blockSize);
}
writer.WriteSymbol((int)Av1PartitionType.Horizontal, partitionTypes[0]);
using IMemoryOwner<byte> encoded = writer.Exit();
using Av1TileReader tileReader = new(Configuration.Default, sequenceHeader, frameHeader);
Assert.Throws<InvalidImageContentException>(() => tileReader.ReadTile(encoded.GetSpan(), 0));
}
[Theory]
[InlineData(TestImages.Heif.XnConvert, 0x010E, 0x03CC, 18, 16)]
[InlineData(TestImages.Heif.Orange4x4, 0x010E, 0x001d, 21, 1)]
@ -190,8 +262,7 @@ public class Av1TilingTests
Span<Av1BlockModeInfo> modeInfos = superblockInfo.GetModeInfos();
Assert.Equal(superblockInfo.BlockCount, modeInfos.Length);
Assert.DoesNotContain(modeInfos.ToArray(), modeInfo => modeInfo is null);
Assert.Same(modeInfos[0], tileReader.FrameInfo.GetModeInfo(superblockPosition));
Assert.Equal(modeInfos[0].ModeInfoIndex, tileReader.FrameInfo.GetModeInfo(superblockPosition).ModeInfoIndex);
foreach (Av1BlockModeInfo modeInfo in modeInfos)
{
@ -203,7 +274,9 @@ public class Av1TilingTests
{
for (int x = 0; x < modeInfo.BlockSize.Get4x4WideCount(); x++)
{
Assert.Same(modeInfo, tileReader.FrameInfo.GetModeInfoAt(new Point(modeInfoPosition.X + x, modeInfoPosition.Y + y)));
Assert.Equal(
modeInfo.ModeInfoIndex,
tileReader.FrameInfo.GetModeInfoAt(new Point(modeInfoPosition.X + x, modeInfoPosition.Y + y)).ModeInfoIndex);
}
}
}

0
tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeWithPaletteMatchesPinnedLibavifPresentation_Rgba32_libavif-palette-draw-points-8b.png → tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeWithPaletteMatchesRetainedPresentationReference_Rgba32_libavif-palette-draw-points-8b.png

28
tests/Images/Input/Heif/Av1/Conformance/README.md

@ -102,7 +102,33 @@ The film-grain Y4M SHA-256 is `A1B553BE140F48ABDDB2A6D39917AB714BA03AC7FFD6359EA
The retained `quantizer-00` and `quantizer-63` streams are the minimum- and maximum-quantizer boundaries from libaom's official eight- and ten-bit test matrices. Their SHA-1 values are `C2E1EC9936B95254187A359E94AA32A9F3DAD1B7`, `2A8AA33513D8E01AE9410C4BF5FE1E471B775482`, `9BBE8499796AA588FF02E313FB0D4349940D2FEA`, and `8B6EB3FFF2E0DB7EAC775B08C745250CA591E2D9`, exactly matching `test/test-data.sha1` at pinned libaom commit `03087864cf4bea6abb0d28f95cf7843511413d8f`. Their SHA-256 values, in the same order, are `6382DBD2BEFBBC93D4EA283586F4FB43FEA5F1C52400E3D2C5281A46B1104C00`, `0E4EC80680F7AF8DE9621B016E0F2D7C0858B2951DEBC173DDA50C6A051547D3`, `FE6053CE4EE20A1C0EC6F7FE35DB097E92AD25D8A3505598BD89162C74D7944F`, and `39759AB77483E1D11049DC38B5F5262158FD9C3CBC9D1F82A02462FC5DF30E0C`.
The native references were generated with the pinned generic `aomdec --threads=1` build. Their SHA-256 values are `D499028E0606DB70CD56A72F151E04F36C09F300A448CCCD8430DD920D3589C5`, `4CC9892B3EE3399B293E31014B9F566C21E0C7A4765FC5F444528769C33E6D67`, `78373C28F401EB95D3E563D146622ED6C714ED96661E5E57C539CE71D7BED599`, and `A9DF86F671B8CF01EFC130660556412D4EBAF31A81D6F26FBDAEB0A7E839D8EA`. Each reference's two raw-frame MD5 values also match the corresponding official `.ivf.md5` file exactly. The tests compare every native sample under normal and scalar `FeatureTestRunner` dispatch and run all four sequences through a 2,560-byte row-aligned constrained tracked allocator.
The native references were originally generated with the historical generic `aomdec --threads=1` build. On 2026-08-31 current official libaom `main` at observed revision `441c439b9916474cac15d2822af47a9ad70674a8` reproduced all four references byte for byte. Their SHA-256 values are `D499028E0606DB70CD56A72F151E04F36C09F300A448CCCD8430DD920D3589C5`, `4CC9892B3EE3399B293E31014B9F566C21E0C7A4765FC5F444528769C33E6D67`, `78373C28F401EB95D3E563D146622ED6C714ED96661E5E57C539CE71D7BED599`, and `A9DF86F671B8CF01EFC130660556412D4EBAF31A81D6F26FBDAEB0A7E839D8EA`. Each reference's two raw-frame MD5 values also match the corresponding official `.ivf.md5` file exactly. The tests compare every native sample under normal and scalar `FeatureTestRunner` dispatch and run all four sequences through a 2,560-byte row-aligned constrained tracked allocator.
## Palette reconstruction fixture
The 42-byte `libaom-palette-draw-points-8b-444.bit` payload has SHA-256
`F412A9E7F19D1C009D0329B993BB503D74CCDA58505C54BAE8FB3C16142181DC`. On 2026-08-31
current official libaom `main` at observed revision `441c439b9916474cac15d2822af47a9ad70674a8`
decoded it with one thread, row threading disabled, raw output, and eight-bit output depth. The resulting
1,089-byte YUV444 output matches the retained native reference exactly at SHA-256
`E05F7C0DF06ECCF0E43869D1D7B03DAA1D635ACD26A766F8940899BE18D53251`.
The production tests require active luma and chroma palette syntax, compare every native sample under
`FeatureTestRunner`, and compare the final AVIF presentation through the established reference-output API.
The retained PNG has SHA-256
`1148EBF6AA4B0F2D069D5E9B9605F6FB2A315E525F18016CDCAE23EFDD81DA84`. A 1 KiB
constrained tracked allocator forces both frame-owned palette map surfaces across multiple memory groups;
the test verifies exact reconstruction and exactly one return for every recorded allocation.
## Official all-intra fixture
On 2026-08-31 current official libaom `main` at observed revision
`441c439b9916474cac15d2822af47a9ad70674a8` reproduced the retained 39-frame all-intra Y4M byte for
byte. The IVF SHA-256 is `5FCD265FD9F9BDD0D3179340B4C4532F1422CA5E5D97741C7481B84CB5DC122F`;
the native reference SHA-256 is
`1211EBEFBC9CCEF9ED19BE4CCE3F807D69FFFE338E95CCA1B5F4CA8023482175`. The production test
decodes all 39 frames in one decoder session, compares every native sample exactly, and requires coverage
of every intra prediction mode and all seven transform types selected by the fixture.
## Official frame-size corner fixtures

Loading…
Cancel
Save