Browse Source

Verify official AV1 two-spatial-layer sequence

pull/2633/head
James Jackson-South 4 days ago
parent
commit
27ebcd54dc
  1. 7
      HEIF_IMPLEMENTATION_PLAN.md
  2. 85
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs
  3. 2
      tests/ImageSharp.Tests/TestImages.cs
  4. 3
      tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1-libaom.y4m
  5. 3
      tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1.ivf

7
HEIF_IMPLEMENTATION_PLAN.md

@ -29,7 +29,7 @@ Checkboxes may be marked complete only when the implementation and the verificat
## Delivery dashboard
Last reconciled with the source tree on 2026-08-29 against production checkpoint `ef93d584511055f8e91e8d801662c44a5b0984d9`. Committed checkpoints include the AV1 transform architecture, OBU framing, intra-block copy, 12-profile reconstruction matrix, corrected twelve-bit inverse-transform SIMD arithmetic, layered-item properties, layered reference/header/CDF/motion-field state, inter-frame intra blocks, SIMD-first translational prediction, complete single-reference inter reconstruction, compound reference trees and modes, paired reference-MV derivation, reference-dependent bounded sequence decoding, allocation-free SIMD-first equal averaging, selected inter-intra prediction, selectable compound blending, OBMC, scaled-reference reconstruction, local warped prediction, non-translational global prediction, official motion-vector conformance, selected spatial-layer presentation, and progressive color and auxiliary-alpha conformance. This dashboard is the authoritative delivery order. The detailed phase checklists below provide subsystem evidence; they do not override the current-stage marker or permit work to skip ahead.
Last reconciled with the source tree on 2026-08-29 against production checkpoint `ea2fa484716944b86fadac5eac3eef23874ac25b`. Committed checkpoints include the AV1 transform architecture, OBU framing, intra-block copy, official extreme-displacement intra-block-copy conformance, the 12-profile reconstruction matrix, corrected twelve-bit inverse-transform SIMD arithmetic, layered-item properties, layered reference/header/CDF/motion-field state, inter-frame intra blocks, SIMD-first translational prediction, complete single-reference inter reconstruction, compound reference trees and modes, paired reference-MV derivation, reference-dependent bounded sequence decoding, allocation-free SIMD-first equal averaging, selected inter-intra prediction, selectable compound blending, OBMC, scaled-reference reconstruction, local warped prediction, non-translational global prediction, official motion-vector conformance, selected spatial-layer presentation, and progressive color and auxiliary-alpha conformance. This dashboard is the authoritative delivery order. The detailed phase checklists below provide subsystem evidence; they do not override the current-stage marker or permit work to skip ahead.
Commit `1c58d855f70b024170ced9eb0a7005f0f9c955ad` records the complete official motion-vector conformance checkpoint. The official IVF has SHA-1 `F064290D7FCD3B3DE19020E8AEC6C43C88D3A505`, matching the pinned libaom test-data manifest, and SHA-256 `222A9050059B254DAB17CFB802FF829C778E3F93AF18961A622C8268576C1395`; its pinned-libaom Y4M has SHA-256 `D97AC78C81782CF1507549368047769DC677DBE205706458D1EE9C807DE6EC78`. Both source targets build with zero warnings and errors, the `net10.0` test-project analyzer build completes with zero errors and 1,014 pre-existing repository warnings, Roslynk reports zero compiler errors, 3,985 focused `net10.0` cases pass without failures or skips, and `git diff --check` is clean.
@ -41,7 +41,7 @@ Commit `ef93d584511055f8e91e8d801662c44a5b0984d9` records the complete twelve-bi
Commit `5ba048be3667064f219ee230be7c8976927ea200` records the complete, verified official syntax-coverage and predictor-architecture checkpoint. SharedInfrastructure commit `a835a9d74e82b2d32b580a7902eb2699ebc47098` is pushed to `origin/main`; the ImageSharp gitlink and synchronized root attributes assign IVF and Y4M fixtures to Git LFS. The official all-intra, CDF-update, and temporal motion-field IVF files have SHA-1 `A9F7EA6312A533CC6426A6145EDD190D45813C37`, `AFCA5502A489692B0A3C120370B0F43B8FC572A1`, and `B48A717C7C003B8DD23C3C2CAED1AC673380FDB3`, exactly matching the pinned libaom manifest. Their SHA-256 values are `5FCD265FD9F9BDD0D3179340B4C4532F1422CA5E5D97741C7481B84CB5DC122F`, `14A3DBF537B6BF15EFC003182D9916D61438C93624A8BD26E6E3AE7EAF33EA82`, and `B59BF9586D8546DFDA81DFEC4EE4E32CEB502C9D22412AB0B63A2ABB534A1F14`; the pinned-libaom Y4M references have SHA-256 `1211EBEFBC9CCEF9ED19BE4CCE3F807D69FFFE338E95CCA1B5F4CA8023482175`, `4FBFF73FF0DE2D9084DAE557D1D4BD677B0486516525BF4D327D2D795D5A7779`, and `F7DB607694818C19E62FD9A27F53E1A3E2D00B72C39C0430C1B26399CC76777D`. Exact native-plane comparison covers all 39 all-intra frames, every intra mode, seven selected transform types, both tile-local and frame-end adaptive CDF publication, all four temporal motion-field frames under normal/scalar dispatch, and constrained tracked motion-field allocation with exactly-once returns. Every distinct AV1 predictor traversal now owns a family-named JPEG-style static operator contract and SIMD traversal instead of nesting separate predictors beneath broad intra/inter families. Compound reference convolution, equal averaging, distance weighting, alpha-mask blending, difference-weighted mask construction, and each intermediate reconstruction or mask traversal have separate family owners and matching `.Operator.cs` contracts; inter-intra mask construction has its own non-operator builder. Filtered intra-block-copy horizontal, vertical, and bilinear phases own separate family-named types, tap-exact static operator contracts, and family-owned SIMD/scalar traversals; the broad dispatcher retains only whole-sample copy and phase routing. Both Release source targets build with zero warnings and errors, Roslynk reports zero compiler errors, 102 final-tree focused predictor and reconstruction cases pass without failures or skips, and `git diff --check` is clean.
The working tree contains a complete, verified official extreme-displacement intra-block-copy checkpoint awaiting commit. The two-frame 1920x1080 YUV420 IVF has SHA-1 `C7F336958E7AF6162C20DDC84D67C7DFA9826910`, exactly matching the pinned libaom manifest, and SHA-256 `0252716FE7A7C7F1EBB1EB23920A6737F45C10DFBA942304BD339AE42B5C576C`; its pinned-libaom Y4M has SHA-256 `8CF7851E112172C62799F8BFDF895505742A3CA77E339C12E547AD2C47AF41B3`. The source corrects the spatial scan's four-mode-info-unit step threshold from 16-sample blocks to libaom's 64-sample boundary, clamps the ranked displacement stack before reference selection, and treats intra-block-copy neighbors as inter for transform-size context. Exact production native-plane comparison covers both complete official frames under normal and scalar `FeatureTestRunner` dispatch, while the existing independent 8/10/12-bit native-plane and presentation matrix remains clean. Both Release source targets build with zero warnings and errors, Roslynk reports zero compiler errors, 170 focused intra-block-copy, frame-buffer, reference-MV, inter-mode, and transform-size tests pass without failures or skips, and `git diff --check` is clean.
Commit `ea2fa484716944b86fadac5eac3eef23874ac25b` records the complete official extreme-displacement intra-block-copy checkpoint. The two-frame 1920x1080 YUV420 IVF has SHA-1 `C7F336958E7AF6162C20DDC84D67C7DFA9826910`, exactly matching the pinned libaom manifest, and SHA-256 `0252716FE7A7C7F1EBB1EB23920A6737F45C10DFBA942304BD339AE42B5C576C`; its pinned-libaom Y4M has SHA-256 `8CF7851E112172C62799F8BFDF895505742A3CA77E339C12E547AD2C47AF41B3`. The source corrects the spatial scan's four-mode-info-unit step threshold from 16-sample blocks to libaom's 64-sample boundary, clamps the ranked displacement stack before reference selection, and treats intra-block-copy neighbors as inter for transform-size context. Exact production native-plane comparison covers both complete official frames under normal and scalar `FeatureTestRunner` dispatch, while the existing independent 8/10/12-bit native-plane and presentation matrix remains clean. Both Release source targets build with zero warnings and errors, Roslynk reports zero compiler errors, 170 focused intra-block-copy, frame-buffer, reference-MV, inter-mode, and transform-size tests pass without failures or skips, and `git diff --check` is clean.
Status meanings:
@ -53,7 +53,7 @@ Status meanings:
Current development stage: **Stage 3 — complete AV1 still-image decoding.** The decoder retains reference/header/CDF/motion-field state, derives frame-level skip-mode references, consumes temporal segment prediction, decodes intra-coded blocks inside inter frames, and reconstructs translational single-reference, compound, inter-intra, OBMC, scaled-reference, local warped, and non-translational global prediction before residual traversal. Commit `1c58d855f70b024170ced9eb0a7005f0f9c955ad` adds exact official four-frame coverage through all ordinary inter modes, all three motion modes, every regular/smooth/sharp dual-filter pair, sub-8x8 chroma prediction, and no-round compound intermediates. Commit `019ac5648b4380a6ff4865072c96fde29ce09fad` adds exact selected-layer native reconstruction and scaled presentation. Neither AV1 nor HEVC production encoding is implemented.
Immediate checkpoint: **finish and commit the official extreme-displacement intra-block-copy conformance correction.** Complete the focused Release matrix, documentation review, and diff validation, then record the exact evidence before advancing to the next production-reachable unsupported valid AV1 still-image syntax path.
Immediate checkpoint: **verification complete; commit pending — official two-spatial-layer AV1 default-operating-point decoding.** The exact pinned-manifest IVF and pinned-libaom eight-frame 1280x720 YUV420 reference pass full-frame native-plane comparison under normal/scalar `FeatureTestRunner` dispatch and constrained tracked allocation with exactly-once returns. The existing decoder required no production correction for this stream. Both Release source targets build with zero warnings and errors, the scoped test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, all five focused compact-sequence cases pass without failures or skips, and `git diff --check` is clean. Only the checkpoint commit remains open.
| Order | Delivery stage | State | Delivered state | Gate that remains open |
| --- | --- | --- | --- | --- |
@ -603,6 +603,7 @@ Implement and verify in dependency order:
- [x] Decode all ordinary inter modes, motion modes, and regular/smooth/sharp dual-filter pairs from the official four-frame 352x288 YUV420 motion-vector sequence. The production path includes sub-8x8 chroma assembly and allocation-free no-round compound intermediates. Exact pinned-libaom native planes, normal/scalar dispatch, constrained tracked allocation, direct compound predictor parity, zero-warning source builds, the zero-error analyzer build, Roslyn compiler diagnostics, 3,985 focused cases, documentation review, and `git diff --check` pass. Commit `1c58d855f70b024170ced9eb0a7005f0f9c955ad` records the checkpoint.
- [x] Verify every connected inter mode and filter with exact independent content. Complete official coverage masks prove all ordinary inter modes, all motion modes, and every switchable regular/smooth/sharp filter pair, while the committed bounded AVIF fixtures independently prove every compound, inter-intra, OBMC, scaled, local-warp, and global-warp reconstruction branch.
- [x] Return an explicitly selected lower spatial layer at the item's presentation extent without rescaling retained references. The essential-`lsel` derivative selects the exact pinned-libaom 40x40 YUV444 base, scales native planes to 80x80 before color conversion, and matches pinned-libavif output. The allocator-owned presentation buffer uses group-safe rows, portable `Vector128` two-times filtering with scalar fallback, exact 8/12-bit libyuv-shaped vectors, constrained-allocation lifetime tracking, and leaves committed final-layer and display-only film-grain results unchanged.
- [x] Decode the official two-spatial-layer L2T1 sequence at its default operating point. The 159,037-byte IVF has manifest-matching SHA-1 `E14825F50FF845B8A6932C64CB254007A0B5E3A1` and SHA-256 `D65824E8E7C19BC958DB810D728B2132B513B5A93CB5F99CA3D7995E19A226E1`; its pinned-libaom Y4M has SHA-256 `7A427631ECBF144F435AA4612F1201415FB1A9BCF9A67BA010AEF830B0C3AB81`. All eight 1280x720 YUV420 frames match exactly under normal and scalar dispatch, and an 8 KiB constrained tracked allocator verifies balanced exactly-once returns. The existing decoder required no production correction. Both Release source targets build with zero warnings and errors, the scoped test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and all five focused compact-sequence cases pass without failures or skips.
- [ ] Lossless and high-bit-depth reconstruction with correct clipping and intermediate precision.
- [x] Route lossless 4x4 blocks through allocation-free reversible inverse Walsh-Hadamard reconstruction for 8/10/12-bit samples, including the DC-only specialization, `Vector128` production traversal, scalar fallback, exact clipping, and `FeatureTestRunner` parity.
- [x] Verify lossless syntax, inverse quantization, prediction, and presented reconstruction with independently encoded 8/10/12-bit AVIF fixtures. The tests require coded residuals with palette and intra-block copy disabled, compare every native YUV sample with the pinned generic libaom-backed decoder, and compare every presented RGBA byte with pinned generic libavif exactly under normal hardware dispatch and the scalar fallback.

85
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs

@ -186,6 +186,21 @@ public class Av1ReconstructionConformanceTests
/// </summary>
private const int OfficialIntraBlockCopyFixtureFrameCount = 2;
/// <summary>
/// The displayed width of the official libaom two-spatial-layer sequence.
/// </summary>
private const int OfficialTwoSpatialLayerFixtureWidth = 1280;
/// <summary>
/// The displayed height of the official libaom two-spatial-layer sequence.
/// </summary>
private const int OfficialTwoSpatialLayerFixtureHeight = 720;
/// <summary>
/// The number of default-operating-point frames in the official libaom two-spatial-layer sequence.
/// </summary>
private const int OfficialTwoSpatialLayerFixtureFrameCount = 8;
/// <summary>
/// The coverage bit representing tile-local adaptive CDF updates.
/// </summary>
@ -1219,6 +1234,56 @@ public class Av1ReconstructionConformanceTests
Assert.NotEqual(0, coverage & ReferenceFrameMotionVectorCoverage);
}
/// <summary>
/// Verifies the default operating point of an official two-spatial-layer sequence against exact pinned-libaom
/// native output under normal and scalar dispatch.
/// </summary>
[Fact]
public void DecodeOfficialTwoSpatialLayerSequenceMatchesPinnedLibaomReference()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(
ValidateOfficialTwoSpatialLayerFixture,
ReconstructionConfigurations);
/// <summary>
/// Verifies the official two-spatial-layer sequence through constrained tracked allocation.
/// </summary>
[Fact]
[ValidateDisposedMemoryAllocations]
public void DecodeOfficialTwoSpatialLayerSequenceWithConstrainedAllocator()
{
TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 8_192 };
allocator.EnableNonThreadSafeLogging();
Configuration configuration = Configuration.Default.Clone();
configuration.MemoryAllocator = allocator;
ValidateOfficialCompactSequence(
configuration,
TestImages.Heif.Av1OfficialTwoSpatialLayerSequence,
TestImages.Heif.Av1OfficialTwoSpatialLayerSequenceNativeReference,
OfficialTwoSpatialLayerFixtureFrameCount,
OfficialTwoSpatialLayerFixtureWidth,
OfficialTwoSpatialLayerFixtureHeight);
Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count);
Assert.All(
allocator.AllocationLog,
allocation => Assert.Single(
allocator.ReturnLog,
returned => returned.AllocationId == allocation.AllocationId));
}
/// <summary>
/// Decodes the default operating point of the official two-spatial-layer sequence.
/// </summary>
private static void ValidateOfficialTwoSpatialLayerFixture()
=> ValidateOfficialCompactSequence(
Configuration.Default,
TestImages.Heif.Av1OfficialTwoSpatialLayerSequence,
TestImages.Heif.Av1OfficialTwoSpatialLayerSequenceNativeReference,
OfficialTwoSpatialLayerFixtureFrameCount,
OfficialTwoSpatialLayerFixtureWidth,
OfficialTwoSpatialLayerFixtureHeight);
/// <summary>
/// Decodes one compact official IVF sequence, compares every native sample, and returns its active frame-state
/// coverage mask.
@ -1227,19 +1292,23 @@ public class Av1ReconstructionConformanceTests
Configuration configuration,
string fixturePath,
string nativeReferencePath,
int expectedFrameCount)
int expectedFrameCount,
int expectedWidth = OfficialMotionVectorFixtureWidth,
int expectedHeight = OfficialMotionVectorFixtureHeight)
{
byte[] ivf = TestFile.Create(fixturePath).Bytes;
byte[] nativeReference = TestFile.Create(nativeReferencePath).Bytes;
ReadOnlySpan<byte> y4mFileHeader = "YUV4MPEG2 W352 H288 F30:1 Ip C420jpeg\n"u8;
byte[] y4mFileHeader =
Encoding.ASCII.GetBytes($"YUV4MPEG2 W{expectedWidth} H{expectedHeight} F30:1 Ip C420jpeg\n");
ReadOnlySpan<byte> y4mFrameHeader = "FRAME\n"u8;
Assert.True(ivf.AsSpan(0, 4).SequenceEqual("DKIF"u8));
Assert.Equal(0, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(4, 2)));
Assert.Equal(32, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(6, 2)));
Assert.True(ivf.AsSpan(8, 4).SequenceEqual("AV01"u8));
Assert.Equal(OfficialMotionVectorFixtureWidth, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(12, 2)));
Assert.Equal(OfficialMotionVectorFixtureHeight, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(14, 2)));
Assert.Equal(expectedWidth, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(12, 2)));
Assert.Equal(expectedHeight, BinaryPrimitives.ReadUInt16LittleEndian(ivf.AsSpan(14, 2)));
Assert.Equal(
expectedFrameCount,
checked((int)BinaryPrimitives.ReadUInt32LittleEndian(ivf.AsSpan(24, 4))));
@ -1249,8 +1318,8 @@ public class Av1ReconstructionConformanceTests
int ivfOffset = 32;
int nativeOffset = y4mFileHeader.Length;
int nativeFrameLength =
(OfficialMotionVectorFixtureWidth * OfficialMotionVectorFixtureHeight) +
(2 * (OfficialMotionVectorFixtureWidth >> 1) * (OfficialMotionVectorFixtureHeight >> 1));
(expectedWidth * expectedHeight) +
(2 * (expectedWidth >> 1) * (expectedHeight >> 1));
int coverage = 0;
using Av1Decoder decoder = new(configuration);
@ -1264,8 +1333,8 @@ public class Av1ReconstructionConformanceTests
null);
ivfOffset += payloadLength;
Assert.Equal(OfficialMotionVectorFixtureWidth, frame.Width);
Assert.Equal(OfficialMotionVectorFixtureHeight, frame.Height);
Assert.Equal(expectedWidth, frame.Width);
Assert.Equal(expectedHeight, frame.Height);
Assert.True(nativeReference.AsSpan(nativeOffset).StartsWith(y4mFrameHeader));
nativeOffset += y4mFrameHeader.Length;

2
tests/ImageSharp.Tests/TestImages.cs

@ -1398,6 +1398,8 @@ public static class TestImages
public const string Av1OfficialMotionFieldSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-06-mfmv-libaom.y4m";
public const string Av1OfficialIntraBlockCopySequence = "Heif/Av1/Conformance/libaom-av1-1-b8-16-intra_only-intrabc-extreme-dv.ivf";
public const string Av1OfficialIntraBlockCopySequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-16-intra_only-intrabc-extreme-dv-libaom.y4m";
public const string Av1OfficialTwoSpatialLayerSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1.ivf";
public const string Av1OfficialTwoSpatialLayerSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1-libaom.y4m";
public const string Av1Cdef8BitPayload = "Heif/Av1/Conformance/libaom-cdef-kodim23-8b.bit";
public const string Av1Cdef8BitReference = "Heif/Av1/Conformance/libaom-cdef-kodim23-8b-libaom.yuv";
public const string Av1Cdef8BitAvif = "Heif/Av1/Conformance/libavif-cdef-kodim23-8b.avif";

3
tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1-libaom.y4m

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:7a427631ecbf144f435aa4612f1201415fb1a9bcf9a67ba010aef830b0c3ab81
size 11059287

3
tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1.ivf

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d65824e8e7c19bc958db810d728b2132b513b5a93cb5f99ca3d7995e19a226e1
size 159037
Loading…
Cancel
Save