Browse Source

Verify official AV1 spatial-temporal layers

pull/2633/head
James Jackson-South 4 days ago
parent
commit
03737411e6
  1. 7
      HEIF_IMPLEMENTATION_PLAN.md
  2. 65
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs
  3. 2
      tests/ImageSharp.Tests/TestImages.cs
  4. 3
      tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T2-libaom.y4m
  5. 3
      tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T2.ivf

7
HEIF_IMPLEMENTATION_PLAN.md

@ -29,7 +29,7 @@ Checkboxes may be marked complete only when the implementation and the verificat
## Delivery dashboard ## Delivery dashboard
Last reconciled with the source tree on 2026-08-29 against production checkpoint `27ebcd54dc732aff8c665ee1ab61e72ed9621b42`. Committed checkpoints include the AV1 transform architecture, OBU framing, intra-block copy, official extreme-displacement intra-block-copy conformance, the 12-profile reconstruction matrix, corrected twelve-bit inverse-transform SIMD arithmetic, layered-item properties, layered reference/header/CDF/motion-field state, inter-frame intra blocks, SIMD-first translational prediction, complete single-reference inter reconstruction, compound reference trees and modes, paired reference-MV derivation, reference-dependent bounded sequence decoding, allocation-free SIMD-first equal averaging, selected inter-intra prediction, selectable compound blending, OBMC, scaled-reference reconstruction, local warped prediction, non-translational global prediction, official motion-vector conformance, official two-spatial-layer default-operating-point conformance, selected spatial-layer presentation, and progressive color and auxiliary-alpha conformance. This dashboard is the authoritative delivery order. The detailed phase checklists below provide subsystem evidence; they do not override the current-stage marker or permit work to skip ahead. Last reconciled with the source tree on 2026-08-29 against production checkpoint `a5e4cc6cc6cf87ee02b41cfe4ca71d1ef3411ed7`. Committed checkpoints include the AV1 transform architecture, OBU framing, intra-block copy, official extreme-displacement intra-block-copy conformance, the 12-profile reconstruction matrix, corrected twelve-bit inverse-transform SIMD arithmetic, layered-item properties, layered reference/header/CDF/motion-field state, inter-frame intra blocks, SIMD-first translational prediction, complete single-reference inter reconstruction, compound reference trees and modes, paired reference-MV derivation, reference-dependent bounded sequence decoding, allocation-free SIMD-first equal averaging, selected inter-intra prediction, selectable compound blending, OBMC, scaled-reference reconstruction, local warped prediction, non-translational global prediction, official motion-vector conformance, official spatial- and temporal-layer default-operating-point conformance, selected spatial-layer presentation, and progressive color and auxiliary-alpha conformance. This dashboard is the authoritative delivery order. The detailed phase checklists below provide subsystem evidence; they do not override the current-stage marker or permit work to skip ahead.
Commit `1c58d855f70b024170ced9eb0a7005f0f9c955ad` records the complete official motion-vector conformance checkpoint. The official IVF has SHA-1 `F064290D7FCD3B3DE19020E8AEC6C43C88D3A505`, matching the pinned libaom test-data manifest, and SHA-256 `222A9050059B254DAB17CFB802FF829C778E3F93AF18961A622C8268576C1395`; its pinned-libaom Y4M has SHA-256 `D97AC78C81782CF1507549368047769DC677DBE205706458D1EE9C807DE6EC78`. Both source targets build with zero warnings and errors, the `net10.0` test-project analyzer build completes with zero errors and 1,014 pre-existing repository warnings, Roslynk reports zero compiler errors, 3,985 focused `net10.0` cases pass without failures or skips, and `git diff --check` is clean. Commit `1c58d855f70b024170ced9eb0a7005f0f9c955ad` records the complete official motion-vector conformance checkpoint. The official IVF has SHA-1 `F064290D7FCD3B3DE19020E8AEC6C43C88D3A505`, matching the pinned libaom test-data manifest, and SHA-256 `222A9050059B254DAB17CFB802FF829C778E3F93AF18961A622C8268576C1395`; its pinned-libaom Y4M has SHA-256 `D97AC78C81782CF1507549368047769DC677DBE205706458D1EE9C807DE6EC78`. Both source targets build with zero warnings and errors, the `net10.0` test-project analyzer build completes with zero errors and 1,014 pre-existing repository warnings, Roslynk reports zero compiler errors, 3,985 focused `net10.0` cases pass without failures or skips, and `git diff --check` is clean.
@ -45,6 +45,8 @@ Commit `ea2fa484716944b86fadac5eac3eef23874ac25b` records the complete official
Commit `27ebcd54dc732aff8c665ee1ab61e72ed9621b42` records the official two-spatial-layer L2T1 default-operating-point checkpoint. The manifest-matching 159,037-byte IVF and pinned-libaom Y4M prove all eight 1280x720 YUV420 frames exactly under normal and scalar dispatch; constrained tracked allocation verifies balanced exactly-once returns. The existing decoder required no production correction. Both Release source targets build with zero warnings and errors, the scoped test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, all five focused compact-sequence cases pass without failures or skips, and `git diff --check` is clean. Commit `27ebcd54dc732aff8c665ee1ab61e72ed9621b42` records the official two-spatial-layer L2T1 default-operating-point checkpoint. The manifest-matching 159,037-byte IVF and pinned-libaom Y4M prove all eight 1280x720 YUV420 frames exactly under normal and scalar dispatch; constrained tracked allocation verifies balanced exactly-once returns. The existing decoder required no production correction. Both Release source targets build with zero warnings and errors, the scoped test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, all five focused compact-sequence cases pass without failures or skips, and `git diff --check` is clean.
Commit `a5e4cc6cc6cf87ee02b41cfe4ca71d1ef3411ed7` records the official one-spatial-layer, two-temporal-layer L1T2 checkpoint. The manifest-matching 48,883-byte IVF and pinned-libaom Y4M prove all eight 640x360 YUV420 frames exactly under normal and scalar dispatch; constrained tracked allocation verifies balanced exactly-once returns. The existing decoder required no production correction. The scoped test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, both focused cases pass without failures or skips, and `git diff --check` is clean.
Status meanings: Status meanings:
- **Complete:** the implementation and its phase exit evidence are recorded. - **Complete:** the implementation and its phase exit evidence are recorded.
@ -55,7 +57,7 @@ Status meanings:
Current development stage: **Stage 3 — complete AV1 still-image decoding.** The decoder retains reference/header/CDF/motion-field state, derives frame-level skip-mode references, consumes temporal segment prediction, decodes intra-coded blocks inside inter frames, and reconstructs translational single-reference, compound, inter-intra, OBMC, scaled-reference, local warped, and non-translational global prediction before residual traversal. Commit `1c58d855f70b024170ced9eb0a7005f0f9c955ad` adds exact official four-frame coverage through all ordinary inter modes, all three motion modes, every regular/smooth/sharp dual-filter pair, sub-8x8 chroma prediction, and no-round compound intermediates. Commit `019ac5648b4380a6ff4865072c96fde29ce09fad` adds exact selected-layer native reconstruction and scaled presentation. Neither AV1 nor HEVC production encoding is implemented. Current development stage: **Stage 3 — complete AV1 still-image decoding.** The decoder retains reference/header/CDF/motion-field state, derives frame-level skip-mode references, consumes temporal segment prediction, decodes intra-coded blocks inside inter frames, and reconstructs translational single-reference, compound, inter-intra, OBMC, scaled-reference, local warped, and non-translational global prediction before residual traversal. Commit `1c58d855f70b024170ced9eb0a7005f0f9c955ad` adds exact official four-frame coverage through all ordinary inter modes, all three motion modes, every regular/smooth/sharp dual-filter pair, sub-8x8 chroma prediction, and no-round compound intermediates. Commit `019ac5648b4380a6ff4865072c96fde29ce09fad` adds exact selected-layer native reconstruction and scaled presentation. Neither AV1 nor HEVC production encoding is implemented.
Immediate checkpoint: **verification complete; commit pending — official one-spatial-layer, two-temporal-layer L1T2 AV1 sequence.** All eight 640x360 YUV420 frames match pinned libaom exactly under normal/scalar dispatch, and constrained tracked allocation returns every allocation exactly once. The existing decoder required no production correction. The scoped test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, both focused cases pass without failures or skips, and `git diff --check` is clean. Only the checkpoint commit remains open. Immediate checkpoint: **verification complete; commit pending — official two-spatial-layer, two-temporal-layer L2T2 AV1 sequence.** All eight 1280x720 YUV420 frames match pinned libaom exactly under normal/scalar dispatch, and constrained tracked allocation returns every allocation exactly once. The existing decoder required no production correction. The scoped test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, both focused cases pass without failures or skips, and `git diff --check` is clean. Only the checkpoint commit remains open.
| Order | Delivery stage | State | Delivered state | Gate that remains open | | Order | Delivery stage | State | Delivered state | Gate that remains open |
| --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- |
@ -607,6 +609,7 @@ Implement and verify in dependency order:
- [x] Return an explicitly selected lower spatial layer at the item's presentation extent without rescaling retained references. The essential-`lsel` derivative selects the exact pinned-libaom 40x40 YUV444 base, scales native planes to 80x80 before color conversion, and matches pinned-libavif output. The allocator-owned presentation buffer uses group-safe rows, portable `Vector128` two-times filtering with scalar fallback, exact 8/12-bit libyuv-shaped vectors, constrained-allocation lifetime tracking, and leaves committed final-layer and display-only film-grain results unchanged. - [x] Return an explicitly selected lower spatial layer at the item's presentation extent without rescaling retained references. The essential-`lsel` derivative selects the exact pinned-libaom 40x40 YUV444 base, scales native planes to 80x80 before color conversion, and matches pinned-libavif output. The allocator-owned presentation buffer uses group-safe rows, portable `Vector128` two-times filtering with scalar fallback, exact 8/12-bit libyuv-shaped vectors, constrained-allocation lifetime tracking, and leaves committed final-layer and display-only film-grain results unchanged.
- [x] Decode the official two-spatial-layer L2T1 sequence at its default operating point. The 159,037-byte IVF has manifest-matching SHA-1 `E14825F50FF845B8A6932C64CB254007A0B5E3A1` and SHA-256 `D65824E8E7C19BC958DB810D728B2132B513B5A93CB5F99CA3D7995E19A226E1`; its pinned-libaom Y4M has SHA-256 `7A427631ECBF144F435AA4612F1201415FB1A9BCF9A67BA010AEF830B0C3AB81`. All eight 1280x720 YUV420 frames match exactly under normal and scalar dispatch, and an 8 KiB constrained tracked allocator verifies balanced exactly-once returns. The existing decoder required no production correction. Both Release source targets build with zero warnings and errors, the scoped test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and all five focused compact-sequence cases pass without failures or skips. - [x] Decode the official two-spatial-layer L2T1 sequence at its default operating point. The 159,037-byte IVF has manifest-matching SHA-1 `E14825F50FF845B8A6932C64CB254007A0B5E3A1` and SHA-256 `D65824E8E7C19BC958DB810D728B2132B513B5A93CB5F99CA3D7995E19A226E1`; its pinned-libaom Y4M has SHA-256 `7A427631ECBF144F435AA4612F1201415FB1A9BCF9A67BA010AEF830B0C3AB81`. All eight 1280x720 YUV420 frames match exactly under normal and scalar dispatch, and an 8 KiB constrained tracked allocator verifies balanced exactly-once returns. The existing decoder required no production correction. Both Release source targets build with zero warnings and errors, the scoped test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and all five focused compact-sequence cases pass without failures or skips.
- [x] Decode the official one-spatial-layer, two-temporal-layer L1T2 sequence at its default operating point. The 48,883-byte IVF has manifest-matching SHA-1 `E94687EB0E90179B3800B6D5E11EB7E9BFB34EEC` and SHA-256 `EADE696AB60415D476E1D2A489EC2CCCD24D5EAAADFA4559C6488B7932274C5E`; its pinned-libaom Y4M has SHA-256 `4012DE2D4AFD095E7BB68EAE18B50B0674781BB4971CECABC0E5471E63373ED3`. All eight 640x360 YUV420 frames match exactly under normal/scalar dispatch, and an 8 KiB constrained tracked allocator verifies balanced exactly-once returns. The existing decoder required no production correction. The scoped test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and both focused cases pass without failures or skips. - [x] Decode the official one-spatial-layer, two-temporal-layer L1T2 sequence at its default operating point. The 48,883-byte IVF has manifest-matching SHA-1 `E94687EB0E90179B3800B6D5E11EB7E9BFB34EEC` and SHA-256 `EADE696AB60415D476E1D2A489EC2CCCD24D5EAAADFA4559C6488B7932274C5E`; its pinned-libaom Y4M has SHA-256 `4012DE2D4AFD095E7BB68EAE18B50B0674781BB4971CECABC0E5471E63373ED3`. All eight 640x360 YUV420 frames match exactly under normal/scalar dispatch, and an 8 KiB constrained tracked allocator verifies balanced exactly-once returns. The existing decoder required no production correction. The scoped test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and both focused cases pass without failures or skips.
- [x] Decode the official two-spatial-layer, two-temporal-layer L2T2 sequence at its default operating point. The 119,907-byte IVF has manifest-matching SHA-1 `32EF2F14EE9CB11A24A22934F4C065E926E5D236` and SHA-256 `14257D5FD3901581DBEB88133378CD79D50EA32506ADB729909AB56B4A52AF3C`; its pinned-libaom Y4M has SHA-256 `1ABB981CFF76BA9557DA437B258D8A95FCA755DED8E3949D857E8388AB1D6AE3`. All eight 1280x720 YUV420 frames match exactly under normal/scalar dispatch, and an 8 KiB constrained tracked allocator verifies balanced exactly-once returns. The existing decoder required no production correction. The scoped test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and both focused cases pass without failures or skips.
- [ ] Lossless and high-bit-depth reconstruction with correct clipping and intermediate precision. - [ ] Lossless and high-bit-depth reconstruction with correct clipping and intermediate precision.
- [x] Route lossless 4x4 blocks through allocation-free reversible inverse Walsh-Hadamard reconstruction for 8/10/12-bit samples, including the DC-only specialization, `Vector128` production traversal, scalar fallback, exact clipping, and `FeatureTestRunner` parity. - [x] Route lossless 4x4 blocks through allocation-free reversible inverse Walsh-Hadamard reconstruction for 8/10/12-bit samples, including the DC-only specialization, `Vector128` production traversal, scalar fallback, exact clipping, and `FeatureTestRunner` parity.
- [x] Verify lossless syntax, inverse quantization, prediction, and presented reconstruction with independently encoded 8/10/12-bit AVIF fixtures. The tests require coded residuals with palette and intra-block copy disabled, compare every native YUV sample with the pinned generic libaom-backed decoder, and compare every presented RGBA byte with pinned generic libavif exactly under normal hardware dispatch and the scalar fallback. - [x] Verify lossless syntax, inverse quantization, prediction, and presented reconstruction with independently encoded 8/10/12-bit AVIF fixtures. The tests require coded residuals with palette and intra-block copy disabled, compare every native YUV sample with the pinned generic libaom-backed decoder, and compare every presented RGBA byte with pinned generic libavif exactly under normal hardware dispatch and the scalar fallback.

65
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs

@ -216,6 +216,21 @@ public class Av1ReconstructionConformanceTests
/// </summary> /// </summary>
private const int OfficialTwoTemporalLayerFixtureFrameCount = 8; private const int OfficialTwoTemporalLayerFixtureFrameCount = 8;
/// <summary>
/// The displayed width of the official libaom spatial-and-temporal-layer sequence.
/// </summary>
private const int OfficialSpatialTemporalLayerFixtureWidth = 1280;
/// <summary>
/// The displayed height of the official libaom spatial-and-temporal-layer sequence.
/// </summary>
private const int OfficialSpatialTemporalLayerFixtureHeight = 720;
/// <summary>
/// The number of default-operating-point frames in the official libaom spatial-and-temporal-layer sequence.
/// </summary>
private const int OfficialSpatialTemporalLayerFixtureFrameCount = 8;
/// <summary> /// <summary>
/// The coverage bit representing tile-local adaptive CDF updates. /// The coverage bit representing tile-local adaptive CDF updates.
/// </summary> /// </summary>
@ -1349,6 +1364,56 @@ public class Av1ReconstructionConformanceTests
OfficialTwoTemporalLayerFixtureWidth, OfficialTwoTemporalLayerFixtureWidth,
OfficialTwoTemporalLayerFixtureHeight); OfficialTwoTemporalLayerFixtureHeight);
/// <summary>
/// Verifies the default operating point of an official spatial-and-temporal-layer sequence against exact
/// pinned-libaom native output under normal and scalar dispatch.
/// </summary>
[Fact]
public void DecodeOfficialSpatialTemporalLayerSequenceMatchesPinnedLibaomReference()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(
ValidateOfficialSpatialTemporalLayerFixture,
ReconstructionConfigurations);
/// <summary>
/// Verifies the official spatial-and-temporal-layer sequence through constrained tracked allocation.
/// </summary>
[Fact]
[ValidateDisposedMemoryAllocations]
public void DecodeOfficialSpatialTemporalLayerSequenceWithConstrainedAllocator()
{
TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 8_192 };
allocator.EnableNonThreadSafeLogging();
Configuration configuration = Configuration.Default.Clone();
configuration.MemoryAllocator = allocator;
ValidateOfficialCompactSequence(
configuration,
TestImages.Heif.Av1OfficialSpatialTemporalLayerSequence,
TestImages.Heif.Av1OfficialSpatialTemporalLayerSequenceNativeReference,
OfficialSpatialTemporalLayerFixtureFrameCount,
OfficialSpatialTemporalLayerFixtureWidth,
OfficialSpatialTemporalLayerFixtureHeight);
Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count);
Assert.All(
allocator.AllocationLog,
allocation => Assert.Single(
allocator.ReturnLog,
returned => returned.AllocationId == allocation.AllocationId));
}
/// <summary>
/// Decodes the default operating point of the official spatial-and-temporal-layer sequence.
/// </summary>
private static void ValidateOfficialSpatialTemporalLayerFixture()
=> ValidateOfficialCompactSequence(
Configuration.Default,
TestImages.Heif.Av1OfficialSpatialTemporalLayerSequence,
TestImages.Heif.Av1OfficialSpatialTemporalLayerSequenceNativeReference,
OfficialSpatialTemporalLayerFixtureFrameCount,
OfficialSpatialTemporalLayerFixtureWidth,
OfficialSpatialTemporalLayerFixtureHeight);
/// <summary> /// <summary>
/// Decodes one compact official IVF sequence, compares every native sample, and returns its active frame-state /// Decodes one compact official IVF sequence, compares every native sample, and returns its active frame-state
/// coverage mask. /// coverage mask.

2
tests/ImageSharp.Tests/TestImages.cs

@ -1402,6 +1402,8 @@ public static class TestImages
public const string Av1OfficialTwoSpatialLayerSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1-libaom.y4m"; public const string Av1OfficialTwoSpatialLayerSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T1-libaom.y4m";
public const string Av1OfficialTwoTemporalLayerSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L1T2.ivf"; public const string Av1OfficialTwoTemporalLayerSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L1T2.ivf";
public const string Av1OfficialTwoTemporalLayerSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L1T2-libaom.y4m"; public const string Av1OfficialTwoTemporalLayerSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L1T2-libaom.y4m";
public const string Av1OfficialSpatialTemporalLayerSequence = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T2.ivf";
public const string Av1OfficialSpatialTemporalLayerSequenceNativeReference = "Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T2-libaom.y4m";
public const string Av1Cdef8BitPayload = "Heif/Av1/Conformance/libaom-cdef-kodim23-8b.bit"; public const string Av1Cdef8BitPayload = "Heif/Av1/Conformance/libaom-cdef-kodim23-8b.bit";
public const string Av1Cdef8BitReference = "Heif/Av1/Conformance/libaom-cdef-kodim23-8b-libaom.yuv"; public const string Av1Cdef8BitReference = "Heif/Av1/Conformance/libaom-cdef-kodim23-8b-libaom.yuv";
public const string Av1Cdef8BitAvif = "Heif/Av1/Conformance/libavif-cdef-kodim23-8b.avif"; public const string Av1Cdef8BitAvif = "Heif/Av1/Conformance/libavif-cdef-kodim23-8b.avif";

3
tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T2-libaom.y4m

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:1abb981cff76ba9557da437b258d8a95fca755ded8e3949d857e8388ab1d6ae3
size 11059287

3
tests/Images/Input/Heif/Av1/Conformance/libaom-av1-1-b8-22-svc-L2T2.ivf

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:14257d5fd3901581dbeb88133378cd79d50ea32506adb729909ab56b4a52af3c
size 119907
Loading…
Cancel
Save