Browse Source

Add independent AV1 CDEF reconstruction vectors

pull/2633/head
James Jackson-South 2 months ago
parent
commit
22026c5420
  1. 5
      HEIF_IMPLEMENTATION_PLAN.md
  2. 109
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs
  3. 6
      tests/ImageSharp.Tests/TestImages.cs
  4. 11
      tests/Images/Input/Heif/Av1/Conformance/README.md
  5. 3
      tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-10b-libaom.yuv
  6. 3
      tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-10b.bit
  7. 3
      tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-12b-libaom.yuv
  8. 3
      tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-12b.bit
  9. 3
      tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-kodim23-8b-libaom.yuv
  10. 3
      tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-kodim23-8b.bit

5
HEIF_IMPLEMENTATION_PLAN.md

@ -352,7 +352,7 @@ This assessment was reconciled with the source tree on 2026-08-26. Unless a resu
- The single-still `Av1Decoder` path now parses tile state before allocating and reconstructing one independently decodable frame, and it disposes the reconstruction planes after pixel conversion. It deliberately does not retain animation/video reference frames or implement `show_existing_frame` playback state.
- Transform coefficient entropy derivation and updates now address the above contexts relative to the tile column and the left contexts relative to the current superblock row, preserve luma coordinates independently of chroma subsampling, and test every packed context entry for the libaom any-nonzero rule. Extended vertical partition updates advance the mode-information column rather than the row. The existing multi-superblock 4:4:4 AVIF fixture now completes tile parsing; independent coefficient-context vectors across tile boundaries, chroma layouts, bit depths, and edge-clipped transforms remain required.
- The reconstruction pipeline now records plane-relative transform geometry, preserves tile-local delta-Q and delta-LF predictors, derives segmentation and reference-adjusted filter levels, and runs the exact AV1 4-, 6-, 8-, and 14-tap deblocking kernels in normative vertical-then-horizontal order. Deblocking uses the same closed edge-operator architecture as the HEVC filter, with operators specialized by sample storage and orientation, `Vector128<int>` lanes representing the four rows or columns along an edge, and an allocation-free scalar fallback for disabled intrinsics. Exact native-plane comparison with pinned scalar libaom output now verifies active deblocking and complete reconstruction for real 8-bit 4:2:0, 10-bit 4:4:4, and 12-bit 4:4:4 content; genuine AVIF containers separately verify presentation and public bit-depth metadata. The pipeline then applies CDEF through one semantic filter architecture: paired AVX2 and single-block `Vector128` direction analysis, closed primary/secondary strength operators, packed 4x4/4x8/8x4/8x8 constrained filtering, byte/16-bit output operators, and an exact allocation-free scalar fallback. Decoder orchestration now owns the immutable plane snapshots and clean direction/variance maps through ImageSharp's memory allocator, widens 8-bit source rows with the same AVX2/128-bit/scalar tiers as libaom, lists each unit's non-skipped blocks in fixed inline storage, analyzes listed blocks in pairs, and writes filtered bytes or 16-bit samples directly to the frame planes. Active super-resolution derives the Appendix A bounded coded width and applies the exact 64-phase, 8-tap horizontal filter with aligned reconstruction-edge input, 8/10/12-bit clipping, and the existing cross-platform `Vector128_.MultiplyAddAdjacent` helper. Loop restoration follows super-resolution, preserves the required pre-CDEF deblocked context at internal stripes, and applies decoded Wiener or self-guided units from immutable plane snapshots. The visible still-image path then applies the complete self-contained film-grain parameter set after all in-loop filters. Independent 8-, 10-, and 12-bit vectors exercising CDEF, super-resolution, restoration, and grain remain required before those stages have external pixel-level verification.
- The reconstruction pipeline now records plane-relative transform geometry, preserves tile-local delta-Q and delta-LF predictors, derives segmentation and reference-adjusted filter levels, and runs the exact AV1 4-, 6-, 8-, and 14-tap deblocking kernels in normative vertical-then-horizontal order. Deblocking uses the same closed edge-operator architecture as the HEVC filter, with operators specialized by sample storage and orientation, `Vector128<int>` lanes representing the four rows or columns along an edge, and an allocation-free scalar fallback for disabled intrinsics. Exact native-plane comparison with pinned scalar libaom output now verifies active deblocking and complete reconstruction for real 8-bit 4:2:0, 10-bit 4:4:4, and 12-bit 4:4:4 content; genuine AVIF containers separately verify presentation and public bit-depth metadata. The pipeline then applies CDEF through one semantic filter architecture: paired AVX2 and single-block `Vector128` direction analysis, closed primary/secondary strength operators, packed 4x4/4x8/8x4/8x8 constrained filtering, byte/16-bit output operators, and an exact allocation-free scalar fallback. Decoder orchestration now owns the immutable plane snapshots and clean direction/variance maps through ImageSharp's memory allocator, widens 8-bit source rows with the same AVX2/128-bit/scalar tiers as libaom, lists each unit's non-skipped blocks in fixed inline storage, analyzes listed blocks in pairs, and writes filtered bytes or 16-bit samples directly to the frame planes. Independently encoded active-CDEF 8-bit 4:2:0 and 10/12-bit 4:4:4 streams now match every visible native sample produced by pinned scalar libaom with restoration disabled; matching active-CDEF AVIF presentation fixtures remain required. Active super-resolution derives the Appendix A bounded coded width and applies the exact 64-phase, 8-tap horizontal filter with aligned reconstruction-edge input, 8/10/12-bit clipping, and the existing cross-platform `Vector128_.MultiplyAddAdjacent` helper. Loop restoration follows super-resolution, preserves the required pre-CDEF deblocked context at internal stripes, and applies decoded Wiener or self-guided units from immutable plane snapshots. The visible still-image path then applies the complete self-contained film-grain parameter set after all in-loop filters. Independent 8-, 10-, and 12-bit vectors exercising super-resolution, restoration, and grain remain required before those stages have external pixel-level verification.
- Palette mode now reads the normative luma/chroma mode and size CDFs, neighbor color caches, high-bit-depth color syntax, diagonal color-index maps, and clipped-edge padding. Reconstruction retains libaom's exact indexed-palette contract while using repeated eight-entry tables and native lane-local shuffles across runtime-selected `Vector512`, `Vector256`, and `Vector128` paths before an exact scalar fallback. `FeatureTestRunner` verifies palette sizes 2-8, transform widths 4-64, padded rows, and 8/10/12-bit samples. This path is not production-complete until its syntax and reconstruction are verified with an independently encoded palette AVIF fixture.
- Non-reduced still-image sequence parsing now consumes decoder-model operating parameters, temporal presentation fields, and OBU-layer-applicable buffer-removal fields only to preserve AV1 bit alignment. The scheduling values are not retained, and no movie, track, timing, playback, or generic ISO BMFF surface has been introduced. Existing focused sequence-header coverage exercises only reduced-still syntax, so an independent non-reduced still AVIF vector remains required.
- Loop-restoration unit parsing records tile-local switchable/Wiener/self-guided filter selections and coefficients in frame-owned plane grids, including super-resolution-adjusted unit corners and the corrected conditional 64x64-superblock unit-size bit. The active restoration stage implements the normative unit geometry, striped deblocked boundaries, Wiener filtering, self-guided projection, and 8/10/12-bit clipping, while reusing the existing SIMD-backed super-resolution and adjacent multiply/add primitives. Independently encoded fixtures covering every parameter set, plane layout, bit depth, and frame-edge geometry are still required. Other normative independently decodable still-image syntax paths still contain `NotImplementedException` or equivalent unsupported branches. Tile-local palette CDF adaptation is present; the remaining still-image frame-context behavior requires a separate source audit without introducing sequence playback state.
@ -508,7 +508,8 @@ Implement and verify in dependency order:
- [ ] CDEF.
- [x] Replace the scalar helper bucket with an allocation-free SIMD-first `Av1CdefFilter`: vector direction analysis, 4x4/4x8/8x4/8x8 constrained filtering, byte/16-bit output operators, and one exact scalar fallback verified through `FeatureTestRunner`.
- [x] Move the immutable deblocked-plane snapshot and frame-wide direction/variance maps to allocator-owned decoder memory, preserving luma-first paired direction derivation, fixed-storage skipped-block selection, asymmetric chroma direction conversion, frame-edge sentinels, SIMD source widening, and direct frame-plane output.
- [ ] Verify CDEF syntax, strength derivation, unit/block traversal, subsampling, frame edges, and presented reconstruction with independently encoded active-CDEF AV1/AVIF fixtures across 8/10/12-bit samples.
- [x] Verify CDEF syntax, selected nonzero strengths, unit/block traversal, subsampling, frame edges, and exact native reconstruction with independently encoded active-CDEF AV1 streams across 8/10/12-bit samples and pinned scalar-libaom output.
- [ ] Verify presented reconstruction with independently encoded active-CDEF AVIF containers across 8/10/12-bit samples.
- [ ] Super-resolution scaling.
- [ ] Loop restoration.
- [ ] Frame padding and film-grain synthesis when signaled.

109
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1DeblockingConformanceTests.cs → tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs

@ -8,6 +8,7 @@ using SixLabors.ImageSharp.Formats.Heif.Av1;
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Memory;
using SixLabors.ImageSharp.PixelFormats;
using SixLabors.ImageSharp.Tests.TestUtilities;
namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1;
@ -15,8 +16,13 @@ namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1;
/// Validates complete AV1 reconstruction against independently decoded native component planes.
/// </summary>
[Trait("Format", "Avif")]
public class Av1DeblockingConformanceTests
public class Av1ReconstructionConformanceTests
{
/// <summary>
/// The hardware configurations covering normal SIMD dispatch and the scalar fallback.
/// </summary>
private const HwIntrinsics ReconstructionConfigurations = HwIntrinsics.AllowAll | HwIntrinsics.DisableHWIntrinsic;
/// <summary>
/// Verifies deblocking syntax, filter activation, component traversal, and presentation for real eight-, ten-,
/// and twelve-bit AV1 and AVIF content.
@ -50,11 +56,51 @@ public class Av1DeblockingConformanceTests
1024,
428,
Av1BitDepth.TwelveBit,
Av1ColorFormat.Yuv444);
Av1ColorFormat.Yuv444,
requireActiveCdef: false);
ValidatePresentedImage(TestImages.Heif.Av1Deblocking12BitAvif, 64, 64, HeifBitDepth.Bit12);
}
/// <summary>
/// Verifies active CDEF syntax, strength selection, unit traversal, subsampling, frame edges, and final native
/// samples against scalar libaom for independently encoded eight-, ten-, and twelve-bit still-picture streams
/// under normal SIMD dispatch and with hardware intrinsics disabled.
/// </summary>
[Fact]
public void DecodeWithActiveCdefMatchesPinnedLibaomReference()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateActiveCdefFixtures, ReconstructionConfigurations);
/// <summary>
/// Validates every active-CDEF fixture under the hardware configuration selected by <see cref="FeatureTestRunner"/>.
/// </summary>
private static void ValidateActiveCdefFixtures()
{
ValidateActiveCdefFixture(
TestImages.Heif.Av1Cdef8BitPayload,
TestImages.Heif.Av1Cdef8BitReference,
768,
512,
Av1BitDepth.EightBit,
Av1ColorFormat.Yuv420);
ValidateActiveCdefFixture(
TestImages.Heif.Av1Cdef10BitPayload,
TestImages.Heif.Av1Cdef10BitReference,
1024,
428,
Av1BitDepth.TenBit,
Av1ColorFormat.Yuv444);
ValidateActiveCdefFixture(
TestImages.Heif.Av1Cdef12BitPayload,
TestImages.Heif.Av1Cdef12BitReference,
1024,
428,
Av1BitDepth.TwelveBit,
Av1ColorFormat.Yuv444);
}
/// <summary>
/// Validates one elementary-stream sample and its containing AVIF image.
/// </summary>
@ -76,7 +122,7 @@ public class Av1DeblockingConformanceTests
Av1ColorFormat colorFormat,
HeifBitDepth metadataBitDepth)
{
ValidateNativeFixture(payloadPath, referencePath, width, height, bitDepth, colorFormat);
ValidateNativeFixture(payloadPath, referencePath, width, height, bitDepth, colorFormat, false);
ValidatePresentedImage(imagePath, width, height, metadataBitDepth);
}
@ -89,13 +135,15 @@ public class Av1DeblockingConformanceTests
/// <param name="height">The expected reconstructed height.</param>
/// <param name="bitDepth">The expected AV1 sample precision.</param>
/// <param name="colorFormat">The expected native chroma-sampling layout.</param>
/// <param name="requireActiveCdef">Indicates whether the stream must signal and select nonzero CDEF strengths.</param>
private static void ValidateNativeFixture(
string payloadPath,
string referencePath,
int width,
int height,
Av1BitDepth bitDepth,
Av1ColorFormat colorFormat)
Av1ColorFormat colorFormat,
bool requireActiveCdef)
{
byte[] payload = TestFile.Create(payloadPath).Bytes;
byte[] reference = TestFile.Create(referencePath).Bytes;
@ -114,9 +162,62 @@ public class Av1DeblockingConformanceTests
|| filterParameters.FilterLevelU != 0
|| filterParameters.FilterLevelV != 0);
if (requireActiveCdef)
{
Assert.NotNull(decoder.SequenceHeader);
Assert.True(decoder.SequenceHeader.EnableCdef);
Assert.False(decoder.FrameHeader.LoopRestorationParameters.UsesLoopRestoration);
Assert.NotNull(decoder.FrameInfo);
ObuConstraintDirectionalEnhancementFilterParameters parameters = decoder.FrameHeader.CdefParameters;
bool hasActiveStrength = false;
int superblockSizeLog2 = decoder.SequenceHeader.SuperblockSizeLog2;
int superblockColumnCount = Av1Math.AlignPowerOf2(decoder.SequenceHeader.MaxFrameWidth, superblockSizeLog2) >> superblockSizeLog2;
int superblockRowCount = Av1Math.AlignPowerOf2(decoder.SequenceHeader.MaxFrameHeight, superblockSizeLog2) >> superblockSizeLog2;
for (int superblockRow = 0; superblockRow < superblockRowCount && !hasActiveStrength; superblockRow++)
{
for (int superblockColumn = 0; superblockColumn < superblockColumnCount && !hasActiveStrength; superblockColumn++)
{
Span<int> selectedStrengths = decoder.FrameInfo.GetCdefStrength(new Point(superblockColumn, superblockRow));
// Unassigned entries belong to completely skipped units. Every assigned index must resolve through
// the signaled table before the exact output can establish that CDEF changed reconstructed samples.
foreach (int selectedStrength in selectedStrengths)
{
if (selectedStrength >= 0
&& (parameters.YStrength[selectedStrength] != 0 || parameters.UvStrength[selectedStrength] != 0))
{
hasActiveStrength = true;
break;
}
}
}
}
// The independent output only proves CDEF when the encoded frame selects at least one nonzero strength.
Assert.True(hasActiveStrength);
}
AssertNativePlanesEqual(frameBuffer, reference);
}
/// <summary>
/// Validates one independently encoded stream that activates constrained directional enhancement filtering.
/// </summary>
/// <param name="payloadPath">The AV1 elementary-stream sample.</param>
/// <param name="referencePath">The native planar output produced by the pinned scalar libaom decoder.</param>
/// <param name="width">The expected reconstructed width.</param>
/// <param name="height">The expected reconstructed height.</param>
/// <param name="bitDepth">The expected AV1 sample precision.</param>
/// <param name="colorFormat">The expected native chroma-sampling layout.</param>
private static void ValidateActiveCdefFixture(
string payloadPath,
string referencePath,
int width,
int height,
Av1BitDepth bitDepth,
Av1ColorFormat colorFormat)
=> ValidateNativeFixture(payloadPath, referencePath, width, height, bitDepth, colorFormat, requireActiveCdef: true);
/// <summary>
/// Validates the public presentation and metadata produced from one complete AVIF container.
/// </summary>

6
tests/ImageSharp.Tests/TestImages.cs

@ -1353,6 +1353,12 @@ public static class TestImages
public const string Av1Deblocking12BitAvif = "Heif/Av1/Conformance/libavif-colors-12b.avif";
public const string Av1Deblocking12BitPayload = "Heif/Av1/Conformance/libaom-cosmos1650-12b.bit";
public const string Av1Deblocking12BitReference = "Heif/Av1/Conformance/libaom-cosmos1650-12b-libaom.yuv";
public const string Av1Cdef8BitPayload = "Heif/Av1/Conformance/libaom-cdef-kodim23-8b.bit";
public const string Av1Cdef8BitReference = "Heif/Av1/Conformance/libaom-cdef-kodim23-8b-libaom.yuv";
public const string Av1Cdef10BitPayload = "Heif/Av1/Conformance/libaom-cdef-cosmos-10b.bit";
public const string Av1Cdef10BitReference = "Heif/Av1/Conformance/libaom-cdef-cosmos-10b-libaom.yuv";
public const string Av1Cdef12BitPayload = "Heif/Av1/Conformance/libaom-cdef-cosmos-12b.bit";
public const string Av1Cdef12BitReference = "Heif/Av1/Conformance/libaom-cdef-cosmos-12b-libaom.yuv";
}
public static class Ico

11
tests/Images/Input/Heif/Av1/Conformance/README.md

@ -1,4 +1,4 @@
# AV1 deblocking conformance fixtures
# AV1 reconstruction conformance fixtures
The AVIF files come from `libavif/tests/data` at commit `062e582e8afda88e6baf988fdcf046a801efa0f5`. They retain the licenses recorded in libavif's `tests/data/README.md`: the Kodak image is released for unrestricted use, the Cosmos Laundromat frame uses CC BY 3.0, and the libavif color animation is distributed with the libavif test corpus under its BSD-2-Clause license.
@ -8,10 +8,17 @@ The 8- and 10-bit `.bit` files contain the exact AV1 item payloads from the corr
The `_libaom.yuv` files were decoded from those exact payloads with `aomdec` built from libaom commit `03087864cf4bea6abb0d28f95cf7843511413d8f`. The reference build used `AOM_TARGET_CPU=generic`, so these files come from libaom's scalar decoder rather than ImageSharp or an architecture-specific implementation.
The `libaom-cdef-*` elementary streams were encoded separately with the same pinned generic libaom build so CDEF could be verified independently of the original corpus. The 8-bit stream uses `kodim23_yuv420_8bpc.y4m`; the 10- and 12-bit streams use `cosmos1650_yuv444_10bpc_p3pq.y4m`. Both source files are retained in libavif's test data at commit `062e582e8afda88e6baf988fdcf046a801efa0f5`.
The material encoder options were `--usage=2 --passes=1 --limit=1 --obu --end-usage=q --cq-level=30 --cpu-used=4 --threads=1 --lag-in-frames=0 --full-still-picture-hdr --enable-cdef=1 --enable-restoration=0`. Each command also supplied the matching `--bit-depth`, `--input-bit-depth`, and `--profile` values. The 12-bit stream promotes the 10-bit 4:4:4 input through libaom's native 12-bit pipeline. Loop restoration is explicitly disabled so exact output equality exercises deblocking followed by active CDEF without a later restoration stage changing those samples.
The native reference layouts are:
- `libavif-kodim23-8b-libaom.yuv`: 768x512, 8-bit YUV 4:2:0, planar Y/U/V.
- `libavif-cosmos1650-10b-libaom.yuv`: 1024x428, 10-bit YUV 4:4:4, planar Y/U/V with little-endian 16-bit samples.
- `libaom-cosmos1650-12b-libaom.yuv`: 1024x428, 12-bit YUV 4:4:4, planar Y/U/V with little-endian 16-bit samples.
- `libaom-cdef-kodim23-8b-libaom.yuv`: 768x512, 8-bit YUV 4:2:0, planar Y/U/V.
- `libaom-cdef-cosmos-10b-libaom.yuv`: 1024x428, 10-bit YUV 4:4:4, planar Y/U/V with little-endian 16-bit samples.
- `libaom-cdef-cosmos-12b-libaom.yuv`: 1024x428, 12-bit YUV 4:4:4, planar Y/U/V with little-endian 16-bit samples.
The conformance test compares every visible reconstructed sample with these files. It also verifies that the parsed frame enables deblocking, so a disabled or bypassed loop-filter stage cannot satisfy the test accidentally.
The conformance tests compare every visible reconstructed sample with these files. The deblocking corpus verifies nonzero loop-filter levels. The CDEF corpus additionally verifies sequence-level CDEF enablement, a selected nonzero frame strength, and disabled loop restoration, so a disabled or bypassed CDEF stage cannot satisfy the exact native-plane comparison accidentally.

3
tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-10b-libaom.yuv

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a3e95419a967bb552cd6f80365490f195fdad645114004771d8bb2662ea8f446
size 2629632

3
tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-10b.bit

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:138a1b4a8e4a9d36115bcb6a0e4f21fee245e574e4f5b775d9205cce436c1727
size 26733

3
tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-12b-libaom.yuv

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e0e246a511272743c1a3a28fd58f66d3b17e741e999143d65d23f4ea773407df
size 2629632

3
tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-cosmos-12b.bit

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:862c9e01e379204e348fe5dec62caffe4b0c9275416240b899a0459fe70de95e
size 23157

3
tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-kodim23-8b-libaom.yuv

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c46b370f221dfdc2db1cffc824961e7594659c3e2634656813459d9f058c7ab1
size 589824

3
tests/Images/Input/Heif/Av1/Conformance/libaom-cdef-kodim23-8b.bit

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a1a140b806c7b2570fa7f9e2572eaecd62ac3fd76b59830eb508337e87919dd9
size 16469
Loading…
Cancel
Save