From 18b1c881271a3494489ca6f410ab140544902e2d Mon Sep 17 00:00:00 2001 From: James Jackson-South Date: Mon, 31 Aug 2026 02:16:48 +1000 Subject: [PATCH] Complete AV1 inter-intra decoding checkpoint --- HEIF_IMPLEMENTATION_PLAN.md | 35 +++++++++++++++++-- .../Av1/Av1CompoundInterPredictorTests.cs | 6 ++-- .../Av1/Av1ReconstructionConformanceTests.cs | 4 +-- ...ebp-logo-difference-weighted-compound.png} | 0 ...-webp-logo-distance-weighted-compound.png} | 0 ..._Rgba32_libavif-webp-logo-inter-intra.png} | 0 ...ba32_libavif-webp-logo-wedge-compound.png} | 0 .../Input/Heif/Av1/Conformance/README.md | 27 ++++++++------ 8 files changed, 55 insertions(+), 17 deletions(-) rename tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/{DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-difference-weighted-compound.png => DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-difference-weighted-compound.png} (100%) rename tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/{DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-distance-weighted-compound.png => DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-distance-weighted-compound.png} (100%) rename tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/{DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-inter-intra.png => DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-inter-intra.png} (100%) rename tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/{DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-wedge-compound.png => DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-wedge-compound.png} (100%) diff --git a/HEIF_IMPLEMENTATION_PLAN.md b/HEIF_IMPLEMENTATION_PLAN.md index ce3e5da4e6..36f04adedf 100644 --- a/HEIF_IMPLEMENTATION_PLAN.md +++ b/HEIF_IMPLEMENTATION_PLAN.md @@ -177,8 +177,8 @@ foreach ($aomEntry in $aomEnvironment) The single-reference syntax, buffer, reconstruction, and ownership foundation is verified by `54bb6cbe59bd113058854a3ee31448cf61f462ca`. Reverify the existing downstream implementations in this exact order, treating each as locally implemented but unverified until its current-main evidence is recorded. - [x] Compound reference selection, paired reference-MV derivation, and equal averaging. -- [~] Inter-intra prediction. Current item. -- [~] Distance-weighted compound prediction. +- [x] Inter-intra prediction. +- [~] Distance-weighted compound prediction. Current item. - [~] Wedge compound prediction. - [~] Difference-weighted compound prediction. - [~] OBMC. @@ -220,6 +220,37 @@ Verified equal-average compound checkpoint evidence on 2026-08-31: failures or skips. Scoped analyzer and whitespace verification pass for every changed C# file, Roslynk reports zero compiler errors and no diagnostics in the changed files, and `git diff --check` passes. `.gitattributes` is unchanged. +- [x] The completed checkpoint was committed as `4075a0844836e863a93cb2e2f3ca42d202c7df1b` + with author and committer `James Jackson-South `. + +Verified inter-intra checkpoint evidence on 2026-08-31: + +- [x] Audited syntax against current libaom `av1/decoder/decodemv.c` and + `av1/common/blockd.h`. ImageSharp applies the same sequence enable, skip-mode, block-size, and + single-reference gates, reads the same four-mode CDF, and reads wedge syntax only within libaom's + wedge-supported `BLOCK_8X8` through `BLOCK_32X32` range. +- [x] Audited reconstruction against `ii_weights1d`, `ii_size_scales`, + `build_smooth_interintra_mask`, and `combine_interintra` in current + `av1/common/reconinter.c`. The ImageSharp weights, plane-size scaling, smooth-mask direction, + complemented destination orientation, wedge sign, subsampling, and final 6-bit blend match. No + production change was required. +- [x] The mask tests cover all four inter-intra modes, complemented orientation, row-padding + sentinels, and the 32-wide curve. FeatureTestRunner covers byte and high-bit-depth selectable + blending under SIMD and scalar dispatch, and complete `Av1BlockDecoder.DecodeBlock` tests execute + smooth inter-intra reconstruction at 8, 10, and 12 bits. +- [x] Extracted the fixture's 5,327-byte AV1 `mdat` payload and decoded it with the refreshed current + libaom `aomdec`, using one thread with row threading disabled. All 19 frames decoded. The final + 19,200 YUV444 samples have SHA-256 + `E8B776C2751DC30CA838931A4B74535FC6E681179568A1278747A38CFF2E5BFA` and match the retained + native reference with zero differing samples. +- [x] The real production sequence requires both smooth and wedge inter-intra blocks, compares the + final native Y, Cb, and Cr planes exactly, and compares final RGBA presentation through + ImageSharp's established reference-output API. Its constrained 1,024-byte tracked-allocator run + proves motion-field allocation and exactly one return for every allocation. +- [x] The focused Release checkpoint set passes 50/50 on net10.0 and 50/50 on net11.0, with zero + failures or skips. Scoped analyzer and whitespace verification pass for both changed C# files. + Roslynk reports zero compiler errors and no diagnostics in the changed files, `git diff --check` + passes, and `.gitattributes` is unchanged. For every item: diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs index 6124b5ddcb..4989e3c946 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs @@ -60,7 +60,7 @@ public class Av1CompoundInterPredictorTests /// Verifies the four smooth inter-intra modes and their complemented destination orientation. /// [Fact] - public void SmoothInterIntraMasksMatchPinnedWeights() + public void SmoothInterIntraMasksMatchCurrentLibaomWeights() { ReadOnlySpan weights = [60, 34, 19, 11, 6, 4, 2, 1]; @@ -103,10 +103,10 @@ public class Av1CompoundInterPredictorTests } /// - /// Verifies the pinned horizontal curve at the index exercised by a 32-by-16 inter-intra block. + /// Verifies the current libaom horizontal curve at the index exercised by a 32-by-16 inter-intra block. /// [Fact] - public void HorizontalInterIntraMaskMatchesPinnedThirtyTwoWideCurve() + public void HorizontalInterIntraMaskMatchesCurrentLibaomThirtyTwoWideCurve() { const int width = 32; const int height = 16; diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs index 7adfde8ca9..ac21eedba7 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs @@ -1002,14 +1002,14 @@ public class Av1ReconstructionConformanceTests } /// - /// Verifies every selectable compound and inter-intra production branch against pinned native and presentation references. + /// Verifies every selectable compound and inter-intra production branch against retained native and presentation references. /// [Theory] [WithFile(TestImages.Heif.Av1DistanceWeightedCompoundSequenceAvif, PixelTypes.Rgba32)] [WithFile(TestImages.Heif.Av1WedgeCompoundSequenceAvif, PixelTypes.Rgba32)] [WithFile(TestImages.Heif.Av1DifferenceWeightedCompoundSequenceAvif, PixelTypes.Rgba32)] [WithFile(TestImages.Heif.Av1InterIntraSequenceAvif, PixelTypes.Rgba32)] - public void DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences( + public void DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences( TestImageProvider provider) => FeatureTestRunner.RunWithHwIntrinsicsFeature( diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-difference-weighted-compound.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-difference-weighted-compound.png similarity index 100% rename from tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-difference-weighted-compound.png rename to tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-difference-weighted-compound.png diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-distance-weighted-compound.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-distance-weighted-compound.png similarity index 100% rename from tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-distance-weighted-compound.png rename to tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-distance-weighted-compound.png diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-inter-intra.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-inter-intra.png similarity index 100% rename from tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-inter-intra.png rename to tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-inter-intra.png diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-wedge-compound.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-wedge-compound.png similarity index 100% rename from tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-wedge-compound.png rename to tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-wedge-compound.png diff --git a/tests/Images/Input/Heif/Av1/Conformance/README.md b/tests/Images/Input/Heif/Av1/Conformance/README.md index 605bc5f6ca..ddd3304152 100644 --- a/tests/Images/Input/Heif/Av1/Conformance/README.md +++ b/tests/Images/Input/Heif/Av1/Conformance/README.md @@ -118,7 +118,11 @@ only and is not used to establish AV1 reconstruction arithmetic. ## Selectable compound and inter-intra fixtures -The four selectable-compound fixtures use the same pinned `tests/data/webp_logo_animated.y4m` source and its `0872208D9C19B68B10A1647FA6849CFC4E2B21A19561ACD672E0629C70EFACA2` SHA-256. They were encoded at speed zero after disabling later inter-mode checkpoints. Each command also disables competing prediction tools that would prevent the resulting stream from isolating its named mode: +The four AVIF files are retained solely as interoperability inputs. Their original creation commands +used the same `tests/data/webp_logo_animated.y4m` source and encoded at speed zero after disabling later +inter-mode checkpoints. These commands record container-fixture provenance only; they are not codec +implementation references. Each command also disables competing prediction tools that would prevent the +resulting stream from isolating its named mode: ```text ./avifenc -j 1 -c aom -s 0 -q 80 -a enable-obmc=0 -a enable-warped-motion=0 -a enable-global-motion=0 -a enable-masked-comp=0 -a enable-interintra-comp=0 tests/data/webp_logo_animated.y4m libavif-webp-logo-distance-weighted-compound.avif @@ -127,21 +131,24 @@ The four selectable-compound fixtures use the same pinned `tests/data/webp_logo_ ./avifenc -j 1 -c aom -s 0 -q 80 -a enable-obmc=0 -a enable-warped-motion=0 -a enable-global-motion=0 -a enable-dist-wtd-comp=0 -a enable-masked-comp=0 tests/data/webp_logo_animated.y4m libavif-webp-logo-inter-intra.avif ``` -Pinned scalar libavif generated each final native and presentation reference with: - -```text -./avifdec -j 1 -c aom --index 18 .avif -libaom.y4m -./avifdec -j 1 -c aom --index 18 .avif -libavif.png -``` - -| Fixture | AVIF SHA-256 | Frame-18 Y4M SHA-256 | Frame-18 PNG SHA-256 | +| Fixture | AVIF SHA-256 | Retained frame-18 Y4M SHA-256 | Retained frame-18 PNG SHA-256 | | --- | --- | --- | --- | | `libavif-webp-logo-distance-weighted-compound` | `DA710D11C60F03EEA209E4360E2FC807B89C49AD50671F0DFB1BCF4AD5EF76DD` | `904D1B5B3E7F334CE8D44040F9A7BDCAC1F7773122FF1C5F06A5B4DD31A62A97` | `D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C` | | `libavif-webp-logo-wedge-compound` | `98640640A445055FEA3D9E2F4A78FEEF54E97F8171B472CF57D99156E1C553B2` | `904D1B5B3E7F334CE8D44040F9A7BDCAC1F7773122FF1C5F06A5B4DD31A62A97` | `D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C` | | `libavif-webp-logo-difference-weighted-compound` | `FC6459CD334762D74D9D2654640221E80C463CC01B82B29A5866C9E725ABE273` | `904D1B5B3E7F334CE8D44040F9A7BDCAC1F7773122FF1C5F06A5B4DD31A62A97` | `D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C` | | `libavif-webp-logo-inter-intra` | `71DF22E63626B5BC9001FF1E88076B90F11BB47D18089750853E66F0CBBB084B` | `502265688138641A7B12C8C4190B66C76CD4808D9AED05B06486056B39D7E9A0` | `F0DE4CCDFB6D95A400E69B69FA4C57F0BEEEEE75825722C31613385F0B3FD9FC` | -Pinned libaom block tracing confirms that these streams select distance weighting, both wedge signs, both difference-mask types, and both smooth and wedge inter-intra prediction. The production test independently requires those decoded mode states, decodes all preceding samples, compares the final native Y, U, and V planes exactly, compares the final RGBA presentation exactly, and repeats reconstruction with constrained tracked allocation. +The inter-intra fixture's 5,327-byte AV1 `mdat` payload was decoded directly with the current official +libaom `main` checkout observed at `441c439b9916474cac15d2822af47a9ad70674a8`, using one thread and +with row threading disabled. Current `aomdec` produced all 19 YUV444 frames. The final frame's 19,200 +native samples have SHA-256 `E8B776C2751DC30CA838931A4B74535FC6E681179568A1278747A38CFF2E5BFA` +and match the retained Y4M with zero differing samples. + +The production tests independently require their decoded mode states. The inter-intra input must exercise +both smooth and wedge inter-intra prediction, decode all preceding samples, compare the final native Y, +Cb, and Cr planes exactly, compare final RGBA presentation through ImageSharp's established +reference-output API, and repeat reconstruction with constrained tracked allocation. The retained PNG is +presentation evidence only and is not an AV1 reconstruction reference. ## Overlapping motion-compensation fixture