Browse Source

Complete AV1 inter-intra decoding checkpoint

pull/2633/head
James Jackson-South 2 months ago
parent
commit
18b1c88127
  1. 35
      HEIF_IMPLEMENTATION_PLAN.md
  2. 6
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs
  3. 4
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs
  4. 0
      tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-difference-weighted-compound.png
  5. 0
      tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-distance-weighted-compound.png
  6. 0
      tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-inter-intra.png
  7. 0
      tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-wedge-compound.png
  8. 27
      tests/Images/Input/Heif/Av1/Conformance/README.md

35
HEIF_IMPLEMENTATION_PLAN.md

@ -177,8 +177,8 @@ foreach ($aomEntry in $aomEnvironment)
The single-reference syntax, buffer, reconstruction, and ownership foundation is verified by `54bb6cbe59bd113058854a3ee31448cf61f462ca`. Reverify the existing downstream implementations in this exact order, treating each as locally implemented but unverified until its current-main evidence is recorded. The single-reference syntax, buffer, reconstruction, and ownership foundation is verified by `54bb6cbe59bd113058854a3ee31448cf61f462ca`. Reverify the existing downstream implementations in this exact order, treating each as locally implemented but unverified until its current-main evidence is recorded.
- [x] Compound reference selection, paired reference-MV derivation, and equal averaging. - [x] Compound reference selection, paired reference-MV derivation, and equal averaging.
- [~] Inter-intra prediction. Current item. - [x] Inter-intra prediction.
- [~] Distance-weighted compound prediction. - [~] Distance-weighted compound prediction. Current item.
- [~] Wedge compound prediction. - [~] Wedge compound prediction.
- [~] Difference-weighted compound prediction. - [~] Difference-weighted compound prediction.
- [~] OBMC. - [~] OBMC.
@ -220,6 +220,37 @@ Verified equal-average compound checkpoint evidence on 2026-08-31:
failures or skips. Scoped analyzer and whitespace verification pass for every changed C# file, failures or skips. Scoped analyzer and whitespace verification pass for every changed C# file,
Roslynk reports zero compiler errors and no diagnostics in the changed files, and `git diff --check` Roslynk reports zero compiler errors and no diagnostics in the changed files, and `git diff --check`
passes. `.gitattributes` is unchanged. passes. `.gitattributes` is unchanged.
- [x] The completed checkpoint was committed as `4075a0844836e863a93cb2e2f3ca42d202c7df1b`
with author and committer `James Jackson-South <james_south@hotmail.com>`.
Verified inter-intra checkpoint evidence on 2026-08-31:
- [x] Audited syntax against current libaom `av1/decoder/decodemv.c` and
`av1/common/blockd.h`. ImageSharp applies the same sequence enable, skip-mode, block-size, and
single-reference gates, reads the same four-mode CDF, and reads wedge syntax only within libaom's
wedge-supported `BLOCK_8X8` through `BLOCK_32X32` range.
- [x] Audited reconstruction against `ii_weights1d`, `ii_size_scales`,
`build_smooth_interintra_mask`, and `combine_interintra` in current
`av1/common/reconinter.c`. The ImageSharp weights, plane-size scaling, smooth-mask direction,
complemented destination orientation, wedge sign, subsampling, and final 6-bit blend match. No
production change was required.
- [x] The mask tests cover all four inter-intra modes, complemented orientation, row-padding
sentinels, and the 32-wide curve. FeatureTestRunner covers byte and high-bit-depth selectable
blending under SIMD and scalar dispatch, and complete `Av1BlockDecoder.DecodeBlock` tests execute
smooth inter-intra reconstruction at 8, 10, and 12 bits.
- [x] Extracted the fixture's 5,327-byte AV1 `mdat` payload and decoded it with the refreshed current
libaom `aomdec`, using one thread with row threading disabled. All 19 frames decoded. The final
19,200 YUV444 samples have SHA-256
`E8B776C2751DC30CA838931A4B74535FC6E681179568A1278747A38CFF2E5BFA` and match the retained
native reference with zero differing samples.
- [x] The real production sequence requires both smooth and wedge inter-intra blocks, compares the
final native Y, Cb, and Cr planes exactly, and compares final RGBA presentation through
ImageSharp's established reference-output API. Its constrained 1,024-byte tracked-allocator run
proves motion-field allocation and exactly one return for every allocation.
- [x] The focused Release checkpoint set passes 50/50 on net10.0 and 50/50 on net11.0, with zero
failures or skips. Scoped analyzer and whitespace verification pass for both changed C# files.
Roslynk reports zero compiler errors and no diagnostics in the changed files, `git diff --check`
passes, and `.gitattributes` is unchanged.
For every item: For every item:

6
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs

@ -60,7 +60,7 @@ public class Av1CompoundInterPredictorTests
/// Verifies the four smooth inter-intra modes and their complemented destination orientation. /// Verifies the four smooth inter-intra modes and their complemented destination orientation.
/// </summary> /// </summary>
[Fact] [Fact]
public void SmoothInterIntraMasksMatchPinnedWeights() public void SmoothInterIntraMasksMatchCurrentLibaomWeights()
{ {
ReadOnlySpan<byte> weights = [60, 34, 19, 11, 6, 4, 2, 1]; ReadOnlySpan<byte> weights = [60, 34, 19, 11, 6, 4, 2, 1];
@ -103,10 +103,10 @@ public class Av1CompoundInterPredictorTests
} }
/// <summary> /// <summary>
/// Verifies the pinned horizontal curve at the index exercised by a 32-by-16 inter-intra block. /// Verifies the current libaom horizontal curve at the index exercised by a 32-by-16 inter-intra block.
/// </summary> /// </summary>
[Fact] [Fact]
public void HorizontalInterIntraMaskMatchesPinnedThirtyTwoWideCurve() public void HorizontalInterIntraMaskMatchesCurrentLibaomThirtyTwoWideCurve()
{ {
const int width = 32; const int width = 32;
const int height = 16; const int height = 16;

4
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs

@ -1002,14 +1002,14 @@ public class Av1ReconstructionConformanceTests
} }
/// <summary> /// <summary>
/// Verifies every selectable compound and inter-intra production branch against pinned native and presentation references. /// Verifies every selectable compound and inter-intra production branch against retained native and presentation references.
/// </summary> /// </summary>
[Theory] [Theory]
[WithFile(TestImages.Heif.Av1DistanceWeightedCompoundSequenceAvif, PixelTypes.Rgba32)] [WithFile(TestImages.Heif.Av1DistanceWeightedCompoundSequenceAvif, PixelTypes.Rgba32)]
[WithFile(TestImages.Heif.Av1WedgeCompoundSequenceAvif, PixelTypes.Rgba32)] [WithFile(TestImages.Heif.Av1WedgeCompoundSequenceAvif, PixelTypes.Rgba32)]
[WithFile(TestImages.Heif.Av1DifferenceWeightedCompoundSequenceAvif, PixelTypes.Rgba32)] [WithFile(TestImages.Heif.Av1DifferenceWeightedCompoundSequenceAvif, PixelTypes.Rgba32)]
[WithFile(TestImages.Heif.Av1InterIntraSequenceAvif, PixelTypes.Rgba32)] [WithFile(TestImages.Heif.Av1InterIntraSequenceAvif, PixelTypes.Rgba32)]
public void DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences( public void DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences(
TestImageProvider<Rgba32> provider) TestImageProvider<Rgba32> provider)
=> FeatureTestRunner.RunWithHwIntrinsicsFeature( => FeatureTestRunner.RunWithHwIntrinsicsFeature(

0
tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-difference-weighted-compound.png → tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-difference-weighted-compound.png

0
tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-distance-weighted-compound.png → tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-distance-weighted-compound.png

0
tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-inter-intra.png → tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-inter-intra.png

0
tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesPinnedReferences_Rgba32_libavif-webp-logo-wedge-compound.png → tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifSequencesWithSelectableCompoundAndInterIntraMatchesRetainedReferences_Rgba32_libavif-webp-logo-wedge-compound.png

27
tests/Images/Input/Heif/Av1/Conformance/README.md

@ -118,7 +118,11 @@ only and is not used to establish AV1 reconstruction arithmetic.
## Selectable compound and inter-intra fixtures ## Selectable compound and inter-intra fixtures
The four selectable-compound fixtures use the same pinned `tests/data/webp_logo_animated.y4m` source and its `0872208D9C19B68B10A1647FA6849CFC4E2B21A19561ACD672E0629C70EFACA2` SHA-256. They were encoded at speed zero after disabling later inter-mode checkpoints. Each command also disables competing prediction tools that would prevent the resulting stream from isolating its named mode: The four AVIF files are retained solely as interoperability inputs. Their original creation commands
used the same `tests/data/webp_logo_animated.y4m` source and encoded at speed zero after disabling later
inter-mode checkpoints. These commands record container-fixture provenance only; they are not codec
implementation references. Each command also disables competing prediction tools that would prevent the
resulting stream from isolating its named mode:
```text ```text
./avifenc -j 1 -c aom -s 0 -q 80 -a enable-obmc=0 -a enable-warped-motion=0 -a enable-global-motion=0 -a enable-masked-comp=0 -a enable-interintra-comp=0 tests/data/webp_logo_animated.y4m libavif-webp-logo-distance-weighted-compound.avif ./avifenc -j 1 -c aom -s 0 -q 80 -a enable-obmc=0 -a enable-warped-motion=0 -a enable-global-motion=0 -a enable-masked-comp=0 -a enable-interintra-comp=0 tests/data/webp_logo_animated.y4m libavif-webp-logo-distance-weighted-compound.avif
@ -127,21 +131,24 @@ The four selectable-compound fixtures use the same pinned `tests/data/webp_logo_
./avifenc -j 1 -c aom -s 0 -q 80 -a enable-obmc=0 -a enable-warped-motion=0 -a enable-global-motion=0 -a enable-dist-wtd-comp=0 -a enable-masked-comp=0 tests/data/webp_logo_animated.y4m libavif-webp-logo-inter-intra.avif ./avifenc -j 1 -c aom -s 0 -q 80 -a enable-obmc=0 -a enable-warped-motion=0 -a enable-global-motion=0 -a enable-dist-wtd-comp=0 -a enable-masked-comp=0 tests/data/webp_logo_animated.y4m libavif-webp-logo-inter-intra.avif
``` ```
Pinned scalar libavif generated each final native and presentation reference with: | Fixture | AVIF SHA-256 | Retained frame-18 Y4M SHA-256 | Retained frame-18 PNG SHA-256 |
```text
./avifdec -j 1 -c aom --index 18 <fixture>.avif <fixture>-libaom.y4m
./avifdec -j 1 -c aom --index 18 <fixture>.avif <fixture>-libavif.png
```
| Fixture | AVIF SHA-256 | Frame-18 Y4M SHA-256 | Frame-18 PNG SHA-256 |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| `libavif-webp-logo-distance-weighted-compound` | `DA710D11C60F03EEA209E4360E2FC807B89C49AD50671F0DFB1BCF4AD5EF76DD` | `904D1B5B3E7F334CE8D44040F9A7BDCAC1F7773122FF1C5F06A5B4DD31A62A97` | `D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C` | | `libavif-webp-logo-distance-weighted-compound` | `DA710D11C60F03EEA209E4360E2FC807B89C49AD50671F0DFB1BCF4AD5EF76DD` | `904D1B5B3E7F334CE8D44040F9A7BDCAC1F7773122FF1C5F06A5B4DD31A62A97` | `D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C` |
| `libavif-webp-logo-wedge-compound` | `98640640A445055FEA3D9E2F4A78FEEF54E97F8171B472CF57D99156E1C553B2` | `904D1B5B3E7F334CE8D44040F9A7BDCAC1F7773122FF1C5F06A5B4DD31A62A97` | `D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C` | | `libavif-webp-logo-wedge-compound` | `98640640A445055FEA3D9E2F4A78FEEF54E97F8171B472CF57D99156E1C553B2` | `904D1B5B3E7F334CE8D44040F9A7BDCAC1F7773122FF1C5F06A5B4DD31A62A97` | `D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C` |
| `libavif-webp-logo-difference-weighted-compound` | `FC6459CD334762D74D9D2654640221E80C463CC01B82B29A5866C9E725ABE273` | `904D1B5B3E7F334CE8D44040F9A7BDCAC1F7773122FF1C5F06A5B4DD31A62A97` | `D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C` | | `libavif-webp-logo-difference-weighted-compound` | `FC6459CD334762D74D9D2654640221E80C463CC01B82B29A5866C9E725ABE273` | `904D1B5B3E7F334CE8D44040F9A7BDCAC1F7773122FF1C5F06A5B4DD31A62A97` | `D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C` |
| `libavif-webp-logo-inter-intra` | `71DF22E63626B5BC9001FF1E88076B90F11BB47D18089750853E66F0CBBB084B` | `502265688138641A7B12C8C4190B66C76CD4808D9AED05B06486056B39D7E9A0` | `F0DE4CCDFB6D95A400E69B69FA4C57F0BEEEEE75825722C31613385F0B3FD9FC` | | `libavif-webp-logo-inter-intra` | `71DF22E63626B5BC9001FF1E88076B90F11BB47D18089750853E66F0CBBB084B` | `502265688138641A7B12C8C4190B66C76CD4808D9AED05B06486056B39D7E9A0` | `F0DE4CCDFB6D95A400E69B69FA4C57F0BEEEEE75825722C31613385F0B3FD9FC` |
Pinned libaom block tracing confirms that these streams select distance weighting, both wedge signs, both difference-mask types, and both smooth and wedge inter-intra prediction. The production test independently requires those decoded mode states, decodes all preceding samples, compares the final native Y, U, and V planes exactly, compares the final RGBA presentation exactly, and repeats reconstruction with constrained tracked allocation. The inter-intra fixture's 5,327-byte AV1 `mdat` payload was decoded directly with the current official
libaom `main` checkout observed at `441c439b9916474cac15d2822af47a9ad70674a8`, using one thread and
with row threading disabled. Current `aomdec` produced all 19 YUV444 frames. The final frame's 19,200
native samples have SHA-256 `E8B776C2751DC30CA838931A4B74535FC6E681179568A1278747A38CFF2E5BFA`
and match the retained Y4M with zero differing samples.
The production tests independently require their decoded mode states. The inter-intra input must exercise
both smooth and wedge inter-intra prediction, decode all preceding samples, compare the final native Y,
Cb, and Cr planes exactly, compare final RGBA presentation through ImageSharp's established
reference-output API, and repeat reconstruction with constrained tracked allocation. The retained PNG is
presentation evidence only and is not an AV1 reconstruction reference.
## Overlapping motion-compensation fixture ## Overlapping motion-compensation fixture

Loading…
Cancel
Save