Browse Source

Complete AV1 OBMC decoding checkpoint

pull/2633/head
James Jackson-South 3 days ago
parent
commit
7e7e3cbe64
  1. 42
      HEIF_IMPLEMENTATION_PLAN.md
  2. 9
      src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs
  3. 5
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs
  4. 0
      tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifObmcSequenceMatchesVerifiedReferences_Rgba32_libavif-webp-logo-obmc.png
  5. 30
      tests/Images/Input/Heif/Av1/Conformance/README.md

42
HEIF_IMPLEMENTATION_PLAN.md

@ -30,7 +30,9 @@ Reference checkout evidence on 2026-08-31:
Reconciled with the worktree on 2026-08-31.
- [~] The bounded container reader, still-image path, sequence parser, AV1 decoder, color pipeline, presentation pipeline, and broad AV1 test suite exist locally.
- [~] The inter-frame decoder has verified checkpoints through difference-weighted compound prediction. OBMC, scaled references, local warped motion, and global motion exist locally but remain open until their ordered checkpoints below are completed.
- [~] The inter-frame decoder has verified checkpoints through OBMC. Scaled references, local warped
motion, and global motion exist locally but remain open until their ordered checkpoints below are
completed.
- [~] Loop filtering, CDEF, super-resolution, restoration, film grain, layered presentation, alpha composition, and color conversion exist locally. Shared-source cleanup changed the current tree, so final production-path verification is open.
- [~] AV1 writer primitives, forward transforms, symbol encoding, and tile-writing source exist locally, but they are not connected to the public encoder.
- [ ] The public AV1 encoder is not implemented. HeifEncoderCore.Encode throws NotSupportedException when AV1 is selected.
@ -181,8 +183,8 @@ The single-reference syntax, buffer, reconstruction, and ownership foundation is
- [x] Distance-weighted compound prediction.
- [x] Wedge compound prediction.
- [x] Difference-weighted compound prediction.
- [~] OBMC. Current item.
- [~] Scaled-reference prediction.
- [x] OBMC.
- [~] Scaled-reference prediction. Next item.
- [~] Local warped prediction.
- [~] Non-translational global prediction.
- [~] Inter deblocking decisions and reference/mode deltas.
@ -358,6 +360,40 @@ Verified difference-weighted compound checkpoint evidence on 2026-08-31:
- [x] The focused Release checkpoint set passes 37/37 on net10.0 and 37/37 on net11.0, with zero
failures or skips. Scoped analyzer and whitespace verification pass for every changed C# file.
Roslynk reports zero compiler errors, `git diff --check` passes, and `.gitattributes` is unchanged.
- [x] The completed checkpoint was committed as `fb4c64474e1ced4067a42731384f3b5ad4212a2f`
with author and committer `James Jackson-South <james_south@hotmail.com>`.
Verified OBMC checkpoint evidence on 2026-08-31:
- [x] Audited motion-mode syntax against current libaom `av1/decoder/decodemv.c`,
`av1/common/blockd.h`, `av1/common/reconinter.c`, `av1/common/obmc.h`, and
`av1/common/reconinter_template.inc`. ImageSharp applies the same switchable-mode, skip,
single-reference, inter-intra, minimum-size, overlappable-neighbor, fixed-global-motion, scaled
reference, and projection-sample gates and reads the matching binary or three-way CDF.
- [x] Audited above and left neighbor traversal, 4x4 pairing, neighbor caps, chroma suppression,
prediction rectangles, interpolation filters, first-reference selection, mask tables, and blend
order against current libaom. The existing semantic mask-blend predictor remains the correct
SIMD-first traversal; no OBMC-specific operator family, allocation, or copy was introduced.
- [x] Corrected the unscaled neighbor far-edge UMV clamp. After converting libaom's neighbor-relative
motion-vector limits to an absolute source coordinate, the prediction extent cancels from the
right and bottom limits; the previous code counted it twice.
- [x] Extracted the fixture's 5,387-byte AV1 `mdat` payload at AVIF offset 1,065 and decoded it with
refreshed current libaom `aomdec`, using one thread with row threading disabled. All 19 frames
decoded. The final 19,200 YUV444 samples have SHA-256
`E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained native
reference with zero differing samples.
- [x] The production sequence asserts decoded OBMC mode state, compares final native Y, Cb, and Cr
planes exactly, compares final RGBA presentation through ImageSharp's established reference-output
API under normal and scalar FeatureTestRunner dispatch, and repeats reconstruction with a 1,024-byte
constrained tracked allocator. Direct `DecodeBlock` tests cover above-then-left blending at
8/10/12-bit and 4:2:0 and 4:2:2 chroma geometry.
- [x] Renamed the stale pinned-reference test and its established reference-output PNG together. The
PNG SHA-256 remains
`D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C`; only its contract-derived
filename changed.
- [x] The focused Release checkpoint set passes 18/18 on net10.0 and 18/18 on net11.0, with zero
failures or skips. Scoped analyzer and whitespace verification pass for every changed C# file.
Roslynk reports zero compiler errors, `git diff --check` passes, and `.gitattributes` is unchanged.
For every item:

9
src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs

@ -1962,16 +1962,17 @@ internal sealed class Av1BlockDecoder : IDisposable
int framePlaneWidth = (this.frameHeader.ModeInfoColumnCount << Av1Constants.ModeInfoSizeLog2) >> subX;
int framePlaneHeight = (this.frameHeader.ModeInfoRowCount << Av1Constants.ModeInfoSizeLog2) >> subY;
// OBMC changes the prediction rectangle but not AV1's unrestricted-motion-vector boundary extension.
// Clamping the absolute source coordinate expresses the same edge calculation libaom rebuilds per neighbor.
// libaom clamps the motion vector relative to each neighbor rectangle. Once the neighbor origin is added, the
// prediction extent remains in the left/top limit but cancels from the right/bottom limit. Keeping this
// asymmetry avoids counting the OBMC rectangle twice when the source lies beyond the far frame edge.
sourceColumnQ4 = Av1Math.Clip3(
-horizontalExtensionQ4,
(framePlaneWidth << 4) + horizontalExtensionQ4 - 16,
((framePlaneWidth + 4) << 4) - 16,
sourceColumnQ4);
sourceRowQ4 = Av1Math.Clip3(
-verticalExtensionQ4,
(framePlaneHeight << 4) + verticalExtensionQ4 - 16,
((framePlaneHeight + 4) << 4) - 16,
sourceRowQ4);
int horizontalPhase = sourceColumnQ4 & 15;

5
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs

@ -1054,11 +1054,12 @@ public class Av1ReconstructionConformanceTests
}
/// <summary>
/// Verifies production OBMC reconstruction against pinned native and presentation references.
/// Verifies production OBMC reconstruction against exact native and presentation references. The native reference
/// has been reverified against current official libaom main.
/// </summary>
[Theory]
[WithFile(TestImages.Heif.Av1ObmcSequenceAvif, PixelTypes.Rgba32)]
public void DecodeRealLibavifObmcSequenceMatchesPinnedReferences(TestImageProvider<Rgba32> provider)
public void DecodeRealLibavifObmcSequenceMatchesVerifiedReferences(TestImageProvider<Rgba32> provider)
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(
ValidateObmcSequenceWithDefaultConfiguration,
ReconstructionConfigurations,

0
tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifObmcSequenceMatchesPinnedReferences_Rgba32_libavif-webp-logo-obmc.png → tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifObmcSequenceMatchesVerifiedReferences_Rgba32_libavif-webp-logo-obmc.png

30
tests/Images/Input/Heif/Av1/Conformance/README.md

@ -45,7 +45,7 @@ The reference builds use `AOM_TARGET_CPU=generic` and disable libyuv. Native rec
| `libavif-webp-logo-wedge-compound` | Wedge compound prediction with both signaled mask orientations |
| `libavif-webp-logo-difference-weighted-compound` | Difference-weighted compound prediction with both mask orientations |
| `libavif-webp-logo-inter-intra` | Smooth and wedge inter-intra prediction |
| `libavif-webp-logo-obmc` | Above and left overlapping motion compensation through a 19-frame dependent sequence |
| `libavif-webp-logo-obmc` | Overlapping motion compensation through a 19-frame dependent sequence |
| `libavif-rotating-grid-local-warp` | Multi-sample local affine projection and warped prediction through a two-frame dependent sequence |
| `libavif-rotating-grid-global-warp` | Non-translational rotation/zoom GLOBALMV prediction through a two-frame dependent sequence |
@ -169,20 +169,30 @@ and are not AV1 reconstruction references.
## Overlapping motion-compensation fixture
The `libavif-webp-logo-obmc.avif` fixture uses the same pinned `tests/data/webp_logo_animated.y4m` source and source SHA-256 as the compound fixtures. It was encoded with the pinned scalar toolchain after disabling competing compound, inter-intra, warped, and global prediction modes:
The `libavif-webp-logo-obmc.avif` fixture is retained interoperability input. It was originally
packaged by libavif from the same `tests/data/webp_logo_animated.y4m` source as the compound fixtures,
with libaom's encoder selected and competing compound, inter-intra, warped, and global prediction modes
disabled:
```text
./avifenc -j 1 -c aom -s 0 -q 80 -a max-reference-frames=3 -a enable-dist-wtd-comp=0 -a enable-masked-comp=0 -a enable-interintra-comp=0 -a enable-warped-motion=0 -a enable-global-motion=0 tests/data/webp_logo_animated.y4m libavif-webp-logo-obmc.avif
```
Pinned scalar libavif generated the final native and presentation references with:
```text
./avifdec -j 1 -c aom --index 18 libavif-webp-logo-obmc.avif libavif-webp-logo-obmc-libaom.y4m
./avifdec -j 1 -c aom --index 18 libavif-webp-logo-obmc.avif libavif-webp-logo-obmc-libavif.png
```
The AVIF SHA-256 is `765245F71BD398F7AD87BD83FD5C5C11172981B37A4FABF4F10F65D4E8AEA537`. The retained frame-18 Y4M SHA-256 is `904D1B5B3E7F334CE8D44040F9A7BDCAC1F7773122FF1C5F06A5B4DD31A62A97`, and the frame-18 PNG SHA-256 is `D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C`. Pinned libaom block tracing records more than one hundred actual OBMC blocks across the decoded sequence, including blocks with nonzero horizontal and vertical motion vectors. The production test requires decoded OBMC mode state, decodes every retained-reference dependency, compares the final native Y, U, and V planes exactly, compares the final RGBA presentation exactly through `FeatureTestRunner`, and repeats reconstruction with constrained tracked allocation. Direct production-branch tests separately cover 8/10/12-bit storage and 4:2:0 and 4:2:2 overlap geometry.
On 2026-08-31 the fixture's 5,387-byte AV1 `mdat` payload at AVIF offset 1,065 was decoded with
`aomdec` from a freshly updated clean checkout of current official libaom `main`. One decode thread
was used and row threading remained disabled. All 19 frames decoded. The final frame's 19,200 native
YUV444 samples have SHA-256
`E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match
`libavif-webp-logo-obmc-libaom.y4m` with zero differing samples.
The production test requires decoded OBMC mode state, decodes every retained-reference dependency,
compares final Y, Cb, and Cr planes exactly, compares final RGBA presentation through ImageSharp's
established reference-output API under normal and scalar FeatureTestRunner dispatch, and repeats the
decode with constrained tracked allocation. Direct production-branch tests separately cover
above-then-left blending at 8/10/12-bit and 4:2:0 and 4:2:2 overlap geometry. The retained presentation
PNG's SHA-256 is
`D2CB388C9092EF17C4F0382C0150DD30D6F9D0EE247FF45AB5D7D4D312CEB23C`; its pixels were not changed
when its contract-derived filename was updated with the test name.
## Local warped-motion fixture

Loading…
Cancel
Save