From b17dd999b9099eb02ebb57d140e018238e5565dc Mon Sep 17 00:00:00 2001 From: James Jackson-South Date: Thu, 27 Aug 2026 11:39:11 +1000 Subject: [PATCH] Verify AV1 intra-block copy decoding --- HEIF_IMPLEMENTATION_PLAN.md | 8 +- .../Formats/Heif/Av1/Av1BitStreamWriter.cs | 2 +- .../Heif/Av1/OpenBitstreamUnit/ObuWriter.cs | 2 +- .../Av1/Prediction/Av1PredictionDecoder.cs | 4 +- .../Formats/Heif/Av1/Tiling/Av1MacroBlockD.cs | 2 +- .../Heif/Av1/Tiling/Av1PictureControlSet.cs | 4 +- .../Av1/Transform/Av1Inverse2dTransformer.cs | 32 ++-- .../Av1/Transform/Av1InverseTransformer.cs | 6 +- .../Transform/Av1InverseTransformerFactory.cs | 4 +- .../Formats/Heif/Av1HeifItemDecoder.cs | 2 +- .../Formats/Heif/HeifDecoderCore.cs | 2 +- .../Heif/Hevc/HevcIntraPredictionState.cs | 12 +- .../Formats/Heif/HevcHeifItemDecoder.cs | 2 +- .../Av1/Av1ReconstructionConformanceTests.cs | 174 +++++++++++++++++- tests/ImageSharp.Tests/TestImages.cs | 9 + .../Input/Heif/Av1/Conformance/README.md | 10 + ...libavif-intrabc-abc-10b-444-libaom-y4m.yuv | 3 + .../libavif-intrabc-abc-10b-444.avif | 3 + .../libavif-intrabc-abc-10b-444.png | 3 + ...libavif-intrabc-abc-12b-444-libaom-y4m.yuv | 3 + .../libavif-intrabc-abc-12b-444.avif | 3 + .../libavif-intrabc-abc-12b-444.png | 3 + .../libavif-intrabc-abc-8b-444-libaom-y4m.yuv | 3 + .../libavif-intrabc-abc-8b-444.avif | 3 + .../libavif-intrabc-abc-8b-444.png | 3 + 25 files changed, 263 insertions(+), 39 deletions(-) create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444-libaom-y4m.yuv create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444.avif create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444.png create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444-libaom-y4m.yuv create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444.avif create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444.png create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444-libaom-y4m.yuv create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444.avif create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444.png diff --git a/HEIF_IMPLEMENTATION_PLAN.md b/HEIF_IMPLEMENTATION_PLAN.md index 1e005cd8a0..08e7cd5831 100644 --- a/HEIF_IMPLEMENTATION_PLAN.md +++ b/HEIF_IMPLEMENTATION_PLAN.md @@ -29,7 +29,7 @@ Checkboxes may be marked complete only when the implementation and the verificat ## Delivery dashboard -Last reconciled with the source tree on 2026-08-27 against the worktree based on commit `5a37cfb1a`, including the completed AV1 transform and OBU-framing checkpoints and the current intra-block-copy implementation. This dashboard is the authoritative delivery order. The detailed phase checklists below provide subsystem evidence; they do not override the current-stage marker or permit work to skip ahead. +Last reconciled with the source tree on 2026-08-27 against the worktree based on commit `cf0c81eb3`, including the completed AV1 transform, OBU-framing, and independently verified intra-block-copy checkpoints. This dashboard is the authoritative delivery order. The detailed phase checklists below provide subsystem evidence; they do not override the current-stage marker or permit work to skip ahead. Status meanings: @@ -46,7 +46,7 @@ Immediate checkpoint: **remove every remaining valid AV1 still-image unsupported | --- | --- | --- | --- | --- | | 1 | Baseline, provenance, documentation, and public contract | In progress | Pinned codec references, a bounded image-only scope, encoder options, typed bit depth, decoder-option propagation, and extensive HEIF documentation exist. | Complete the all-file documentation audit, record a fresh Release baseline, finish distinct public HEIC/AVIF save boundaries, and close API review. | | 2 | Bounded HEIF item and image-sequence container | In progress | Still-item parsing, grids, auxiliary alpha, metadata properties, bounded image-sequence tracks, Identify, and all-sync AV1 sequence presentation are connected. | Complete adversarial boundary coverage, remaining item/property behavior, reference-dependent sequence reconstruction, and the bounded sequence writer. | -| 3 | Still-image AV1 and HEVC decoding | **Current** | HEVC reconstruction reaches exact HM/libheif fixtures across the recorded 8/10/12-bit and chroma cases. AV1 includes bounded OBU framing, reconstruction, filters, grain, color, transforms, and intra-block-copy syntax and prediction through SIMD-first static-generic operators. | Add independent intra-block-copy AV1/AVIF fixtures, remove every other valid AV1 still-image unsupported branch with independent vectors, then complete the remaining HEVC profile and Range Extensions matrix. | +| 3 | Still-image AV1 and HEVC decoding | **Current** | HEVC reconstruction reaches exact HM/libheif fixtures across the recorded 8/10/12-bit and chroma cases. AV1 includes bounded OBU framing, reconstruction, filters, grain, color, transforms, and independently verified intra-block-copy syntax and prediction through SIMD-first static-generic operators. | Remove every other valid AV1 still-image unsupported branch with independent vectors, then complete the remaining HEVC profile and Range Extensions matrix. | | 4 | Complete decoded presentation and animation | In progress | Shared SIMD-first AV1/HEVC color conversion, ICC application, grids, transforms, direct planar alpha composition, frame metadata, repetition, and independently decodable AV1 sequence samples exist. | Close the full color/ICC cross-product, HEVC sequence decoding, AV1/HEVC reference-dependent samples, frame-local metadata/alpha behavior, and independent animated decode vectors. | | 5 | AV1/AVIF encoding | Not started | RGB-to-planar conversion, forward transforms, OBU writer foundations, options, and container-writing infrastructure exist. | `HeifEncoderCore` still rejects AV1. Implement a real independently decodable lossy/lossless AV1 payload and the complete AVIF item/metadata matrix. | | 6 | HEVC/HEIC encoding | Not started | Shared input color conversion, options, and HEIF writer infrastructure exist. | `HeifEncoderCore` still rejects HEVC. Implement a real independently decodable lossy/lossless HEVC payload and the complete HEIC item/metadata matrix. | @@ -504,9 +504,9 @@ Implement and verify in dependency order: - [x] Implement SIMD-first chroma-from-luma storage, 4:4:4/4:2:2/4:2:0 subsampling, rounded mean subtraction, and 8/10/12-bit prediction with exact scalar fallback and `FeatureTestRunner` parity. - [x] Implement allocation-free SIMD-first palette reconstruction for palette sizes 2-8, transform widths 4-64, and 8/10/12-bit samples with exact scalar fallback and `FeatureTestRunner` parity. - [x] Verify palette mode syntax and presented reconstruction with an independently encoded palette AVIF fixture. -- [ ] Intra-block copy for still-image intra frames. +- [x] Intra-block copy for still-image intra frames. - [x] Decode tile-adaptive integer displacement vectors, derive and validate spatial references, apply the inter transform sets, and reconstruct luma and subsampled chroma through allocation-free `Vector512`/`Vector256`/`Vector128` operators with exact-width stores and scalar fallback. `FeatureTestRunner` verifies every transform width, bit-depth storage path, chroma phase, intrinsic tier, scalar continuation, and destination-padding boundary. - - [ ] Verify displacement-vector entropy, spatial candidate ordering, wavefront legality, native 8/10/12-bit reconstruction, and presented output against independently encoded opaque intra-block-copy AV1 and AVIF fixtures from the pinned libaom/libavif reference. + - [x] Verify displacement-vector entropy, spatial candidate ordering, wavefront legality, native 8/10/12-bit reconstruction, and presented output against independently encoded opaque intra-block-copy AVIF fixtures from the pinned generic libaom/libavif reference. The fixtures require actual intra-block-copy block selection, compare every native plane sample from retained scalar-decoder Y4M output, and compare every presented RGBA byte exactly under normal hardware dispatch and each narrower fallback configuration without a tolerance. - [ ] Lossless and high-bit-depth reconstruction with correct clipping and intermediate precision. - [x] Route lossless 4x4 blocks through allocation-free reversible inverse Walsh-Hadamard reconstruction for 8/10/12-bit samples, including the DC-only specialization, `Vector128` production traversal, scalar fallback, exact clipping, and `FeatureTestRunner` parity. - [x] Verify lossless syntax, inverse quantization, prediction, and presented reconstruction with independently encoded 8/10/12-bit AVIF fixtures. The tests require coded residuals with palette and intra-block copy disabled, compare every native YUV sample with the pinned generic libaom-backed decoder, and compare every presented RGBA byte with pinned generic libavif exactly under normal hardware dispatch and the scalar fallback. diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1BitStreamWriter.cs b/src/ImageSharp/Formats/Heif/Av1/Av1BitStreamWriter.cs index 3c31d21a87..e4612db831 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Av1BitStreamWriter.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Av1BitStreamWriter.cs @@ -231,7 +231,7 @@ internal ref struct Av1BitStreamWriter /// Writes a byte-aligned entropy-coded tile payload. /// /// The tile payload. - internal void WriteBlob(Span tileData) + internal void WriteBlob(ReadOnlySpan tileData) { DebugGuard.IsTrue(Av1Math.Modulus8(this.BitPosition) == 0, "Writing of Tile Data only allowed on byte alignment"); diff --git a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuWriter.cs b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuWriter.cs index 5d943d8e9f..727dd373fd 100644 --- a/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuWriter.cs +++ b/src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuWriter.cs @@ -73,7 +73,7 @@ internal class ObuWriter /// The destination stream. /// The OBU payload type. /// The complete OBU payload. - private static void WriteObuHeaderAndSize(Stream stream, ObuType type, Span payload) + private static void WriteObuHeaderAndSize(Stream stream, ObuType type, ReadOnlySpan payload) { stream.WriteByte(WriteObuHeader(type)); diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs index 79d50aa07b..f9ffd12111 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs @@ -334,7 +334,7 @@ internal class Av1PredictionDecoder Span pixelBuffer, int pixelBufferStride, Span topNeighbor, - Span leftNeighbor, + ReadOnlySpan leftNeighbor, int referenceStride, Av1PredictionMode mode, int blockModeInfoColumnOffset, @@ -754,7 +754,7 @@ internal class Av1PredictionDecoder private void DecodeBuildIntraPredictors( Av1PartitionInfo partitionInfo, Span aboveNeighbor, - Span leftNeighbor, + ReadOnlySpan leftNeighbor, nuint referenceStride, Span destination, nuint destinationStride, diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1MacroBlockD.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1MacroBlockD.cs index e7edd06225..8d6b8c5f0f 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1MacroBlockD.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1MacroBlockD.cs @@ -16,7 +16,7 @@ internal class Av1MacroBlockD /// /// Gets or sets the mode-information entries for the current block and its mapped neighbors. /// - public required Span ModeInfo + public required ReadOnlySpan ModeInfo { get => this.modeInfo; internal set diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PictureControlSet.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PictureControlSet.cs index e347ab95ad..c0a158d781 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PictureControlSet.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1PictureControlSet.cs @@ -88,7 +88,7 @@ internal class Av1PictureControlSet /// /// The frame position in 4x4 mode-information units. /// The mode-information entries to map. - public void SetModeInfoGridRow(Point position, Span span) + public void SetModeInfoGridRow(Point position, ReadOnlySpan span) => this.SetModeInfoGridRow((position.Y * this.ModeInfoStride) + position.X, span); /// @@ -96,7 +96,7 @@ internal class Av1PictureControlSet /// /// The linear grid offset. /// The mode-information entries to map. - public void SetModeInfoGridRow(int offset, Span span) + public void SetModeInfoGridRow(int offset, ReadOnlySpan span) { // Grid entries own their arrays because the source span can refer to temporary traversal state. this.ModeInfoGrid[offset] = new Av1ModeInfo[span.Length]; diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Inverse2dTransformer.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Inverse2dTransformer.cs index 12f5a131f4..23ddc7e2ff 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Inverse2dTransformer.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1Inverse2dTransformer.cs @@ -31,8 +31,8 @@ internal static class Av1Inverse2dTransformer /// The reusable transform workspace. /// The coded sample bit depth. public static void Transform2dAdd( - Span input, - Span outputForRead, + ReadOnlySpan input, + ReadOnlySpan outputForRead, int strideForRead, Span outputForWrite, int strideForWrite, @@ -60,8 +60,8 @@ internal static class Av1Inverse2dTransformer /// The per-axis transform, flip, shift, and range configuration. /// The reusable transform workspace. public static void Transform2dAdd( - Span input, - Span outputForRead, + ReadOnlySpan input, + ReadOnlySpan outputForRead, int strideForRead, Span outputForWrite, int strideForWrite, @@ -81,8 +81,8 @@ internal static class Av1Inverse2dTransformer /// Initializes the transform ranges and selects the concrete column operator. /// private static void Transform2dAdd( - Span input, - Span outputForRead, + ReadOnlySpan input, + ReadOnlySpan outputForRead, int strideForRead, Span outputForWrite, int strideForWrite, @@ -164,8 +164,8 @@ internal static class Av1Inverse2dTransformer /// Selects the concrete row operator after the column operator has been specialized. /// private static void DispatchRow( - Span input, - Span outputForRead, + ReadOnlySpan input, + ReadOnlySpan outputForRead, int strideForRead, Span outputForWrite, int strideForWrite, @@ -247,8 +247,8 @@ internal static class Av1Inverse2dTransformer /// Applies the specialized operator pair using the production lane width selected for the block and processor. /// private static void Transform2d( - Span input, - Span outputForRead, + ReadOnlySpan input, + ReadOnlySpan outputForRead, int strideForRead, Span outputForWrite, int strideForWrite, @@ -299,8 +299,8 @@ internal static class Av1Inverse2dTransformer /// The reusable storage for SIMD vectors and transposed coefficients. /// The coded sample bit depth used to clamp reconstructed values. public static void Transform2dVector256( - Span input, - Span outputForRead, + ReadOnlySpan input, + ReadOnlySpan outputForRead, int strideForRead, Span outputForWrite, int strideForWrite, @@ -455,8 +455,8 @@ internal static class Av1Inverse2dTransformer /// The reusable storage for SIMD vectors and transposed coefficients. /// The coded sample bit depth used to clamp reconstructed values. public static void Transform2dVector128( - Span input, - Span outputForRead, + ReadOnlySpan input, + ReadOnlySpan outputForRead, int strideForRead, Span outputForWrite, int strideForWrite, @@ -587,8 +587,8 @@ internal static class Av1Inverse2dTransformer /// The reusable storage for transform stages and transposed coefficients. /// The coded sample bit depth used to clamp reconstructed values. public static void Transform2dScalar( - Span input, - Span outputForRead, + ReadOnlySpan input, + ReadOnlySpan outputForRead, int strideForRead, Span outputForWrite, int strideForWrite, diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.cs index aa0802d435..5ba9ca4c80 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformer.cs @@ -21,7 +21,7 @@ internal class Av1InverseTransformer /// Whether the segment uses lossless transform rules. /// The reusable transform workspace for the containing block decode. public static void Reconstruct8Bit( - Span coefficientsBuffer, + ReadOnlySpan coefficientsBuffer, Span reconstructionBuffer, int reconstructionStride, Av1TransformSize transformSize, @@ -60,7 +60,7 @@ internal class Av1InverseTransformer /// Whether the segment uses lossless transform rules. /// The reusable transform workspace for the containing block decode. public static void Reconstruct8Bit( - Span coefficientsBuffer, + ReadOnlySpan coefficientsBuffer, Span reconstructionBufferRead, int reconstructionReadStride, Span reconstructionBufferWrite, @@ -111,7 +111,7 @@ internal class Av1InverseTransformer /// The reusable transform workspace for the containing block decode. /// Implements the reconstruction operation in AV1 section 7.11.2. public static void ReconstructHighBitDepth( - Span coefficientsBuffer, + ReadOnlySpan coefficientsBuffer, Span reconstructionBuffer, int reconstructionStride, Av1TransformSize transformSize, diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformerFactory.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformerFactory.cs index 744f439297..2578574431 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformerFactory.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1InverseTransformerFactory.cs @@ -19,7 +19,7 @@ internal static class Av1InverseTransformerFactory /// The transform type, dimensions, bit depth, and pipeline selection. /// The reusable transform workspace for the containing decode operation. public static void InverseTransformAdd( - Span coefficients, + ReadOnlySpan coefficients, Span readBuffer, int readStride, Span writeBuffer, @@ -63,7 +63,7 @@ internal static class Av1InverseTransformerFactory /// The transform type, dimensions, bit depth, and pipeline selection. /// The reusable transform workspace for the containing decode operation. public static void InverseTransformAdd( - Span coefficients, + ReadOnlySpan coefficients, Span readBuffer, int readStride, Span writeBuffer, diff --git a/src/ImageSharp/Formats/Heif/Av1HeifItemDecoder.cs b/src/ImageSharp/Formats/Heif/Av1HeifItemDecoder.cs index fb8c179c09..cc8741f05b 100644 --- a/src/ImageSharp/Formats/Heif/Av1HeifItemDecoder.cs +++ b/src/ImageSharp/Formats/Heif/Av1HeifItemDecoder.cs @@ -104,7 +104,7 @@ internal class Av1HeifItemDecoder : IHeifItemDecoder, IHeifAlpha private static Av1CodecConfiguration ValidateItemData( DecoderOptions options, HeifItem item, - Span data, + ReadOnlySpan data, out HeifContentLightLevel? obuContentLightLevel, out HeifMasteringDisplayColorVolume? obuMasteringDisplayColorVolume) { diff --git a/src/ImageSharp/Formats/Heif/HeifDecoderCore.cs b/src/ImageSharp/Formats/Heif/HeifDecoderCore.cs index fdb36ea0f7..926e40a46c 100644 --- a/src/ImageSharp/Formats/Heif/HeifDecoderCore.cs +++ b/src/ImageSharp/Formats/Heif/HeifDecoderCore.cs @@ -2709,7 +2709,7 @@ internal sealed class HeifDecoderCore : ImageDecoderCore /// The zero-based field offset. /// The field width in bytes. /// The diagnostic name used for malformed input errors. - private static void EnsureBufferRemaining(Span buffer, int offset, int count, string boxName) + private static void EnsureBufferRemaining(ReadOnlySpan buffer, int offset, int count, string boxName) { if ((uint)offset > (uint)buffer.Length || (uint)count > (uint)(buffer.Length - offset)) { diff --git a/src/ImageSharp/Formats/Heif/Hevc/HevcIntraPredictionState.cs b/src/ImageSharp/Formats/Heif/Hevc/HevcIntraPredictionState.cs index ec2ccae2f7..7b9c5656f9 100644 --- a/src/ImageSharp/Formats/Heif/Hevc/HevcIntraPredictionState.cs +++ b/src/ImageSharp/Formats/Heif/Hevc/HevcIntraPredictionState.cs @@ -340,17 +340,23 @@ internal sealed class HevcIntraPredictionState : IDisposable { if (values[0] > values[1]) { - (values[0], values[1]) = (values[1], values[0]); + byte value = values[0]; + values[0] = values[1]; + values[1] = value; } if (values[0] > values[2]) { - (values[0], values[2]) = (values[2], values[0]); + byte value = values[0]; + values[0] = values[2]; + values[2] = value; } if (values[1] > values[2]) { - (values[1], values[2]) = (values[2], values[1]); + byte value = values[1]; + values[1] = values[2]; + values[2] = value; } } diff --git a/src/ImageSharp/Formats/Heif/HevcHeifItemDecoder.cs b/src/ImageSharp/Formats/Heif/HevcHeifItemDecoder.cs index 23a74809fd..903ea7c9c8 100644 --- a/src/ImageSharp/Formats/Heif/HevcHeifItemDecoder.cs +++ b/src/ImageSharp/Formats/Heif/HevcHeifItemDecoder.cs @@ -147,7 +147,7 @@ internal sealed class HevcHeifItemDecoder : IHeifItemDecoder, IH private static HevcPictureDecoder DecodePicture( DecoderOptions options, HeifItem item, - Span data, + ReadOnlySpan data, CicpProfile? colorProfile, CancellationToken cancellationToken, out HevcCodecConfiguration codecConfiguration, diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs index 00bb87af4f..a918d1126e 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs @@ -31,6 +31,12 @@ public class Av1ReconstructionConformanceTests private const HwIntrinsics PaletteConfigurations = HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + /// + /// The hardware configurations covering normal dispatch, narrower vector fallbacks, and scalar intra-block copy. + /// + private const HwIntrinsics IntraBlockCopyConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + /// /// The hardware configurations covering the 128-bit and scalar lossless inverse-transform paths. /// @@ -160,6 +166,22 @@ public class Av1ReconstructionConformanceTests public void DecodeWithPaletteMatchesPinnedLibavifPresentation() => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidatePalettePresentedFixture, PresentationConfigurations); + /// + /// Verifies selected intra-block-copy prediction and exact native samples against scalar libaom for an + /// independently encoded AV1 still pictures across every available vector width and the scalar fallback. + /// + [Fact] + public void DecodeWithIntraBlockCopyMatchesPinnedLibaomReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateIntraBlockCopyNativeFixtures, IntraBlockCopyConfigurations); + + /// + /// Verifies exact presented pixels for independently encoded intra-block-copy AVIF images across the available + /// vector widths and the scalar fallback. + /// + [Fact] + public void DecodeWithIntraBlockCopyMatchesPinnedLibavifPresentation() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateIntraBlockCopyPresentedFixtures, PresentationConfigurations); + /// /// Verifies lossless syntax, residual reconstruction, and exact native samples against scalar libaom for /// independently encoded eight-, ten-, and twelve-bit AVIF images. @@ -357,6 +379,100 @@ public class Av1ReconstructionConformanceTests requireActiveLoopFilter: false, requirePalette: true); + /// + /// Validates every active intra-block-copy native fixture under the hardware configuration selected by + /// . + /// + private static void ValidateIntraBlockCopyNativeFixtures() + { + ValidateIntraBlockCopyNativeFixture( + TestImages.Heif.Av1IntraBlockCopy8BitAvif, + TestImages.Heif.Av1IntraBlockCopy8BitReference, + Av1BitDepth.EightBit); + + ValidateIntraBlockCopyNativeFixture( + TestImages.Heif.Av1IntraBlockCopy10BitAvif, + TestImages.Heif.Av1IntraBlockCopy10BitReference, + Av1BitDepth.TenBit); + + ValidateIntraBlockCopyNativeFixture( + TestImages.Heif.Av1IntraBlockCopy12BitAvif, + TestImages.Heif.Av1IntraBlockCopy12BitReference, + Av1BitDepth.TwelveBit); + } + + /// + /// Validates one independently encoded intra-block-copy AVIF against its native Y4M reference. + /// + /// The complete AVIF container. + /// The native Y4M output produced by the pinned scalar libaom-backed decoder. + /// The expected AV1 sample precision. + private static void ValidateIntraBlockCopyNativeFixture(string imagePath, string referencePath, Av1BitDepth bitDepth) + { + byte[] imageBytes = TestFile.Create(imagePath).Bytes; + byte[] referenceBytes = TestFile.Create(referencePath).Bytes; + ReadOnlySpan fileHeader = bitDepth switch + { + Av1BitDepth.EightBit => "YUV4MPEG2 W512 H256 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=FULL\n"u8, + Av1BitDepth.TenBit => "YUV4MPEG2 W512 H256 F25:1 Ip A0:0 C444p10 XYSCSS=444P10 XCOLORRANGE=FULL\n"u8, + _ => "YUV4MPEG2 W512 H256 F25:1 Ip A0:0 C444p12 XYSCSS=444P12 XCOLORRANGE=FULL\n"u8 + }; + + ReadOnlySpan frameHeader = "FRAME\n"u8; + + // The retained Y4M header locks the independently decoded reference to the expected dimensions, sampling, + // bit depth, and full range. Only the following frame payload contains the planar Y, U, and V samples. + ReadOnlySpan nativeReference = referenceBytes; + Assert.True(nativeReference.StartsWith(fileHeader)); + nativeReference = nativeReference[fileHeader.Length..]; + Assert.True(nativeReference.StartsWith(frameHeader)); + nativeReference = nativeReference[frameHeader.Length..]; + + Span payload = GetSoleAv1ItemPayload(imageBytes); + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _); + + Assert.Equal(512, frameBuffer.Width); + Assert.Equal(256, frameBuffer.Height); + Assert.Equal(bitDepth, frameBuffer.BitDepth); + Assert.Equal(Av1ColorFormat.Yuv444, frameBuffer.ColorFormat); + Assert.NotNull(decoder.FrameHeader); + Assert.True(decoder.FrameHeader.AllowIntraBlockCopy); + Assert.NotEqual(0, GetIntraBlockCopyBlockCount(decoder)); + AssertNativePlanesEqual(frameBuffer, nativeReference); + } + + /// + /// Validates every active intra-block-copy presentation fixture under the hardware configuration selected by + /// . + /// + private static void ValidateIntraBlockCopyPresentedFixtures() + { + ValidatePresentedFixture( + TestImages.Heif.Av1IntraBlockCopy8BitAvif, + TestImages.Heif.Av1IntraBlockCopy8BitPresentationReference, + 512, + 256, + HeifBitDepth.Bit8, + requireIntraBlockCopy: true); + + ValidatePresentedFixture( + TestImages.Heif.Av1IntraBlockCopy10BitAvif, + TestImages.Heif.Av1IntraBlockCopy10BitPresentationReference, + 512, + 256, + HeifBitDepth.Bit10, + requireIntraBlockCopy: true); + + ValidatePresentedFixture( + TestImages.Heif.Av1IntraBlockCopy12BitAvif, + TestImages.Heif.Av1IntraBlockCopy12BitPresentationReference, + 512, + 256, + HeifBitDepth.Bit12, + requireIntraBlockCopy: true); + } + /// /// Validates every lossless native fixture under the hardware configuration selected by /// . @@ -1010,6 +1126,7 @@ public class Av1ReconstructionConformanceTests /// The expected public HEIF sample precision. /// Whether the AV1 item must upscale from a narrower coded frame. /// Whether the AV1 item must select palette prediction for luma and chroma. + /// Whether the AV1 item must select intra-block-copy prediction. private static void ValidatePresentedFixture( string imagePath, string referencePath, @@ -1017,7 +1134,8 @@ public class Av1ReconstructionConformanceTests int height, HeifBitDepth metadataBitDepth, bool requireSuperResolution = false, - bool requirePalette = false) + bool requirePalette = false, + bool requireIntraBlockCopy = false) { DecoderOptions options = new() { MaxFrames = 1 }; byte[] imageBytes = TestFile.Create(imagePath).Bytes; @@ -1033,6 +1151,11 @@ public class Av1ReconstructionConformanceTests AssertUsesPalette(imageBytes); } + if (requireIntraBlockCopy) + { + AssertUsesIntraBlockCopy(imageBytes); + } + using Image image = Image.Load(options, imageBytes); using Image reference = Image.Load(referenceBytes); @@ -1074,6 +1197,21 @@ public class Av1ReconstructionConformanceTests Assert.Equal(RequiredPaletteCoverage, GetPaletteCoverage(decoder)); } + /// + /// Verifies that the sole AV1 image item in an independently encoded AVIF selects intra-block-copy prediction. + /// + /// The complete AVIF file. + private static void AssertUsesIntraBlockCopy(Span imageBytes) + { + Span payload = GetSoleAv1ItemPayload(imageBytes); + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _); + + Assert.NotNull(decoder.FrameHeader); + Assert.True(decoder.FrameHeader.AllowIntraBlockCopy); + Assert.NotEqual(0, GetIntraBlockCopyBlockCount(decoder)); + } + /// /// Decodes the sole image item in an independently generated AVIF fixture and returns its restoration coverage. /// @@ -1195,6 +1333,40 @@ public class Av1ReconstructionConformanceTests return paletteCoverage; } + /// + /// Counts the final coding blocks that select intra-block-copy prediction. + /// + /// The decoder after tile parsing and reconstruction. + /// The number of selected intra-block-copy coding blocks. + private static int GetIntraBlockCopyBlockCount(Av1Decoder decoder) + { + Assert.NotNull(decoder.SequenceHeader); + Assert.NotNull(decoder.FrameInfo); + int superblockSizeLog2 = decoder.SequenceHeader.SuperblockSizeLog2; + int superblockColumnCount = Av1Math.AlignPowerOf2(decoder.SequenceHeader.MaxFrameWidth, superblockSizeLog2) >> superblockSizeLog2; + int superblockRowCount = Av1Math.AlignPowerOf2(decoder.SequenceHeader.MaxFrameHeight, superblockSizeLog2) >> superblockSizeLog2; + int blockCount = 0; + + // Mode records retain final coding blocks in bitstream order. Traversing each record once counts selected + // intra-block-copy operations without repeatedly visiting the 4x4 cells covered by a larger block. + for (int superblockRow = 0; superblockRow < superblockRowCount; superblockRow++) + { + for (int superblockColumn = 0; superblockColumn < superblockColumnCount; superblockColumn++) + { + Av1SuperblockInfo superblock = decoder.FrameInfo.GetSuperblock(new Point(superblockColumn, superblockRow)); + foreach (Av1BlockModeInfo modeInfo in superblock.GetModeInfos()) + { + if (modeInfo.UseIntraBlockCopy) + { + blockCount++; + } + } + } + } + + return blockCount; + } + /// /// Returns the restoration algorithms selected by the decoded frame's unit grids. /// diff --git a/tests/ImageSharp.Tests/TestImages.cs b/tests/ImageSharp.Tests/TestImages.cs index 273eb20dd7..1895b7ef15 100644 --- a/tests/ImageSharp.Tests/TestImages.cs +++ b/tests/ImageSharp.Tests/TestImages.cs @@ -1369,6 +1369,15 @@ public static class TestImages public const string Av1Palette8BitReference = "Heif/Av1/Conformance/libaom-palette-draw-points-8b-444-libaom.yuv"; public const string Av1Palette8BitAvif = "Heif/Av1/Conformance/libavif-palette-draw-points-8b.avif"; public const string Av1Palette8BitPresentationReference = "Heif/Av1/Conformance/libavif-palette-draw-points-8b.png"; + public const string Av1IntraBlockCopy8BitAvif = "Heif/Av1/Conformance/libavif-intrabc-abc-8b-444.avif"; + public const string Av1IntraBlockCopy8BitReference = "Heif/Av1/Conformance/libavif-intrabc-abc-8b-444-libaom-y4m.yuv"; + public const string Av1IntraBlockCopy8BitPresentationReference = "Heif/Av1/Conformance/libavif-intrabc-abc-8b-444.png"; + public const string Av1IntraBlockCopy10BitAvif = "Heif/Av1/Conformance/libavif-intrabc-abc-10b-444.avif"; + public const string Av1IntraBlockCopy10BitReference = "Heif/Av1/Conformance/libavif-intrabc-abc-10b-444-libaom-y4m.yuv"; + public const string Av1IntraBlockCopy10BitPresentationReference = "Heif/Av1/Conformance/libavif-intrabc-abc-10b-444.png"; + public const string Av1IntraBlockCopy12BitAvif = "Heif/Av1/Conformance/libavif-intrabc-abc-12b-444.avif"; + public const string Av1IntraBlockCopy12BitReference = "Heif/Av1/Conformance/libavif-intrabc-abc-12b-444-libaom-y4m.yuv"; + public const string Av1IntraBlockCopy12BitPresentationReference = "Heif/Av1/Conformance/libavif-intrabc-abc-12b-444.png"; public const string Av1Lossless8BitAvif = "Heif/Av1/Conformance/libavif-lossless-circle-8b-444.avif"; public const string Av1Lossless8BitReference = "Heif/Av1/Conformance/libavif-lossless-circle-8b-444-libaom.yuv"; public const string Av1Lossless8BitPresentationReference = "Heif/Av1/Conformance/libavif-lossless-circle-8b-444.png"; diff --git a/tests/Images/Input/Heif/Av1/Conformance/README.md b/tests/Images/Input/Heif/Av1/Conformance/README.md index b01dd46b07..98dfbee372 100644 --- a/tests/Images/Input/Heif/Av1/Conformance/README.md +++ b/tests/Images/Input/Heif/Av1/Conformance/README.md @@ -53,6 +53,16 @@ The palette fixture was encoded independently from ImageSharp using `tests/data/ `libaom-palette-draw-points-8b-444.bit` is the exact sole AV1 item extracted from `libavif-palette-draw-points-8b.avif`. The matching native YUV reference was decoded from that payload by the pinned scalar `aomdec --rawvideo` build. The presented PNG was decoded from the complete AVIF container by the pinned scalar `avifdec -j 1 -d 8` build. The tests require both palette planes to be selected, compare every native YUV sample exactly, and compare every presented RGBA byte exactly across the available vector widths and scalar fallback. No tolerance is used. +## Intra-block-copy coverage + +The intra-block-copy fixtures were encoded independently from ImageSharp using `tests/data/abc.png` from the pinned libavif revision. This real 512x256 screen-content image provides repeated glyph and background regions beyond AV1's required 256-pixel reconstruction delay. Each AVIF is opaque YUV 4:4:4 with palette prediction disabled, so selected screen-content reuse must traverse intra-block-copy syntax and prediction rather than palette reconstruction. + +The common pinned generic `avifenc` options were `-j 1 -s 0 -l --ignore-alpha -y 444 -a enable-palette=0 -a enable-intrabc=1 -a tune-content=screen`. The 8-bit fixture uses `--cicp 1/13/0`; the 10- and 12-bit fixtures add the matching `-d` value and use `--cicp 12/16/0`. The high-depth encodes promote the 8-bit source, so `-l` configures lossless codec quantization but does not claim reversible conversion back to the original 8-bit PNG. + +The matching Y4M files were decoded from the complete AVIF containers with the pinned generic `avifdec -j 1` build. Their retained headers record the 512x256 full-range YUV 4:4:4 layouts at 8, 10, and 12 bits, followed by one planar frame. The matching PNG files were decoded with `avifdec -j 1 -d 8`. The build uses `AOM_TARGET_CPU=generic` and `AVIF_LIBYUV=OFF`, so the native planes and presented pixels come from the pinned scalar libaom/libavif paths. + +Tests require the frame header to allow intra-block copy and at least one final coding block to select it. They then compare every native Y, U, and V sample and every presented RGBA byte exactly under normal hardware dispatch, with AVX-512 disabled, with AVX disabled, and with all hardware intrinsics disabled. Displacement-vector entropy, spatial reference derivation, legal reconstruction order, inter transform selection, and prediction must therefore agree with the independent decoder for all three supported bit depths. No tolerance is used. + ## Lossless coverage The lossless fixtures were encoded independently from ImageSharp using `tests/data/circle-trns-after-plte.png` from the pinned libavif revision. Alpha was intentionally ignored so the native references isolate color-plane reconstruction. The 8-bit input uses CICP 1/13/0; the 10- and 12-bit YUV 4:4:4 inputs use CICP 12/16/0. The material `avifenc` options were `-j 1 -s 0 -l --ignore-alpha -y 444 -a enable-palette=0 -a enable-intrabc=0`, together with the matching depth and CICP values. Disabling palette and intra-block copy ensures the exact result traverses ordinary prediction, coefficient decoding, inverse quantization, and the reversible lossless transform. diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444-libaom-y4m.yuv new file mode 100644 index 0000000000..6b158dc65a --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c6c5b5be6295bbe45a24480d3959fcb6f97bcec2b7d572361d93a6373c99c971 +size 786511 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444.avif new file mode 100644 index 0000000000..fc246fcb61 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8620d552eee66e4f99606a509cc6af1a7d365025493d483a7bde4fb4c03e5c14 +size 14778 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444.png b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444.png new file mode 100644 index 0000000000..18a7d748dc --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-10b-444.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1c8025d05ee94764db4dc52bf2b19a4b2a9478c7cb33bd5ad57ad5414417efe0 +size 6433 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444-libaom-y4m.yuv new file mode 100644 index 0000000000..ef8fb6146a --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:89368788da9af213ed16d4e9b9bcae4f4a9f8460497e19542e8aa84e781391ae +size 786511 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444.avif new file mode 100644 index 0000000000..4076167e71 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b5fc8307611313acd6b8df5fb91cf7a98199b3df1d88a1971e543472f6485fdd +size 20247 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444.png b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444.png new file mode 100644 index 0000000000..18a7d748dc --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-12b-444.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1c8025d05ee94764db4dc52bf2b19a4b2a9478c7cb33bd5ad57ad5414417efe0 +size 6433 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444-libaom-y4m.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444-libaom-y4m.yuv new file mode 100644 index 0000000000..d1fcc73af9 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444-libaom-y4m.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:938814c6efdfecb4d3c630787f6435208ab03fd96e1505a2fa190571ab2298fc +size 393289 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444.avif new file mode 100644 index 0000000000..612f45ddbe --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1fd33226d8c0fd011f7a39ff91e34eebb088fadd46c491c55110b6ae70ef7418 +size 10068 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444.png b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444.png new file mode 100644 index 0000000000..4f1d508cc6 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-intrabc-abc-8b-444.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4cd476ce81e54eed777e4218f9b0ad38cc509d60d1c38bb0ba2ccc013d4ee053 +size 6462