From 25295683d39a2336e9b98484c9fd54f33107ea66 Mon Sep 17 00:00:00 2001 From: James Jackson-South Date: Mon, 31 Aug 2026 06:06:39 +1000 Subject: [PATCH] Complete AV1 global motion prediction checkpoint --- HEIF_IMPLEMENTATION_PLAN.md | 45 ++++- .../Inter/Av1WarpedInterPredictor.cs | 160 ++++++++++++++++++ .../Heif/Av1/Transform/Av1BlockDecoder.cs | 76 ++++++--- .../Heif/Av1/Av1CompoundBlockDecoderTests.cs | 51 +++--- .../Av1/Av1GlobalMotionParametersTests.cs | 4 +- .../Av1/Av1ReconstructionConformanceTests.cs | 4 +- .../Heif/Av1/Av1WarpedInterPredictorTests.cs | 119 +++++++++++-- ...a32_libavif-rotating-grid-global-warp.png} | 0 .../Input/Heif/Av1/Conformance/README.md | 35 +++- 9 files changed, 420 insertions(+), 74 deletions(-) rename tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/{DecodeRealLibavifGlobalWarpSequenceMatchesPinnedReferences_Rgba32_libavif-rotating-grid-global-warp.png => DecodeRealLibavifGlobalWarpSequenceMatchesCurrentLibaomReferences_Rgba32_libavif-rotating-grid-global-warp.png} (100%) diff --git a/HEIF_IMPLEMENTATION_PLAN.md b/HEIF_IMPLEMENTATION_PLAN.md index 79330e2cc..e7cd65e81 100644 --- a/HEIF_IMPLEMENTATION_PLAN.md +++ b/HEIF_IMPLEMENTATION_PLAN.md @@ -30,8 +30,8 @@ Reference checkout evidence on 2026-08-31: Reconciled with the worktree on 2026-08-31. - [~] The bounded container reader, still-image path, sequence parser, AV1 decoder, color pipeline, presentation pipeline, and broad AV1 test suite exist locally. -- [~] The inter-frame decoder has verified checkpoints through local warped prediction. Global motion exists - locally but remains open until its ordered checkpoint below is completed. +- [~] The inter-frame decoder has verified checkpoints through non-translational global prediction. Inter + deblocking decisions and reference/mode deltas remain open as the next ordered checkpoint. - [~] Loop filtering, CDEF, super-resolution, restoration, film grain, layered presentation, alpha composition, and color conversion exist locally. Shared-source cleanup changed the current tree, so final production-path verification is open. - [~] AV1 writer primitives, forward transforms, symbol encoding, and tile-writing source exist locally, but they are not connected to the public encoder. - [ ] The public AV1 encoder is not implemented. HeifEncoderCore.Encode throws NotSupportedException when AV1 is selected. @@ -185,8 +185,8 @@ The single-reference syntax, buffer, reconstruction, and ownership foundation is - [x] OBMC. - [x] Scaled-reference prediction. - [x] Local warped prediction. -- [~] Non-translational global prediction. Next item. -- [~] Inter deblocking decisions and reference/mode deltas. +- [x] Non-translational global prediction. +- [~] Inter deblocking decisions and reference/mode deltas. Current item. Verified equal-average compound checkpoint evidence on 2026-08-31: @@ -465,6 +465,43 @@ Verified local warped-prediction checkpoint evidence on 2026-08-31: - [x] The focused Release checkpoint set passes 4/4 on net10.0 and 4/4 on net11.0, with zero failures or skips. Scoped analyzer verification passes for both changed C# files. Roslynk reports zero compiler errors, `git diff --check` passes, and `.gitattributes` is unchanged. +- [x] The completed checkpoint was committed as `27a522424fe7aaea25078e705d71a501da110727` + with author and committer `James Jackson-South `. + +Verified non-translational global-prediction checkpoint evidence on 2026-08-31: + +- [x] Audited global-motion syntax, coefficient decoding, previous-reference recentering, shear validation, + motion-vector projection, and warped-prediction eligibility against current official libaom `main` at the + observed revision `441c439b9916474cac15d2822af47a9ad70674a8`. The implementation matches + `read_global_motion_params`, `read_global_motion_model`, `gm_get_motion_vector`, `is_global_mv_block`, + and the WARP_PRED selection in `av1/common/reconinter.c`. +- [x] Corrected high-bit-depth compound warped/global prediction to retain both references in libaom's + unsigned no-round compound domain. Current `get_conv_params_no_round`, `av1_warp_plane`, and + `av1_highbd_warp_affine_c` require the 12-bit first-round adjustment while retaining a seven-bit second + round; native clipping now occurs only after the compound blend. +- [x] The independent scalar libaom transcription validates native and no-round compound output for byte, + 8-bit, 10-bit, and 12-bit sources, including tail widths and destination-stride preservation. All cases pass + through AVX-512, AVX, 128-bit, and scalar dispatch with `FeatureTestRunner`. Direct + `Av1BlockDecoder.DecodeBlock()` coverage validates `GLOBAL_GLOBALMV` compound reconstruction at all + supported bit depths. +- [x] Extracted the fixture's exact 38,475-byte AV1 `mdat` payload at AVIF offset 997. Its SHA-256 is + `6AC7EC9984B1FF5C00403D7E3858441E9CEE75128F7414101D06DEEE59A351D0`. Current + official libaom decoded both 256x256 YUV444 frames with one thread, row threading disabled, and all layers + enabled. The complete Y4M SHA-256 is + `84754DE0B9FABC4F3F8F344C848183EC17B625BFD87E4519C3D8AD7DEFD20F2C`; the final + frame's 196,608 native samples have SHA-256 + `FEC89E2DE7496980389806B194425042F3800C7BAA817249D1A51D44A2B37A8E` and match the + retained native reference with zero differences. +- [x] The real two-frame fixture exercises the production decoder, requires decoded non-translational global + motion, compares final native Y, U, and V planes exactly, compares the retained presentation through + ImageSharp's established reference-output API, and passes the constrained tracked-allocator path. +- [x] Renamed the stale pinned-reference test and its contract-derived PNG together without changing the PNG + bytes. Its SHA-256 remains + `F7D27ABF79450DFA311F72106FD1DA80997EABC0937F2F5578EF627119FF83B0`, and Git + attributes select the LFS filter and diff driver. +- [x] The focused Release checkpoint set passes 11/11 on net10.0 and 11/11 on net11.0, with zero failures or + skips. Scoped analyzer verification passes for all six changed C# files. Roslynk reports zero compiler + errors, `git diff --check` passes, and `.gitattributes` is unchanged. For every item: diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WarpedInterPredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WarpedInterPredictor.cs index 7848f8b2c..5d3aa45b5 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WarpedInterPredictor.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WarpedInterPredictor.cs @@ -147,6 +147,43 @@ internal static partial class Av1WarpedInterPredictor scratch, useHardwareIntrinsics: true); + /// + /// Reconstructs a high-bit-depth affine warped reference into AV1's unsigned compound intermediate format. + /// + public static void PredictWarpedCompound( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + int bitDepth, + Av1GlobalMotionParameters parameters, + Span scratch) + => PredictWarpedCompound( + source, + sourceStride, + sourceOrigin, + sourceWidth, + sourceHeight, + destination, + destinationStride, + destinationPosition, + width, + height, + subsamplingX, + subsamplingY, + bitDepth, + parameters, + scratch, + useHardwareIntrinsics: true); + /// /// Reconstructs an 8-bit affine warped prediction without explicit hardware intrinsics. /// @@ -254,6 +291,43 @@ internal static partial class Av1WarpedInterPredictor scratch, useHardwareIntrinsics: false); + /// + /// Reconstructs a high-bit-depth affine warped reference into compound intermediates without explicit hardware intrinsics. + /// + public static void PredictWarpedCompoundScalar( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + int bitDepth, + Av1GlobalMotionParameters parameters, + Span scratch) + => PredictWarpedCompound( + source, + sourceStride, + sourceOrigin, + sourceWidth, + sourceHeight, + destination, + destinationStride, + destinationPosition, + width, + height, + subsamplingX, + subsamplingY, + bitDepth, + parameters, + scratch, + useHardwareIntrinsics: false); + /// /// Reconstructs one 8-bit warped block through a closed convolution operator. /// @@ -912,6 +986,92 @@ internal static partial class Av1WarpedInterPredictor } } + /// + /// Reconstructs one high-bit-depth warped reference without discarding the compound convolution precision. + /// + private static void PredictWarpedCompound( + ReadOnlySpan source, + int sourceStride, + Point sourceOrigin, + int sourceWidth, + int sourceHeight, + Span destination, + int destinationStride, + Point destinationPosition, + int width, + int height, + int subsamplingX, + int subsamplingY, + int bitDepth, + Av1GlobalMotionParameters parameters, + Span scratch, + bool useHardwareIntrinsics) + where TOperator : struct, IAv1WarpedPredictionOperator + { + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + Span intermediate = MemoryMarshal.Cast(scratch)[..WarpedScratchLength]; + + // Twelve-bit input raises round0 so every biased horizontal sample fits in the shared 16-bit scratch tile. + // Compound prediction keeps round1 at seven; its final blend removes the remaining two normative bits. + int intermediateRange = bitDepth + FilterBits - Round0Bits + 2; + int round0 = Round0Bits + Math.Max(intermediateRange - 16, 0); + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + int verticalBias = 1 << (bitDepth + (2 * FilterBits) - round0); + + for (int tileRow = destinationPosition.Y; tileRow < destinationPosition.Y + height; tileRow += WarpedTileSize) + { + for (int tileColumn = destinationPosition.X; tileColumn < destinationPosition.X + width; tileColumn += WarpedTileSize) + { + DeriveWarpedTilePosition( + parameters, + tileColumn, + tileRow, + subsamplingX, + subsamplingY, + out int integerX, + out int integerY, + out int phaseX, + out int phaseY); + + FilterWarpedHorizontal( + ref sourceBase, + sourceStride, + sourceOrigin, + sourceHeight, + integerX, + integerY, + phaseX, + parameters, + intermediate, + horizontalBias, + round0, + useHardwareIntrinsics); + + int tileHeight = Math.Min(WarpedTileSize, destinationPosition.Y + height - tileRow); + int tileWidth = Math.Min(WarpedTileSize, destinationPosition.X + width - tileColumn); + for (int row = 0; row < tileHeight; row++) + { + int phase = phaseY + (parameters.Delta * row); + int destinationRowOffset = (tileRow - destinationPosition.Y + row) * destinationStride; + ref ushort intermediateSource = ref intermediate[row * WarpedTileSize]; + ref ushort destinationRow = ref Unsafe.Add( + ref destinationBase, + destinationRowOffset + tileColumn - destinationPosition.X); + + FilterWarpedCompoundVertical( + ref intermediateSource, + ref destinationRow, + tileWidth, + phase, + parameters.Gamma, + verticalBias, + useHardwareIntrinsics); + } + } + } + } + /// /// Reconstructs one high-bit-depth warped block through a closed convolution operator. /// diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs index 95ed03966..ba3f2ab66 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs @@ -363,17 +363,15 @@ internal sealed class Av1BlockDecoder : IDisposable int referenceCount = usesSub8x8ChromaPrediction ? 0 : isCompound ? 2 : 1; - // Compound convolution is combined before its final rounding step. Scaled and ordinary translational - // predictors share that no-round domain; high-bit-depth warped/global models retain their own kernels. + // Every compound predictor is combined before its final rounding step. Warped prediction has its own + // convolution kernels, but current libaom writes their output into the same unsigned no-round domain. bool useHighBitDepthCompoundIntermediates = highBitDepth && - modeInfo.CompoundType is + modeInfo.CompoundType is ( Av1CompoundType.Average or Av1CompoundType.DistanceWeighted or Av1CompoundType.Wedge or - Av1CompoundType.DifferenceWeighted && - modeInfo.MotionMode != Av1MotionMode.Warped && - modeInfo.YMode != Av1PredictionMode.GlobalGlobalMotionVector; + Av1CompoundType.DifferenceWeighted); bool useCompoundIntermediates = isCompound && @@ -445,27 +443,53 @@ internal sealed class Av1BlockDecoder : IDisposable out int sourceStride, out Point sourceOrigin); - Span destination = MemoryMarshal.Cast( - referenceIndex == 0 - ? highBitDepthBlockReconstructionBuffer[reconstructionStride..] - : secondPredictionStorage); + if (useCompoundIntermediates) + { + Span destination = referenceIndex == 0 + ? firstCompoundPrediction + : highBitDepthSecondPrediction; - Av1WarpedInterPredictor.PredictWarped( - source, - sourceStride, - sourceOrigin, - referencePlaneWidth, - referencePlaneHeight, - destination, - destinationStride, - pixelPosition, - predictionWidth, - predictionHeight, - subX, - subY, - this.frameBuffer.BitDepth.GetBitCount(), - warpedMotionParameters, - predictionScratch); + Av1WarpedInterPredictor.PredictWarpedCompound( + source, + sourceStride, + sourceOrigin, + referencePlaneWidth, + referencePlaneHeight, + destination, + destinationStride, + pixelPosition, + predictionWidth, + predictionHeight, + subX, + subY, + this.frameBuffer.BitDepth.GetBitCount(), + warpedMotionParameters, + predictionScratch); + } + else + { + Span destination = MemoryMarshal.Cast( + referenceIndex == 0 + ? highBitDepthBlockReconstructionBuffer[reconstructionStride..] + : secondPredictionStorage); + + Av1WarpedInterPredictor.PredictWarped( + source, + sourceStride, + sourceOrigin, + referencePlaneWidth, + referencePlaneHeight, + destination, + destinationStride, + pixelPosition, + predictionWidth, + predictionHeight, + subX, + subY, + this.frameBuffer.BitDepth.GetBitCount(), + warpedMotionParameters, + predictionScratch); + } } else { diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs index 6c87e95ed..80108b1d1 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs @@ -1271,7 +1271,7 @@ public class Av1CompoundBlockDecoderTests } /// - /// Reconstructs one compound global-warp block and compares it with independently invoked scalar predictors. + /// Reconstructs one compound global-warp block and compares it with direct scalar compound predictors. /// /// The native sample depth. private static void ValidateCompoundGlobalWarpPredictionAtBitDepth(Av1BitDepth bitDepth) @@ -1279,8 +1279,20 @@ public class Av1CompoundBlockDecoderTests const int frameSize = 32; const int blockOrigin = 8; const int blockSize = 8; - const int compoundRoundBits = 4; - const int compoundRoundOffset = (1 << 12) + (1 << 11); + int bitDepthValue = bitDepth.GetBitCount(); + int intermediateRange = bitDepthValue + Av1InterPredictor.FilterBits - Av1InterPredictor.Round0Bits + 2; + int round0 = Av1InterPredictor.Round0Bits + Math.Max(intermediateRange - 16, 0); + int compoundRoundBits = + (2 * Av1InterPredictor.FilterBits) - round0 - Av1CompoundInterPredictor.CompoundRound1Bits; + + int compoundOffsetBits = + bitDepthValue + + (2 * Av1InterPredictor.FilterBits) - + round0 - + Av1CompoundInterPredictor.CompoundRound1Bits; + + int compoundRoundOffset = (1 << compoundOffsetBits) + (1 << (compoundOffsetBits - 1)); + int maximum = (1 << bitDepthValue) - 1; ObuSequenceHeader sequenceHeader = CreateSequenceHeader(bitDepth, frameSize); ObuFrameHeader frameHeader = CreateFrameHeader(frameSize); frameHeader.GetReferenceFrameIndices()[0] = 0; @@ -1381,8 +1393,7 @@ public class Av1CompoundBlockDecoderTests out int secondStride, out Point secondOrigin); - int bitDepthValue = bitDepth.GetBitCount(); - Av1WarpedInterPredictor.PredictWarpedScalar( + Av1WarpedInterPredictor.PredictWarpedCompoundScalar( firstSource, firstStride, firstOrigin, @@ -1399,7 +1410,7 @@ public class Av1CompoundBlockDecoderTests globalMotionParameters, firstScratch); - Av1WarpedInterPredictor.PredictWarpedScalar( + Av1WarpedInterPredictor.PredictWarpedCompoundScalar( secondSource, secondStride, secondOrigin, @@ -1463,28 +1474,26 @@ public class Av1CompoundBlockDecoderTests for (int column = 0; column < blockSize; column++) { int predictionIndex = (row * blockSize) + column; - if (bitDepth == Av1BitDepth.EightBit) - { - // Libaom truncates the equal average before removing the Q4 compound bias, then performs the - // sole final rounding step. Averaging two already reconstructed pixels can differ by one. - int intermediate = ((firstHighBitDepthPrediction[predictionIndex] + - secondHighBitDepthPrediction[predictionIndex]) >> 1) - compoundRoundOffset; - byte expected = (byte)Math.Clamp( - (intermediate + (1 << (compoundRoundBits - 1))) >> compoundRoundBits, - 0, - byte.MaxValue); + // Libaom truncates the equal average before removing the compound bias, then performs the sole final + // rounding step. Reconstructing each reference to native pixels first can differ from this result. + int intermediate = ((firstHighBitDepthPrediction[predictionIndex] + + secondHighBitDepthPrediction[predictionIndex]) >> 1) - compoundRoundOffset; + int expected = Math.Clamp( + (intermediate + (1 << (compoundRoundBits - 1))) >> compoundRoundBits, + 0, + maximum); + + if (bitDepth == Av1BitDepth.EightBit) + { Span samples = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(blockOrigin + row); - Assert.Equal(expected, samples[blockOrigin + column]); + Assert.Equal((byte)expected, samples[blockOrigin + column]); } else { - ushort expected = (ushort)((firstHighBitDepthPrediction[predictionIndex] + - secondHighBitDepthPrediction[predictionIndex] + 1) >> 1); - Span samples = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, blockOrigin + row, 0, 0); - Assert.Equal(expected, samples[blockOrigin + column]); + Assert.Equal((ushort)expected, samples[blockOrigin + column]); } } } diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1GlobalMotionParametersTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1GlobalMotionParametersTests.cs index af4f9d49d..d6a5ac615 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1GlobalMotionParametersTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1GlobalMotionParametersTests.cs @@ -84,10 +84,10 @@ public class Av1GlobalMotionParametersTests } /// - /// Verifies local least-squares projection against a multi-sample model traced from pinned libaom. + /// Verifies local least-squares projection against a multi-sample model traced from current libaom. /// [Fact] - public void LocalProjectionMatchesPinnedLibaomMultiSampleModel() + public void LocalProjectionMatchesCurrentLibaomMultiSampleModel() { Point[] sourcePoints = [new(24, -40), new(-40, 24), new(-24, -24), new(72, -24)]; Point[] referencePoints = [new(-16, -8), new(-72, 64), new(-64, 16), new(32, 8)]; diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs index e7d9ac9dd..20cf55e73 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs @@ -1120,11 +1120,11 @@ public class Av1ReconstructionConformanceTests } /// - /// Verifies production non-translational global-motion reconstruction against pinned native and presentation references. + /// Verifies production non-translational global motion against current-libaom native and retained presentation references. /// [Theory] [WithFile(TestImages.Heif.Av1GlobalWarpSequenceAvif, PixelTypes.Rgba32)] - public void DecodeRealLibavifGlobalWarpSequenceMatchesPinnedReferences(TestImageProvider provider) + public void DecodeRealLibavifGlobalWarpSequenceMatchesCurrentLibaomReferences(TestImageProvider provider) => FeatureTestRunner.RunWithHwIntrinsicsFeature( ValidateGlobalWarpSequenceWithDefaultConfiguration, ReconstructionConfigurations, diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1WarpedInterPredictorTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1WarpedInterPredictorTests.cs index 5ebe40bac..ae190e285 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1WarpedInterPredictorTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1WarpedInterPredictorTests.cs @@ -21,17 +21,17 @@ public class Av1WarpedInterPredictorTests HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; /// - /// Verifies exact 8-bit prediction against the current libaom scalar equations. + /// Verifies exact 8-bit native and compound prediction against the current libaom scalar equations. /// [Fact] - public void BytePredictionMatchesCurrentLibaomOracleAcrossIntrinsicConfigurations() + public void ByteNativeAndCompoundPredictionMatchesCurrentLibaomOracleAcrossIntrinsicConfigurations() => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateBytePrediction, PredictorConfigurations); /// - /// Verifies exact 8-, 10-, and 12-bit prediction against the current libaom scalar equations. + /// Verifies exact 8-, 10-, and 12-bit native and compound prediction against the current libaom scalar equations. /// [Fact] - public void HighBitDepthPredictionMatchesCurrentLibaomOracleAcrossIntrinsicConfigurations() + public void HighBitDepthNativeAndCompoundPredictionMatchesCurrentLibaomOracleAcrossIntrinsicConfigurations() => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateHighBitDepthPrediction, PredictorConfigurations); /// @@ -78,7 +78,8 @@ public class Av1WarpedInterPredictorTests subsampling, subsampling, 8, - parameters); + parameters, + compound: false); Av1WarpedInterPredictor.PredictWarped( source, @@ -97,6 +98,46 @@ public class Av1WarpedInterPredictorTests actualScratch); Assert.Equal(expected, actual); + + ushort[] expectedCompound = new ushort[destinationStride * height]; + ushort[] actualCompound = new ushort[destinationStride * height]; + Array.Fill(expectedCompound, (ushort)0xDEAD); + Array.Fill(actualCompound, (ushort)0xDEAD); + + PredictCurrentLibaomReference( + source, + sourceStride, + new Point(padding, padding), + activeSize, + activeSize, + expectedCompound, + destinationStride, + destinationPosition, + width, + height, + subsampling, + subsampling, + 8, + parameters, + compound: true); + + Av1WarpedInterPredictor.PredictWarpedCompound( + source, + sourceStride, + new Point(padding, padding), + activeSize, + activeSize, + actualCompound, + destinationStride, + destinationPosition, + width, + height, + subsampling, + subsampling, + parameters, + actualScratch); + + Assert.Equal(expectedCompound, actualCompound); } } @@ -148,7 +189,8 @@ public class Av1WarpedInterPredictorTests subsampling, subsampling, bitDepth, - parameters); + parameters, + compound: false); Av1WarpedInterPredictor.PredictWarped( source, @@ -168,6 +210,47 @@ public class Av1WarpedInterPredictorTests actualScratch); Assert.Equal(expected, actual); + + ushort[] expectedCompound = new ushort[destinationStride * height]; + ushort[] actualCompound = new ushort[destinationStride * height]; + Array.Fill(expectedCompound, (ushort)0xDEAD); + Array.Fill(actualCompound, (ushort)0xDEAD); + + PredictCurrentLibaomReference( + source, + sourceStride, + new Point(padding, padding), + activeSize, + activeSize, + expectedCompound, + destinationStride, + destinationPosition, + width, + height, + subsampling, + subsampling, + bitDepth, + parameters, + compound: true); + + Av1WarpedInterPredictor.PredictWarpedCompound( + source, + sourceStride, + new Point(padding, padding), + activeSize, + activeSize, + actualCompound, + destinationStride, + destinationPosition, + width, + height, + subsampling, + subsampling, + bitDepth, + parameters, + actualScratch); + + Assert.Equal(expectedCompound, actualCompound); } } } @@ -175,13 +258,13 @@ public class Av1WarpedInterPredictorTests /// /// Reconstructs one warped block by directly transcribing current libaom's scalar affine loops. /// - private static void PredictCurrentLibaomReference( - ReadOnlySpan source, + private static void PredictCurrentLibaomReference( + ReadOnlySpan source, int sourceStride, Point sourceOrigin, int sourceWidth, int sourceHeight, - Span destination, + Span destination, int destinationStride, Point destinationPosition, int width, @@ -189,8 +272,10 @@ public class Av1WarpedInterPredictorTests int subsamplingX, int subsamplingY, int bitDepth, - Av1GlobalMotionParameters parameters) - where TPixel : unmanaged, IBinaryInteger + Av1GlobalMotionParameters parameters, + bool compound) + where TSource : unmanaged, IBinaryInteger + where TDestination : unmanaged, IBinaryInteger { const int filterBits = 7; const int filterTaps = 8; @@ -204,7 +289,7 @@ public class Av1WarpedInterPredictorTests // remains representable in 16 bits, then removes those two bits from the vertical rounding. int intermediateRange = bitDepth + filterBits - 3 + 2; int round0 = 3 + Math.Max(intermediateRange - 16, 0); - int verticalRound = (2 * filterBits) - round0; + int verticalRound = compound ? 7 : (2 * filterBits) - round0; int horizontalBias = 1 << (bitDepth + filterBits - 1); int verticalBias = 1 << (bitDepth + (2 * filterBits) - round0); int maximum = (1 << bitDepth) - 1; @@ -273,7 +358,13 @@ public class Av1WarpedInterPredictorTests sum += intermediate[intermediateIndex] * CurrentLibaomWarpedFilter[coefficientOffset + tap]; } - int value = RoundPowerOfTwo(sum, verticalRound) - (1 << (bitDepth - 1)) - (1 << bitDepth); + int value = RoundPowerOfTwo(sum, verticalRound); + if (!compound) + { + value -= (1 << (bitDepth - 1)) + (1 << bitDepth); + value = Math.Clamp(value, 0, maximum); + } + int destinationIndex = ((tileRow - destinationPosition.Y + row + 4) * destinationStride) + tileColumn - @@ -281,7 +372,7 @@ public class Av1WarpedInterPredictorTests column + 4; - destination[destinationIndex] = TPixel.CreateChecked(Math.Clamp(value, 0, maximum)); + destination[destinationIndex] = TDestination.CreateChecked(value); phase += parameters.Gamma; } } diff --git a/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifGlobalWarpSequenceMatchesPinnedReferences_Rgba32_libavif-rotating-grid-global-warp.png b/tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifGlobalWarpSequenceMatchesCurrentLibaomReferences_Rgba32_libavif-rotating-grid-global-warp.png similarity index 100% rename from tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifGlobalWarpSequenceMatchesPinnedReferences_Rgba32_libavif-rotating-grid-global-warp.png rename to tests/Images/External/ReferenceOutput/Av1ReconstructionConformanceTests/DecodeRealLibavifGlobalWarpSequenceMatchesCurrentLibaomReferences_Rgba32_libavif-rotating-grid-global-warp.png diff --git a/tests/Images/Input/Heif/Av1/Conformance/README.md b/tests/Images/Input/Heif/Av1/Conformance/README.md index 49167989c..352ca28e6 100644 --- a/tests/Images/Input/Heif/Av1/Conformance/README.md +++ b/tests/Images/Input/Heif/Av1/Conformance/README.md @@ -20,7 +20,7 @@ The reference builds use `AOM_TARGET_CPU=generic` and disable libyuv. Native rec - `-libaom.yuv` files contain headerless planar Y, U, and V reference samples. Samples above eight bits are stored as little-endian 16-bit values. - `-libaom-y4m.yuv` files retain the Y4M header together with the native planar frame. - `-libaom.y4m` files retain the Y4M header together with the native sequence frames selected for comparison. -- Reference-output `.png` files contain the eight-bit RGBA presentation produced by the pinned scalar libavif build. Their names combine the public test method, `Rgba32`, and the input AVIF basename so `CompareToReferenceOutput` resolves them directly. +- Reference-output `.png` files contain retained eight-bit RGBA presentations. Their names combine the public test method, `Rgba32`, and the input AVIF basename so `CompareToReferenceOutput` resolves them directly. ## Coverage @@ -265,15 +265,40 @@ AVX, 128-bit, and scalar execution. ## Global warped-motion fixture -The `libavif-rotating-grid-global-warp.avif` fixture uses the same deterministic two-frame 256x256 limited-range YUV444 source and its `82C1468C95C996B05165590417184F59373D67896F8C7B0EB398582C29C9C7A7` SHA-256. Pinned libavif commit `062e582e8afda88e6baf988fdcf046a801efa0f5` and libaom commit `03087864cf4bea6abb0d28f95cf7843511413d8f` generated the fixture and references with: +The `libavif-rotating-grid-global-warp.avif` file is retained solely as interoperability input. It uses +the same deterministic two-frame 256x256 limited-range YUV444 source as the local-warp fixture. The original +packaging command records fixture provenance only; libavif is not used as an AV1 implementation or +reconstruction reference: ```text ./avifenc -j 1 -s 0 -q 100 -a color:enable-warped-motion=0 -a color:enable-global-motion=1 -a color:enable-obmc=0 rotating-grid-256-two-frame.y4m libavif-rotating-grid-global-warp.avif -./avifdec -j 1 --index 1 libavif-rotating-grid-global-warp.avif libavif-rotating-grid-global-warp-libaom.y4m -./avifdec -j 1 --index 1 libavif-rotating-grid-global-warp.avif libavif-rotating-grid-global-warp-libavif.png ``` -The AVIF SHA-256 is `EE8CDF6DF36FB2999A17D2A86C41040D8BE13B958A1D5AE53E37343C6FB49E0E`. The retained frame-1 Y4M SHA-256 is `A36D445778BB2D37D69526A1058E3569DA24F3913317EFE22ADB61748A0F7511`, and the frame-1 PNG SHA-256 is `F7D27ABF79450DFA311F72106FD1DA80997EABC0937F2F5578EF627119FF83B0`. Pinned libaom tracing records seven actual `GLOBALMV` blocks using the valid rotation/zoom matrix `[-357376, 372736, 65468, 2856, -2856, 65468]` and reduced shear `[-64, 2880, -2880, 64]`. The production sequence test requires that decoded model and mode state, compares every final native Y, U, and V sample and the final RGBA presentation exactly, runs normal and scalar dispatch through `FeatureTestRunner`, and repeats reconstruction with constrained tracked allocation. A direct production-branch test independently drives both references of `GLOBAL_GLOBALMV` through the matrix predictor and compound averaging at 8, 10, and 12 bits. +On 2026-08-31 the clean official libaom `main` checkout was refreshed from its upstream remote. At the +observed revision `441c439b9916474cac15d2822af47a9ad70674a8`, the fixture's 38,475-byte AV1 +`mdat` payload at AVIF offset 997 has SHA-256 +`6AC7EC9984B1FF5C00403D7E3858441E9CEE75128F7414101D06DEEE59A351D0` and was decoded with: + +```text +aomdec --codec=av1 --threads=1 --row-mt=0 --all-layers --output-bit-depth=8 -o global-warp-current-main.y4m global-warp-current-main.obu +``` + +Current libaom produced two 256x256 YUV444 frames. The complete Y4M has SHA-256 +`84754DE0B9FABC4F3F8F344C848183EC17B625BFD87E4519C3D8AD7DEFD20F2C`. Its final +frame's 196,608 native samples have SHA-256 +`FEC89E2DE7496980389806B194425042F3800C7BAA817249D1A51D44A2B37A8E` and match the +retained `libavif-rotating-grid-global-warp-libaom.y4m` reference with zero differing samples. + +The decoded stream contains non-translational `GLOBALMV` blocks using rotation/zoom matrix +`[-357376, 372736, 65468, 2856, -2856, 65468]` and reduced shear +`[-64, 2880, -2880, 64]`. The production sequence test requires that decoded model and mode state, +compares every final native Y, U, and V sample exactly, compares the retained final presentation through +the established image-reference API, runs intrinsic and scalar dispatch through `FeatureTestRunner`, and +repeats reconstruction with constrained tracked allocation. A direct `Av1BlockDecoder.DecodeBlock()` +test drives both references of `GLOBAL_GLOBALMV` through matrix prediction and no-round compound averaging +at 8, 10, and 12 bits. High-bit-depth warped references remain in current libaom's unsigned +`CONV_BUF_TYPE` domain until the final blend; 12-bit prediction uses the corresponding adjusted round0 and +two-bit final rounding. ## Updating fixtures