diff --git a/HEIF_IMPLEMENTATION_PLAN.md b/HEIF_IMPLEMENTATION_PLAN.md index 2c45cab31..8f2a955f0 100644 --- a/HEIF_IMPLEMENTATION_PLAN.md +++ b/HEIF_IMPLEMENTATION_PLAN.md @@ -30,13 +30,13 @@ Reference checkout evidence on 2026-08-31: Reconciled with the worktree on 2026-08-31. - [~] The bounded container reader, still-image path, sequence parser, AV1 decoder, color pipeline, presentation pipeline, and broad AV1 test suite exist locally. -- [~] The inter-frame decoder contains implementations for single-reference prediction, compound references, inter-intra prediction, selectable compound blending, OBMC, scaled references, local warped motion, and global motion. These downstream paths must not be called verified until the single-reference checkpoint below is corrected and rerun. +- [~] The inter-frame decoder has verified checkpoints through wedge compound prediction. Difference-weighted compound prediction, OBMC, scaled references, local warped motion, and global motion exist locally but remain open until their ordered checkpoints below are completed. - [~] Loop filtering, CDEF, super-resolution, restoration, film grain, layered presentation, alpha composition, and color conversion exist locally. Shared-source cleanup changed the current tree, so final production-path verification is open. - [~] AV1 writer primitives, forward transforms, symbol encoding, and tile-writing source exist locally, but they are not connected to the public encoder. - [ ] The public AV1 encoder is not implemented. HeifEncoderCore.Encode throws NotSupportedException when AV1 is selected. - [x] Patented codec production code, registrations, tests, benchmarks, fixtures, reference outputs, and notices were manually deleted and committed by `78a74d448`. - [x] Remaining task-created HM, HEVC, libheif, GPAC, Nokia, FFmpeg, Pillow HEIF, libavif-build, and libjpeg-build directories were traced to their creation commands in the recovered Codex session history and deleted on 2026-08-31. The user-provided repositories and all libaom-only source, build, and reference data were left untouched. -- [~] A PNG metadata-suppression fix and three HEIF/AV1 diagnostic-save call-site corrections are implemented in the current worktree. The exact 34 cases that failed in the net11.0 ARM CI job now pass in Release, but these changes are not yet committed and are not decoder or encoder completion evidence. +- [x] The PNG metadata-suppression fix and three HEIF/AV1 diagnostic-save call-site corrections passed the exact 34 net11.0 ARM CI cases and were committed with the single-reference checkpoint as `54bb6cbe59bd113058854a3ee31448cf61f462ca`. They are infrastructure evidence, not decoder or encoder completion evidence. - [ ] The complete decoder and encoder release matrix is not complete. ## Immediate execution queue @@ -50,10 +50,10 @@ Work must proceed in this order. Do not skip to a later item while an earlier ch - [x] Retain the official current-main libaom checkout and libaom-only build artifacts required for AV1 verification. - [x] Retain user-supplied AV1 fixtures and their recorded expected outputs. - [x] Audit production source, tests, benchmarks, assets, project files, notices, and documentation for stale removed-code references. -- [~] The prior cleanup tree built in Release with restore disabled, build servers disabled, and one MSBuild node. A fresh current-worktree build remains required after the local PNG cICP fix. -- [~] The prior focused AV1/container set passed, but it missed the net11.0 PNG cICP diagnostic-save failures. The exact 34 CI failures now pass after the local fix; the final current-worktree checkpoint set remains required. -- [~] Roslynk currently reports zero compiler errors for the local fix, and `git diff --check` passes. Scoped StyleCop and final current-worktree inspection remain required. -- [~] Record the final current-worktree cleanup and cICP evidence after the checkpoint verification completes. +- [x] The cleanup and cICP tree built in Release for net10.0 and net11.0 with restore disabled, build servers disabled, and one MSBuild node. +- [x] The exact 34 net11.0 ARM CI failures pass after the cICP correction, and the subsequent single-reference checkpoint set passes on net10.0 and net11.0. +- [x] Roslynk, scoped StyleCop, whitespace, and `git diff --check` accepted the cleanup and cICP checkpoint. +- [x] The cleanup and cICP evidence was recorded and committed with the single-reference checkpoint. Historical cleanup evidence from 2026-08-30, retained with its limitation: @@ -179,8 +179,8 @@ The single-reference syntax, buffer, reconstruction, and ownership foundation is - [x] Compound reference selection, paired reference-MV derivation, and equal averaging. - [x] Inter-intra prediction. - [x] Distance-weighted compound prediction. -- [~] Wedge compound prediction. Current item. -- [~] Difference-weighted compound prediction. +- [x] Wedge compound prediction. +- [~] Difference-weighted compound prediction. Current item. - [~] OBMC. - [~] Scaled-reference prediction. - [~] Local warped prediction. @@ -286,6 +286,42 @@ Verified distance-weighted compound checkpoint evidence on 2026-08-31: or skips. Scoped analyzer and whitespace verification pass for every changed C# file. Roslynk reports zero compiler errors and no diagnostics in the changed files, `git diff --check` passes, and `.gitattributes` is unchanged. +- [x] The completed checkpoint was committed as `7e2de7a2c25852acc374b17936a1a644464f77f3` + with author and committer `James Jackson-South `. + +Verified wedge compound checkpoint evidence on 2026-08-31: + +- [x] Audited mask generation against current libaom `tools/gen_wedge_masks_data.py` and + `av1/common/reconinter.c`, including the master prototypes, direction transforms, block-size + codebooks, sign flips, offsets, and luma/chroma mask sampling. ImageSharp's generated masks match + those definitions; only stale “pinned” documentation required correction. +- [x] Audited reconstruction against current libaom `aom_dsp/blend_a64_mask.c`. The high-bit-depth + d16 path applies the Q6 mask to both no-round intermediates before bias removal, the sole final + rounding step, and clipping. +- [x] Corrected the production high-bit-depth intermediate eligibility gate, which admitted only + equal-average blocks and made the distance-weighted and wedge no-round finalizers unreachable. + Average, distance-weighted, and wedge subpixel blocks now retain both intermediates until their + signaled finalizer; difference-weighted blending remains excluded for its next ordered checkpoint. +- [x] Added high-bit-depth traversal to the existing semantic mask-blend predictor and readonly + operator family with descending Vector512, Vector256, Vector128, and scalar dispatch. Unsigned + widening preserves the biased 12-bit intermediate range. No per-block, per-row, or per-scanline + allocation or copy was added. +- [x] Extended FeatureTestRunner coverage with an independent Q6 mask oracle across 10/12-bit copy, + horizontal, vertical, and separable subpixel prediction, widths 9, 17, 33, and 65, all mask weights + from 0 through 64, and row-padding sentinels. A complete `Av1BlockDecoder.DecodeBlock` regression + verifies the current-libaom 8x8 wedge mask and the production no-round branch. +- [x] Extracted the fixture's 5,374-byte AV1 `mdat` payload and decoded it with refreshed current + libaom `aomdec`, using one thread with row threading disabled. All 19 frames decoded. The final + 19,200 YUV444 samples have SHA-256 + `E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained native + reference with zero differing samples. +- [x] The real 19-frame production sequence requires both wedge-mask orientations, compares final + native Y, Cb, and Cr planes exactly, compares final RGBA presentation through ImageSharp's + established reference-output API, and repeats the complete decode with a 1,024-byte constrained + tracked allocator and exactly-once return checks. +- [x] The focused Release checkpoint set passes 35/35 on net10.0 and 35/35 on net11.0, with zero + failures or skips. Scoped analyzer and whitespace verification pass for every changed C# file. + Roslynk reports zero compiler errors, `git diff --check` passes, and `.gitattributes` is unchanged. For every item: diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.Operator.cs index 93be9b072..6ffd9e7f2 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.Operator.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.Operator.cs @@ -89,6 +89,99 @@ internal static partial class Av1CompoundIntermediateMaskBlendPredictor Vector512 alpha, int roundBits, int roundOffset); + + /// + /// Alpha-blends and finalizes one pair of high-bit-depth compound intermediate samples. + /// + /// The first compound intermediate. + /// The second compound intermediate. + /// The first-predictor weight in the AV1 mask range. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The reconstructed sample. + public static abstract ushort BlendHighBitDepth( + ushort first, + ushort second, + byte alpha, + int roundBits, + int roundOffset, + int maximum); + + /// + /// Alpha-blends and finalizes 128 bits of high-bit-depth compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The first-predictor weights in the AV1 mask range. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The lower reconstructed samples. + /// The upper reconstructed samples. + public static abstract void BlendHighBitDepth( + Vector128 first0, + Vector128 first1, + Vector128 second0, + Vector128 second1, + Vector128 alpha, + int roundBits, + int roundOffset, + int maximum, + out Vector128 result0, + out Vector128 result1); + + /// + /// Alpha-blends and finalizes 256 bits of high-bit-depth compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The first-predictor weights in the AV1 mask range. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The lower reconstructed samples. + /// The upper reconstructed samples. + public static abstract void BlendHighBitDepth( + Vector256 first0, + Vector256 first1, + Vector256 second0, + Vector256 second1, + Vector256 alpha, + int roundBits, + int roundOffset, + int maximum, + out Vector256 result0, + out Vector256 result1); + + /// + /// Alpha-blends and finalizes 512 bits of high-bit-depth compound intermediate samples. + /// + /// The lower first-predictor intermediates. + /// The upper first-predictor intermediates. + /// The lower second-predictor intermediates. + /// The upper second-predictor intermediates. + /// The first-predictor weights in the AV1 mask range. + /// The final reconstruction shift. + /// The compound intermediate bias. + /// The maximum reconstructed sample value. + /// The lower reconstructed samples. + /// The upper reconstructed samples. + public static abstract void BlendHighBitDepth( + Vector512 first0, + Vector512 first1, + Vector512 second0, + Vector512 second1, + Vector512 alpha, + int roundBits, + int roundOffset, + int maximum, + out Vector512 result0, + out Vector512 result1); } /// @@ -106,6 +199,21 @@ internal static partial class Av1CompoundIntermediateMaskBlendPredictor return (byte)Math.Clamp(RoundPowerOfTwo(result, roundBits), 0, byte.MaxValue); } + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static ushort BlendHighBitDepth( + ushort first, + ushort second, + byte alpha, + int roundBits, + int roundOffset, + int maximum) + { + int result = ((alpha * first) + ((MaximumMaskAlpha - alpha) * second)) >> MaskWeightBits; + result -= roundOffset; + return (ushort)Math.Clamp(RoundPowerOfTwo(result, roundBits), 0, maximum); + } + /// [MethodImpl(MethodImplOptions.AggressiveInlining)] public static Vector128 Blend( @@ -148,6 +256,99 @@ internal static partial class Av1CompoundIntermediateMaskBlendPredictor Blend(first0, second0, Vector512.WidenLower(alpha), roundBits, roundOffset), Blend(first1, second1, Vector512.WidenUpper(alpha), roundBits, roundOffset)); + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void BlendHighBitDepth( + Vector128 first0, + Vector128 first1, + Vector128 second0, + Vector128 second1, + Vector128 alpha, + int roundBits, + int roundOffset, + int maximum, + out Vector128 result0, + out Vector128 result1) + { + result0 = BlendHighBitDepth( + first0, + second0, + Vector128.WidenLower(alpha), + roundBits, + roundOffset, + maximum); + + result1 = BlendHighBitDepth( + first1, + second1, + Vector128.WidenUpper(alpha), + roundBits, + roundOffset, + maximum); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void BlendHighBitDepth( + Vector256 first0, + Vector256 first1, + Vector256 second0, + Vector256 second1, + Vector256 alpha, + int roundBits, + int roundOffset, + int maximum, + out Vector256 result0, + out Vector256 result1) + { + result0 = BlendHighBitDepth( + first0, + second0, + Vector256.WidenLower(alpha), + roundBits, + roundOffset, + maximum); + + result1 = BlendHighBitDepth( + first1, + second1, + Vector256.WidenUpper(alpha), + roundBits, + roundOffset, + maximum); + } + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void BlendHighBitDepth( + Vector512 first0, + Vector512 first1, + Vector512 second0, + Vector512 second1, + Vector512 alpha, + int roundBits, + int roundOffset, + int maximum, + out Vector512 result0, + out Vector512 result1) + { + result0 = BlendHighBitDepth( + first0, + second0, + Vector512.WidenLower(alpha), + roundBits, + roundOffset, + maximum); + + result1 = BlendHighBitDepth( + first1, + second1, + Vector512.WidenUpper(alpha), + roundBits, + roundOffset, + maximum); + } + /// /// Alpha-blends 128-bit lanes after widening every product to signed 32-bit precision. /// @@ -213,5 +414,98 @@ internal static partial class Av1CompoundIntermediateMaskBlendPredictor Vector512 upper = ((alphaUpper * firstUpper) + ((maximum - alphaUpper) * secondUpper)) >> MaskWeightBits; return Vector512.Narrow(FinalizeIntermediate(lower, roundBits, roundOffset), FinalizeIntermediate(upper, roundBits, roundOffset)).AsUInt16(); } + + /// + /// Alpha-blends 128-bit high-bit-depth lanes without narrowing the unsigned intermediate range. + /// + private static Vector128 BlendHighBitDepth( + Vector128 first, + Vector128 second, + Vector128 alpha, + int roundBits, + int roundOffset, + int maximum) + { + Vector128 firstLower = Vector128.WidenLower(first); + Vector128 firstUpper = Vector128.WidenUpper(first); + Vector128 secondLower = Vector128.WidenLower(second); + Vector128 secondUpper = Vector128.WidenUpper(second); + Vector128 alphaLower = Vector128.WidenLower(alpha); + Vector128 alphaUpper = Vector128.WidenUpper(alpha); + Vector128 maximumAlpha = Vector128.Create((uint)MaximumMaskAlpha); + Vector128 lower = + ((alphaLower * firstLower) + ((maximumAlpha - alphaLower) * secondLower)) >> MaskWeightBits; + + Vector128 upper = + ((alphaUpper * firstUpper) + ((maximumAlpha - alphaUpper) * secondUpper)) >> MaskWeightBits; + + return FinalizeHighBitDepthIntermediate( + Vector128.Narrow(lower, upper), + roundBits, + roundOffset, + maximum); + } + + /// + /// Alpha-blends 256-bit high-bit-depth lanes without narrowing the unsigned intermediate range. + /// + private static Vector256 BlendHighBitDepth( + Vector256 first, + Vector256 second, + Vector256 alpha, + int roundBits, + int roundOffset, + int maximum) + { + Vector256 firstLower = Vector256.WidenLower(first); + Vector256 firstUpper = Vector256.WidenUpper(first); + Vector256 secondLower = Vector256.WidenLower(second); + Vector256 secondUpper = Vector256.WidenUpper(second); + Vector256 alphaLower = Vector256.WidenLower(alpha); + Vector256 alphaUpper = Vector256.WidenUpper(alpha); + Vector256 maximumAlpha = Vector256.Create((uint)MaximumMaskAlpha); + Vector256 lower = + ((alphaLower * firstLower) + ((maximumAlpha - alphaLower) * secondLower)) >> MaskWeightBits; + + Vector256 upper = + ((alphaUpper * firstUpper) + ((maximumAlpha - alphaUpper) * secondUpper)) >> MaskWeightBits; + + return FinalizeHighBitDepthIntermediate( + Vector256.Narrow(lower, upper), + roundBits, + roundOffset, + maximum); + } + + /// + /// Alpha-blends 512-bit high-bit-depth lanes without narrowing the unsigned intermediate range. + /// + private static Vector512 BlendHighBitDepth( + Vector512 first, + Vector512 second, + Vector512 alpha, + int roundBits, + int roundOffset, + int maximum) + { + Vector512 firstLower = Vector512.WidenLower(first); + Vector512 firstUpper = Vector512.WidenUpper(first); + Vector512 secondLower = Vector512.WidenLower(second); + Vector512 secondUpper = Vector512.WidenUpper(second); + Vector512 alphaLower = Vector512.WidenLower(alpha); + Vector512 alphaUpper = Vector512.WidenUpper(alpha); + Vector512 maximumAlpha = Vector512.Create((uint)MaximumMaskAlpha); + Vector512 lower = + ((alphaLower * firstLower) + ((maximumAlpha - alphaLower) * secondLower)) >> MaskWeightBits; + + Vector512 upper = + ((alphaUpper * firstUpper) + ((maximumAlpha - alphaUpper) * secondUpper)) >> MaskWeightBits; + + return FinalizeHighBitDepthIntermediate( + Vector512.Narrow(lower, upper), + roundBits, + roundOffset, + maximum); + } } } diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.cs index 3c98a060a..04e82a059 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.cs @@ -152,6 +152,205 @@ internal static partial class Av1CompoundIntermediateMaskBlendPredictor } } + /// + /// Blends two high-bit-depth compound intermediates through a luma-resolution mask. + /// + public static void BlendIntermediate( + Span destination, + int destinationStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + ReadOnlySpan mask, + int maskStride, + int width, + int height, + int subX, + int subY, + int bitDepth) + => BlendIntermediate( + destination, + destinationStride, + first, + firstStride, + second, + secondStride, + mask, + maskStride, + width, + height, + subX, + subY, + bitDepth); + + /// + /// Executes one closed high-bit-depth alpha-blend compound-intermediate operator. + /// + /// The compound-intermediate operator. + private static void BlendIntermediate( + Span destination, + int destinationStride, + ReadOnlySpan first, + int firstStride, + ReadOnlySpan second, + int secondStride, + ReadOnlySpan mask, + int maskStride, + int width, + int height, + int subX, + int subY, + int bitDepth) + where TOperator : struct, IAv1CompoundIntermediateMaskBlendOperator + { + GetIntermediateRounding(bitDepth, out int roundBits, out int roundOffset); + int maximum = (1 << bitDepth) - 1; + + for (int row = 0; row < height; row++) + { + Span destinationRow = destination.Slice(row * destinationStride, width); + ReadOnlySpan firstRow = first.Slice(row * firstStride, width); + ReadOnlySpan secondRow = second.Slice(row * secondStride, width); + ref ushort destinationReference = ref MemoryMarshal.GetReference(destinationRow); + ref ushort firstReference = ref MemoryMarshal.GetReference(firstRow); + ref ushort secondReference = ref MemoryMarshal.GetReference(secondRow); + int column = 0; + + if (Vector512.IsHardwareAccelerated && subX == 0 && subY == 0) + { + ref byte maskReference = ref MemoryMarshal.GetReference(mask); + int maskRowOffset = row * maskStride; + int vectorEnd = width - Vector512.Count; + for (; column <= vectorEnd; column += Vector512.Count) + { + Vector512 first0 = Vector512.LoadUnsafe(ref firstReference, (nuint)column); + Vector512 first1 = Vector512.LoadUnsafe( + ref firstReference, + (nuint)(column + Vector512.Count)); + + Vector512 second0 = Vector512.LoadUnsafe(ref secondReference, (nuint)column); + Vector512 second1 = Vector512.LoadUnsafe( + ref secondReference, + (nuint)(column + Vector512.Count)); + + Vector512 alpha = Vector512.LoadUnsafe( + ref maskReference, + (nuint)(maskRowOffset + column)); + + TOperator.BlendHighBitDepth( + first0, + first1, + second0, + second1, + alpha, + roundBits, + roundOffset, + maximum, + out Vector512 result0, + out Vector512 result1); + + result0.StoreUnsafe(ref destinationReference, (nuint)column); + result1.StoreUnsafe( + ref destinationReference, + (nuint)(column + Vector512.Count)); + } + } + + if (Vector256.IsHardwareAccelerated && subX == 0 && subY == 0) + { + ref byte maskReference = ref MemoryMarshal.GetReference(mask); + int maskRowOffset = row * maskStride; + int vectorEnd = width - Vector256.Count; + for (; column <= vectorEnd; column += Vector256.Count) + { + Vector256 first0 = Vector256.LoadUnsafe(ref firstReference, (nuint)column); + Vector256 first1 = Vector256.LoadUnsafe( + ref firstReference, + (nuint)(column + Vector256.Count)); + + Vector256 second0 = Vector256.LoadUnsafe(ref secondReference, (nuint)column); + Vector256 second1 = Vector256.LoadUnsafe( + ref secondReference, + (nuint)(column + Vector256.Count)); + + Vector256 alpha = Vector256.LoadUnsafe( + ref maskReference, + (nuint)(maskRowOffset + column)); + + TOperator.BlendHighBitDepth( + first0, + first1, + second0, + second1, + alpha, + roundBits, + roundOffset, + maximum, + out Vector256 result0, + out Vector256 result1); + + result0.StoreUnsafe(ref destinationReference, (nuint)column); + result1.StoreUnsafe( + ref destinationReference, + (nuint)(column + Vector256.Count)); + } + } + + if (Vector128.IsHardwareAccelerated && subX == 0 && subY == 0) + { + ref byte maskReference = ref MemoryMarshal.GetReference(mask); + int maskRowOffset = row * maskStride; + int vectorEnd = width - Vector128.Count; + for (; column <= vectorEnd; column += Vector128.Count) + { + Vector128 first0 = Vector128.LoadUnsafe(ref firstReference, (nuint)column); + Vector128 first1 = Vector128.LoadUnsafe( + ref firstReference, + (nuint)(column + Vector128.Count)); + + Vector128 second0 = Vector128.LoadUnsafe(ref secondReference, (nuint)column); + Vector128 second1 = Vector128.LoadUnsafe( + ref secondReference, + (nuint)(column + Vector128.Count)); + + Vector128 alpha = Vector128.LoadUnsafe( + ref maskReference, + (nuint)(maskRowOffset + column)); + + TOperator.BlendHighBitDepth( + first0, + first1, + second0, + second1, + alpha, + roundBits, + roundOffset, + maximum, + out Vector128 result0, + out Vector128 result1); + + result0.StoreUnsafe(ref destinationReference, (nuint)column); + result1.StoreUnsafe( + ref destinationReference, + (nuint)(column + Vector128.Count)); + } + } + + for (; column < width; column++) + { + byte alpha = (byte)GetSubsampledMaskAlpha(mask, maskStride, row, column, subX, subY); + destinationRow[column] = TOperator.BlendHighBitDepth( + firstRow[column], + secondRow[column], + alpha, + roundBits, + roundOffset, + maximum); + } + } + } + /// /// Gets the mask alpha for one plane sample, averaging its two or four luma samples when required. /// diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WedgeMask.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WedgeMask.cs index 0d4654a1c..ebef29bfc 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WedgeMask.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WedgeMask.cs @@ -14,7 +14,7 @@ internal static class Av1WedgeMask private const int MasterSize = 64; /// - /// Gets the odd-row oblique prototype from pinned libaom. + /// Gets the odd-row oblique prototype defined by libaom. /// private static ReadOnlySpan MasterObliqueOdd => [ @@ -25,7 +25,7 @@ internal static class Av1WedgeMask ]; /// - /// Gets the even-row oblique prototype from pinned libaom. + /// Gets the even-row oblique prototype defined by libaom. /// private static ReadOnlySpan MasterObliqueEven => [ @@ -36,7 +36,7 @@ internal static class Av1WedgeMask ]; /// - /// Gets the vertical prototype from pinned libaom. + /// Gets the vertical prototype defined by libaom. /// private static ReadOnlySpan MasterVertical => [ diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs index 9394e594b..eeb8d6d78 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs @@ -372,7 +372,10 @@ internal sealed class Av1BlockDecoder : IDisposable // warped/global models have separate kernels and remain with their owning later prediction checkpoints. bool useHighBitDepthCompoundIntermediates = highBitDepth && - modeInfo.CompoundType == Av1CompoundType.Average && + modeInfo.CompoundType is + Av1CompoundType.Average or + Av1CompoundType.DistanceWeighted or + Av1CompoundType.Wedge && modeInfo.MotionMode != Av1MotionMode.Warped && modeInfo.YMode != Av1PredictionMode.GlobalGlobalMotionVector; @@ -731,6 +734,35 @@ internal sealed class Av1BlockDecoder : IDisposable secondCompoundWeight, this.frameBuffer.BitDepth.GetBitCount()); } + else if (modeInfo.CompoundType == Av1CompoundType.Wedge) + { + Av1WedgeMask.Fill( + compoundMask, + predictionWidth, + blockSize, + modeInfo.CompoundWedgeIndex, + modeInfo.CompoundWedgeSign, + subX, + subY, + invert: false); + + // Masked compound prediction must blend the same no-round intermediates as libaom's + // high-bit-depth d16 path so the mask is applied before the sole final rounding step. + Av1CompoundIntermediateMaskBlendPredictor.BlendIntermediate( + highBitDepthDestination, + reconstructionStride, + first, + predictionWidth, + highBitDepthSecondPrediction, + predictionWidth, + compoundMask, + predictionWidth, + predictionWidth, + predictionHeight, + subX: 0, + subY: 0, + this.frameBuffer.BitDepth.GetBitCount()); + } else { Av1CompoundIntermediateAveragePredictor.AverageIntermediate( diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs index ff34b5368..0c0e92079 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs @@ -141,6 +141,15 @@ public class Av1CompoundBlockDecoderTests ValidateSubpixelHighBitDepthDistanceWeightedCompoundPrediction, CompoundPredictionConfigurations); + /// + /// Verifies that high-bit-depth subpixel predictors retain no-round precision until wedge blending. + /// + [Fact] + public void DecodeBlockReconstructsSubpixelHighBitDepthWedgeCompoundPrediction() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateSubpixelHighBitDepthWedgeCompoundPrediction, + CompoundPredictionConfigurations); + /// /// Verifies that both references of a GLOBAL_GLOBALMV block use their complete matrix before compound averaging. /// @@ -673,6 +682,17 @@ public class Av1CompoundBlockDecoderTests } } + /// + /// Reconstructs the high-bit-depth subpixel wedge regression at every supported source precision. + /// + private static void ValidateSubpixelHighBitDepthWedgeCompoundPrediction() + { + foreach (Av1BitDepth bitDepth in new[] { Av1BitDepth.TenBit, Av1BitDepth.TwelveBit }) + { + ValidateSubpixelHighBitDepthCompoundPredictionAtBitDepth(bitDepth, Av1CompoundType.Wedge); + } + } + /// /// Reconstructs one high-bit-depth half-sample compound block and compares it with the scalar no-round pipeline. /// @@ -712,6 +732,8 @@ public class Av1CompoundBlockDecoderTests YMode = Av1PredictionMode.NearestNearestMotionVector, CompoundIndex = compoundType != Av1CompoundType.DistanceWeighted, CompoundType = compoundType, + CompoundWedgeIndex = 0, + CompoundWedgeSign = true, }; modeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last; @@ -727,6 +749,9 @@ public class Av1CompoundBlockDecoderTests ushort[] expectedFirst = new ushort[blockSize * blockSize]; ushort[] expectedSecond = new ushort[blockSize * blockSize]; Span expectedPredictions = expectedFirst; + ushort[] actualFirst = new ushort[blockSize * blockSize]; + ushort[] actualSecond = new ushort[blockSize * blockSize]; + Span actualPredictions = actualFirst; short[] predictionScratch = new short[128 * (blockSize + 8)]; for (int referenceIndex = 0; referenceIndex < 2; referenceIndex++) { @@ -759,9 +784,28 @@ public class Av1CompoundBlockDecoderTests bitDepth.GetBitCount(), predictionScratch); + Av1CompoundInterPredictor.PredictCompound( + source, + sourceStride, + sourceIndex, + actualPredictions, + blockSize, + blockSize, + blockSize, + Av1InterpolationFilter.Bilinear, + Av1InterpolationFilter.Bilinear, + sourceColumnQ4 & 15, + sourceRowQ4 & 15, + bitDepth.GetBitCount(), + predictionScratch); + expectedPredictions = expectedSecond; + actualPredictions = actualSecond; } + Assert.Equal(expectedFirst, actualFirst); + Assert.Equal(expectedSecond, actualSecond); + ushort[] expected = new ushort[blockSize * blockSize]; if (compoundType == Av1CompoundType.DistanceWeighted) { @@ -788,6 +832,50 @@ public class Av1CompoundBlockDecoderTests Assert.NotEqual((ushort)60, expected[0]); } + else if (compoundType == Av1CompoundType.Wedge) + { + ReadOnlySpan wedgeMask = + [ + 0, 0, 0, 1, 1, 2, 4, 6, + 0, 1, 1, 2, 4, 6, 11, 18, + 1, 2, 4, 6, 11, 18, 27, 37, + 4, 6, 11, 18, 27, 37, 46, 53, + 11, 18, 27, 37, 46, 53, 58, 60, + 27, 37, 46, 53, 58, 60, 62, 63, + 46, 53, 58, 60, 62, 63, 63, 64, + 58, 60, 62, 63, 63, 64, 64, 64, + ]; + + byte[] generatedWedgeMask = new byte[blockSize * blockSize]; + Av1WedgeMask.Fill( + generatedWedgeMask, + blockSize, + Av1BlockSize.Block8x8, + wedgeIndex: 0, + wedgeSign: true, + subX: 0, + subY: 0, + invert: false); + + Assert.Equal(wedgeMask, generatedWedgeMask); + + Av1CompoundIntermediateMaskBlendPredictor.BlendIntermediate( + expected, + blockSize, + expectedFirst, + blockSize, + expectedSecond, + blockSize, + wedgeMask, + blockSize, + blockSize, + blockSize, + subX: 0, + subY: 0, + bitDepth.GetBitCount()); + + Assert.NotEqual((ushort)60, expected[0]); + } else { Av1CompoundIntermediateAveragePredictor.AverageIntermediate( diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs index 76f318a73..cde1731cf 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs @@ -593,6 +593,47 @@ public class Av1CompoundInterPredictorTests Assert.Equal(expectedWeighted, actualWeighted); } + + int maskStride = width + 5; + byte[] mask = new byte[maskStride * height]; + ushort[] expectedMasked = new ushort[destinationStride * height]; + ushort[] actualMasked = new ushort[destinationStride * height]; + expectedMasked.AsSpan().Fill(0xA5A5); + actualMasked.AsSpan().Fill(0xA5A5); + + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + byte alpha = (byte)(((row * 29) + (column * 17) + 3) % 65); + mask[(row * maskStride) + column] = alpha; + int intermediateIndex = (row * intermediateStride) + column; + int result = ((alpha * expectedFirst[intermediateIndex]) + + ((64 - alpha) * expectedSecond[intermediateIndex])) >> 6; + + result -= roundOffset; + result = (result + (1 << (roundBits - 1))) >> roundBits; + expectedMasked[(row * destinationStride) + column] = + (ushort)Math.Clamp(result, 0, maximum); + } + } + + Av1CompoundIntermediateMaskBlendPredictor.BlendIntermediate( + actualMasked, + destinationStride, + actualFirst, + intermediateStride, + actualSecond, + intermediateStride, + mask, + maskStride, + width, + height, + subX: 0, + subY: 0, + bitDepth); + + Assert.Equal(expectedMasked, actualMasked); } } } diff --git a/tests/Images/Input/Heif/Av1/Conformance/README.md b/tests/Images/Input/Heif/Av1/Conformance/README.md index b867e3ca6..eb7af8ab6 100644 --- a/tests/Images/Input/Heif/Av1/Conformance/README.md +++ b/tests/Images/Input/Heif/Av1/Conformance/README.md @@ -149,12 +149,18 @@ conditions. Current `aomdec` produced all 19 YUV444 frames. The final frame's 19 SHA-256 `E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained Y4M with zero differing samples. +The wedge fixture's 5,374-byte AV1 `mdat` payload was decoded under the same current-libaom conditions. +Current `aomdec` produced all 19 YUV444 frames. The final frame's 19,200 native samples have SHA-256 +`E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained Y4M +with zero differing samples. + The production tests independently require their decoded mode states. The distance-weighted input must -exercise distance-weighted compound prediction and the inter-intra input must exercise both smooth and -wedge inter-intra prediction. The tests decode all preceding samples, compare final native Y, Cb, and Cr -planes exactly, compare final RGBA presentation through ImageSharp's established reference-output API, -and repeat reconstruction with constrained tracked allocation. The retained PNG files are presentation -evidence only and are not AV1 reconstruction references. +exercise distance-weighted compound prediction, the wedge input must exercise both mask orientations, +and the inter-intra input must exercise both smooth and wedge inter-intra prediction. The tests decode all +preceding samples, compare final native Y, Cb, and Cr planes exactly, compare final RGBA presentation +through ImageSharp's established reference-output API, and repeat reconstruction with constrained tracked +allocation. The retained PNG files are presentation evidence only and are not AV1 reconstruction +references. ## Overlapping motion-compensation fixture