Browse Source

Complete AV1 wedge compound decoding checkpoint

pull/2633/head
James Jackson-South 3 days ago
parent
commit
9883a24dc3
  1. 52
      HEIF_IMPLEMENTATION_PLAN.md
  2. 294
      src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.Operator.cs
  3. 199
      src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.cs
  4. 6
      src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WedgeMask.cs
  5. 34
      src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs
  6. 88
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs
  7. 41
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs
  8. 16
      tests/Images/Input/Heif/Av1/Conformance/README.md

52
HEIF_IMPLEMENTATION_PLAN.md

@ -30,13 +30,13 @@ Reference checkout evidence on 2026-08-31:
Reconciled with the worktree on 2026-08-31. Reconciled with the worktree on 2026-08-31.
- [~] The bounded container reader, still-image path, sequence parser, AV1 decoder, color pipeline, presentation pipeline, and broad AV1 test suite exist locally. - [~] The bounded container reader, still-image path, sequence parser, AV1 decoder, color pipeline, presentation pipeline, and broad AV1 test suite exist locally.
- [~] The inter-frame decoder contains implementations for single-reference prediction, compound references, inter-intra prediction, selectable compound blending, OBMC, scaled references, local warped motion, and global motion. These downstream paths must not be called verified until the single-reference checkpoint below is corrected and rerun. - [~] The inter-frame decoder has verified checkpoints through wedge compound prediction. Difference-weighted compound prediction, OBMC, scaled references, local warped motion, and global motion exist locally but remain open until their ordered checkpoints below are completed.
- [~] Loop filtering, CDEF, super-resolution, restoration, film grain, layered presentation, alpha composition, and color conversion exist locally. Shared-source cleanup changed the current tree, so final production-path verification is open. - [~] Loop filtering, CDEF, super-resolution, restoration, film grain, layered presentation, alpha composition, and color conversion exist locally. Shared-source cleanup changed the current tree, so final production-path verification is open.
- [~] AV1 writer primitives, forward transforms, symbol encoding, and tile-writing source exist locally, but they are not connected to the public encoder. - [~] AV1 writer primitives, forward transforms, symbol encoding, and tile-writing source exist locally, but they are not connected to the public encoder.
- [ ] The public AV1 encoder is not implemented. HeifEncoderCore.Encode throws NotSupportedException when AV1 is selected. - [ ] The public AV1 encoder is not implemented. HeifEncoderCore.Encode throws NotSupportedException when AV1 is selected.
- [x] Patented codec production code, registrations, tests, benchmarks, fixtures, reference outputs, and notices were manually deleted and committed by `78a74d448`. - [x] Patented codec production code, registrations, tests, benchmarks, fixtures, reference outputs, and notices were manually deleted and committed by `78a74d448`.
- [x] Remaining task-created HM, HEVC, libheif, GPAC, Nokia, FFmpeg, Pillow HEIF, libavif-build, and libjpeg-build directories were traced to their creation commands in the recovered Codex session history and deleted on 2026-08-31. The user-provided repositories and all libaom-only source, build, and reference data were left untouched. - [x] Remaining task-created HM, HEVC, libheif, GPAC, Nokia, FFmpeg, Pillow HEIF, libavif-build, and libjpeg-build directories were traced to their creation commands in the recovered Codex session history and deleted on 2026-08-31. The user-provided repositories and all libaom-only source, build, and reference data were left untouched.
- [~] A PNG metadata-suppression fix and three HEIF/AV1 diagnostic-save call-site corrections are implemented in the current worktree. The exact 34 cases that failed in the net11.0 ARM CI job now pass in Release, but these changes are not yet committed and are not decoder or encoder completion evidence. - [x] The PNG metadata-suppression fix and three HEIF/AV1 diagnostic-save call-site corrections passed the exact 34 net11.0 ARM CI cases and were committed with the single-reference checkpoint as `54bb6cbe59bd113058854a3ee31448cf61f462ca`. They are infrastructure evidence, not decoder or encoder completion evidence.
- [ ] The complete decoder and encoder release matrix is not complete. - [ ] The complete decoder and encoder release matrix is not complete.
## Immediate execution queue ## Immediate execution queue
@ -50,10 +50,10 @@ Work must proceed in this order. Do not skip to a later item while an earlier ch
- [x] Retain the official current-main libaom checkout and libaom-only build artifacts required for AV1 verification. - [x] Retain the official current-main libaom checkout and libaom-only build artifacts required for AV1 verification.
- [x] Retain user-supplied AV1 fixtures and their recorded expected outputs. - [x] Retain user-supplied AV1 fixtures and their recorded expected outputs.
- [x] Audit production source, tests, benchmarks, assets, project files, notices, and documentation for stale removed-code references. - [x] Audit production source, tests, benchmarks, assets, project files, notices, and documentation for stale removed-code references.
- [~] The prior cleanup tree built in Release with restore disabled, build servers disabled, and one MSBuild node. A fresh current-worktree build remains required after the local PNG cICP fix. - [x] The cleanup and cICP tree built in Release for net10.0 and net11.0 with restore disabled, build servers disabled, and one MSBuild node.
- [~] The prior focused AV1/container set passed, but it missed the net11.0 PNG cICP diagnostic-save failures. The exact 34 CI failures now pass after the local fix; the final current-worktree checkpoint set remains required. - [x] The exact 34 net11.0 ARM CI failures pass after the cICP correction, and the subsequent single-reference checkpoint set passes on net10.0 and net11.0.
- [~] Roslynk currently reports zero compiler errors for the local fix, and `git diff --check` passes. Scoped StyleCop and final current-worktree inspection remain required. - [x] Roslynk, scoped StyleCop, whitespace, and `git diff --check` accepted the cleanup and cICP checkpoint.
- [~] Record the final current-worktree cleanup and cICP evidence after the checkpoint verification completes. - [x] The cleanup and cICP evidence was recorded and committed with the single-reference checkpoint.
Historical cleanup evidence from 2026-08-30, retained with its limitation: Historical cleanup evidence from 2026-08-30, retained with its limitation:
@ -179,8 +179,8 @@ The single-reference syntax, buffer, reconstruction, and ownership foundation is
- [x] Compound reference selection, paired reference-MV derivation, and equal averaging. - [x] Compound reference selection, paired reference-MV derivation, and equal averaging.
- [x] Inter-intra prediction. - [x] Inter-intra prediction.
- [x] Distance-weighted compound prediction. - [x] Distance-weighted compound prediction.
- [~] Wedge compound prediction. Current item. - [x] Wedge compound prediction.
- [~] Difference-weighted compound prediction. - [~] Difference-weighted compound prediction. Current item.
- [~] OBMC. - [~] OBMC.
- [~] Scaled-reference prediction. - [~] Scaled-reference prediction.
- [~] Local warped prediction. - [~] Local warped prediction.
@ -286,6 +286,42 @@ Verified distance-weighted compound checkpoint evidence on 2026-08-31:
or skips. Scoped analyzer and whitespace verification pass for every changed C# file. Roslynk reports or skips. Scoped analyzer and whitespace verification pass for every changed C# file. Roslynk reports
zero compiler errors and no diagnostics in the changed files, `git diff --check` passes, and zero compiler errors and no diagnostics in the changed files, `git diff --check` passes, and
`.gitattributes` is unchanged. `.gitattributes` is unchanged.
- [x] The completed checkpoint was committed as `7e2de7a2c25852acc374b17936a1a644464f77f3`
with author and committer `James Jackson-South <james_south@hotmail.com>`.
Verified wedge compound checkpoint evidence on 2026-08-31:
- [x] Audited mask generation against current libaom `tools/gen_wedge_masks_data.py` and
`av1/common/reconinter.c`, including the master prototypes, direction transforms, block-size
codebooks, sign flips, offsets, and luma/chroma mask sampling. ImageSharp's generated masks match
those definitions; only stale “pinned” documentation required correction.
- [x] Audited reconstruction against current libaom `aom_dsp/blend_a64_mask.c`. The high-bit-depth
d16 path applies the Q6 mask to both no-round intermediates before bias removal, the sole final
rounding step, and clipping.
- [x] Corrected the production high-bit-depth intermediate eligibility gate, which admitted only
equal-average blocks and made the distance-weighted and wedge no-round finalizers unreachable.
Average, distance-weighted, and wedge subpixel blocks now retain both intermediates until their
signaled finalizer; difference-weighted blending remains excluded for its next ordered checkpoint.
- [x] Added high-bit-depth traversal to the existing semantic mask-blend predictor and readonly
operator family with descending Vector512, Vector256, Vector128, and scalar dispatch. Unsigned
widening preserves the biased 12-bit intermediate range. No per-block, per-row, or per-scanline
allocation or copy was added.
- [x] Extended FeatureTestRunner coverage with an independent Q6 mask oracle across 10/12-bit copy,
horizontal, vertical, and separable subpixel prediction, widths 9, 17, 33, and 65, all mask weights
from 0 through 64, and row-padding sentinels. A complete `Av1BlockDecoder.DecodeBlock` regression
verifies the current-libaom 8x8 wedge mask and the production no-round branch.
- [x] Extracted the fixture's 5,374-byte AV1 `mdat` payload and decoded it with refreshed current
libaom `aomdec`, using one thread with row threading disabled. All 19 frames decoded. The final
19,200 YUV444 samples have SHA-256
`E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained native
reference with zero differing samples.
- [x] The real 19-frame production sequence requires both wedge-mask orientations, compares final
native Y, Cb, and Cr planes exactly, compares final RGBA presentation through ImageSharp's
established reference-output API, and repeats the complete decode with a 1,024-byte constrained
tracked allocator and exactly-once return checks.
- [x] The focused Release checkpoint set passes 35/35 on net10.0 and 35/35 on net11.0, with zero
failures or skips. Scoped analyzer and whitespace verification pass for every changed C# file.
Roslynk reports zero compiler errors, `git diff --check` passes, and `.gitattributes` is unchanged.
For every item: For every item:

294
src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.Operator.cs

@ -89,6 +89,99 @@ internal static partial class Av1CompoundIntermediateMaskBlendPredictor
Vector512<byte> alpha, Vector512<byte> alpha,
int roundBits, int roundBits,
int roundOffset); int roundOffset);
/// <summary>
/// Alpha-blends and finalizes one pair of high-bit-depth compound intermediate samples.
/// </summary>
/// <param name="first">The first compound intermediate.</param>
/// <param name="second">The second compound intermediate.</param>
/// <param name="alpha">The first-predictor weight in the AV1 mask range.</param>
/// <param name="roundBits">The final reconstruction shift.</param>
/// <param name="roundOffset">The compound intermediate bias.</param>
/// <param name="maximum">The maximum reconstructed sample value.</param>
/// <returns>The reconstructed sample.</returns>
public static abstract ushort BlendHighBitDepth(
ushort first,
ushort second,
byte alpha,
int roundBits,
int roundOffset,
int maximum);
/// <summary>
/// Alpha-blends and finalizes 128 bits of high-bit-depth compound intermediate samples.
/// </summary>
/// <param name="first0">The lower first-predictor intermediates.</param>
/// <param name="first1">The upper first-predictor intermediates.</param>
/// <param name="second0">The lower second-predictor intermediates.</param>
/// <param name="second1">The upper second-predictor intermediates.</param>
/// <param name="alpha">The first-predictor weights in the AV1 mask range.</param>
/// <param name="roundBits">The final reconstruction shift.</param>
/// <param name="roundOffset">The compound intermediate bias.</param>
/// <param name="maximum">The maximum reconstructed sample value.</param>
/// <param name="result0">The lower reconstructed samples.</param>
/// <param name="result1">The upper reconstructed samples.</param>
public static abstract void BlendHighBitDepth(
Vector128<ushort> first0,
Vector128<ushort> first1,
Vector128<ushort> second0,
Vector128<ushort> second1,
Vector128<byte> alpha,
int roundBits,
int roundOffset,
int maximum,
out Vector128<ushort> result0,
out Vector128<ushort> result1);
/// <summary>
/// Alpha-blends and finalizes 256 bits of high-bit-depth compound intermediate samples.
/// </summary>
/// <param name="first0">The lower first-predictor intermediates.</param>
/// <param name="first1">The upper first-predictor intermediates.</param>
/// <param name="second0">The lower second-predictor intermediates.</param>
/// <param name="second1">The upper second-predictor intermediates.</param>
/// <param name="alpha">The first-predictor weights in the AV1 mask range.</param>
/// <param name="roundBits">The final reconstruction shift.</param>
/// <param name="roundOffset">The compound intermediate bias.</param>
/// <param name="maximum">The maximum reconstructed sample value.</param>
/// <param name="result0">The lower reconstructed samples.</param>
/// <param name="result1">The upper reconstructed samples.</param>
public static abstract void BlendHighBitDepth(
Vector256<ushort> first0,
Vector256<ushort> first1,
Vector256<ushort> second0,
Vector256<ushort> second1,
Vector256<byte> alpha,
int roundBits,
int roundOffset,
int maximum,
out Vector256<ushort> result0,
out Vector256<ushort> result1);
/// <summary>
/// Alpha-blends and finalizes 512 bits of high-bit-depth compound intermediate samples.
/// </summary>
/// <param name="first0">The lower first-predictor intermediates.</param>
/// <param name="first1">The upper first-predictor intermediates.</param>
/// <param name="second0">The lower second-predictor intermediates.</param>
/// <param name="second1">The upper second-predictor intermediates.</param>
/// <param name="alpha">The first-predictor weights in the AV1 mask range.</param>
/// <param name="roundBits">The final reconstruction shift.</param>
/// <param name="roundOffset">The compound intermediate bias.</param>
/// <param name="maximum">The maximum reconstructed sample value.</param>
/// <param name="result0">The lower reconstructed samples.</param>
/// <param name="result1">The upper reconstructed samples.</param>
public static abstract void BlendHighBitDepth(
Vector512<ushort> first0,
Vector512<ushort> first1,
Vector512<ushort> second0,
Vector512<ushort> second1,
Vector512<byte> alpha,
int roundBits,
int roundOffset,
int maximum,
out Vector512<ushort> result0,
out Vector512<ushort> result1);
} }
/// <summary> /// <summary>
@ -106,6 +199,21 @@ internal static partial class Av1CompoundIntermediateMaskBlendPredictor
return (byte)Math.Clamp(RoundPowerOfTwo(result, roundBits), 0, byte.MaxValue); return (byte)Math.Clamp(RoundPowerOfTwo(result, roundBits), 0, byte.MaxValue);
} }
/// <inheritdoc/>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static ushort BlendHighBitDepth(
ushort first,
ushort second,
byte alpha,
int roundBits,
int roundOffset,
int maximum)
{
int result = ((alpha * first) + ((MaximumMaskAlpha - alpha) * second)) >> MaskWeightBits;
result -= roundOffset;
return (ushort)Math.Clamp(RoundPowerOfTwo(result, roundBits), 0, maximum);
}
/// <inheritdoc/> /// <inheritdoc/>
[MethodImpl(MethodImplOptions.AggressiveInlining)] [MethodImpl(MethodImplOptions.AggressiveInlining)]
public static Vector128<byte> Blend( public static Vector128<byte> Blend(
@ -148,6 +256,99 @@ internal static partial class Av1CompoundIntermediateMaskBlendPredictor
Blend(first0, second0, Vector512.WidenLower(alpha), roundBits, roundOffset), Blend(first0, second0, Vector512.WidenLower(alpha), roundBits, roundOffset),
Blend(first1, second1, Vector512.WidenUpper(alpha), roundBits, roundOffset)); Blend(first1, second1, Vector512.WidenUpper(alpha), roundBits, roundOffset));
/// <inheritdoc/>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void BlendHighBitDepth(
Vector128<ushort> first0,
Vector128<ushort> first1,
Vector128<ushort> second0,
Vector128<ushort> second1,
Vector128<byte> alpha,
int roundBits,
int roundOffset,
int maximum,
out Vector128<ushort> result0,
out Vector128<ushort> result1)
{
result0 = BlendHighBitDepth(
first0,
second0,
Vector128.WidenLower(alpha),
roundBits,
roundOffset,
maximum);
result1 = BlendHighBitDepth(
first1,
second1,
Vector128.WidenUpper(alpha),
roundBits,
roundOffset,
maximum);
}
/// <inheritdoc/>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void BlendHighBitDepth(
Vector256<ushort> first0,
Vector256<ushort> first1,
Vector256<ushort> second0,
Vector256<ushort> second1,
Vector256<byte> alpha,
int roundBits,
int roundOffset,
int maximum,
out Vector256<ushort> result0,
out Vector256<ushort> result1)
{
result0 = BlendHighBitDepth(
first0,
second0,
Vector256.WidenLower(alpha),
roundBits,
roundOffset,
maximum);
result1 = BlendHighBitDepth(
first1,
second1,
Vector256.WidenUpper(alpha),
roundBits,
roundOffset,
maximum);
}
/// <inheritdoc/>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void BlendHighBitDepth(
Vector512<ushort> first0,
Vector512<ushort> first1,
Vector512<ushort> second0,
Vector512<ushort> second1,
Vector512<byte> alpha,
int roundBits,
int roundOffset,
int maximum,
out Vector512<ushort> result0,
out Vector512<ushort> result1)
{
result0 = BlendHighBitDepth(
first0,
second0,
Vector512.WidenLower(alpha),
roundBits,
roundOffset,
maximum);
result1 = BlendHighBitDepth(
first1,
second1,
Vector512.WidenUpper(alpha),
roundBits,
roundOffset,
maximum);
}
/// <summary> /// <summary>
/// Alpha-blends 128-bit lanes after widening every product to signed 32-bit precision. /// Alpha-blends 128-bit lanes after widening every product to signed 32-bit precision.
/// </summary> /// </summary>
@ -213,5 +414,98 @@ internal static partial class Av1CompoundIntermediateMaskBlendPredictor
Vector512<int> upper = ((alphaUpper * firstUpper) + ((maximum - alphaUpper) * secondUpper)) >> MaskWeightBits; Vector512<int> upper = ((alphaUpper * firstUpper) + ((maximum - alphaUpper) * secondUpper)) >> MaskWeightBits;
return Vector512.Narrow(FinalizeIntermediate(lower, roundBits, roundOffset), FinalizeIntermediate(upper, roundBits, roundOffset)).AsUInt16(); return Vector512.Narrow(FinalizeIntermediate(lower, roundBits, roundOffset), FinalizeIntermediate(upper, roundBits, roundOffset)).AsUInt16();
} }
/// <summary>
/// Alpha-blends 128-bit high-bit-depth lanes without narrowing the unsigned intermediate range.
/// </summary>
private static Vector128<ushort> BlendHighBitDepth(
Vector128<ushort> first,
Vector128<ushort> second,
Vector128<ushort> alpha,
int roundBits,
int roundOffset,
int maximum)
{
Vector128<uint> firstLower = Vector128.WidenLower(first);
Vector128<uint> firstUpper = Vector128.WidenUpper(first);
Vector128<uint> secondLower = Vector128.WidenLower(second);
Vector128<uint> secondUpper = Vector128.WidenUpper(second);
Vector128<uint> alphaLower = Vector128.WidenLower(alpha);
Vector128<uint> alphaUpper = Vector128.WidenUpper(alpha);
Vector128<uint> maximumAlpha = Vector128.Create((uint)MaximumMaskAlpha);
Vector128<uint> lower =
((alphaLower * firstLower) + ((maximumAlpha - alphaLower) * secondLower)) >> MaskWeightBits;
Vector128<uint> upper =
((alphaUpper * firstUpper) + ((maximumAlpha - alphaUpper) * secondUpper)) >> MaskWeightBits;
return FinalizeHighBitDepthIntermediate(
Vector128.Narrow(lower, upper),
roundBits,
roundOffset,
maximum);
}
/// <summary>
/// Alpha-blends 256-bit high-bit-depth lanes without narrowing the unsigned intermediate range.
/// </summary>
private static Vector256<ushort> BlendHighBitDepth(
Vector256<ushort> first,
Vector256<ushort> second,
Vector256<ushort> alpha,
int roundBits,
int roundOffset,
int maximum)
{
Vector256<uint> firstLower = Vector256.WidenLower(first);
Vector256<uint> firstUpper = Vector256.WidenUpper(first);
Vector256<uint> secondLower = Vector256.WidenLower(second);
Vector256<uint> secondUpper = Vector256.WidenUpper(second);
Vector256<uint> alphaLower = Vector256.WidenLower(alpha);
Vector256<uint> alphaUpper = Vector256.WidenUpper(alpha);
Vector256<uint> maximumAlpha = Vector256.Create((uint)MaximumMaskAlpha);
Vector256<uint> lower =
((alphaLower * firstLower) + ((maximumAlpha - alphaLower) * secondLower)) >> MaskWeightBits;
Vector256<uint> upper =
((alphaUpper * firstUpper) + ((maximumAlpha - alphaUpper) * secondUpper)) >> MaskWeightBits;
return FinalizeHighBitDepthIntermediate(
Vector256.Narrow(lower, upper),
roundBits,
roundOffset,
maximum);
}
/// <summary>
/// Alpha-blends 512-bit high-bit-depth lanes without narrowing the unsigned intermediate range.
/// </summary>
private static Vector512<ushort> BlendHighBitDepth(
Vector512<ushort> first,
Vector512<ushort> second,
Vector512<ushort> alpha,
int roundBits,
int roundOffset,
int maximum)
{
Vector512<uint> firstLower = Vector512.WidenLower(first);
Vector512<uint> firstUpper = Vector512.WidenUpper(first);
Vector512<uint> secondLower = Vector512.WidenLower(second);
Vector512<uint> secondUpper = Vector512.WidenUpper(second);
Vector512<uint> alphaLower = Vector512.WidenLower(alpha);
Vector512<uint> alphaUpper = Vector512.WidenUpper(alpha);
Vector512<uint> maximumAlpha = Vector512.Create((uint)MaximumMaskAlpha);
Vector512<uint> lower =
((alphaLower * firstLower) + ((maximumAlpha - alphaLower) * secondLower)) >> MaskWeightBits;
Vector512<uint> upper =
((alphaUpper * firstUpper) + ((maximumAlpha - alphaUpper) * secondUpper)) >> MaskWeightBits;
return FinalizeHighBitDepthIntermediate(
Vector512.Narrow(lower, upper),
roundBits,
roundOffset,
maximum);
}
} }
} }

199
src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1CompoundIntermediateMaskBlendPredictor.cs

@ -152,6 +152,205 @@ internal static partial class Av1CompoundIntermediateMaskBlendPredictor
} }
} }
/// <summary>
/// Blends two high-bit-depth compound intermediates through a luma-resolution mask.
/// </summary>
public static void BlendIntermediate(
Span<ushort> destination,
int destinationStride,
ReadOnlySpan<ushort> first,
int firstStride,
ReadOnlySpan<ushort> second,
int secondStride,
ReadOnlySpan<byte> mask,
int maskStride,
int width,
int height,
int subX,
int subY,
int bitDepth)
=> BlendIntermediate<CompoundIntermediateMaskBlendOperator>(
destination,
destinationStride,
first,
firstStride,
second,
secondStride,
mask,
maskStride,
width,
height,
subX,
subY,
bitDepth);
/// <summary>
/// Executes one closed high-bit-depth alpha-blend compound-intermediate operator.
/// </summary>
/// <typeparam name="TOperator">The compound-intermediate operator.</typeparam>
private static void BlendIntermediate<TOperator>(
Span<ushort> destination,
int destinationStride,
ReadOnlySpan<ushort> first,
int firstStride,
ReadOnlySpan<ushort> second,
int secondStride,
ReadOnlySpan<byte> mask,
int maskStride,
int width,
int height,
int subX,
int subY,
int bitDepth)
where TOperator : struct, IAv1CompoundIntermediateMaskBlendOperator
{
GetIntermediateRounding(bitDepth, out int roundBits, out int roundOffset);
int maximum = (1 << bitDepth) - 1;
for (int row = 0; row < height; row++)
{
Span<ushort> destinationRow = destination.Slice(row * destinationStride, width);
ReadOnlySpan<ushort> firstRow = first.Slice(row * firstStride, width);
ReadOnlySpan<ushort> secondRow = second.Slice(row * secondStride, width);
ref ushort destinationReference = ref MemoryMarshal.GetReference(destinationRow);
ref ushort firstReference = ref MemoryMarshal.GetReference(firstRow);
ref ushort secondReference = ref MemoryMarshal.GetReference(secondRow);
int column = 0;
if (Vector512.IsHardwareAccelerated && subX == 0 && subY == 0)
{
ref byte maskReference = ref MemoryMarshal.GetReference(mask);
int maskRowOffset = row * maskStride;
int vectorEnd = width - Vector512<byte>.Count;
for (; column <= vectorEnd; column += Vector512<byte>.Count)
{
Vector512<ushort> first0 = Vector512.LoadUnsafe(ref firstReference, (nuint)column);
Vector512<ushort> first1 = Vector512.LoadUnsafe(
ref firstReference,
(nuint)(column + Vector512<ushort>.Count));
Vector512<ushort> second0 = Vector512.LoadUnsafe(ref secondReference, (nuint)column);
Vector512<ushort> second1 = Vector512.LoadUnsafe(
ref secondReference,
(nuint)(column + Vector512<ushort>.Count));
Vector512<byte> alpha = Vector512.LoadUnsafe(
ref maskReference,
(nuint)(maskRowOffset + column));
TOperator.BlendHighBitDepth(
first0,
first1,
second0,
second1,
alpha,
roundBits,
roundOffset,
maximum,
out Vector512<ushort> result0,
out Vector512<ushort> result1);
result0.StoreUnsafe(ref destinationReference, (nuint)column);
result1.StoreUnsafe(
ref destinationReference,
(nuint)(column + Vector512<ushort>.Count));
}
}
if (Vector256.IsHardwareAccelerated && subX == 0 && subY == 0)
{
ref byte maskReference = ref MemoryMarshal.GetReference(mask);
int maskRowOffset = row * maskStride;
int vectorEnd = width - Vector256<byte>.Count;
for (; column <= vectorEnd; column += Vector256<byte>.Count)
{
Vector256<ushort> first0 = Vector256.LoadUnsafe(ref firstReference, (nuint)column);
Vector256<ushort> first1 = Vector256.LoadUnsafe(
ref firstReference,
(nuint)(column + Vector256<ushort>.Count));
Vector256<ushort> second0 = Vector256.LoadUnsafe(ref secondReference, (nuint)column);
Vector256<ushort> second1 = Vector256.LoadUnsafe(
ref secondReference,
(nuint)(column + Vector256<ushort>.Count));
Vector256<byte> alpha = Vector256.LoadUnsafe(
ref maskReference,
(nuint)(maskRowOffset + column));
TOperator.BlendHighBitDepth(
first0,
first1,
second0,
second1,
alpha,
roundBits,
roundOffset,
maximum,
out Vector256<ushort> result0,
out Vector256<ushort> result1);
result0.StoreUnsafe(ref destinationReference, (nuint)column);
result1.StoreUnsafe(
ref destinationReference,
(nuint)(column + Vector256<ushort>.Count));
}
}
if (Vector128.IsHardwareAccelerated && subX == 0 && subY == 0)
{
ref byte maskReference = ref MemoryMarshal.GetReference(mask);
int maskRowOffset = row * maskStride;
int vectorEnd = width - Vector128<byte>.Count;
for (; column <= vectorEnd; column += Vector128<byte>.Count)
{
Vector128<ushort> first0 = Vector128.LoadUnsafe(ref firstReference, (nuint)column);
Vector128<ushort> first1 = Vector128.LoadUnsafe(
ref firstReference,
(nuint)(column + Vector128<ushort>.Count));
Vector128<ushort> second0 = Vector128.LoadUnsafe(ref secondReference, (nuint)column);
Vector128<ushort> second1 = Vector128.LoadUnsafe(
ref secondReference,
(nuint)(column + Vector128<ushort>.Count));
Vector128<byte> alpha = Vector128.LoadUnsafe(
ref maskReference,
(nuint)(maskRowOffset + column));
TOperator.BlendHighBitDepth(
first0,
first1,
second0,
second1,
alpha,
roundBits,
roundOffset,
maximum,
out Vector128<ushort> result0,
out Vector128<ushort> result1);
result0.StoreUnsafe(ref destinationReference, (nuint)column);
result1.StoreUnsafe(
ref destinationReference,
(nuint)(column + Vector128<ushort>.Count));
}
}
for (; column < width; column++)
{
byte alpha = (byte)GetSubsampledMaskAlpha(mask, maskStride, row, column, subX, subY);
destinationRow[column] = TOperator.BlendHighBitDepth(
firstRow[column],
secondRow[column],
alpha,
roundBits,
roundOffset,
maximum);
}
}
}
/// <summary> /// <summary>
/// Gets the mask alpha for one plane sample, averaging its two or four luma samples when required. /// Gets the mask alpha for one plane sample, averaging its two or four luma samples when required.
/// </summary> /// </summary>

6
src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1WedgeMask.cs

@ -14,7 +14,7 @@ internal static class Av1WedgeMask
private const int MasterSize = 64; private const int MasterSize = 64;
/// <summary> /// <summary>
/// Gets the odd-row oblique prototype from pinned libaom. /// Gets the odd-row oblique prototype defined by libaom.
/// </summary> /// </summary>
private static ReadOnlySpan<byte> MasterObliqueOdd => private static ReadOnlySpan<byte> MasterObliqueOdd =>
[ [
@ -25,7 +25,7 @@ internal static class Av1WedgeMask
]; ];
/// <summary> /// <summary>
/// Gets the even-row oblique prototype from pinned libaom. /// Gets the even-row oblique prototype defined by libaom.
/// </summary> /// </summary>
private static ReadOnlySpan<byte> MasterObliqueEven => private static ReadOnlySpan<byte> MasterObliqueEven =>
[ [
@ -36,7 +36,7 @@ internal static class Av1WedgeMask
]; ];
/// <summary> /// <summary>
/// Gets the vertical prototype from pinned libaom. /// Gets the vertical prototype defined by libaom.
/// </summary> /// </summary>
private static ReadOnlySpan<byte> MasterVertical => private static ReadOnlySpan<byte> MasterVertical =>
[ [

34
src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs

@ -372,7 +372,10 @@ internal sealed class Av1BlockDecoder : IDisposable
// warped/global models have separate kernels and remain with their owning later prediction checkpoints. // warped/global models have separate kernels and remain with their owning later prediction checkpoints.
bool useHighBitDepthCompoundIntermediates = bool useHighBitDepthCompoundIntermediates =
highBitDepth && highBitDepth &&
modeInfo.CompoundType == Av1CompoundType.Average && modeInfo.CompoundType is
Av1CompoundType.Average or
Av1CompoundType.DistanceWeighted or
Av1CompoundType.Wedge &&
modeInfo.MotionMode != Av1MotionMode.Warped && modeInfo.MotionMode != Av1MotionMode.Warped &&
modeInfo.YMode != Av1PredictionMode.GlobalGlobalMotionVector; modeInfo.YMode != Av1PredictionMode.GlobalGlobalMotionVector;
@ -731,6 +734,35 @@ internal sealed class Av1BlockDecoder : IDisposable
secondCompoundWeight, secondCompoundWeight,
this.frameBuffer.BitDepth.GetBitCount()); this.frameBuffer.BitDepth.GetBitCount());
} }
else if (modeInfo.CompoundType == Av1CompoundType.Wedge)
{
Av1WedgeMask.Fill(
compoundMask,
predictionWidth,
blockSize,
modeInfo.CompoundWedgeIndex,
modeInfo.CompoundWedgeSign,
subX,
subY,
invert: false);
// Masked compound prediction must blend the same no-round intermediates as libaom's
// high-bit-depth d16 path so the mask is applied before the sole final rounding step.
Av1CompoundIntermediateMaskBlendPredictor.BlendIntermediate(
highBitDepthDestination,
reconstructionStride,
first,
predictionWidth,
highBitDepthSecondPrediction,
predictionWidth,
compoundMask,
predictionWidth,
predictionWidth,
predictionHeight,
subX: 0,
subY: 0,
this.frameBuffer.BitDepth.GetBitCount());
}
else else
{ {
Av1CompoundIntermediateAveragePredictor.AverageIntermediate( Av1CompoundIntermediateAveragePredictor.AverageIntermediate(

88
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs

@ -141,6 +141,15 @@ public class Av1CompoundBlockDecoderTests
ValidateSubpixelHighBitDepthDistanceWeightedCompoundPrediction, ValidateSubpixelHighBitDepthDistanceWeightedCompoundPrediction,
CompoundPredictionConfigurations); CompoundPredictionConfigurations);
/// <summary>
/// Verifies that high-bit-depth subpixel predictors retain no-round precision until wedge blending.
/// </summary>
[Fact]
public void DecodeBlockReconstructsSubpixelHighBitDepthWedgeCompoundPrediction()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(
ValidateSubpixelHighBitDepthWedgeCompoundPrediction,
CompoundPredictionConfigurations);
/// <summary> /// <summary>
/// Verifies that both references of a GLOBAL_GLOBALMV block use their complete matrix before compound averaging. /// Verifies that both references of a GLOBAL_GLOBALMV block use their complete matrix before compound averaging.
/// </summary> /// </summary>
@ -673,6 +682,17 @@ public class Av1CompoundBlockDecoderTests
} }
} }
/// <summary>
/// Reconstructs the high-bit-depth subpixel wedge regression at every supported source precision.
/// </summary>
private static void ValidateSubpixelHighBitDepthWedgeCompoundPrediction()
{
foreach (Av1BitDepth bitDepth in new[] { Av1BitDepth.TenBit, Av1BitDepth.TwelveBit })
{
ValidateSubpixelHighBitDepthCompoundPredictionAtBitDepth(bitDepth, Av1CompoundType.Wedge);
}
}
/// <summary> /// <summary>
/// Reconstructs one high-bit-depth half-sample compound block and compares it with the scalar no-round pipeline. /// Reconstructs one high-bit-depth half-sample compound block and compares it with the scalar no-round pipeline.
/// </summary> /// </summary>
@ -712,6 +732,8 @@ public class Av1CompoundBlockDecoderTests
YMode = Av1PredictionMode.NearestNearestMotionVector, YMode = Av1PredictionMode.NearestNearestMotionVector,
CompoundIndex = compoundType != Av1CompoundType.DistanceWeighted, CompoundIndex = compoundType != Av1CompoundType.DistanceWeighted,
CompoundType = compoundType, CompoundType = compoundType,
CompoundWedgeIndex = 0,
CompoundWedgeSign = true,
}; };
modeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last; modeInfo.ReferenceFrames[0] = Av1ReferenceFrameType.Last;
@ -727,6 +749,9 @@ public class Av1CompoundBlockDecoderTests
ushort[] expectedFirst = new ushort[blockSize * blockSize]; ushort[] expectedFirst = new ushort[blockSize * blockSize];
ushort[] expectedSecond = new ushort[blockSize * blockSize]; ushort[] expectedSecond = new ushort[blockSize * blockSize];
Span<ushort> expectedPredictions = expectedFirst; Span<ushort> expectedPredictions = expectedFirst;
ushort[] actualFirst = new ushort[blockSize * blockSize];
ushort[] actualSecond = new ushort[blockSize * blockSize];
Span<ushort> actualPredictions = actualFirst;
short[] predictionScratch = new short[128 * (blockSize + 8)]; short[] predictionScratch = new short[128 * (blockSize + 8)];
for (int referenceIndex = 0; referenceIndex < 2; referenceIndex++) for (int referenceIndex = 0; referenceIndex < 2; referenceIndex++)
{ {
@ -759,9 +784,28 @@ public class Av1CompoundBlockDecoderTests
bitDepth.GetBitCount(), bitDepth.GetBitCount(),
predictionScratch); predictionScratch);
Av1CompoundInterPredictor.PredictCompound(
source,
sourceStride,
sourceIndex,
actualPredictions,
blockSize,
blockSize,
blockSize,
Av1InterpolationFilter.Bilinear,
Av1InterpolationFilter.Bilinear,
sourceColumnQ4 & 15,
sourceRowQ4 & 15,
bitDepth.GetBitCount(),
predictionScratch);
expectedPredictions = expectedSecond; expectedPredictions = expectedSecond;
actualPredictions = actualSecond;
} }
Assert.Equal(expectedFirst, actualFirst);
Assert.Equal(expectedSecond, actualSecond);
ushort[] expected = new ushort[blockSize * blockSize]; ushort[] expected = new ushort[blockSize * blockSize];
if (compoundType == Av1CompoundType.DistanceWeighted) if (compoundType == Av1CompoundType.DistanceWeighted)
{ {
@ -788,6 +832,50 @@ public class Av1CompoundBlockDecoderTests
Assert.NotEqual((ushort)60, expected[0]); Assert.NotEqual((ushort)60, expected[0]);
} }
else if (compoundType == Av1CompoundType.Wedge)
{
ReadOnlySpan<byte> wedgeMask =
[
0, 0, 0, 1, 1, 2, 4, 6,
0, 1, 1, 2, 4, 6, 11, 18,
1, 2, 4, 6, 11, 18, 27, 37,
4, 6, 11, 18, 27, 37, 46, 53,
11, 18, 27, 37, 46, 53, 58, 60,
27, 37, 46, 53, 58, 60, 62, 63,
46, 53, 58, 60, 62, 63, 63, 64,
58, 60, 62, 63, 63, 64, 64, 64,
];
byte[] generatedWedgeMask = new byte[blockSize * blockSize];
Av1WedgeMask.Fill(
generatedWedgeMask,
blockSize,
Av1BlockSize.Block8x8,
wedgeIndex: 0,
wedgeSign: true,
subX: 0,
subY: 0,
invert: false);
Assert.Equal(wedgeMask, generatedWedgeMask);
Av1CompoundIntermediateMaskBlendPredictor.BlendIntermediate(
expected,
blockSize,
expectedFirst,
blockSize,
expectedSecond,
blockSize,
wedgeMask,
blockSize,
blockSize,
blockSize,
subX: 0,
subY: 0,
bitDepth.GetBitCount());
Assert.NotEqual((ushort)60, expected[0]);
}
else else
{ {
Av1CompoundIntermediateAveragePredictor.AverageIntermediate( Av1CompoundIntermediateAveragePredictor.AverageIntermediate(

41
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundInterPredictorTests.cs

@ -593,6 +593,47 @@ public class Av1CompoundInterPredictorTests
Assert.Equal(expectedWeighted, actualWeighted); Assert.Equal(expectedWeighted, actualWeighted);
} }
int maskStride = width + 5;
byte[] mask = new byte[maskStride * height];
ushort[] expectedMasked = new ushort[destinationStride * height];
ushort[] actualMasked = new ushort[destinationStride * height];
expectedMasked.AsSpan().Fill(0xA5A5);
actualMasked.AsSpan().Fill(0xA5A5);
for (int row = 0; row < height; row++)
{
for (int column = 0; column < width; column++)
{
byte alpha = (byte)(((row * 29) + (column * 17) + 3) % 65);
mask[(row * maskStride) + column] = alpha;
int intermediateIndex = (row * intermediateStride) + column;
int result = ((alpha * expectedFirst[intermediateIndex]) +
((64 - alpha) * expectedSecond[intermediateIndex])) >> 6;
result -= roundOffset;
result = (result + (1 << (roundBits - 1))) >> roundBits;
expectedMasked[(row * destinationStride) + column] =
(ushort)Math.Clamp(result, 0, maximum);
}
}
Av1CompoundIntermediateMaskBlendPredictor.BlendIntermediate(
actualMasked,
destinationStride,
actualFirst,
intermediateStride,
actualSecond,
intermediateStride,
mask,
maskStride,
width,
height,
subX: 0,
subY: 0,
bitDepth);
Assert.Equal(expectedMasked, actualMasked);
} }
} }
} }

16
tests/Images/Input/Heif/Av1/Conformance/README.md

@ -149,12 +149,18 @@ conditions. Current `aomdec` produced all 19 YUV444 frames. The final frame's 19
SHA-256 `E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained SHA-256 `E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained
Y4M with zero differing samples. Y4M with zero differing samples.
The wedge fixture's 5,374-byte AV1 `mdat` payload was decoded under the same current-libaom conditions.
Current `aomdec` produced all 19 YUV444 frames. The final frame's 19,200 native samples have SHA-256
`E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899` and match the retained Y4M
with zero differing samples.
The production tests independently require their decoded mode states. The distance-weighted input must The production tests independently require their decoded mode states. The distance-weighted input must
exercise distance-weighted compound prediction and the inter-intra input must exercise both smooth and exercise distance-weighted compound prediction, the wedge input must exercise both mask orientations,
wedge inter-intra prediction. The tests decode all preceding samples, compare final native Y, Cb, and Cr and the inter-intra input must exercise both smooth and wedge inter-intra prediction. The tests decode all
planes exactly, compare final RGBA presentation through ImageSharp's established reference-output API, preceding samples, compare final native Y, Cb, and Cr planes exactly, compare final RGBA presentation
and repeat reconstruction with constrained tracked allocation. The retained PNG files are presentation through ImageSharp's established reference-output API, and repeat reconstruction with constrained tracked
evidence only and are not AV1 reconstruction references. allocation. The retained PNG files are presentation evidence only and are not AV1 reconstruction
references.
## Overlapping motion-compensation fixture ## Overlapping motion-compensation fixture

Loading…
Cancel
Save