Browse Source

Optimize AV1 CDEF decoder integration

pull/2633/head
James Jackson-South 1 week ago
parent
commit
20df9115d0
  1. 4
      HEIF_IMPLEMENTATION_PLAN.md
  2. 278
      src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefDecoder.cs
  3. 122
      src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.cs
  4. 67
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CdefFilterTests.cs

4
HEIF_IMPLEMENTATION_PLAN.md

@ -351,7 +351,7 @@ This assessment was reconciled with the source tree on 2026-08-26. Unless a resu
- The single-still `Av1Decoder` path now parses tile state before allocating and reconstructing one independently decodable frame, and it disposes the reconstruction planes after pixel conversion. It deliberately does not retain animation/video reference frames or implement `show_existing_frame` playback state.
- Transform coefficient entropy derivation and updates now address the above contexts relative to the tile column and the left contexts relative to the current superblock row, preserve luma coordinates independently of chroma subsampling, and test every packed context entry for the libaom any-nonzero rule. Extended vertical partition updates advance the mode-information column rather than the row. The existing multi-superblock 4:4:4 AVIF fixture now completes tile parsing; independent coefficient-context vectors across tile boundaries, chroma layouts, bit depths, and edge-clipped transforms remain required.
- The reconstruction pipeline now records plane-relative transform geometry, preserves tile-local delta-Q and delta-LF predictors, derives segmentation and reference-adjusted filter levels, and runs the exact AV1 4-, 6-, 8-, and 14-tap deblocking kernels in normative vertical-then-horizontal order. Deblocking uses the same closed edge-operator architecture as the HEVC filter, with operators specialized by sample storage and orientation, `Vector128<int>` lanes representing the four rows or columns along an edge, and an allocation-free scalar fallback for disabled intrinsics. Exact native-plane comparison with pinned scalar libaom output now verifies active deblocking and complete reconstruction for real 8-bit 4:2:0, 10-bit 4:4:4, and 12-bit 4:4:4 content; genuine AVIF containers separately verify presentation and public bit-depth metadata. The pipeline then applies CDEF through one semantic filter architecture: paired AVX2 and single-block `Vector128` direction analysis, closed primary/secondary strength operators, packed 4x4/4x8/8x4/8x8 constrained filtering, byte/16-bit output operators, and an exact allocation-free scalar fallback. The decoder still uses heap-allocated immutable plane snapshots and direction/variance maps; moving those to allocator-owned memory and using paired direction traversal is the next CDEF step. Active super-resolution derives the Appendix A bounded coded width and applies the exact 64-phase, 8-tap horizontal filter with aligned reconstruction-edge input, 8/10/12-bit clipping, and the existing cross-platform `Vector128_.MultiplyAddAdjacent` helper. Loop restoration follows super-resolution, preserves the required pre-CDEF deblocked context at internal stripes, and applies decoded Wiener or self-guided units from immutable plane snapshots. The visible still-image path then applies the complete self-contained film-grain parameter set after all in-loop filters. Independent 8-, 10-, and 12-bit vectors exercising CDEF, super-resolution, restoration, and grain remain required before those stages have external pixel-level verification.
- The reconstruction pipeline now records plane-relative transform geometry, preserves tile-local delta-Q and delta-LF predictors, derives segmentation and reference-adjusted filter levels, and runs the exact AV1 4-, 6-, 8-, and 14-tap deblocking kernels in normative vertical-then-horizontal order. Deblocking uses the same closed edge-operator architecture as the HEVC filter, with operators specialized by sample storage and orientation, `Vector128<int>` lanes representing the four rows or columns along an edge, and an allocation-free scalar fallback for disabled intrinsics. Exact native-plane comparison with pinned scalar libaom output now verifies active deblocking and complete reconstruction for real 8-bit 4:2:0, 10-bit 4:4:4, and 12-bit 4:4:4 content; genuine AVIF containers separately verify presentation and public bit-depth metadata. The pipeline then applies CDEF through one semantic filter architecture: paired AVX2 and single-block `Vector128` direction analysis, closed primary/secondary strength operators, packed 4x4/4x8/8x4/8x8 constrained filtering, byte/16-bit output operators, and an exact allocation-free scalar fallback. Decoder orchestration now owns the immutable plane snapshots and clean direction/variance maps through ImageSharp's memory allocator, widens 8-bit source rows with the same AVX2/128-bit/scalar tiers as libaom, lists each unit's non-skipped blocks in fixed inline storage, analyzes listed blocks in pairs, and writes filtered bytes or 16-bit samples directly to the frame planes. Active super-resolution derives the Appendix A bounded coded width and applies the exact 64-phase, 8-tap horizontal filter with aligned reconstruction-edge input, 8/10/12-bit clipping, and the existing cross-platform `Vector128_.MultiplyAddAdjacent` helper. Loop restoration follows super-resolution, preserves the required pre-CDEF deblocked context at internal stripes, and applies decoded Wiener or self-guided units from immutable plane snapshots. The visible still-image path then applies the complete self-contained film-grain parameter set after all in-loop filters. Independent 8-, 10-, and 12-bit vectors exercising CDEF, super-resolution, restoration, and grain remain required before those stages have external pixel-level verification.
- Palette mode now reads the normative luma/chroma mode and size CDFs, neighbor color caches, high-bit-depth color syntax, diagonal color-index maps, and clipped-edge padding. Reconstruction retains libaom's exact indexed-palette contract while using repeated eight-entry tables and native lane-local shuffles across runtime-selected `Vector512`, `Vector256`, and `Vector128` paths before an exact scalar fallback. `FeatureTestRunner` verifies palette sizes 2-8, transform widths 4-64, padded rows, and 8/10/12-bit samples. This path is not production-complete until its syntax and reconstruction are verified with an independently encoded palette AVIF fixture.
- Non-reduced still-image sequence parsing now consumes decoder-model operating parameters, temporal presentation fields, and OBU-layer-applicable buffer-removal fields only to preserve AV1 bit alignment. The scheduling values are not retained, and no movie, track, timing, playback, or generic ISO BMFF surface has been introduced. Existing focused sequence-header coverage exercises only reduced-still syntax, so an independent non-reduced still AVIF vector remains required.
- Loop-restoration unit parsing records tile-local switchable/Wiener/self-guided filter selections and coefficients in frame-owned plane grids, including super-resolution-adjusted unit corners and the corrected conditional 64x64-superblock unit-size bit. The active restoration stage implements the normative unit geometry, striped deblocked boundaries, Wiener filtering, self-guided projection, and 8/10/12-bit clipping, while reusing the existing SIMD-backed super-resolution and adjacent multiply/add primitives. Independently encoded fixtures covering every parameter set, plane layout, bit depth, and frame-edge geometry are still required. Other normative independently decodable still-image syntax paths still contain `NotImplementedException` or equivalent unsupported branches. Tile-local palette CDF adaptation is present; the remaining still-image frame-context behavior requires a separate source audit without introducing sequence playback state.
@ -506,7 +506,7 @@ Implement and verify in dependency order:
- [x] Verify deblocking syntax, filter-level derivation, and boundary traversal with independently encoded 8/10/12-bit AV1 samples and exact scalar-libaom planes; verify presented reconstruction and public precision with genuine AVIF containers at every supported bit depth.
- [ ] CDEF.
- [x] Replace the scalar helper bucket with an allocation-free SIMD-first `Av1CdefFilter`: vector direction analysis, 4x4/4x8/8x4/8x8 constrained filtering, byte/16-bit output operators, and one exact scalar fallback verified through `FeatureTestRunner`.
- [ ] Move the immutable deblocked-plane snapshot and frame-wide direction/variance maps to allocator-owned decoder memory, preserving luma-first direction derivation, skipped-block selection, asymmetric chroma direction conversion, and frame-edge sentinels.
- [x] Move the immutable deblocked-plane snapshot and frame-wide direction/variance maps to allocator-owned decoder memory, preserving luma-first paired direction derivation, fixed-storage skipped-block selection, asymmetric chroma direction conversion, frame-edge sentinels, SIMD source widening, and direct frame-plane output.
- [ ] Verify CDEF syntax, strength derivation, unit/block traversal, subsampling, frame edges, and presented reconstruction with independently encoded active-CDEF AV1/AVIF fixtures across 8/10/12-bit samples.
- [ ] Super-resolution scaling.
- [ ] Loop restoration.

278
src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefDecoder.cs

@ -1,9 +1,12 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using System.Buffers;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.Cdef;
@ -22,6 +25,11 @@ internal class Av1CdefDecoder
/// </summary>
private const int SourceBorder = 2;
/// <summary>
/// The maximum number of non-skipped 8x8 luma blocks in one 64x64 CDEF unit.
/// </summary>
private const int MaximumBlocksPerUnit = 8 * 8;
/// <summary>
/// The sequence-level superblock, bit-depth, and color configuration.
/// </summary>
@ -91,8 +99,12 @@ internal class Av1CdefDecoder
int lumaBlockColumnCount = this.frameHeader.ModeInfoColumnCount >> 1;
int lumaBlockRowCount = this.frameHeader.ModeInfoRowCount >> 1;
int[] directions = new int[lumaBlockColumnCount * lumaBlockRowCount];
int[] variances = new int[directions.Length];
int mapLength = lumaBlockColumnCount * lumaBlockRowCount;
MemoryAllocator allocator = this.frameBuffer.MemoryAllocator;
using IMemoryOwner<int> directionOwner = allocator.Allocate<int>(mapLength, AllocationOptions.Clean);
using IMemoryOwner<int> varianceOwner = allocator.Allocate<int>(mapLength, AllocationOptions.Clean);
Span<int> directions = directionOwner.Memory.Span[..mapLength];
Span<int> variances = varianceOwner.Memory.Span[..mapLength];
ObuColorConfig colorConfig = this.sequenceHeader.ColorConfig;
// Luma must be processed first even when its strengths are zero because chroma CDEF
@ -119,18 +131,21 @@ internal class Av1CdefDecoder
Av1Plane plane,
int subsamplingX,
int subsamplingY,
int[] directions,
int[] variances,
Span<int> directions,
Span<int> variances,
int lumaBlockColumnCount)
{
int planeWidth = this.frameHeader.ModeInfoColumnCount << (Av1Constants.ModeInfoSizeLog2 - subsamplingX);
int planeHeight = this.frameHeader.ModeInfoRowCount << (Av1Constants.ModeInfoSizeLog2 - subsamplingY);
int sourceStride = planeWidth + (SourceBorder * 2);
int sourceLength = (planeHeight + (SourceBorder * 2)) * sourceStride;
using IMemoryOwner<ushort> sourceOwner = this.frameBuffer.MemoryAllocator.Allocate<ushort>(sourceLength);
Span<ushort> source = sourceOwner.Memory.Span[..sourceLength];
// CDEF output must never become input to a later block. The sentinel border also makes
// frame-edge taps follow AV1 without exposing the frame buffer's prediction padding.
ushort[] source = new ushort[(planeHeight + (SourceBorder * 2)) * sourceStride];
Array.Fill(source, Av1CdefFilter.VeryLarge);
// frame-edge taps follow AV1 without exposing the frame buffer's prediction padding. Every
// sample in the requested working allocation is initialized before any filter can read it.
source.Fill(Av1CdefFilter.VeryLarge);
Span<byte> lowBitDepthDestination = default;
Span<ushort> highBitDepthDestination = default;
@ -156,25 +171,31 @@ internal class Av1CdefDecoder
out destinationStride);
}
for (int row = 0; row < planeHeight; row++)
int sourceOffset = (SourceBorder * sourceStride) + SourceBorder;
int destinationOffset = destinationStride;
if (this.frameBuffer.BytesPerSample == 2)
{
Span<ushort> sourceRow = source.AsSpan(
((row + SourceBorder) * sourceStride) + SourceBorder,
planeWidth);
int destinationOffset = destinationStride + (row * destinationStride);
if (this.frameBuffer.BytesPerSample == 2)
{
highBitDepthDestination.Slice(destinationOffset, planeWidth).CopyTo(sourceRow);
}
else
{
Span<byte> destinationRow = lowBitDepthDestination.Slice(destinationOffset, planeWidth);
for (int column = 0; column < planeWidth; column++)
{
sourceRow[column] = destinationRow[column];
}
}
Av1CdefFilter.CopyPlane(
highBitDepthDestination,
destinationOffset,
destinationStride,
source,
sourceOffset,
sourceStride,
planeWidth,
planeHeight);
}
else
{
Av1CdefFilter.CopyPlane(
lowBitDepthDestination,
destinationOffset,
destinationStride,
source,
sourceOffset,
sourceStride,
planeWidth,
planeHeight);
}
ObuConstraintDirectionalEnhancementFilterParameters parameters = this.frameHeader.CdefParameters;
@ -183,7 +204,6 @@ internal class Av1CdefDecoder
int blockHeight = 8 >> subsamplingY;
int unitColumnCount = (this.frameHeader.ModeInfoColumnCount + CdefUnitModeInfoSize - 1) / CdefUnitModeInfoSize;
int unitRowCount = (this.frameHeader.ModeInfoRowCount + CdefUnitModeInfoSize - 1) / CdefUnitModeInfoSize;
Span<ushort> filteredBlock = stackalloc ushort[8 * 8];
for (int unitRow = 0; unitRow < unitRowCount; unitRow++)
{
@ -217,6 +237,8 @@ internal class Av1CdefDecoder
int damping = parameters.Damping + coefficientShift - (plane == Av1Plane.Y ? 0 : 1);
int unitModeInfoRowEnd = Math.Min(unitModeInfoRow + CdefUnitModeInfoSize, this.frameHeader.ModeInfoRowCount);
int unitModeInfoColumnEnd = Math.Min(unitModeInfoColumn + CdefUnitModeInfoSize, this.frameHeader.ModeInfoColumnCount);
CdefBlockList blocks = default;
int blockCount = 0;
for (int blockModeInfoRow = unitModeInfoRow; blockModeInfoRow < unitModeInfoRowEnd; blockModeInfoRow += 2)
{
@ -227,50 +249,105 @@ internal class Av1CdefDecoder
continue;
}
int lumaBlockRow = blockModeInfoRow >> 1;
int lumaBlockColumn = blockModeInfoColumn >> 1;
int directionIndex = (lumaBlockRow * lumaBlockColumnCount) + lumaBlockColumn;
int planeColumn = (blockModeInfoColumn << Av1Constants.ModeInfoSizeLog2) >> subsamplingX;
int planeRow = (blockModeInfoRow << Av1Constants.ModeInfoSizeLog2) >> subsamplingY;
int sourceOffset = ((planeRow + SourceBorder) * sourceStride) + planeColumn + SourceBorder;
blocks[blockCount++] = new CdefBlock(blockModeInfoColumn, blockModeInfoRow);
}
}
if (plane == Av1Plane.Y)
{
directions[directionIndex] = Av1CdefFilter.FindDirection(
source,
sourceOffset,
sourceStride,
coefficientShift,
out variances[directionIndex]);
}
if (plane == Av1Plane.Y)
{
int blockIndex = 0;
if (codedStrength == 0)
{
continue;
}
// libaom analyzes two listed 8x8 blocks together. The per-unit fixed list preserves that traversal
// without allocating a managed block list or repeating four skip-map lookups during filtering.
for (; blockIndex < blockCount - 1; blockIndex += 2)
{
CdefBlock firstBlock = blocks[blockIndex];
CdefBlock secondBlock = blocks[blockIndex + 1];
int firstDirectionIndex = firstBlock.GetDirectionIndex(lumaBlockColumnCount);
int secondDirectionIndex = secondBlock.GetDirectionIndex(lumaBlockColumnCount);
int filteredPrimaryStrength = plane == Av1Plane.Y
? Av1CdefFilter.AdjustStrength(primaryStrength, variances[directionIndex])
: primaryStrength;
Av1CdefFilter.FindDirections(
source,
firstBlock.GetSourceOffset(sourceStride, SourceBorder),
secondBlock.GetSourceOffset(sourceStride, SourceBorder),
sourceStride,
coefficientShift,
out directions[firstDirectionIndex],
out variances[firstDirectionIndex],
out directions[secondDirectionIndex],
out variances[secondDirectionIndex]);
}
if (filteredPrimaryStrength == 0 && secondaryStrength == 0)
{
continue;
}
if (blockIndex < blockCount)
{
CdefBlock block = blocks[blockIndex];
int directionIndex = block.GetDirectionIndex(lumaBlockColumnCount);
directions[directionIndex] = Av1CdefFilter.FindDirection(
source,
block.GetSourceOffset(sourceStride, SourceBorder),
sourceStride,
coefficientShift,
out variances[directionIndex]);
}
}
for (int blockIndex = 0; blockIndex < blockCount; blockIndex++)
{
CdefBlock block = blocks[blockIndex];
int directionIndex = block.GetDirectionIndex(lumaBlockColumnCount);
if (codedStrength == 0)
{
continue;
}
int filteredPrimaryStrength = plane == Av1Plane.Y
? Av1CdefFilter.AdjustStrength(primaryStrength, variances[directionIndex])
: primaryStrength;
// Secondary-only filtering uses direction zero; otherwise chroma remaps the
// luma direction into its asymmetrically subsampled sample grid when required.
int direction = primaryStrength != 0
? Av1CdefFilter.ConvertDirection(directions[directionIndex], subsamplingX, subsamplingY)
: 0;
if (filteredPrimaryStrength == 0 && secondaryStrength == 0)
{
continue;
}
// Secondary-only filtering uses direction zero; otherwise chroma remaps the
// luma direction into its asymmetrically subsampled sample grid when required.
int direction = primaryStrength != 0
? Av1CdefFilter.ConvertDirection(directions[directionIndex], subsamplingX, subsamplingY)
: 0;
int planeColumn = (block.ModeInfoColumn << Av1Constants.ModeInfoSizeLog2) >> subsamplingX;
int planeRow = (block.ModeInfoRow << Av1Constants.ModeInfoSizeLog2) >> subsamplingY;
int blockSourceOffset = ((planeRow + SourceBorder) * sourceStride) + planeColumn + SourceBorder;
int blockDestinationOffset = destinationStride + (planeRow * destinationStride) + planeColumn;
if (this.frameBuffer.BytesPerSample == 2)
{
Av1CdefFilter.FilterBlock(
source,
sourceOffset,
blockSourceOffset,
sourceStride,
filteredBlock,
0,
highBitDepthDestination,
blockDestinationOffset,
destinationStride,
filteredPrimaryStrength,
secondaryStrength,
direction,
damping,
damping,
coefficientShift,
blockWidth,
blockHeight);
}
else
{
Av1CdefFilter.FilterBlock(
source,
blockSourceOffset,
sourceStride,
lowBitDepthDestination,
blockDestinationOffset,
destinationStride,
filteredPrimaryStrength,
secondaryStrength,
direction,
@ -279,27 +356,6 @@ internal class Av1CdefDecoder
coefficientShift,
blockWidth,
blockHeight);
int destinationOffset = destinationStride + (planeRow * destinationStride) + planeColumn;
for (int row = 0; row < blockHeight; row++)
{
ReadOnlySpan<ushort> filteredRow = filteredBlock.Slice(row * blockWidth, blockWidth);
if (this.frameBuffer.BytesPerSample == 2)
{
filteredRow.CopyTo(highBitDepthDestination.Slice(destinationOffset + (row * destinationStride), blockWidth));
}
else
{
Span<byte> destinationRow = lowBitDepthDestination.Slice(
destinationOffset + (row * destinationStride),
blockWidth);
for (int column = 0; column < blockWidth; column++)
{
destinationRow[column] = (byte)filteredRow[column];
}
}
}
}
}
}
@ -349,4 +405,64 @@ internal class Av1CdefDecoder
return true;
}
/// <summary>
/// Stores the non-skipped blocks in one CDEF unit without a managed allocation.
/// </summary>
[InlineArray(MaximumBlocksPerUnit)]
private struct CdefBlockList
{
/// <summary>
/// The first block in the inline storage.
/// </summary>
private CdefBlock element0;
}
/// <summary>
/// Identifies one 8x8 luma block by its frame-relative mode-information coordinates.
/// </summary>
private readonly struct CdefBlock
{
/// <summary>
/// Initializes a new instance of the <see cref="CdefBlock"/> struct.
/// </summary>
/// <param name="modeInfoColumn">The frame-relative column in 4x4 luma units.</param>
/// <param name="modeInfoRow">The frame-relative row in 4x4 luma units.</param>
public CdefBlock(int modeInfoColumn, int modeInfoRow)
{
this.ModeInfoColumn = modeInfoColumn;
this.ModeInfoRow = modeInfoRow;
}
/// <summary>
/// Gets the frame-relative column in 4x4 luma units.
/// </summary>
public int ModeInfoColumn { get; }
/// <summary>
/// Gets the frame-relative row in 4x4 luma units.
/// </summary>
public int ModeInfoRow { get; }
/// <summary>
/// Gets the frame-wide direction-map index for this block.
/// </summary>
/// <param name="lumaBlockColumnCount">The number of 8x8 blocks in an aligned luma row.</param>
/// <returns>The direction-map index.</returns>
public int GetDirectionIndex(int lumaBlockColumnCount)
=> ((this.ModeInfoRow >> 1) * lumaBlockColumnCount) + (this.ModeInfoColumn >> 1);
/// <summary>
/// Gets the offset of this luma block in the bordered CDEF source plane.
/// </summary>
/// <param name="sourceStride">The number of samples between adjacent source rows.</param>
/// <param name="sourceBorder">The number of unavailable samples surrounding the source.</param>
/// <returns>The source-plane offset.</returns>
public int GetSourceOffset(int sourceStride, int sourceBorder)
{
int planeColumn = this.ModeInfoColumn << Av1Constants.ModeInfoSizeLog2;
int planeRow = this.ModeInfoRow << Av1Constants.ModeInfoSizeLog2;
return ((planeRow + sourceBorder) * sourceStride) + planeColumn + sourceBorder;
}
}
}

122
src/ImageSharp/Formats/Heif/Av1/Pipeline/Cdef/Av1CdefFilter.cs

@ -60,6 +60,93 @@ internal static class Av1CdefFilter
public static abstract bool EnableSecondary { get; }
}
/// <summary>
/// Copies an eight-bit sample rectangle into the 16-bit CDEF working plane.
/// </summary>
/// <param name="source">The source sample plane.</param>
/// <param name="sourceOffset">The offset of the rectangle's top-left source sample.</param>
/// <param name="sourceStride">The number of samples between adjacent source rows.</param>
/// <param name="destination">The destination CDEF working plane.</param>
/// <param name="destinationOffset">The offset of the rectangle's top-left destination sample.</param>
/// <param name="destinationStride">The number of samples between adjacent destination rows.</param>
/// <param name="width">The rectangle width in samples.</param>
/// <param name="height">The even rectangle height in samples.</param>
public static void CopyPlane(
ReadOnlySpan<byte> source,
int sourceOffset,
int sourceStride,
Span<ushort> destination,
int destinationOffset,
int destinationStride,
int width,
int height)
{
ref byte sourceBase = ref MemoryMarshal.GetReference(source);
ref ushort destinationBase = ref MemoryMarshal.GetReference(destination);
if (Avx2.IsSupported)
{
// AV1 plane dimensions are multiples of four, so the CDEF copy has an even row count. Processing two rows
// together follows libaom's AVX2 scheduling while each conversion widens sixteen unsigned samples exactly.
for (int row = 0; row < height; row += 2)
{
int firstSourceRow = sourceOffset + (row * sourceStride);
int secondSourceRow = firstSourceRow + sourceStride;
int firstDestinationRow = destinationOffset + (row * destinationStride);
int secondDestinationRow = firstDestinationRow + destinationStride;
int column = 0;
for (; column <= width - Vector128<byte>.Count; column += Vector128<byte>.Count)
{
Vector128<byte> first = Vector128.LoadUnsafe(ref sourceBase, (nuint)(firstSourceRow + column));
Vector128<byte> second = Vector128.LoadUnsafe(ref sourceBase, (nuint)(secondSourceRow + column));
Avx2.ConvertToVector256Int16(first).AsUInt16().StoreUnsafe(ref destinationBase, (nuint)(firstDestinationRow + column));
Avx2.ConvertToVector256Int16(second).AsUInt16().StoreUnsafe(ref destinationBase, (nuint)(secondDestinationRow + column));
}
CopyRemainingSamples(ref sourceBase, firstSourceRow, ref destinationBase, firstDestinationRow, column, width);
CopyRemainingSamples(ref sourceBase, secondSourceRow, ref destinationBase, secondDestinationRow, column, width);
}
return;
}
for (int row = 0; row < height; row++)
{
int sourceRow = sourceOffset + (row * sourceStride);
int destinationRow = destinationOffset + (row * destinationStride);
CopyRemainingSamples(ref sourceBase, sourceRow, ref destinationBase, destinationRow, 0, width);
}
}
/// <summary>
/// Copies a 16-bit sample rectangle into the CDEF working plane.
/// </summary>
/// <param name="source">The source sample plane.</param>
/// <param name="sourceOffset">The offset of the rectangle's top-left source sample.</param>
/// <param name="sourceStride">The number of samples between adjacent source rows.</param>
/// <param name="destination">The destination CDEF working plane.</param>
/// <param name="destinationOffset">The offset of the rectangle's top-left destination sample.</param>
/// <param name="destinationStride">The number of samples between adjacent destination rows.</param>
/// <param name="width">The rectangle width in samples.</param>
/// <param name="height">The rectangle height in samples.</param>
public static void CopyPlane(
ReadOnlySpan<ushort> source,
int sourceOffset,
int sourceStride,
Span<ushort> destination,
int destinationOffset,
int destinationStride,
int width,
int height)
{
for (int row = 0; row < height; row++)
{
ReadOnlySpan<ushort> sourceRow = source.Slice(sourceOffset + (row * sourceStride), width);
sourceRow.CopyTo(destination.Slice(destinationOffset + (row * destinationStride), width));
}
}
/// <summary>
/// Finds the dominant direction of an 8x8 luma block and its directional variance.
/// </summary>
@ -1442,6 +1529,41 @@ internal static class Av1CdefFilter
return Vector128.Create(Unsafe.ReadUnaligned<ulong>(ref sourceBytes), 0UL).AsInt16();
}
/// <summary>
/// Widens the remaining samples in one eight-bit source row into the CDEF working plane.
/// </summary>
/// <param name="source">The first element in the source plane.</param>
/// <param name="sourceRow">The offset of the first source sample in the row.</param>
/// <param name="destination">The first element in the destination plane.</param>
/// <param name="destinationRow">The offset of the first destination sample in the row.</param>
/// <param name="column">The first column not already processed by a wider vector path.</param>
/// <param name="width">The rectangle width in samples.</param>
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static void CopyRemainingSamples(
ref byte source,
int sourceRow,
ref ushort destination,
int destinationRow,
int column,
int width)
{
if (Vector128.IsHardwareAccelerated)
{
for (; column <= width - Vector64<byte>.Count; column += Vector64<byte>.Count)
{
ref byte sourceBytes = ref Unsafe.Add(ref source, sourceRow + column);
ulong packed = Unsafe.ReadUnaligned<ulong>(ref sourceBytes);
Vector128<ushort> widened = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsByte());
widened.StoreUnsafe(ref destination, (nuint)(destinationRow + column));
}
}
for (; column < width; column++)
{
Unsafe.Add(ref destination, destinationRow + column) = Unsafe.Add(ref source, sourceRow + column);
}
}
/// <summary>
/// Loads two 8-wide rows or four 4-wide rows into the independent row groups of one 256-bit vector.
/// </summary>

67
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CdefFilterTests.cs

@ -16,7 +16,7 @@ public class Av1CdefFilterTests
/// <summary>
/// The hardware configurations required to exercise packed filtering and the scalar fallback.
/// </summary>
private const HwIntrinsics Configurations = HwIntrinsics.AllowAll | HwIntrinsics.DisableHWIntrinsic;
private const HwIntrinsics Configurations = HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX2 | HwIntrinsics.DisableHWIntrinsic;
/// <summary>
/// The row stride of the bordered source plane used by the filter tests.
@ -42,6 +42,13 @@ public class Av1CdefFilterTests
public void FilterBlockMatchesIndependentDefinitionAcrossIntrinsicTiers()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateFilters, Configurations);
/// <summary>
/// Verifies eight-bit widening and 16-bit copying across packed and scalar execution tiers.
/// </summary>
[Fact]
public void CopyPlaneMatchesIndependentDefinitionAcrossIntrinsicTiers()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidatePlaneCopies, Configurations);
/// <summary>
/// Verifies the complete asymmetric chroma direction mappings and the unchanged symmetric mappings.
/// </summary>
@ -140,6 +147,64 @@ public class Av1CdefFilterTests
}
}
/// <summary>
/// Exercises CDEF source-plane preparation across vector boundaries and row padding.
/// </summary>
private static void ValidatePlaneCopies()
{
const int height = 6;
foreach (int width in new[] { 4, 7, 8, 15, 16, 31, 32, 37 })
{
int sourceStride = width + 9;
int destinationStride = width + 11;
int sourceOffset = sourceStride + 3;
int destinationOffset = destinationStride + 5;
byte[] byteSource = new byte[(height + 2) * sourceStride];
ushort[] ushortSource = new ushort[byteSource.Length];
ushort[] expectedByteDestination = Enumerable.Repeat((ushort)0x7A7A, (height + 2) * destinationStride).ToArray();
ushort[] expectedUShortDestination = Enumerable.Repeat((ushort)0x5A5A, expectedByteDestination.Length).ToArray();
for (int row = 0; row < height; row++)
{
for (int column = 0; column < width; column++)
{
int index = sourceOffset + (row * sourceStride) + column;
byteSource[index] = (byte)((row * 47) + (column * 13));
ushortSource[index] = (ushort)(byteSource[index] * 17);
expectedByteDestination[destinationOffset + (row * destinationStride) + column] = byteSource[index];
expectedUShortDestination[destinationOffset + (row * destinationStride) + column] = ushortSource[index];
}
}
ushort[] actualByteDestination = Enumerable.Repeat((ushort)0x7A7A, expectedByteDestination.Length).ToArray();
ushort[] actualUShortDestination = Enumerable.Repeat((ushort)0x5A5A, expectedUShortDestination.Length).ToArray();
Av1CdefFilter.CopyPlane(
byteSource,
sourceOffset,
sourceStride,
actualByteDestination,
destinationOffset,
destinationStride,
width,
height);
Av1CdefFilter.CopyPlane(
ushortSource,
sourceOffset,
sourceStride,
actualUShortDestination,
destinationOffset,
destinationStride,
width,
height);
Assert.Equal(expectedByteDestination, actualByteDestination);
Assert.Equal(expectedUShortDestination, actualUShortDestination);
}
}
/// <summary>
/// Populates two adjacent 8x8 blocks with deterministic directional samples.
/// </summary>

Loading…
Cancel
Save