Browse Source

Implement SIMD-first AV1 palette reconstruction

pull/2633/head
James Jackson-South 7 days ago
parent
commit
99e42d1a6a
  1. 7
      HEIF_IMPLEMENTATION_PLAN.md
  2. 268
      src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PalettePredictor.cs
  3. 15
      src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs
  4. 149
      tests/ImageSharp.Benchmarks/Codecs/Heif/Av1PalettePredictionBenchmarks.cs
  5. 138
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PalettePredictorTests.cs

7
HEIF_IMPLEMENTATION_PLAN.md

@ -352,13 +352,13 @@ This assessment was reconciled with the source tree on 2026-08-26. Unless a resu
- The single-still `Av1Decoder` path now parses tile state before allocating and reconstructing one independently decodable frame, and it disposes the reconstruction planes after pixel conversion. It deliberately does not retain animation/video reference frames or implement `show_existing_frame` playback state.
- Transform coefficient entropy derivation and updates now address the above contexts relative to the tile column and the left contexts relative to the current superblock row, preserve luma coordinates independently of chroma subsampling, and test every packed context entry for the libaom any-nonzero rule. Extended vertical partition updates advance the mode-information column rather than the row. The existing multi-superblock 4:4:4 AVIF fixture now completes tile parsing; independent coefficient-context vectors across tile boundaries, chroma layouts, bit depths, and edge-clipped transforms remain required.
- The reconstruction pipeline now records plane-relative transform geometry, preserves tile-local delta-Q and delta-LF predictors, derives segmentation and reference-adjusted filter levels, and runs the exact scalar low/high-bit-depth AV1 4-, 6-, 8-, and 14-tap deblocking kernels in normative vertical-then-horizontal order. It then applies scalar CDEF direction search, luma variance adjustment, primary and secondary constrained taps, chroma direction conversion, high-bit-depth scaling, skipped-block selection, and frame-edge sentinel handling from immutable per-plane snapshots. Active super-resolution derives the Appendix A bounded coded width and applies the exact 64-phase, 8-tap horizontal filter with aligned reconstruction-edge input, 8/10/12-bit clipping, and the existing cross-platform `Vector128_.MultiplyAddAdjacent` helper. Loop restoration follows super-resolution, preserves the required pre-CDEF deblocked context at internal stripes, and applies decoded Wiener or self-guided units from immutable plane snapshots. The visible still-image path then applies the complete self-contained film-grain parameter set after all in-loop filters. Independent 8-, 10-, and 12-bit AVIF vectors exercising every active filter and grain stage remain required before these paths have external pixel-level verification.
- Palette mode now reads the normative luma/chroma mode and size CDFs, neighbor color caches, high-bit-depth color syntax, diagonal color-index maps, clipped-edge padding, and direct sample prediction through the existing reconstruction pipeline. The current scalar behavior matches the pinned libaom source, but this path is not production-complete until it is expressed through the SIMD-first predictor architecture and verified with an independently encoded palette AVIF fixture.
- Palette mode now reads the normative luma/chroma mode and size CDFs, neighbor color caches, high-bit-depth color syntax, diagonal color-index maps, and clipped-edge padding. Reconstruction retains libaom's exact indexed-palette contract while using repeated eight-entry tables and native lane-local shuffles across runtime-selected `Vector512`, `Vector256`, and `Vector128` paths before an exact scalar fallback. `FeatureTestRunner` verifies palette sizes 2-8, transform widths 4-64, padded rows, and 8/10/12-bit samples. This path is not production-complete until its syntax and reconstruction are verified with an independently encoded palette AVIF fixture.
- Non-reduced still-image sequence parsing now consumes decoder-model operating parameters, temporal presentation fields, and OBU-layer-applicable buffer-removal fields only to preserve AV1 bit alignment. The scheduling values are not retained, and no movie, track, timing, playback, or generic ISO BMFF surface has been introduced. Existing focused sequence-header coverage exercises only reduced-still syntax, so an independent non-reduced still AVIF vector remains required.
- Loop-restoration unit parsing records tile-local switchable/Wiener/self-guided filter selections and coefficients in frame-owned plane grids, including super-resolution-adjusted unit corners and the corrected conditional 64x64-superblock unit-size bit. The active restoration stage implements the normative unit geometry, striped deblocked boundaries, Wiener filtering, self-guided projection, and 8/10/12-bit clipping, while reusing the existing SIMD-backed super-resolution and adjacent multiply/add primitives. Independently encoded fixtures covering every parameter set, plane layout, bit depth, and frame-edge geometry are still required. Other normative independently decodable still-image syntax paths still contain `NotImplementedException` or equivalent unsupported branches. Tile-local palette CDF adaptation is present; the remaining still-image frame-context behavior requires a separate source audit without introducing sequence playback state.
- The frame buffer now establishes two-byte native sample storage, logical plane rows, and sample-unit block strides for 10/12-bit frames. The active intra-prediction, inverse-transform, and block-reconstruction path selects native 16-bit samples for 10/12-bit frames and has focused pipeline wiring coverage. Chroma-from-luma storage, 4:4:4/4:2:2/4:2:0 subsampling, rounded mean subtraction, U/V sharing, and 8/10/12-bit prediction now traverse AVX2 and cross-platform `Vector128` paths before an exact scalar fallback. `FeatureTestRunner` verifies every tier against independent fixed-point definitions across every supported block width; independently encoded high-bit-depth and chroma-from-luma AVIF conformance files are still required.
- `Av1YuvConverter` now consumes the signaled full or limited range, every non-reserved AV1 H.273 matrix coefficient, transfer characteristics where the matrix definition requires them, subsampling, and chroma sample position for 8, 10, and 12-bit output. Its high-bit-depth decode and encode paths use allocator-backed `Rgb48` rows and the existing `PixelOperations<TPixel>` conversions, avoiding the former eight-bit intermediate. Encoder conversion covers monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with libavif-compatible box averaging. Identity, full/limited-range YCgCo, the fixed non-constant-luminance matrices, both fixed and chromaticity-derived constant/non-constant-luminance systems, SMPTE ST 2085, and PQ/HLG ICtCp are active in both directions. Independent vectors for every matrix, transfer, range, bit depth, sampling layout, and chroma position remain required before the complete color matrix is externally verified.
- Forward and inverse transforms use operation-owned workspace, explicit sequential fixed storage, and stateless static-generic operator structs for every valid DCT, ADST, and identity size. Named configuration factories keep the encoder's three shifts and variable cosine precision separate from the decoder's two shifts, fixed 12-bit cosine precision, and 8/10/12-bit clamp ranges. Forward traversal shares one libaom-shaped stage network across scalar, `Vector128`, `Vector256`, and `Vector512`; inverse production traversal uses the verified scalar, `Vector128`, and `Vector256` tiers. The focused 511-case Release matrix passes across the `FeatureTestRunner` hardware configurations, and the production 32x32 benchmark reports zero allocation with the paired stage implementation.
- Core intra prediction and chroma-from-luma now use SIMD-first operator or packed-kernel architectures with exact scalar fallbacks. Palette reconstruction and nonlinear self-guided restoration remain scalar and therefore do not meet the completion architecture. They must be redesigned around vector-friendly storage, shared operator contracts, and SIMD-first traversal with benchmark-gated lane selection, with their current results retained only as fallback/parity behavior. Transform traversal is already SIMD-first, while normative super-resolution and Wiener horizontal products reuse ImageSharp's cross-platform adjacent multiply/add SIMD helper with exact scalar fallbacks.
- Core intra prediction, chroma-from-luma, and palette reconstruction now use SIMD-first operator or packed-kernel architectures with exact scalar fallbacks. Nonlinear self-guided restoration remains scalar and therefore does not meet the completion architecture. It must be redesigned around vector-friendly storage, shared operator contracts, and SIMD-first traversal with benchmark-gated lane selection, with its current results retained only as fallback/parity behavior. Transform traversal is already SIMD-first, while normative super-resolution and Wiener horizontal products reuse ImageSharp's cross-platform adjacent multiply/add SIMD helper with exact scalar fallbacks.
### AV1 encoder
@ -496,6 +496,8 @@ Implement and verify in dependency order:
- [ ] Tile partitioning, mode information, segmentation, delta quantization, transform-size selection, coefficient token decode, inverse quantization, and inverse transforms.
- [ ] Intra prediction, including every directional, smooth, Paeth, CFL, filter-intra, and palette case permitted by AV1.
- [x] Implement SIMD-first chroma-from-luma storage, 4:4:4/4:2:2/4:2:0 subsampling, rounded mean subtraction, and 8/10/12-bit prediction with exact scalar fallback and `FeatureTestRunner` parity.
- [x] Implement allocation-free SIMD-first palette reconstruction for palette sizes 2-8, transform widths 4-64, and 8/10/12-bit samples with exact scalar fallback and `FeatureTestRunner` parity.
- [ ] Verify palette mode syntax and presented reconstruction with an independently encoded palette AVIF fixture.
- [ ] Lossless and high-bit-depth reconstruction with correct clipping and intermediate precision.
- [ ] Deblocking loop filter.
- [ ] CDEF.
@ -636,6 +638,7 @@ Tasks:
- [x] Add a permanent frame-wide HEVC residual-reconstruction benchmark. On .NET 10, dense 32x32 twelve-bit transform skip, horizontal RDPCM, and vertical RDPCM measured 79.64, 275.61, and 110.81 microseconds per padded 1920x1088 frame, compared with forced-scalar timings of 571.5 microseconds, 1.512 milliseconds, and 1.257 milliseconds: 7.2, 5.5, and 11.3 times faster with zero managed allocations.
- [x] Add a permanent frame-wide HEVC grouped coefficient-scan benchmark. On .NET 10, generating the complete diagonal 32x32 scan for all 2,040 transform blocks in a padded 1920x1088 frame measured 1.432 milliseconds with zero managed allocations. CABAC decisions remain serial and require an independently encoded complete-slice workload after transform-unit traversal is connected.
- [x] Add a permanent frame-wide AV1 chroma-from-luma prediction benchmark. The lane-selection run rejected the explicit `Vector512` experiment because AVX2 was 2.6% faster at 8-bit and 4.8% faster at 12-bit on the test system. With that path removed, the selected hardware and forced-AVX2 paths are equivalent; the final .NET 10 run measured 88.76 and 74.89 microseconds per padded 1920x1088 frame at 8 and 12 bits, compared with forced-scalar timings of 1.594 and 1.604 milliseconds: 18.0 and 21.4 times faster with zero managed allocations.
- [x] Add a permanent frame-wide AV1 palette-reconstruction benchmark. On .NET 10, runtime-selected hardware measured 20.40 and 62.99 microseconds per padded 1920x1088 frame at 8 and 12 bits, compared with forced-scalar timings of 488.96 and 509.58 microseconds: 24.0 and 8.1 times faster with zero managed allocations. Forced `Vector512` measured 20.81 and 62.70 microseconds, so production continues to honor the runtime's preferred vector width instead of overriding this machine's `Vector256` selection.
- [ ] Implement vector paths only for confirmed hot loops, using existing `Vector128`, `Vector256`, and `Vector512` helper and dispatch patterns where supported.
- [ ] Prioritize shared color conversion and pixel packing, chroma upsampling, inverse-transform add-and-clip, intra predictors, HEVC deblock/SAO, AV1 loop filter/CDEF/restoration, and contiguous grid copies.
- [ ] Benchmark the complete decode color pipeline on representative 8/10/12-bit AVIF and HEIC images with and without embedded ICC profiles. Report absolute end-to-end timings and allocations in addition to the isolated YUV/CICP and ICC stage costs.

268
src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PalettePredictor.cs

@ -0,0 +1,268 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Runtime.Intrinsics;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
/// <summary>
/// Reconstructs AV1 palette-predicted sample blocks from decoded color-index maps.
/// </summary>
internal static class Av1PalettePredictor
{
/// <summary>
/// The repeated byte offsets that select both bytes of eight 16-bit palette entries.
/// </summary>
private const ushort PaletteByteOffsetMultiplier = 0x0202;
/// <summary>
/// The high-byte increment that selects the second byte of each 16-bit palette entry.
/// </summary>
private const ushort PaletteHighByteOffset = 0x0100;
/// <summary>
/// Reconstructs an 8-bit palette-predicted block.
/// </summary>
/// <param name="paletteColors">The decoded palette colors in prediction-index order.</param>
/// <param name="colorIndexMap">The color-index map beginning at the prediction block origin.</param>
/// <param name="colorIndexMapStride">The distance, in indices, between map rows.</param>
/// <param name="destination">The destination beginning at the prediction block origin.</param>
/// <param name="destinationStride">The distance, in samples, between destination rows.</param>
/// <param name="width">The prediction width in samples.</param>
/// <param name="height">The prediction height in samples.</param>
public static void Predict(
ReadOnlySpan<ushort> paletteColors,
ReadOnlySpan<byte> colorIndexMap,
int colorIndexMapStride,
Span<byte> destination,
int destinationStride,
int width,
int height)
{
ref byte mapBase = ref MemoryMarshal.GetReference(colorIndexMap);
ref byte destinationBase = ref MemoryMarshal.GetReference(destination);
// An AV1 palette contains at most eight colors. Packing it once into the low 64 bits and repeating it in
// every 128-bit lane turns reconstruction into the lane-local table lookup implemented by ShuffleNative.
ulong packedPalette = 0;
for (int index = 0; index < paletteColors.Length; index++)
{
packedPalette |= (ulong)(byte)paletteColors[index] << (index * 8);
}
Vector128<byte> palette128 = Vector128.Create(packedPalette, packedPalette).AsByte();
if (Vector512.IsHardwareAccelerated && width >= Vector512<byte>.Count)
{
Vector256<byte> palette256 = Vector256.Create(palette128, palette128);
Vector512<byte> palette512 = Vector512.Create(palette256, palette256);
for (int row = 0; row < height; row++)
{
ref byte mapRow = ref Unsafe.Add(ref mapBase, row * colorIndexMapStride);
ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride);
for (int column = 0; column < width; column += Vector512<byte>.Count)
{
Vector512<byte> indices = Vector512.LoadUnsafe(ref mapRow, (nuint)column);
Vector512.ShuffleNative(palette512, indices).StoreUnsafe(ref destinationRow, (nuint)column);
}
}
return;
}
if (Vector256.IsHardwareAccelerated && width >= Vector256<byte>.Count)
{
Vector256<byte> palette256 = Vector256.Create(palette128, palette128);
for (int row = 0; row < height; row++)
{
ref byte mapRow = ref Unsafe.Add(ref mapBase, row * colorIndexMapStride);
ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride);
for (int column = 0; column < width; column += Vector256<byte>.Count)
{
Vector256<byte> indices = Vector256.LoadUnsafe(ref mapRow, (nuint)column);
Vector256.ShuffleNative(palette256, indices).StoreUnsafe(ref destinationRow, (nuint)column);
}
}
return;
}
if (Vector128.IsHardwareAccelerated)
{
for (int row = 0; row < height; row++)
{
ref byte mapRow = ref Unsafe.Add(ref mapBase, row * colorIndexMapStride);
ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride);
int column = 0;
for (; column <= width - Vector128<byte>.Count; column += Vector128<byte>.Count)
{
Vector128<byte> indices = Vector128.LoadUnsafe(ref mapRow, (nuint)column);
Vector128.ShuffleNative(palette128, indices).StoreUnsafe(ref destinationRow, (nuint)column);
}
if (column < width)
{
// Transform widths are powers of two. After complete vector chunks, only a four- or eight-byte
// row tail remains, so an exact-width load and store keeps neighboring transforms untouched.
int remaining = width - column;
ulong packedIndices = remaining == 4
? Unsafe.ReadUnaligned<uint>(ref Unsafe.Add(ref mapRow, column))
: Unsafe.ReadUnaligned<ulong>(ref Unsafe.Add(ref mapRow, column));
Vector128<byte> result = Vector128.ShuffleNative(palette128, Vector128.CreateScalarUnsafe(packedIndices).AsByte());
if (remaining == 4)
{
Unsafe.WriteUnaligned(ref Unsafe.Add(ref destinationRow, column), result.AsUInt32().ToScalar());
}
else
{
Unsafe.WriteUnaligned(ref Unsafe.Add(ref destinationRow, column), result.AsUInt64().ToScalar());
}
column = width;
}
}
return;
}
for (int row = 0; row < height; row++)
{
ref byte mapRow = ref Unsafe.Add(ref mapBase, row * colorIndexMapStride);
ref byte destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride);
for (int column = 0; column < width; column++)
{
Unsafe.Add(ref destinationRow, column) = (byte)paletteColors[Unsafe.Add(ref mapRow, column)];
}
}
}
/// <summary>
/// Reconstructs a high-bit-depth palette-predicted block.
/// </summary>
/// <param name="paletteColors">The decoded palette colors in prediction-index order.</param>
/// <param name="colorIndexMap">The color-index map beginning at the prediction block origin.</param>
/// <param name="colorIndexMapStride">The distance, in indices, between map rows.</param>
/// <param name="destination">The destination beginning at the prediction block origin.</param>
/// <param name="destinationStride">The distance, in samples, between destination rows.</param>
/// <param name="width">The prediction width in samples.</param>
/// <param name="height">The prediction height in samples.</param>
public static void Predict(
ReadOnlySpan<ushort> paletteColors,
ReadOnlySpan<byte> colorIndexMap,
int colorIndexMapStride,
Span<short> destination,
int destinationStride,
int width,
int height)
{
ref byte mapBase = ref MemoryMarshal.GetReference(colorIndexMap);
ref short destinationBase = ref MemoryMarshal.GetReference(destination);
InlineArray8<ushort> paletteStorage = default;
paletteColors.CopyTo(paletteStorage);
// Each index is expanded to the byte offsets 2n and 2n+1. Repeating the complete 16-byte palette in every
// 128-bit lane then permits the same native byte-table shuffle on x86, Arm, and WebAssembly.
Vector128<byte> palette128 = Vector128.LoadUnsafe(ref paletteStorage[0]).AsByte();
if (Vector512.IsHardwareAccelerated && width >= Vector512<ushort>.Count)
{
Vector256<byte> palette256 = Vector256.Create(palette128, palette128);
Vector512<byte> palette512 = Vector512.Create(palette256, palette256);
Vector512<ushort> multiplier = Vector512.Create(PaletteByteOffsetMultiplier);
Vector512<ushort> increment = Vector512.Create(PaletteHighByteOffset);
for (int row = 0; row < height; row++)
{
ref byte mapRow = ref Unsafe.Add(ref mapBase, row * colorIndexMapStride);
ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride);
for (int column = 0; column < width; column += Vector512<ushort>.Count)
{
(Vector256<ushort> lower, Vector256<ushort> upper) = Vector256.Widen(Vector256.LoadUnsafe(ref mapRow, (nuint)column));
Vector512<ushort> indices = Vector512.Create(lower, upper);
Vector512<byte> controls = ((indices * multiplier) + increment).AsByte();
Vector512.ShuffleNative(palette512, controls).AsInt16().StoreUnsafe(ref destinationRow, (nuint)column);
}
}
return;
}
if (Vector256.IsHardwareAccelerated && width >= Vector256<ushort>.Count)
{
Vector256<byte> palette256 = Vector256.Create(palette128, palette128);
Vector256<ushort> multiplier = Vector256.Create(PaletteByteOffsetMultiplier);
Vector256<ushort> increment = Vector256.Create(PaletteHighByteOffset);
for (int row = 0; row < height; row++)
{
ref byte mapRow = ref Unsafe.Add(ref mapBase, row * colorIndexMapStride);
ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride);
for (int column = 0; column < width; column += Vector256<ushort>.Count)
{
(Vector128<ushort> lower, Vector128<ushort> upper) = Vector128.Widen(Vector128.LoadUnsafe(ref mapRow, (nuint)column));
Vector256<ushort> indices = Vector256.Create(lower, upper);
Vector256<byte> controls = ((indices * multiplier) + increment).AsByte();
Vector256.ShuffleNative(palette256, controls).AsInt16().StoreUnsafe(ref destinationRow, (nuint)column);
}
}
return;
}
if (Vector128.IsHardwareAccelerated)
{
Vector128<ushort> multiplier = Vector128.Create(PaletteByteOffsetMultiplier);
Vector128<ushort> increment = Vector128.Create(PaletteHighByteOffset);
for (int row = 0; row < height; row++)
{
ref byte mapRow = ref Unsafe.Add(ref mapBase, row * colorIndexMapStride);
ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride);
int column = 0;
for (; column <= width - Vector128<ushort>.Count; column += Vector128<ushort>.Count)
{
ulong packedIndices = Unsafe.ReadUnaligned<ulong>(ref Unsafe.Add(ref mapRow, column));
Vector128<ushort> indices = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packedIndices).AsByte());
Vector128<byte> controls = ((indices * multiplier) + increment).AsByte();
Vector128.ShuffleNative(palette128, controls).AsInt16().StoreUnsafe(ref destinationRow, (nuint)column);
}
if (column < width)
{
uint packedIndices = Unsafe.ReadUnaligned<uint>(ref Unsafe.Add(ref mapRow, column));
Vector128<ushort> indices = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packedIndices).AsByte());
Vector128<byte> controls = ((indices * multiplier) + increment).AsByte();
Vector128<short> result = Vector128.ShuffleNative(palette128, controls).AsInt16();
Unsafe.WriteUnaligned(ref Unsafe.As<short, byte>(ref Unsafe.Add(ref destinationRow, column)), result.AsUInt64().ToScalar());
column += 4;
}
}
return;
}
for (int row = 0; row < height; row++)
{
ref byte mapRow = ref Unsafe.Add(ref mapBase, row * colorIndexMapStride);
ref short destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride);
for (int column = 0; column < width; column++)
{
Unsafe.Add(ref destinationRow, column) = (short)paletteColors[Unsafe.Add(ref mapRow, column)];
}
}
}
}

15
src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs

@ -362,14 +362,15 @@ internal class Av1PredictionDecoder
// Every transform reconstructs its own window of the block-level palette map. Keeping the map padded to
// the coded block dimensions lets edge transforms use the same addressing rule as interior transforms.
for (int row = 0; row < transformHeight; row++)
if (typeof(T) == typeof(byte))
{
Span<T> destinationRow = pixelBuffer.Slice(row * pixelBufferStride, transformWidth);
ReadOnlySpan<byte> mapRow = colorIndexMap.Slice(mapOffset + (row * paletteStride), transformWidth);
for (int column = 0; column < transformWidth; column++)
{
destinationRow[column] = T.CreateChecked(paletteColors[mapRow[column]]);
}
Span<byte> byteDestination = MemoryMarshal.Cast<T, byte>(pixelBuffer);
Av1PalettePredictor.Predict(paletteColors, colorIndexMap[mapOffset..], paletteStride, byteDestination, pixelBufferStride, transformWidth, transformHeight);
}
else
{
Span<short> highBitDepthDestination = MemoryMarshal.Cast<T, short>(pixelBuffer);
Av1PalettePredictor.Predict(paletteColors, colorIndexMap[mapOffset..], paletteStride, highBitDepthDestination, pixelBufferStride, transformWidth, transformHeight);
}
return;

149
tests/ImageSharp.Benchmarks/Codecs/Heif/Av1PalettePredictionBenchmarks.cs

@ -0,0 +1,149 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Columns;
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Jobs;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif;
/// <summary>
/// Measures frame-wide AV1 palette reconstruction at each available intrinsic tier.
/// </summary>
[Config(typeof(Configuration))]
[MemoryDiagnoser(displayGenColumns: false)]
[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)]
[CategoriesColumn]
public class Av1PalettePredictionBenchmarks
{
/// <summary>
/// The coded frame width, which is an exact multiple of the maximum palette block side.
/// </summary>
private const int Width = 1920;
/// <summary>
/// The coded frame height including the final padded coding-tree row for a 1080-line presentation.
/// </summary>
private const int Height = 1088;
/// <summary>
/// The maximum AV1 palette block side in samples.
/// </summary>
private const int BlockSize = 64;
/// <summary>
/// The eight-entry 8-bit palette reused by each benchmark block.
/// </summary>
private readonly ushort[] palette8 = [3, 37, 71, 109, 143, 181, 217, 251];
/// <summary>
/// The eight-entry 12-bit palette reused by each benchmark block.
/// </summary>
private readonly ushort[] palette12 = [17, 509, 1001, 1493, 1985, 2477, 2969, 4095];
/// <summary>
/// The decoded color-index map for one maximum-size palette block.
/// </summary>
private readonly byte[] colorIndexMap = new byte[BlockSize * BlockSize];
/// <summary>
/// The frame-wide 8-bit reconstruction surface.
/// </summary>
private readonly byte[] destination8 = new byte[Width * Height];
/// <summary>
/// The frame-wide 12-bit reconstruction surface.
/// </summary>
private readonly short[] destination12 = new short[Width * Height];
/// <summary>
/// Populates a deterministic, spatially varying color-index map outside the measured traversal.
/// </summary>
[GlobalSetup]
public void Setup()
{
for (int row = 0; row < BlockSize; row++)
{
for (int column = 0; column < BlockSize; column++)
{
this.colorIndexMap[(row * BlockSize) + column] = (byte)(((row * 5) + (column * 3)) & 7);
}
}
}
/// <summary>
/// Measures frame-wide 8-bit palette reconstruction.
/// </summary>
/// <returns>The final reconstructed sample, keeping the frame output observable.</returns>
[Benchmark]
[BenchmarkCategory("8Bit")]
public byte Predict8BitFrame()
{
for (int row = 0; row < Height; row += BlockSize)
{
for (int column = 0; column < Width; column += BlockSize)
{
Av1PalettePredictor.Predict(this.palette8, this.colorIndexMap, BlockSize, this.destination8.AsSpan((row * Width) + column), Width, BlockSize, BlockSize);
}
}
return this.destination8[^1];
}
/// <summary>
/// Measures frame-wide 12-bit palette reconstruction.
/// </summary>
/// <returns>The final reconstructed sample, keeping the frame output observable.</returns>
[Benchmark]
[BenchmarkCategory("12Bit")]
public short Predict12BitFrame()
{
for (int row = 0; row < Height; row += BlockSize)
{
for (int column = 0; column < Width; column += BlockSize)
{
Av1PalettePredictor.Predict(this.palette12, this.colorIndexMap, BlockSize, this.destination12.AsSpan((row * Width) + column), Width, BlockSize, BlockSize);
}
}
return this.destination12[^1];
}
/// <summary>
/// Configures production-process measurements for hardware, forced Vector512, AVX2, Vector128, and scalar paths.
/// </summary>
public sealed class Configuration : ManualConfig
{
/// <summary>
/// Initializes a new instance of the <see cref="Configuration"/> class.
/// </summary>
public Configuration()
{
this.AddJob(Job.ShortRun.WithId("Hardware").AsBaseline());
this.AddJob(
Job.ShortRun
.WithId("Vector512")
.WithEnvironmentVariable("DOTNET_PreferredVectorBitWidth", "512")
.WithEnvironmentVariable("COMPlus_PreferredVectorBitWidth", "512"));
this.AddJob(
Job.ShortRun
.WithId("Avx2")
.WithEnvironmentVariable("DOTNET_EnableAVX512F", "0"));
this.AddJob(
Job.ShortRun
.WithId("Vector128")
.WithEnvironmentVariable("DOTNET_EnableAVX512F", "0")
.WithEnvironmentVariable("DOTNET_EnableAVX2", "0"));
this.AddJob(
Job.ShortRun
.WithId("Scalar")
.WithEnvironmentVariable("DOTNET_EnableHWIntrinsic", "0"));
}
}
}

138
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1PalettePredictorTests.cs

@ -0,0 +1,138 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using SixLabors.ImageSharp.Formats.Heif.Av1;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
using SixLabors.ImageSharp.Tests.TestUtilities;
namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1;
/// <summary>
/// Verifies AV1 palette reconstruction across every supported sample precision and intrinsic tier.
/// </summary>
[Trait("Format", "Avif")]
public class Av1PalettePredictorTests
{
/// <summary>
/// The hardware configurations required to exercise each packed width and the scalar fallback.
/// </summary>
private const HwIntrinsics Configurations =
HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX512F | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic;
/// <summary>
/// Verifies exact indexed reconstruction and destination-padding preservation for every palette size.
/// </summary>
[Fact]
public void PredictMatchesIndependentDefinitionAcrossIntrinsicWidths()
=> FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidatePredictors, Configurations);
/// <summary>
/// Exercises all palette sizes and transform widths at 8, 10, and 12 bits.
/// </summary>
private static void ValidatePredictors()
{
int[] widths = [4, 8, 16, 32, 64];
foreach (int paletteSize in Enumerable.Range(2, Av1Constants.PaletteMaxSize - 1))
{
foreach (int width in widths)
{
int height = width == 64 ? 16 : width;
int mapStride = width + 5;
int destinationStride = width + 9;
byte[] colorIndexMap = CreateColorIndexMap(mapStride, height, width, paletteSize);
ushort[] bytePalette = CreatePalette(paletteSize, 8);
byte[] expectedBytes = Enumerable.Repeat((byte)251, destinationStride * height).ToArray();
byte[] actualBytes = (byte[])expectedBytes.Clone();
ApplyReference(bytePalette, colorIndexMap, mapStride, expectedBytes, destinationStride, width, height);
Av1PalettePredictor.Predict(bytePalette, colorIndexMap, mapStride, actualBytes, destinationStride, width, height);
Assert.Equal(expectedBytes, actualBytes);
foreach (int bitDepth in new[] { 10, 12 })
{
ushort[] palette = CreatePalette(paletteSize, bitDepth);
short[] expected = Enumerable.Repeat((short)-1, destinationStride * height).ToArray();
short[] actual = (short[])expected.Clone();
ApplyReference(palette, colorIndexMap, mapStride, expected, destinationStride, width, height);
Av1PalettePredictor.Predict(palette, colorIndexMap, mapStride, actual, destinationStride, width, height);
Assert.Equal(expected, actual);
}
}
}
}
/// <summary>
/// Creates a deterministic palette spanning the legal range for the requested bit depth.
/// </summary>
private static ushort[] CreatePalette(int paletteSize, int bitDepth)
{
ushort[] result = new ushort[paletteSize];
int maximum = (1 << bitDepth) - 1;
for (int index = 0; index < result.Length; index++)
{
result[index] = (ushort)(((index * 977) + 37) & maximum);
}
return result;
}
/// <summary>
/// Creates deterministic active indices and invalid padding indices for each map row.
/// </summary>
private static byte[] CreateColorIndexMap(int stride, int height, int width, int paletteSize)
{
byte[] result = Enumerable.Repeat((byte)Av1Constants.PaletteMaxSize, stride * height).ToArray();
for (int row = 0; row < height; row++)
{
for (int column = 0; column < width; column++)
{
result[(row * stride) + column] = (byte)(((row * 5) + (column * 3)) % paletteSize);
}
}
return result;
}
/// <summary>
/// Applies independent scalar palette lookup to an 8-bit destination.
/// </summary>
private static void ApplyReference(
ReadOnlySpan<ushort> palette,
ReadOnlySpan<byte> colorIndexMap,
int mapStride,
Span<byte> destination,
int destinationStride,
int width,
int height)
{
for (int row = 0; row < height; row++)
{
for (int column = 0; column < width; column++)
{
destination[(row * destinationStride) + column] = (byte)palette[colorIndexMap[(row * mapStride) + column]];
}
}
}
/// <summary>
/// Applies independent scalar palette lookup to a high-bit-depth destination.
/// </summary>
private static void ApplyReference(
ReadOnlySpan<ushort> palette,
ReadOnlySpan<byte> colorIndexMap,
int mapStride,
Span<short> destination,
int destinationStride,
int width,
int height)
{
for (int row = 0; row < height; row++)
{
for (int column = 0; column < width; column++)
{
destination[(row * destinationStride) + column] = (short)palette[colorIndexMap[(row * mapStride) + column]];
}
}
}
}
Loading…
Cancel
Save