diff --git a/HEIF_IMPLEMENTATION_PLAN.md b/HEIF_IMPLEMENTATION_PLAN.md index f733d7b6c..baf7727a4 100644 --- a/HEIF_IMPLEMENTATION_PLAN.md +++ b/HEIF_IMPLEMENTATION_PLAN.md @@ -352,14 +352,14 @@ This assessment was reconciled with the source tree on 2026-08-26. Unless a resu - The single-still `Av1Decoder` path now parses tile state before allocating and reconstructing one independently decodable frame, and it disposes the reconstruction planes after pixel conversion. It deliberately does not retain animation/video reference frames or implement `show_existing_frame` playback state. - Transform coefficient entropy derivation and updates now address the above contexts relative to the tile column and the left contexts relative to the current superblock row, preserve luma coordinates independently of chroma subsampling, and test every packed context entry for the libaom any-nonzero rule. Extended vertical partition updates advance the mode-information column rather than the row. The existing multi-superblock 4:4:4 AVIF fixture now completes tile parsing; independent coefficient-context vectors across tile boundaries, chroma layouts, bit depths, and edge-clipped transforms remain required. -- The reconstruction pipeline now records plane-relative transform geometry, preserves tile-local delta-Q and delta-LF predictors, derives segmentation and reference-adjusted filter levels, and runs the exact AV1 4-, 6-, 8-, and 14-tap deblocking kernels in normative vertical-then-horizontal order. Deblocking uses the same closed edge-operator architecture as the HEVC filter, with operators specialized by sample storage and orientation, `Vector128` lanes representing the four rows or columns along an edge, and an allocation-free scalar fallback for disabled intrinsics. Exact native-plane comparison with pinned scalar libaom output now verifies active deblocking and complete reconstruction for real 8-bit 4:2:0, 10-bit 4:4:4, and 12-bit 4:4:4 content; genuine AVIF containers separately verify presentation and public bit-depth metadata. The pipeline then applies CDEF through one semantic filter architecture: paired AVX2 and single-block `Vector128` direction analysis, closed primary/secondary strength operators, packed 4x4/4x8/8x4/8x8 constrained filtering, byte/16-bit output operators, and an exact allocation-free scalar fallback. Decoder orchestration now owns the immutable plane snapshots and clean direction/variance maps through ImageSharp's memory allocator, widens 8-bit source rows with the same AVX2/128-bit/scalar tiers as libaom, lists each unit's non-skipped blocks in fixed inline storage, analyzes listed blocks in pairs, and writes filtered bytes or 16-bit samples directly to the frame planes. Independently encoded active-CDEF 8-bit 4:2:0 and 10/12-bit 4:4:4 streams match every visible native sample produced by pinned scalar libaom with restoration disabled. Independently encoded AVIF containers at the same three bit depths also match pinned scalar-libavif presentation exactly under normal, 256-bit, 128-bit, and scalar color-conversion dispatch. Active super-resolution derives the Appendix A bounded coded width and applies the exact 64-phase, 8-tap horizontal filter with aligned reconstruction-edge input, 8/10/12-bit clipping, and the existing cross-platform `Vector128_.MultiplyAddAdjacent` helper. Independently encoded active-super-resolution AV1 streams at 8, 10, and 12 bits match every visible native sample produced by pinned scalar libaom under normal and forced-scalar dispatch; equivalent AVIF presentation fixtures remain required. Loop restoration follows super-resolution, preserves the required pre-CDEF deblocked context at internal stripes, and applies decoded Wiener or self-guided units from immutable plane snapshots. The visible still-image path then applies the complete self-contained film-grain parameter set after all in-loop filters. Independent restoration and film-grain vectors remain required before those stages have external pixel-level verification. +- The reconstruction pipeline now records plane-relative transform geometry, preserves tile-local delta-Q and delta-LF predictors, derives segmentation and reference-adjusted filter levels, and runs the exact AV1 4-, 6-, 8-, and 14-tap deblocking kernels in normative vertical-then-horizontal order. Deblocking uses the same closed edge-operator architecture as the HEVC filter, with operators specialized by sample storage and orientation, `Vector128` lanes representing the four rows or columns along an edge, and an allocation-free scalar fallback for disabled intrinsics. Exact native-plane comparison with pinned scalar libaom output now verifies active deblocking and complete reconstruction for real 8-bit 4:2:0, 10-bit 4:4:4, and 12-bit 4:4:4 content; genuine AVIF containers separately verify presentation and public bit-depth metadata. The pipeline then applies CDEF through one semantic filter architecture: paired AVX2 and single-block `Vector128` direction analysis, closed primary/secondary strength operators, packed 4x4/4x8/8x4/8x8 constrained filtering, byte/16-bit output operators, and an exact allocation-free scalar fallback. Decoder orchestration now owns the immutable plane snapshots and clean direction/variance maps through ImageSharp's memory allocator, widens 8-bit source rows with the same AVX2/128-bit/scalar tiers as libaom, lists each unit's non-skipped blocks in fixed inline storage, analyzes listed blocks in pairs, and writes filtered bytes or 16-bit samples directly to the frame planes. Independently encoded active-CDEF 8-bit 4:2:0 and 10/12-bit 4:4:4 streams match every visible native sample produced by pinned scalar libaom with restoration disabled. Independently encoded AVIF containers at the same three bit depths also match pinned scalar-libavif presentation exactly under normal, 256-bit, 128-bit, and scalar color-conversion dispatch. Active super-resolution derives the Appendix A bounded coded width and applies the exact 64-phase, 8-tap horizontal filter with aligned reconstruction-edge input, 8/10/12-bit clipping, and the existing cross-platform `Vector128_.MultiplyAddAdjacent` helper. Independently encoded active-super-resolution AV1 streams at 8, 10, and 12 bits match every visible native sample produced by pinned scalar libaom under normal and forced-scalar dispatch; equivalent AVIF presentation fixtures remain required. Loop restoration follows super-resolution, preserves the required pre-CDEF deblocked context at internal stripes, and applies decoded Wiener or self-guided units from immutable plane snapshots. Independently encoded active-restoration streams at 8, 10, and 12 bits now match every native sample from pinned scalar libaom across AVX2, 128-bit, and scalar dispatch, with the fixture matrix proving both Wiener and self-guided unit selection. The matching AVIF containers also match pinned scalar-libavif presentation exactly, and an independent direct-window definition verifies all sixteen self-guided parameter sets at each supported bit depth across vector and scalar dispatch. Combined 8-bit 4:2:0, 10-bit 4:2:2, and 12-bit 4:4:4 streams additionally verify restoration-unit boundaries after super-resolution, including clipped chroma transform traversal at a coded-frame edge. The visible still-image path then applies the complete self-contained film-grain parameter set after all in-loop filters; independent film-grain vectors remain required before that stage has external pixel-level verification. - Palette mode now reads the normative luma/chroma mode and size CDFs, neighbor color caches, high-bit-depth color syntax, diagonal color-index maps, and clipped-edge padding. Reconstruction retains libaom's exact indexed-palette contract while using repeated eight-entry tables and native lane-local shuffles across runtime-selected `Vector512`, `Vector256`, and `Vector128` paths before an exact scalar fallback. `FeatureTestRunner` verifies palette sizes 2-8, transform widths 4-64, padded rows, and 8/10/12-bit samples. This path is not production-complete until its syntax and reconstruction are verified with an independently encoded palette AVIF fixture. - Non-reduced still-image sequence parsing now consumes decoder-model operating parameters, temporal presentation fields, and OBU-layer-applicable buffer-removal fields only to preserve AV1 bit alignment. The scheduling values are not retained, and no movie, track, timing, playback, or generic ISO BMFF surface has been introduced. Existing focused sequence-header coverage exercises only reduced-still syntax, so an independent non-reduced still AVIF vector remains required. -- Loop-restoration unit parsing records tile-local switchable/Wiener/self-guided filter selections and coefficients in frame-owned plane grids, including super-resolution-adjusted unit corners and the corrected conditional 64x64-superblock unit-size bit. The active restoration stage implements the normative unit geometry, striped deblocked boundaries, Wiener filtering, self-guided projection, and 8/10/12-bit clipping, while reusing the existing SIMD-backed super-resolution and adjacent multiply/add primitives. Independently encoded fixtures covering every parameter set, plane layout, bit depth, and frame-edge geometry are still required. Other normative independently decodable still-image syntax paths still contain `NotImplementedException` or equivalent unsupported branches. Tile-local palette CDF adaptation is present; the remaining still-image frame-context behavior requires a separate source audit without introducing sequence playback state. +- Loop-restoration unit parsing records tile-local switchable/Wiener/self-guided filter selections and coefficients in frame-owned plane grids, including super-resolution-adjusted unit corners and the corrected conditional 64x64-superblock unit-size bit. The active restoration stage implements the normative unit geometry, striped deblocked boundaries, Wiener filtering, self-guided projection, and 8/10/12-bit clipping. Self-guided filtering now follows libaom's summed-area-table, coefficient-grid, alternating-row radius-two, full radius-one, and projection stages through AVX2 and cross-platform 128-bit traversals with one exact scalar fallback and caller-owned scratch. Independently encoded fixtures cover active Wiener and self-guided reconstruction and exact AVIF presentation at every supported bit depth. An independent direct-window definition covers all sixteen self-guided parameter sets, narrow and odd processing units, both vector-tail widths, padded strides, and the scalar fallback. Combined active-restoration and super-resolution fixtures cover 4:2:0, 4:2:2, and 4:4:4 at 8, 10, and 12 bits, including restoration-unit boundaries and clipped chroma transform traversal. Other normative independently decodable still-image syntax paths still contain `NotImplementedException` or equivalent unsupported branches. Tile-local palette CDF adaptation is present; the remaining still-image frame-context behavior requires a separate source audit without introducing sequence playback state. - The frame buffer now establishes two-byte native sample storage, logical plane rows, and sample-unit block strides for 10/12-bit frames. The active intra-prediction, inverse-transform, and block-reconstruction path selects native 16-bit samples for 10/12-bit frames and has focused pipeline wiring coverage. Chroma-from-luma storage, 4:4:4/4:2:2/4:2:0 subsampling, rounded mean subtraction, U/V sharing, and 8/10/12-bit prediction now traverse AVX2 and cross-platform `Vector128` paths before an exact scalar fallback. `FeatureTestRunner` verifies every tier against independent fixed-point definitions across every supported block width; independently encoded high-bit-depth and chroma-from-luma AVIF conformance files are still required. - `Av1YuvConverter` now consumes the signaled full or limited range, every non-reserved AV1 H.273 matrix coefficient, transfer characteristics where the matrix definition requires them, subsampling, and chroma sample position for 8, 10, and 12-bit output. Its high-bit-depth decode and encode paths use allocator-backed `Rgb48` rows and the existing `PixelOperations` conversions, avoiding the former eight-bit intermediate. Encoder conversion covers monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with libavif-compatible box averaging. Identity, full/limited-range YCgCo, the fixed non-constant-luminance matrices, both fixed and chromaticity-derived constant/non-constant-luminance systems, SMPTE ST 2085, and PQ/HLG ICtCp are active in both directions. Independent vectors for every matrix, transfer, range, bit depth, sampling layout, and chroma position remain required before the complete color matrix is externally verified. - Forward and inverse transforms use operation-owned workspace, explicit sequential fixed storage, and stateless static-generic operator structs for every valid DCT, ADST, and identity size. Named configuration factories keep the encoder's three shifts and variable cosine precision separate from the decoder's two shifts, fixed 12-bit cosine precision, and 8/10/12-bit clamp ranges. Forward traversal shares one libaom-shaped stage network across scalar, `Vector128`, `Vector256`, and `Vector512`; inverse production traversal uses the verified scalar, `Vector128`, and `Vector256` tiers. Lossless segments now bypass the DCT pipeline and apply the reversible four-by-four inverse Walsh-Hadamard transform through the same byte/high-bit-depth output operators, with a `Vector128` production path and caller-workspace scalar fallback. The focused Release matrix passes across the `FeatureTestRunner` hardware configurations, and the production transform benchmarks report zero allocation. -- Core intra prediction, chroma-from-luma, and palette reconstruction now use SIMD-first operator or packed-kernel architectures with exact scalar fallbacks. Nonlinear self-guided restoration remains scalar and therefore does not meet the completion architecture. It must be redesigned around vector-friendly storage, shared operator contracts, and SIMD-first traversal with benchmark-gated lane selection, with its current results retained only as fallback/parity behavior. Transform traversal is already SIMD-first, while normative super-resolution and Wiener horizontal products reuse ImageSharp's cross-platform adjacent multiply/add SIMD helper with exact scalar fallbacks. +- Core intra prediction, chroma-from-luma, palette reconstruction, and nonlinear self-guided restoration now use SIMD-first operator or packed-kernel architectures with exact scalar fallbacks. Self-guided restoration keeps one semantic type while internal overloads select the libaom-shaped AVX2 or cross-platform 128-bit traversal; no namespace, file, or type name exposes SIMD width, ISA, storage, or bit depth. Transform traversal is already SIMD-first, while normative super-resolution and Wiener horizontal products reuse ImageSharp's cross-platform adjacent multiply/add SIMD helper with exact scalar fallbacks. ### AV1 encoder @@ -516,7 +516,12 @@ Implement and verify in dependency order: - [x] Replace the per-output dot product with the libaom-shaped four-output SIMD filter, exact scalar remainder, compile-time coefficient span, direct decoder-padding source rows, and one allocator-owned overlap row shared by every plane. - [x] Verify exact 8/10/12-bit native reconstruction against independently encoded active-super-resolution AV1 streams under normal SIMD dispatch and with hardware intrinsics disabled. - [ ] Verify exact presented reconstruction with independently packaged AVIF containers across 8/10/12-bit samples. -- [ ] Loop restoration. +- [x] Loop restoration. + - [x] Replace scalar self-guided restoration with the libaom-shaped summed-area, coefficient, alternating-row radius-two, radius-one, and projection pipeline using AVX2, cross-platform 128-bit vectors, and an exact scalar fallback through one semantic `Av1SelfGuidedFilter` contract. + - [x] Verify active Wiener and self-guided unit selection plus exact 8/10/12-bit native reconstruction against independently encoded pinned-libaom streams under AVX2, 128-bit, and scalar dispatch through `FeatureTestRunner`. + - [x] Verify exact presented reconstruction with independently packaged active-restoration AVIF containers across 8/10/12-bit samples. + - [x] Verify all sixteen self-guided parameter sets at 8, 10, and 12 bits against an independent direct-window definition, including narrow, odd, AVX2-tail, 128-bit-tail, and padded-stride processing units through `FeatureTestRunner`. + - [x] Extend independent integration coverage across restoration-unit boundary shapes, chroma subsampling layouts, and super-resolution-adjusted unit geometry. - [ ] Frame padding and film-grain synthesis when signaled. For each SIMD-suitable item, design the data layout, operator contract, scratch ownership, traversal, and descending-width dispatch around the production SIMD path first. Implement the behaviorally equivalent scalar fallback through the same contract, then verify every supported vector width and the scalar path against normative or independent results before enabling the stage. Serial adaptive work such as entropy decoding remains scalar where data dependencies prevent meaningful vectorization. Remove constant feature-disable flags and unsupported branches only when their replacement is verified. Unsupported syntax must produce a codec-specific invalid-image error; it must never silently skip a normative reconstruction stage. @@ -656,6 +661,7 @@ Tasks: - [x] Add a permanent frame-wide AV1 lossless inverse-transform benchmark. Dense four-by-four reconstruction across a padded 1920x1088 frame measured 595.6 microseconds at 8 bits and 512.4 microseconds at 12 bits, compared with forced-scalar timings of 2.074 and 2.270 milliseconds: 3.5 and 4.4 times faster with zero managed allocations. - [x] Add a permanent frame-wide AV1 deblocking benchmark. Complete vertical and horizontal fourteen-tap passes across a padded 1920x1088 frame measured 4.040 milliseconds at 8 bits and 4.184 milliseconds at 12 bits, compared with forced-scalar timings of 9.416 and 9.410 milliseconds: 2.33 and 2.25 times faster with zero managed allocations. - [x] Add a permanent frame-wide AV1 super-resolution benchmark. Upscaling a 1280x1080 coded luma plane to 1920x1080 measured 2.052 milliseconds at 8 bits and 2.045 milliseconds at 12 bits, compared with forced-scalar timings of 10.627 and 10.634 milliseconds: 5.18 and 5.20 times faster with zero managed allocations. BenchmarkDotNet could not change the Windows power plan, so retain these figures as an uncontrolled local baseline rather than a release comparison. + - [x] Add a permanent full-HD-equivalent AV1 self-guided restoration benchmark. AVX2 measured 6.674 milliseconds at 8 bits and 7.076 milliseconds at 12 bits, compared with 11.425 and 11.119 milliseconds for the 128-bit path and 40.386 and 40.522 milliseconds for the scalar fallback: AVX2 is 6.05 and 5.73 times faster than scalar with zero managed allocations. BenchmarkDotNet could not change the Windows power plan or query the processor name, so retain these figures as an uncontrolled local baseline rather than a release comparison. - [ ] Implement vector paths only for confirmed hot loops, using existing `Vector128`, `Vector256`, and `Vector512` helper and dispatch patterns where supported. - [ ] Prioritize shared color conversion and pixel packing, chroma upsampling, inverse-transform add-and-clip, intra predictors, HEVC deblock/SAO, AV1 loop filter/CDEF/restoration, and contiguous grid copies. - [ ] Benchmark the complete decode color pipeline on representative 8/10/12-bit AVIF and HEIC images with and without embedded ICC profiles. Report absolute end-to-end timings and allocations in addition to the isolated YUV/CICP and ICC stage costs. diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1SelfGuidedFilter.Operations.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1SelfGuidedFilter.Operations.cs new file mode 100644 index 000000000..6fd1a085d --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1SelfGuidedFilter.Operations.cs @@ -0,0 +1,1384 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopRestoration; + +internal static partial class Av1SelfGuidedFilter +{ + /// + /// Applies self-guided restoration with the AVX2 traversal used by libaom. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The destination processing-unit rectangle. + /// The number of samples between destination rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The encoded sample bit depth. + /// The decoded self-guided parameter-set index. + /// The two transmitted projection coefficients. + /// The caller-owned work storage. + /// The overload-selection value. + private static void FilterBlock( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + int bitDepth, + int parameterSetIndex, + ReadOnlySpan projectionCoefficients, + Span scratch, + Vector256 vector) + { + int filteredLength = width * height; + int bufferLength = GetBufferLength(width, height); + int bufferStride = GetBufferStride(width); + Span filtered0 = scratch[..filteredLength]; + Span filtered1 = scratch.Slice(filteredLength, filteredLength); + Span blendFactors = scratch.Slice(filteredLength * 2, bufferLength); + Span localMeans = scratch.Slice((filteredLength * 2) + bufferLength, bufferLength); + Span squareIntegral = scratch.Slice((filteredLength * 2) + (bufferLength * 2), bufferLength); + Span sumIntegral = scratch.Slice((filteredLength * 2) + (bufferLength * 3), bufferLength); + + BuildIntegralImages(source, sourceStride, width + (Border * 2), height + (Border * 2), bufferStride, squareIntegral, sumIntegral, vector); + + int parameterOffset = parameterSetIndex * 2; + ReadOnlySpan radii = ParameterRadii.Slice(parameterOffset, 2); + ReadOnlySpan scales = ParameterScales.Slice(parameterOffset, 2); + if (radii[0] > 0) + { + CalculateIntermediateCoefficients( + width, + height, + bitDepth, + radii[0], + scales[0], + skipAlternateRows: true, + bufferStride, + squareIntegral, + sumIntegral, + blendFactors, + localMeans, + vector); + + CalculateRadiusTwoFilter(source, sourceStride, width, height, bufferStride, blendFactors, localMeans, filtered0, vector); + } + + if (radii[1] > 0) + { + CalculateIntermediateCoefficients( + width, + height, + bitDepth, + radii[1], + scales[1], + skipAlternateRows: false, + bufferStride, + squareIntegral, + sumIntegral, + blendFactors, + localMeans, + vector); + + CalculateRadiusOneFilter(source, sourceStride, width, height, bufferStride, blendFactors, localMeans, filtered1, vector); + } + + DecodeProjectionCoefficients(radii, projectionCoefficients, out int projection0, out int projection1); + Project( + source, + sourceStride, + destination, + destinationStride, + width, + height, + bitDepth, + radii, + projection0, + projection1, + filtered0, + filtered1, + vector); + } + + /// + /// Applies self-guided restoration with the cross-platform 128-bit traversal. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The destination processing-unit rectangle. + /// The number of samples between destination rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The encoded sample bit depth. + /// The decoded self-guided parameter-set index. + /// The two transmitted projection coefficients. + /// The caller-owned work storage. + /// The overload-selection value. + private static void FilterBlock( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + int bitDepth, + int parameterSetIndex, + ReadOnlySpan projectionCoefficients, + Span scratch, + Vector128 vector) + { + int filteredLength = width * height; + int bufferLength = GetBufferLength(width, height); + int bufferStride = GetBufferStride(width); + Span filtered0 = scratch[..filteredLength]; + Span filtered1 = scratch.Slice(filteredLength, filteredLength); + Span blendFactors = scratch.Slice(filteredLength * 2, bufferLength); + Span localMeans = scratch.Slice((filteredLength * 2) + bufferLength, bufferLength); + Span squareIntegral = scratch.Slice((filteredLength * 2) + (bufferLength * 2), bufferLength); + Span sumIntegral = scratch.Slice((filteredLength * 2) + (bufferLength * 3), bufferLength); + + BuildIntegralImages(source, sourceStride, width + (Border * 2), height + (Border * 2), bufferStride, squareIntegral, sumIntegral, vector); + + int parameterOffset = parameterSetIndex * 2; + ReadOnlySpan radii = ParameterRadii.Slice(parameterOffset, 2); + ReadOnlySpan scales = ParameterScales.Slice(parameterOffset, 2); + if (radii[0] > 0) + { + CalculateIntermediateCoefficients( + width, + height, + bitDepth, + radii[0], + scales[0], + skipAlternateRows: true, + bufferStride, + squareIntegral, + sumIntegral, + blendFactors, + localMeans, + vector); + + CalculateRadiusTwoFilter(source, sourceStride, width, height, bufferStride, blendFactors, localMeans, filtered0, vector); + } + + if (radii[1] > 0) + { + CalculateIntermediateCoefficients( + width, + height, + bitDepth, + radii[1], + scales[1], + skipAlternateRows: false, + bufferStride, + squareIntegral, + sumIntegral, + blendFactors, + localMeans, + vector); + + CalculateRadiusOneFilter(source, sourceStride, width, height, bufferStride, blendFactors, localMeans, filtered1, vector); + } + + DecodeProjectionCoefficients(radii, projectionCoefficients, out int projection0, out int projection1); + Project( + source, + sourceStride, + destination, + destinationStride, + width, + height, + bitDepth, + radii, + projection0, + projection1, + filtered0, + filtered1, + vector); + } + + /// + /// Builds the summed-area tables consumed by the AVX2 coefficient stage. + /// + /// The complete bordered source rectangle. + /// The number of samples between source rows. + /// The bordered source width. + /// The bordered source height. + /// The padded work-buffer row stride. + /// The destination integral image of squared samples. + /// The destination integral image of samples. + /// The overload-selection value. + private static void BuildIntegralImages( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + int bufferStride, + Span squareIntegral, + Span sumIntegral, + Vector256 vector) + { + squareIntegral[..(width + 1)].Clear(); + sumIntegral[..(width + 1)].Clear(); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref int squareBase = ref MemoryMarshal.GetReference(squareIntegral); + ref int sumBase = ref MemoryMarshal.GetReference(sumIntegral); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = row * sourceStride; + int previousRowOffset = row * bufferStride; + int currentRowOffset = previousRowOffset + bufferStride; + squareIntegral[currentRowOffset] = 0; + sumIntegral[currentRowOffset] = 0; + Vector256 squareCarry = Vector256.Zero; + Vector256 sumCarry = Vector256.Zero; + int column = 0; + int vectorEnd = width - Vector256.Count; + for (; column <= vectorEnd; column += Vector256.Count) + { + // Eight packed 16-bit samples become eight 32-bit lanes. The prefix scans mirror + // libaom's scan_32, and the replicated carry joins consecutive vector batches. + Vector128 packed = Vector128.LoadUnsafe(ref sourceBase, (nuint)(sourceRowOffset + column)); + Vector256 samples = Avx2.ConvertToVector256Int32(packed); + Vector256 squares = samples * samples; + Vector256 scannedSums = Scan(samples); + Vector256 scannedSquares = Scan(squares); + Vector256 sumsAbove = Vector256.LoadUnsafe(ref sumBase, (nuint)(previousRowOffset + column + 1)); + Vector256 squaresAbove = Vector256.LoadUnsafe(ref squareBase, (nuint)(previousRowOffset + column + 1)); + Vector256 rowSums = scannedSums + sumsAbove + sumCarry; + Vector256 rowSquares = scannedSquares + squaresAbove + squareCarry; + rowSums.StoreUnsafe(ref sumBase, (nuint)(currentRowOffset + column + 1)); + rowSquares.StoreUnsafe(ref squareBase, (nuint)(currentRowOffset + column + 1)); + + sumCarry = Vector256.Create(rowSums.GetElement(Vector256.Count - 1) - sumsAbove.GetElement(Vector256.Count - 1)); + squareCarry = Vector256.Create(rowSquares.GetElement(Vector256.Count - 1) - squaresAbove.GetElement(Vector256.Count - 1)); + } + + int runningSum = sumCarry.GetElement(0); + int runningSquareSum = squareCarry.GetElement(0); + for (; column < width; column++) + { + int sample = source[sourceRowOffset + column]; + runningSum += sample; + runningSquareSum += sample * sample; + sumIntegral[currentRowOffset + column + 1] = sumIntegral[previousRowOffset + column + 1] + runningSum; + squareIntegral[currentRowOffset + column + 1] = squareIntegral[previousRowOffset + column + 1] + runningSquareSum; + } + } + } + + /// + /// Builds the summed-area tables consumed by the cross-platform coefficient stage. + /// + /// The complete bordered source rectangle. + /// The number of samples between source rows. + /// The bordered source width. + /// The bordered source height. + /// The padded work-buffer row stride. + /// The destination integral image of squared samples. + /// The destination integral image of samples. + /// The overload-selection value. + private static void BuildIntegralImages( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + int bufferStride, + Span squareIntegral, + Span sumIntegral, + Vector128 vector) + { + squareIntegral[..(width + 1)].Clear(); + sumIntegral[..(width + 1)].Clear(); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref int squareBase = ref MemoryMarshal.GetReference(squareIntegral); + ref int sumBase = ref MemoryMarshal.GetReference(sumIntegral); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = row * sourceStride; + int previousRowOffset = row * bufferStride; + int currentRowOffset = previousRowOffset + bufferStride; + squareIntegral[currentRowOffset] = 0; + sumIntegral[currentRowOffset] = 0; + Vector128 squareCarry = Vector128.Zero; + Vector128 sumCarry = Vector128.Zero; + int column = 0; + int vectorEnd = width - Vector128.Count; + for (; column <= vectorEnd; column += Vector128.Count) + { + // Loading through Vector64 avoids reading beyond the four samples owned by this + // batch. Widening is normalized by the runtime for both x86 and Arm64 targets. + ref ushort sourceReference = ref Unsafe.Add(ref sourceBase, sourceRowOffset + column); + Vector64 packed = Unsafe.As>(ref sourceReference); + Vector128 samples = Vector128.WidenLower(Vector128.Create(packed, Vector64.Zero)).AsInt32(); + Vector128 squares = samples * samples; + Vector128 scannedSums = Scan(samples); + Vector128 scannedSquares = Scan(squares); + Vector128 sumsAbove = Vector128.LoadUnsafe(ref sumBase, (nuint)(previousRowOffset + column + 1)); + Vector128 squaresAbove = Vector128.LoadUnsafe(ref squareBase, (nuint)(previousRowOffset + column + 1)); + Vector128 rowSums = scannedSums + sumsAbove + sumCarry; + Vector128 rowSquares = scannedSquares + squaresAbove + squareCarry; + rowSums.StoreUnsafe(ref sumBase, (nuint)(currentRowOffset + column + 1)); + rowSquares.StoreUnsafe(ref squareBase, (nuint)(currentRowOffset + column + 1)); + + sumCarry = Vector128.Create(rowSums.GetElement(Vector128.Count - 1) - sumsAbove.GetElement(Vector128.Count - 1)); + squareCarry = Vector128.Create(rowSquares.GetElement(Vector128.Count - 1) - squaresAbove.GetElement(Vector128.Count - 1)); + } + + int runningSum = sumCarry.GetElement(0); + int runningSquareSum = squareCarry.GetElement(0); + for (; column < width; column++) + { + int sample = source[sourceRowOffset + column]; + runningSum += sample; + runningSquareSum += sample * sample; + sumIntegral[currentRowOffset + column + 1] = sumIntegral[previousRowOffset + column + 1] + runningSum; + squareIntegral[currentRowOffset + column + 1] = squareIntegral[previousRowOffset + column + 1] + runningSquareSum; + } + } + } + + /// + /// Computes inclusive prefix sums for eight 32-bit lanes. + /// + /// The independent input values. + /// The inclusive prefix sum in each lane. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 Scan(Vector256 values) + { + // AVX2 byte shifts operate independently on the two 128-bit halves. After the two + // within-half scans, the lower-half total is added to every lane of the upper half. + Vector256 scan = values + Avx2.ShiftLeftLogical128BitLane(values.AsByte(), sizeof(int)).AsInt32(); + scan += Avx2.ShiftLeftLogical128BitLane(scan.AsByte(), sizeof(int) * 2).AsInt32(); + Vector256 lowerTotal = Vector256.Create(Vector128.Zero, Vector128.Create(scan.GetElement(3))); + return scan + lowerTotal; + } + + /// + /// Computes inclusive prefix sums for four 32-bit lanes. + /// + /// The independent input values. + /// The inclusive prefix sum in each lane. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 Scan(Vector128 values) + { + // The portable shuffle is required here because its out-of-range indices produce zero. + // ShuffleNative may mask those indices and wrap them back into the input on some ISAs. + Vector128 scan = values + Vector128.Shuffle(values, Vector128.Create(4, 0, 1, 2)); + return scan + Vector128.Shuffle(scan, Vector128.Create(4, 4, 0, 1)); + } + + /// + /// Calculates the coefficient grid in eight-sample AVX2 batches. + /// + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The encoded sample bit depth. + /// The square-window radius. + /// The variance scale for the selected parameter set. + /// Whether only alternate coefficient rows are required. + /// The padded work-buffer row stride. + /// The integral image of squared samples. + /// The integral image of samples. + /// The destination local blend factors. + /// The destination scaled local means. + /// The overload-selection value. + private static void CalculateIntermediateCoefficients( + int width, + int height, + int bitDepth, + int radius, + int scale, + bool skipAlternateRows, + int bufferStride, + ReadOnlySpan squareIntegral, + ReadOnlySpan sumIntegral, + Span blendFactors, + Span localMeans, + Vector256 vector) + { + int windowDiameter = (radius * 2) + 1; + int windowArea = windowDiameter * windowDiameter; + int reciprocal = OneByX[windowArea - 1]; + int rowStep = skipAlternateRows ? 2 : 1; + int bufferOrigin = (Border + 1) * (bufferStride + 1); + Vector256 windowAreaVector = Vector256.Create(windowArea); + Vector256 scaleVector = Vector256.Create((uint)scale); + Vector256 reciprocalVector = Vector256.Create(reciprocal); + Vector256 varianceRounding = Vector256.Create(1U << (ScaleBits - 1)); + Vector256 meanRounding = Vector256.Create(1U << (ReciprocalBits - 1)); + Vector256 maximumTableIndex = Vector256.Create(255U); + Vector256 selfGuidedScale = Vector256.Create(SelfGuidedScale); + ref int blendBase = ref MemoryMarshal.GetReference(blendFactors); + ref int meanBase = ref MemoryMarshal.GetReference(localMeans); + + for (int row = -1; row < height + 1; row += rowStep) + { + int column = -1; + int remaining = width + 2; + for (; remaining >= Vector256.Count; column += Vector256.Count, remaining -= Vector256.Count) + { + Vector256 sums = BoxSum(sumIntegral, bufferOrigin + (row * bufferStride) + column, bufferStride, radius, vector); + Vector256 squareSums = BoxSum(squareIntegral, bufferOrigin + (row * bufferStride) + column, bufferStride, radius, vector); + Vector256 variance = CalculateVariance(sums, squareSums, bitDepth, windowAreaVector, vector); + + // The fixed-point product is intentionally unsigned. Its legal range can set the + // sign bit even though the normative value remains a non-negative 32-bit integer. + Vector256 tableIndices = Vector256.Min( + Vector256.ShiftRightLogical((variance * scaleVector) + varianceRounding, ScaleBits), + maximumTableIndex); + + Vector256 factors = LookupBlendFactors(tableIndices); + Vector256 meanProducts = ((selfGuidedScale - factors) * reciprocalVector * sums).AsUInt32(); + Vector256 means = Vector256.ShiftRightLogical(meanProducts + meanRounding, ReciprocalBits).AsInt32(); + int coefficientOffset = bufferOrigin + (row * bufferStride) + column; + factors.StoreUnsafe(ref blendBase, (nuint)coefficientOffset); + means.StoreUnsafe(ref meanBase, (nuint)coefficientOffset); + } + + for (; remaining > 0; column++, remaining--) + { + CalculateIntermediateCoefficient( + squareIntegral, + sumIntegral, + bufferOrigin + (row * bufferStride) + column, + bufferStride, + bitDepth, + radius, + windowArea, + scale, + reciprocal, + out int blendFactor, + out int localMean); + + int coefficientOffset = bufferOrigin + (row * bufferStride) + column; + blendFactors[coefficientOffset] = blendFactor; + localMeans[coefficientOffset] = localMean; + } + } + } + + /// + /// Calculates the coefficient grid in four-sample cross-platform batches. + /// + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The encoded sample bit depth. + /// The square-window radius. + /// The variance scale for the selected parameter set. + /// Whether only alternate coefficient rows are required. + /// The padded work-buffer row stride. + /// The integral image of squared samples. + /// The integral image of samples. + /// The destination local blend factors. + /// The destination scaled local means. + /// The overload-selection value. + private static void CalculateIntermediateCoefficients( + int width, + int height, + int bitDepth, + int radius, + int scale, + bool skipAlternateRows, + int bufferStride, + ReadOnlySpan squareIntegral, + ReadOnlySpan sumIntegral, + Span blendFactors, + Span localMeans, + Vector128 vector) + { + int windowDiameter = (radius * 2) + 1; + int windowArea = windowDiameter * windowDiameter; + int reciprocal = OneByX[windowArea - 1]; + int rowStep = skipAlternateRows ? 2 : 1; + int bufferOrigin = (Border + 1) * (bufferStride + 1); + Vector128 windowAreaVector = Vector128.Create(windowArea); + Vector128 scaleVector = Vector128.Create((uint)scale); + Vector128 reciprocalVector = Vector128.Create(reciprocal); + Vector128 varianceRounding = Vector128.Create(1U << (ScaleBits - 1)); + Vector128 meanRounding = Vector128.Create(1U << (ReciprocalBits - 1)); + Vector128 maximumTableIndex = Vector128.Create(255U); + Vector128 selfGuidedScale = Vector128.Create(SelfGuidedScale); + ref int blendBase = ref MemoryMarshal.GetReference(blendFactors); + ref int meanBase = ref MemoryMarshal.GetReference(localMeans); + + for (int row = -1; row < height + 1; row += rowStep) + { + int column = -1; + int remaining = width + 2; + for (; remaining >= Vector128.Count; column += Vector128.Count, remaining -= Vector128.Count) + { + Vector128 sums = BoxSum(sumIntegral, bufferOrigin + (row * bufferStride) + column, bufferStride, radius, vector); + Vector128 squareSums = BoxSum(squareIntegral, bufferOrigin + (row * bufferStride) + column, bufferStride, radius, vector); + Vector128 variance = CalculateVariance(sums, squareSums, bitDepth, windowAreaVector, vector); + Vector128 tableIndices = Vector128.Min( + Vector128.ShiftRightLogical((variance * scaleVector) + varianceRounding, ScaleBits), + maximumTableIndex); + + Vector128 factors = LookupBlendFactors(tableIndices); + Vector128 meanProducts = ((selfGuidedScale - factors) * reciprocalVector * sums).AsUInt32(); + Vector128 means = Vector128.ShiftRightLogical(meanProducts + meanRounding, ReciprocalBits).AsInt32(); + int coefficientOffset = bufferOrigin + (row * bufferStride) + column; + factors.StoreUnsafe(ref blendBase, (nuint)coefficientOffset); + means.StoreUnsafe(ref meanBase, (nuint)coefficientOffset); + } + + for (; remaining > 0; column++, remaining--) + { + CalculateIntermediateCoefficient( + squareIntegral, + sumIntegral, + bufferOrigin + (row * bufferStride) + column, + bufferStride, + bitDepth, + radius, + windowArea, + scale, + reciprocal, + out int blendFactor, + out int localMean); + + int coefficientOffset = bufferOrigin + (row * bufferStride) + column; + blendFactors[coefficientOffset] = blendFactor; + localMeans[coefficientOffset] = localMean; + } + } + } + + /// + /// Calculates eight adjacent box sums from one integral image. + /// + /// The source integral image. + /// The integral-image offset corresponding to the first box center. + /// The integral-image row stride. + /// The square-box radius. + /// The overload-selection value. + /// The eight adjacent box sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 BoxSum(ReadOnlySpan integral, int centerOffset, int stride, int radius, Vector256 vector) + { + ref int integralBase = ref MemoryMarshal.GetReference(integral); + int upperOffset = centerOffset - ((radius + 1) * stride); + int lowerOffset = centerOffset + (radius * stride); + Vector256 topLeft = Vector256.LoadUnsafe(ref integralBase, (nuint)(upperOffset - radius - 1)); + Vector256 topRight = Vector256.LoadUnsafe(ref integralBase, (nuint)(upperOffset + radius)); + Vector256 bottomLeft = Vector256.LoadUnsafe(ref integralBase, (nuint)(lowerOffset - radius - 1)); + Vector256 bottomRight = Vector256.LoadUnsafe(ref integralBase, (nuint)(lowerOffset + radius)); + return (bottomRight - bottomLeft) - (topRight - topLeft); + } + + /// + /// Calculates four adjacent box sums from one integral image. + /// + /// The source integral image. + /// The integral-image offset corresponding to the first box center. + /// The integral-image row stride. + /// The square-box radius. + /// The overload-selection value. + /// The four adjacent box sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 BoxSum(ReadOnlySpan integral, int centerOffset, int stride, int radius, Vector128 vector) + { + ref int integralBase = ref MemoryMarshal.GetReference(integral); + int upperOffset = centerOffset - ((radius + 1) * stride); + int lowerOffset = centerOffset + (radius * stride); + Vector128 topLeft = Vector128.LoadUnsafe(ref integralBase, (nuint)(upperOffset - radius - 1)); + Vector128 topRight = Vector128.LoadUnsafe(ref integralBase, (nuint)(upperOffset + radius)); + Vector128 bottomLeft = Vector128.LoadUnsafe(ref integralBase, (nuint)(lowerOffset - radius - 1)); + Vector128 bottomRight = Vector128.LoadUnsafe(ref integralBase, (nuint)(lowerOffset + radius)); + return (bottomRight - bottomLeft) - (topRight - topLeft); + } + + /// + /// Converts eight window sums into the bounded variance measure defined by AV1. + /// + /// The sample sums. + /// The squared-sample sums. + /// The encoded sample bit depth. + /// The replicated square-window area. + /// The overload-selection value. + /// The non-negative variance measure. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 CalculateVariance( + Vector256 sums, + Vector256 squareSums, + int bitDepth, + Vector256 windowArea, + Vector256 vector) + { + if (bitDepth > 8) + { + int depthShift = bitDepth - 8; + squareSums = Vector256.ShiftRightLogical( + squareSums.AsUInt32() + Vector256.Create(1U << ((depthShift * 2) - 1)), depthShift * 2).AsInt32(); + + sums = Vector256.ShiftRightLogical(sums.AsUInt32() + Vector256.Create(1U << (depthShift - 1)), depthShift).AsInt32(); + } + + Vector256 squareOfSums = sums * sums; + Vector256 scaledSquareSums = squareSums * windowArea; + + // Rounding high-bit-depth inputs can put the squared mean one step above + // the mean square. AV1 saturates that artifact before applying the scale. + return Vector256.Max(scaledSquareSums, squareOfSums).AsUInt32() - squareOfSums.AsUInt32(); + } + + /// + /// Converts four window sums into the bounded variance measure defined by AV1. + /// + /// The sample sums. + /// The squared-sample sums. + /// The encoded sample bit depth. + /// The replicated square-window area. + /// The overload-selection value. + /// The non-negative variance measure. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 CalculateVariance( + Vector128 sums, + Vector128 squareSums, + int bitDepth, + Vector128 windowArea, + Vector128 vector) + { + if (bitDepth > 8) + { + int depthShift = bitDepth - 8; + squareSums = Vector128.ShiftRightLogical( + squareSums.AsUInt32() + Vector128.Create(1U << ((depthShift * 2) - 1)), depthShift * 2).AsInt32(); + + sums = Vector128.ShiftRightLogical(sums.AsUInt32() + Vector128.Create(1U << (depthShift - 1)), depthShift).AsInt32(); + } + + Vector128 squareOfSums = sums * sums; + Vector128 scaledSquareSums = squareSums * windowArea; + return Vector128.Max(scaledSquareSums, squareOfSums).AsUInt32() - squareOfSums.AsUInt32(); + } + + /// + /// Maps eight bounded variance indices to their normative blend factors. + /// + /// The table indices. + /// The gathered blend factors. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static unsafe Vector256 LookupBlendFactors(Vector256 indices) + { + fixed (int* table = XByXPlusOne) + { + return Avx2.GatherVector256(table, indices.AsInt32(), sizeof(int)); + } + } + + /// + /// Maps four bounded variance indices to their normative blend factors. + /// + /// The table indices. + /// The gathered blend factors. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 LookupBlendFactors(Vector128 indices) + { + ReadOnlySpan table = XByXPlusOne; + return Vector128.Create( + table[(int)indices.GetElement(0)], + table[(int)indices.GetElement(1)], + table[(int)indices.GetElement(2)], + table[(int)indices.GetElement(3)]); + } + + /// + /// Calculates one coefficient pair for a vector remainder. + /// + /// The integral image of squared samples. + /// The integral image of samples. + /// The integral-image offset corresponding to the box center. + /// The integral-image row stride. + /// The encoded sample bit depth. + /// The square-window radius. + /// The square-window area. + /// The variance scale for the selected parameter set. + /// The fixed-point reciprocal of the window area. + /// The calculated local blend factor. + /// The calculated scaled local mean. + private static void CalculateIntermediateCoefficient( + ReadOnlySpan squareIntegral, + ReadOnlySpan sumIntegral, + int centerOffset, + int stride, + int bitDepth, + int radius, + int windowArea, + int scale, + int reciprocal, + out int blendFactor, + out int localMean) + { + int sum = BoxSum(sumIntegral, centerOffset, stride, radius); + int squareSum = BoxSum(squareIntegral, centerOffset, stride, radius); + int normalizedSquareSum = RoundPowerOfTwo(squareSum, 2 * (bitDepth - 8)); + int normalizedSum = RoundPowerOfTwo(sum, bitDepth - 8); + uint squareOfSum = (uint)normalizedSum * (uint)normalizedSum; + uint scaledSquareSum = (uint)normalizedSquareSum * (uint)windowArea; + uint variance = scaledSquareSum < squareOfSum ? 0 : scaledSquareSum - squareOfSum; + uint varianceIndex = RoundPowerOfTwo(variance * (uint)scale, ScaleBits); + blendFactor = XByXPlusOne[(int)Math.Min(varianceIndex, 255U)]; + uint meanProduct = (uint)(SelfGuidedScale - blendFactor) * (uint)reciprocal * (uint)sum; + localMean = (int)RoundPowerOfTwo(meanProduct, ReciprocalBits); + } + + /// + /// Calculates one square-window sum from an integral image. + /// + /// The source integral image. + /// The integral-image offset corresponding to the box center. + /// The integral-image row stride. + /// The square-box radius. + /// The square-window sum. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int BoxSum(ReadOnlySpan integral, int centerOffset, int stride, int radius) + { + int upperOffset = centerOffset - ((radius + 1) * stride); + int lowerOffset = centerOffset + (radius * stride); + int top = integral[upperOffset + radius] - integral[upperOffset - radius - 1]; + int bottom = integral[lowerOffset + radius] - integral[lowerOffset - radius - 1]; + return bottom - top; + } + + /// + /// Produces the radius-two filtered values in eight-sample AVX2 batches. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The padded coefficient-buffer row stride. + /// The local sample blend factors. + /// The scaled local means. + /// The destination fixed-point filtered values. + /// The overload-selection value. + private static void CalculateRadiusTwoFilter( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + int bufferStride, + ReadOnlySpan blendFactors, + ReadOnlySpan localMeans, + Span filtered, + Vector256 vector) + { + int bufferOrigin = (Border + 1) * (bufferStride + 1); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref int filteredBase = ref MemoryMarshal.GetReference(filtered); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = ((row + Border) * sourceStride) + Border; + int filteredRowOffset = row * width; + int coefficientRowOffset = bufferOrigin + (row * bufferStride); + int roundingBits = SelfGuidedBits + ((row & 1) == 0 ? 5 : 4) - RestorationBits; + Vector256 rounding = Vector256.Create(1 << (roundingBits - 1)); + int column = 0; + int vectorEnd = width - Vector256.Count; + for (; column <= vectorEnd; column += Vector256.Count) + { + Vector256 factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride, row, vector); + Vector256 means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride, row, vector); + Vector128 packed = Vector128.LoadUnsafe(ref sourceBase, (nuint)(sourceRowOffset + column)); + Vector256 samples = Avx2.ConvertToVector256Int32(packed); + Vector256 values = Vector256.ShiftRightArithmetic((factors * samples) + means + rounding, roundingBits); + values.StoreUnsafe(ref filteredBase, (nuint)(filteredRowOffset + column)); + } + + for (; column < width; column++) + { + int factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride, row); + int means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride, row); + int value = (factors * source[sourceRowOffset + column]) + means; + filtered[filteredRowOffset + column] = RoundPowerOfTwo(value, roundingBits); + } + } + } + + /// + /// Produces the radius-two filtered values in four-sample cross-platform batches. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The padded coefficient-buffer row stride. + /// The local sample blend factors. + /// The scaled local means. + /// The destination fixed-point filtered values. + /// The overload-selection value. + private static void CalculateRadiusTwoFilter( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + int bufferStride, + ReadOnlySpan blendFactors, + ReadOnlySpan localMeans, + Span filtered, + Vector128 vector) + { + int bufferOrigin = (Border + 1) * (bufferStride + 1); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref int filteredBase = ref MemoryMarshal.GetReference(filtered); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = ((row + Border) * sourceStride) + Border; + int filteredRowOffset = row * width; + int coefficientRowOffset = bufferOrigin + (row * bufferStride); + int roundingBits = SelfGuidedBits + ((row & 1) == 0 ? 5 : 4) - RestorationBits; + Vector128 rounding = Vector128.Create(1 << (roundingBits - 1)); + int column = 0; + int vectorEnd = width - Vector128.Count; + for (; column <= vectorEnd; column += Vector128.Count) + { + Vector128 factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride, row, vector); + Vector128 means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride, row, vector); + ref ushort sourceReference = ref Unsafe.Add(ref sourceBase, sourceRowOffset + column); + Vector64 packed = Unsafe.As>(ref sourceReference); + Vector128 samples = Vector128.WidenLower(Vector128.Create(packed, Vector64.Zero)).AsInt32(); + Vector128 values = Vector128.ShiftRightArithmetic((factors * samples) + means + rounding, roundingBits); + values.StoreUnsafe(ref filteredBase, (nuint)(filteredRowOffset + column)); + } + + for (; column < width; column++) + { + int factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride, row); + int means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride, row); + int value = (factors * source[sourceRowOffset + column]) + means; + filtered[filteredRowOffset + column] = RoundPowerOfTwo(value, roundingBits); + } + } + } + + /// + /// Produces the radius-one filtered values in eight-sample AVX2 batches. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The padded coefficient-buffer row stride. + /// The local sample blend factors. + /// The scaled local means. + /// The destination fixed-point filtered values. + /// The overload-selection value. + private static void CalculateRadiusOneFilter( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + int bufferStride, + ReadOnlySpan blendFactors, + ReadOnlySpan localMeans, + Span filtered, + Vector256 vector) + { + int bufferOrigin = (Border + 1) * (bufferStride + 1); + int roundingBits = SelfGuidedBits + 5 - RestorationBits; + Vector256 rounding = Vector256.Create(1 << (roundingBits - 1)); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref int filteredBase = ref MemoryMarshal.GetReference(filtered); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = ((row + Border) * sourceStride) + Border; + int filteredRowOffset = row * width; + int coefficientRowOffset = bufferOrigin + (row * bufferStride); + int column = 0; + int vectorEnd = width - Vector256.Count; + for (; column <= vectorEnd; column += Vector256.Count) + { + Vector256 factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride, vector); + Vector256 means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride, vector); + Vector128 packed = Vector128.LoadUnsafe(ref sourceBase, (nuint)(sourceRowOffset + column)); + Vector256 samples = Avx2.ConvertToVector256Int32(packed); + Vector256 values = Vector256.ShiftRightArithmetic((factors * samples) + means + rounding, roundingBits); + values.StoreUnsafe(ref filteredBase, (nuint)(filteredRowOffset + column)); + } + + for (; column < width; column++) + { + int factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride); + int means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride); + int value = (factors * source[sourceRowOffset + column]) + means; + filtered[filteredRowOffset + column] = RoundPowerOfTwo(value, roundingBits); + } + } + } + + /// + /// Produces the radius-one filtered values in four-sample cross-platform batches. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The padded coefficient-buffer row stride. + /// The local sample blend factors. + /// The scaled local means. + /// The destination fixed-point filtered values. + /// The overload-selection value. + private static void CalculateRadiusOneFilter( + ReadOnlySpan source, + int sourceStride, + int width, + int height, + int bufferStride, + ReadOnlySpan blendFactors, + ReadOnlySpan localMeans, + Span filtered, + Vector128 vector) + { + int bufferOrigin = (Border + 1) * (bufferStride + 1); + int roundingBits = SelfGuidedBits + 5 - RestorationBits; + Vector128 rounding = Vector128.Create(1 << (roundingBits - 1)); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref int filteredBase = ref MemoryMarshal.GetReference(filtered); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = ((row + Border) * sourceStride) + Border; + int filteredRowOffset = row * width; + int coefficientRowOffset = bufferOrigin + (row * bufferStride); + int column = 0; + int vectorEnd = width - Vector128.Count; + for (; column <= vectorEnd; column += Vector128.Count) + { + Vector128 factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride, vector); + Vector128 means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride, vector); + ref ushort sourceReference = ref Unsafe.Add(ref sourceBase, sourceRowOffset + column); + Vector64 packed = Unsafe.As>(ref sourceReference); + Vector128 samples = Vector128.WidenLower(Vector128.Create(packed, Vector64.Zero)).AsInt32(); + Vector128 values = Vector128.ShiftRightArithmetic((factors * samples) + means + rounding, roundingBits); + values.StoreUnsafe(ref filteredBase, (nuint)(filteredRowOffset + column)); + } + + for (; column < width; column++) + { + int factors = CrossSum(blendFactors, coefficientRowOffset + column, bufferStride); + int means = CrossSum(localMeans, coefficientRowOffset + column, bufferStride); + int value = (factors * source[sourceRowOffset + column]) + means; + filtered[filteredRowOffset + column] = RoundPowerOfTwo(value, roundingBits); + } + } + } + + /// + /// Calculates eight radius-one weighted cross sums. + /// + /// The coefficient buffer. + /// The first center coefficient. + /// The coefficient-buffer row stride. + /// The overload-selection value. + /// The weighted cross sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 CrossSum(ReadOnlySpan buffer, int offset, int stride, Vector256 vector) + { + ref int bufferBase = ref MemoryMarshal.GetReference(buffer); + Vector256 topLeft = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - stride - 1)); + Vector256 top = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - stride)); + Vector256 topRight = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - stride + 1)); + Vector256 left = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - 1)); + Vector256 center = Vector256.LoadUnsafe(ref bufferBase, (nuint)offset); + Vector256 right = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + 1)); + Vector256 bottomLeft = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + stride - 1)); + Vector256 bottom = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + stride)); + Vector256 bottomRight = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + stride + 1)); + Vector256 corners = topLeft + topRight + bottomLeft + bottomRight; + Vector256 remainder = left + top + center + right + bottom; + return Vector256.ShiftLeft(corners + remainder, 2) - corners; + } + + /// + /// Calculates four radius-one weighted cross sums. + /// + /// The coefficient buffer. + /// The first center coefficient. + /// The coefficient-buffer row stride. + /// The overload-selection value. + /// The weighted cross sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 CrossSum(ReadOnlySpan buffer, int offset, int stride, Vector128 vector) + { + ref int bufferBase = ref MemoryMarshal.GetReference(buffer); + Vector128 topLeft = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - stride - 1)); + Vector128 top = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - stride)); + Vector128 topRight = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - stride + 1)); + Vector128 left = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - 1)); + Vector128 center = Vector128.LoadUnsafe(ref bufferBase, (nuint)offset); + Vector128 right = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + 1)); + Vector128 bottomLeft = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + stride - 1)); + Vector128 bottom = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + stride)); + Vector128 bottomRight = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + stride + 1)); + Vector128 corners = topLeft + topRight + bottomLeft + bottomRight; + Vector128 remainder = left + top + center + right + bottom; + return Vector128.ShiftLeft(corners + remainder, 2) - corners; + } + + /// + /// Calculates eight radius-two weighted cross sums from the required coefficient rows. + /// + /// The coefficient buffer. + /// The first center coefficient. + /// The coefficient-buffer row stride. + /// The destination row selecting the even or odd kernel. + /// The overload-selection value. + /// The weighted cross sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 CrossSum(ReadOnlySpan buffer, int offset, int stride, int row, Vector256 vector) + { + ref int bufferBase = ref MemoryMarshal.GetReference(buffer); + if ((row & 1) != 0) + { + Vector256 left = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - 1)); + Vector256 center = Vector256.LoadUnsafe(ref bufferBase, (nuint)offset); + Vector256 right = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + 1)); + Vector256 combined = left + center + right; + return Vector256.ShiftLeft(combined, 2) + combined + center; + } + + Vector256 topLeft = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - stride - 1)); + Vector256 top = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - stride)); + Vector256 topRight = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset - stride + 1)); + Vector256 bottomLeft = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + stride - 1)); + Vector256 bottom = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + stride)); + Vector256 bottomRight = Vector256.LoadUnsafe(ref bufferBase, (nuint)(offset + stride + 1)); + Vector256 centers = top + bottom; + Vector256 combinedRows = topLeft + topRight + bottomLeft + bottomRight + centers; + return Vector256.ShiftLeft(combinedRows, 2) + combinedRows + centers; + } + + /// + /// Calculates four radius-two weighted cross sums from the required coefficient rows. + /// + /// The coefficient buffer. + /// The first center coefficient. + /// The coefficient-buffer row stride. + /// The destination row selecting the even or odd kernel. + /// The overload-selection value. + /// The weighted cross sums. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 CrossSum(ReadOnlySpan buffer, int offset, int stride, int row, Vector128 vector) + { + ref int bufferBase = ref MemoryMarshal.GetReference(buffer); + if ((row & 1) != 0) + { + Vector128 left = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - 1)); + Vector128 center = Vector128.LoadUnsafe(ref bufferBase, (nuint)offset); + Vector128 right = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + 1)); + Vector128 combined = left + center + right; + return Vector128.ShiftLeft(combined, 2) + combined + center; + } + + Vector128 topLeft = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - stride - 1)); + Vector128 top = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - stride)); + Vector128 topRight = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset - stride + 1)); + Vector128 bottomLeft = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + stride - 1)); + Vector128 bottom = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + stride)); + Vector128 bottomRight = Vector128.LoadUnsafe(ref bufferBase, (nuint)(offset + stride + 1)); + Vector128 centers = top + bottom; + Vector128 combinedRows = topLeft + topRight + bottomLeft + bottomRight + centers; + return Vector128.ShiftLeft(combinedRows, 2) + combinedRows + centers; + } + + /// + /// Calculates one scalar radius-one weighted cross sum. + /// + /// The coefficient buffer. + /// The center coefficient. + /// The coefficient-buffer row stride. + /// The weighted cross sum. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int CrossSum(ReadOnlySpan buffer, int offset, int stride) + { + int corners = buffer[offset - stride - 1] + buffer[offset - stride + 1] + buffer[offset + stride - 1] + buffer[offset + stride + 1]; + int remainder = buffer[offset - 1] + buffer[offset - stride] + buffer[offset] + buffer[offset + 1] + buffer[offset + stride]; + return ((corners + remainder) << 2) - corners; + } + + /// + /// Calculates one scalar radius-two weighted cross sum. + /// + /// The coefficient buffer. + /// The center coefficient. + /// The coefficient-buffer row stride. + /// The destination row selecting the even or odd kernel. + /// The weighted cross sum. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int CrossSum(ReadOnlySpan buffer, int offset, int stride, int row) + { + if ((row & 1) != 0) + { + int center = buffer[offset]; + int combined = buffer[offset - 1] + center + buffer[offset + 1]; + return (combined << 2) + combined + center; + } + + int centers = buffer[offset - stride] + buffer[offset + stride]; + int combinedRows = buffer[offset - stride - 1] + buffer[offset - stride + 1] + + buffer[offset + stride - 1] + buffer[offset + stride + 1] + centers; + + return (combinedRows << 2) + combinedRows + centers; + } + + /// + /// Decodes the transmitted projection coefficients for the active radius pair. + /// + /// The two selected filter radii. + /// The two transmitted projection coefficients. + /// The first decoded projection coefficient. + /// The second decoded projection coefficient. + private static void DecodeProjectionCoefficients(ReadOnlySpan radii, ReadOnlySpan transmitted, out int projection0, out int projection1) + { + if (radii[0] == 0) + { + projection0 = 0; + projection1 = (1 << ProjectionBits) - transmitted[1]; + } + else if (radii[1] == 0) + { + projection0 = transmitted[0]; + projection1 = 0; + } + else + { + projection0 = transmitted[0]; + projection1 = (1 << ProjectionBits) - projection0 - transmitted[1]; + } + } + + /// + /// Projects the two restored signals in eight-sample AVX2 batches. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The destination processing-unit rectangle. + /// The number of samples between destination rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The encoded sample bit depth. + /// The two selected filter radii. + /// The first decoded projection coefficient. + /// The second decoded projection coefficient. + /// The first fixed-point restored signal. + /// The second fixed-point restored signal. + /// The overload-selection value. + private static void Project( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + int bitDepth, + ReadOnlySpan radii, + int projection0, + int projection1, + ReadOnlySpan filtered0, + ReadOnlySpan filtered1, + Vector256 vector) + { + const int projectionShift = ProjectionBits + RestorationBits; + Vector256 projection0Vector = Vector256.Create(projection0); + Vector256 projection1Vector = Vector256.Create(projection1); + Vector256 rounding = Vector256.Create(1 << (projectionShift - 1)); + Vector256 maximumSample = Vector256.Create((1 << bitDepth) - 1); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref int filtered0Base = ref MemoryMarshal.GetReference(filtered0); + ref int filtered1Base = ref MemoryMarshal.GetReference(filtered1); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = ((row + Border) * sourceStride) + Border; + int destinationRowOffset = row * destinationStride; + int filteredRowOffset = row * width; + int column = 0; + int vectorEnd = width - Vector256.Count; + for (; column <= vectorEnd; column += Vector256.Count) + { + Vector128 packed = Vector128.LoadUnsafe(ref sourceBase, (nuint)(sourceRowOffset + column)); + Vector256 samples = Avx2.ConvertToVector256Int32(packed); + Vector256 unfiltered = Vector256.ShiftLeft(samples, RestorationBits); + Vector256 projected = Vector256.ShiftLeft(unfiltered, ProjectionBits); + if (radii[0] > 0) + { + Vector256 restored = Vector256.LoadUnsafe(ref filtered0Base, (nuint)(filteredRowOffset + column)); + projected += projection0Vector * (restored - unfiltered); + } + + if (radii[1] > 0) + { + Vector256 restored = Vector256.LoadUnsafe(ref filtered1Base, (nuint)(filteredRowOffset + column)); + projected += projection1Vector * (restored - unfiltered); + } + + Vector256 result = Vector256.ShiftRightArithmetic(projected + rounding, projectionShift); + result = Vector256.Min(Vector256.Max(result, Vector256.Zero), maximumSample); + + // Narrowing the result with a zero upper vector places the eight ordered samples + // in the lower 128 bits, which can be stored without the AVX2 pack permutation. + Vector128 narrowed = Vector256.Narrow(result.AsUInt32(), Vector256.Zero).GetLower(); + narrowed.StoreUnsafe(ref destinationBase, (nuint)(destinationRowOffset + column)); + } + + for (; column < width; column++) + { + int filteredOffset = filteredRowOffset + column; + int sample = source[sourceRowOffset + column]; + int restored0 = radii[0] > 0 ? filtered0[filteredOffset] : 0; + int restored1 = radii[1] > 0 ? filtered1[filteredOffset] : 0; + destination[destinationRowOffset + column] = ProjectSample( + sample, + bitDepth, + radii, + projection0, + projection1, + restored0, + restored1); + } + } + } + + /// + /// Projects the two restored signals in four-sample cross-platform batches. + /// + /// The bordered processing-unit source rectangle. + /// The number of samples between source rows. + /// The destination processing-unit rectangle. + /// The number of samples between destination rows. + /// The processing-unit width in samples. + /// The processing-unit height in samples. + /// The encoded sample bit depth. + /// The two selected filter radii. + /// The first decoded projection coefficient. + /// The second decoded projection coefficient. + /// The first fixed-point restored signal. + /// The second fixed-point restored signal. + /// The overload-selection value. + private static void Project( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + int bitDepth, + ReadOnlySpan radii, + int projection0, + int projection1, + ReadOnlySpan filtered0, + ReadOnlySpan filtered1, + Vector128 vector) + { + const int projectionShift = ProjectionBits + RestorationBits; + Vector128 projection0Vector = Vector128.Create(projection0); + Vector128 projection1Vector = Vector128.Create(projection1); + Vector128 rounding = Vector128.Create(1 << (projectionShift - 1)); + Vector128 maximumSample = Vector128.Create((1 << bitDepth) - 1); + ref ushort sourceBase = ref MemoryMarshal.GetReference(source); + ref ushort destinationBase = ref MemoryMarshal.GetReference(destination); + ref int filtered0Base = ref MemoryMarshal.GetReference(filtered0); + ref int filtered1Base = ref MemoryMarshal.GetReference(filtered1); + + for (int row = 0; row < height; row++) + { + int sourceRowOffset = ((row + Border) * sourceStride) + Border; + int destinationRowOffset = row * destinationStride; + int filteredRowOffset = row * width; + int column = 0; + int vectorEnd = width - Vector128.Count; + for (; column <= vectorEnd; column += Vector128.Count) + { + ref ushort sourceReference = ref Unsafe.Add(ref sourceBase, sourceRowOffset + column); + Vector64 packed = Unsafe.As>(ref sourceReference); + Vector128 samples = Vector128.WidenLower(Vector128.Create(packed, Vector64.Zero)).AsInt32(); + Vector128 unfiltered = Vector128.ShiftLeft(samples, RestorationBits); + Vector128 projected = Vector128.ShiftLeft(unfiltered, ProjectionBits); + if (radii[0] > 0) + { + Vector128 restored = Vector128.LoadUnsafe(ref filtered0Base, (nuint)(filteredRowOffset + column)); + projected += projection0Vector * (restored - unfiltered); + } + + if (radii[1] > 0) + { + Vector128 restored = Vector128.LoadUnsafe(ref filtered1Base, (nuint)(filteredRowOffset + column)); + projected += projection1Vector * (restored - unfiltered); + } + + Vector128 result = Vector128.ShiftRightArithmetic(projected + rounding, projectionShift); + result = Vector128.Min(Vector128.Max(result, Vector128.Zero), maximumSample); + Vector64 narrowed = Vector128.Narrow(result.AsUInt32(), Vector128.Zero).GetLower(); + ref ushort destinationReference = ref Unsafe.Add(ref destinationBase, destinationRowOffset + column); + Unsafe.As>(ref destinationReference) = narrowed; + } + + for (; column < width; column++) + { + int filteredOffset = filteredRowOffset + column; + int sample = source[sourceRowOffset + column]; + int restored0 = radii[0] > 0 ? filtered0[filteredOffset] : 0; + int restored1 = radii[1] > 0 ? filtered1[filteredOffset] : 0; + destination[destinationRowOffset + column] = ProjectSample( + sample, + bitDepth, + radii, + projection0, + projection1, + restored0, + restored1); + } + } + } + + /// + /// Projects one scalar remainder sample from the active restored signals. + /// + /// The unfiltered source sample. + /// The encoded sample bit depth. + /// The two selected filter radii. + /// The first decoded projection coefficient. + /// The second decoded projection coefficient. + /// The first fixed-point restored value. + /// The second fixed-point restored value. + /// The clipped projected sample. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static ushort ProjectSample( + int sample, + int bitDepth, + ReadOnlySpan radii, + int projection0, + int projection1, + int filtered0, + int filtered1) + { + int unfiltered = sample << RestorationBits; + int projected = unfiltered << ProjectionBits; + if (radii[0] > 0) + { + projected += projection0 * (filtered0 - unfiltered); + } + + if (radii[1] > 0) + { + projected += projection1 * (filtered1 - unfiltered); + } + + return (ushort)Av1Math.Clip3(0, (1 << bitDepth) - 1, RoundPowerOfTwo(projected, ProjectionBits + RestorationBits)); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1SelfGuidedFilter.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1SelfGuidedFilter.cs index ffc15d5d8..b1f173e08 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1SelfGuidedFilter.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopRestoration/Av1SelfGuidedFilter.cs @@ -2,13 +2,15 @@ // Licensed under the Six Labors Split License. using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopRestoration; /// /// Applies the normative AV1 self-guided restoration filter and projection. /// -internal static class Av1SelfGuidedFilter +internal static partial class Av1SelfGuidedFilter { /// /// The number of source samples required on every side of a filtered processing unit. @@ -40,6 +42,16 @@ internal static class Av1SelfGuidedFilter /// private const int ReciprocalBits = 12; + /// + /// The base-two exponent used to align work-buffer rows for every supported vector width. + /// + private const int BufferAlignmentLog2 = 4; + + /// + /// The extra columns separating integral-image rows to avoid adverse cache aliasing. + /// + private const int BufferPadding = 16; + /// /// The complete fixed-point self-guided blend range. /// @@ -70,7 +82,7 @@ internal static class Av1SelfGuidedFilter /// /// Gets the table mapping a bounded variance measure to its fixed-point local sample blend factor. /// - private static ReadOnlySpan XByXPlusOne => + private static ReadOnlySpan XByXPlusOne => [ 1, 128, 171, 192, 205, 213, 219, 224, 228, 230, 233, 235, 236, 238, 239, 240, 241, 242, 243, 243, 244, 244, 245, 245, 246, 246, 247, 247, 247, 247, @@ -110,8 +122,8 @@ internal static class Av1SelfGuidedFilter public static int GetScratchLength(int width, int height) { int filteredLength = width * height; - int coefficientLength = GetCoefficientBufferLength(width, height); - return (filteredLength * 2) + (coefficientLength * 2); + int bufferLength = GetBufferLength(width, height); + return (filteredLength * 2) + (bufferLength * 4); } /// @@ -139,6 +151,42 @@ internal static class Av1SelfGuidedFilter ReadOnlySpan projectionCoefficients, Span scratch) { + if (Avx2.IsSupported) + { + FilterBlock( + source, + sourceStride, + destination, + destinationStride, + width, + height, + bitDepth, + parameterSetIndex, + projectionCoefficients, + scratch, + Vector256.Zero); + + return; + } + + if (Vector128.IsHardwareAccelerated) + { + FilterBlock( + source, + sourceStride, + destination, + destinationStride, + width, + height, + bitDepth, + parameterSetIndex, + projectionCoefficients, + scratch, + Vector128.Zero); + + return; + } + int filteredLength = width * height; Span filtered0 = scratch[..filteredLength]; Span filtered1 = scratch.Slice(filteredLength, filteredLength); @@ -261,7 +309,7 @@ internal static class Av1SelfGuidedFilter int windowDiameter = (radius * 2) + 1; int windowArea = windowDiameter * windowDiameter; int rowStep = skipAlternateRows ? 2 : 1; - ReadOnlySpan xByXPlusOne = XByXPlusOne; + ReadOnlySpan xByXPlusOne = XByXPlusOne; ReadOnlySpan oneByX = OneByX; for (int row = -1; row < height + 1; row += rowStep) @@ -466,6 +514,23 @@ internal static class Av1SelfGuidedFilter [MethodImpl(MethodImplOptions.AggressiveInlining)] private static int GetCoefficientBufferLength(int width, int height) => (width + 2) * (height + 2); + /// + /// Gets the padded row stride shared by coefficient and integral-image buffers. + /// + /// The filtered processing-unit width. + /// The aligned number of integers reserved for each work-buffer row. + private static int GetBufferStride(int width) + => Av1Math.AlignPowerOf2(width + (Border * 2) + BufferPadding, BufferAlignmentLog2); + + /// + /// Gets the number of integers reserved for one padded work buffer. + /// + /// The filtered processing-unit width. + /// The filtered processing-unit height. + /// The required work-buffer length. + private static int GetBufferLength(int width, int height) + => GetBufferStride(width) * (height + (Border * 2) + 1); + /// /// Rounds a signed fixed-point value to the requested lower precision. /// diff --git a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs index 2e72a21c5..36ad66125 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs @@ -1290,9 +1290,14 @@ internal sealed class Av1TileReader : IAv1TileReader, IDisposable unitHeight = Av1Math.RoundPowerOf2(Math.Min(height + idy, maxBlockHigh), subY ? 1 : 0); unitWidth = Av1Math.RoundPowerOf2(Math.Min(width + idx, maxBlockWide), subX ? 1 : 0); - for (int blockRow = idy; blockRow < unitHeight; blockRow += stepRow) + int planeRow = idy >> (subY ? 1 : 0); + int planeColumn = idx >> (subX ? 1 : 0); + + // The 64x64 region cursor is expressed on the luma grid. Chroma transform offsets use the + // target plane's 4x4 grid, matching libaom's row/column subsampling before transform traversal. + for (int blockRow = planeRow; blockRow < unitHeight; blockRow += stepRow) { - for (int blockColumn = idx; blockColumn < unitWidth; blockColumn += stepColumn) + for (int blockColumn = planeColumn; blockColumn < unitWidth; blockColumn += stepColumn) { chromaTransformInfo[transformInfoUvIndex] = new Av1TransformInfo( transformSizeUv, blockColumn, blockRow); diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1LoopRestorationBenchmarks.cs b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1LoopRestorationBenchmarks.cs new file mode 100644 index 000000000..a2c335cbb --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1LoopRestorationBenchmarks.cs @@ -0,0 +1,169 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Columns; +using BenchmarkDotNet.Configs; +using BenchmarkDotNet.Jobs; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopRestoration; + +namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; + +/// +/// Measures normative AV1 self-guided restoration across a full-HD-equivalent luma workload. +/// +[Config(typeof(Configuration))] +[MemoryDiagnoser(displayGenColumns: false)] +[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] +[CategoriesColumn] +public class Av1LoopRestorationBenchmarks +{ + /// + /// The width of one normative self-guided processing unit. + /// + private const int Width = 64; + + /// + /// The height of one normative self-guided processing unit. + /// + private const int Height = 64; + + /// + /// The three source samples required on each side of a processing unit. + /// + private const int Border = 3; + + /// + /// The number of processing units covering a 1920 by 1080 luma plane. + /// + private const int ProcessingUnitCount = 30 * 17; + + /// + /// The bordered source-row stride. + /// + private const int SourceStride = Width + (Border * 2); + + /// + /// The self-guided parameter set activating both radius-two and radius-one filtering. + /// + private const int ParameterSetIndex = 0; + + /// + /// The deterministic bordered eight-bit source block. + /// + private readonly ushort[] source8 = new ushort[SourceStride * (Height + (Border * 2))]; + + /// + /// The deterministic bordered twelve-bit source block. + /// + private readonly ushort[] source12 = new ushort[SourceStride * (Height + (Border * 2))]; + + /// + /// The restored processing-unit destination. + /// + private readonly ushort[] destination = new ushort[Width * Height]; + + /// + /// The caller-owned self-guided work storage. + /// + private readonly int[] scratch = new int[Av1SelfGuidedFilter.GetScratchLength(Width, Height)]; + + /// + /// Gets the two transmitted projection coefficients used by the measured parameter set. + /// + private static ReadOnlySpan ProjectionCoefficients => [31, -7]; + + /// + /// Populates deterministic bordered source blocks outside the measured traversal. + /// + [GlobalSetup] + public void Setup() + { + for (int row = 0; row < Height + (Border * 2); row++) + { + for (int column = 0; column < SourceStride; column++) + { + int sample = ((row * 4051) + (column * 7919) + 127) & byte.MaxValue; + int offset = (row * SourceStride) + column; + this.source8[offset] = (ushort)sample; + this.source12[offset] = (ushort)(sample << 4); + } + } + } + + /// + /// Measures eight-bit self-guided restoration for a full-HD-equivalent luma plane. + /// + /// The final restored sample, keeping the output observable. + [Benchmark] + [BenchmarkCategory("8Bit")] + public ushort Restore8BitPlane() + { + for (int unit = 0; unit < ProcessingUnitCount; unit++) + { + Av1SelfGuidedFilter.FilterBlock( + this.source8, + SourceStride, + this.destination, + Width, + Width, + Height, + 8, + ParameterSetIndex, + ProjectionCoefficients, + this.scratch); + } + + return this.destination[^1]; + } + + /// + /// Measures twelve-bit self-guided restoration for a full-HD-equivalent luma plane. + /// + /// The final restored sample, keeping the output observable. + [Benchmark] + [BenchmarkCategory("12Bit")] + public ushort Restore12BitPlane() + { + for (int unit = 0; unit < ProcessingUnitCount; unit++) + { + Av1SelfGuidedFilter.FilterBlock( + this.source12, + SourceStride, + this.destination, + Width, + Width, + Height, + 12, + ParameterSetIndex, + ProjectionCoefficients, + this.scratch); + } + + return this.destination[^1]; + } + + /// + /// Configures production-process measurements for hardware, 128-bit, and scalar filtering. + /// + public sealed class Configuration : ManualConfig + { + /// + /// Initializes a new instance of the class. + /// + public Configuration() + { + this.AddJob(Job.ShortRun.WithId("Hardware").AsBaseline()); + + this.AddJob( + Job.ShortRun + .WithId("Vector128") + .WithEnvironmentVariable("DOTNET_EnableAVX", "0")); + + this.AddJob( + Job.ShortRun + .WithId("Scalar") + .WithEnvironmentVariable("DOTNET_EnableHWIntrinsic", "0")); + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs index 2589e31a6..50694fe0d 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs @@ -6,6 +6,7 @@ using SixLabors.ImageSharp.Formats; using SixLabors.ImageSharp.Formats.Heif; using SixLabors.ImageSharp.Formats.Heif.Av1; using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; using SixLabors.ImageSharp.Memory; using SixLabors.ImageSharp.PixelFormats; using SixLabors.ImageSharp.Tests.TestUtilities; @@ -24,6 +25,22 @@ public class Av1ReconstructionConformanceTests /// private const HwIntrinsics ReconstructionConfigurations = HwIntrinsics.AllowAll | HwIntrinsics.DisableHWIntrinsic; + /// + /// The hardware configurations covering the 256-bit, 128-bit, and scalar loop-restoration paths. + /// + private const HwIntrinsics LoopRestorationConfigurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// The coverage bit representing an active Wiener restoration unit. + /// + private const int WienerRestorationCoverage = 1 << (int)Av1RestorationFilterType.Wiener; + + /// + /// The coverage bit representing an active self-guided restoration unit. + /// + private const int SelfGuidedRestorationCoverage = 1 << (int)Av1RestorationFilterType.SgrProjection; + /// /// The hardware configurations covering the available vector widths and the scalar color-conversion fallback. /// @@ -94,6 +111,63 @@ public class Av1ReconstructionConformanceTests public void DecodeWithSuperResolutionMatchesPinnedLibaomReference() => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateSuperResolutionFixtures, ReconstructionConfigurations); + /// + /// Verifies active normative loop restoration and exact native samples against scalar libaom for independently + /// encoded eight-, ten-, and twelve-bit still-picture streams. + /// + [Fact] + public void DecodeWithLoopRestorationMatchesPinnedLibaomReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateLoopRestorationFixtures, LoopRestorationConfigurations); + + /// + /// Verifies combined super-resolution and loop-restoration geometry for independently encoded 8-bit 4:2:0 content. + /// + [Fact] + public void DecodeWithLoopRestorationAndSuperResolutionMatchesPinnedLibaomReference8Bit420() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateLoopRestorationAndSuperResolution8Bit420, + LoopRestorationConfigurations); + + /// + /// Verifies combined super-resolution and loop-restoration geometry for independently encoded 10-bit 4:2:2 content. + /// + [Fact] + public void DecodeWithLoopRestorationAndSuperResolutionMatchesPinnedLibaomReference10Bit422() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateLoopRestorationAndSuperResolution10Bit422, + LoopRestorationConfigurations); + + /// + /// Verifies combined super-resolution and loop-restoration geometry for independently encoded 12-bit 4:4:4 content. + /// + [Fact] + public void DecodeWithLoopRestorationAndSuperResolutionMatchesPinnedLibaomReference12Bit444() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateLoopRestorationAndSuperResolution12Bit444, + LoopRestorationConfigurations); + + /// + /// Verifies exact presented pixels and public metadata for independently encoded eight-, ten-, and twelve-bit + /// active-restoration AVIF images across the available vector widths and the scalar fallback. + /// + [Fact] + public void DecodeWithLoopRestorationMatchesPinnedLibavifPresentation() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateRestorationPresentedFixtures, PresentationConfigurations); + + /// + /// Verifies that the independently encoded AVIF presentation fixtures collectively select both restoration algorithms. + /// + [Fact] + public void LoopRestorationPresentationFixturesSelectBothAlgorithms() + { + int restorationCoverage = GetRestorationCoverageFromAvif(TestFile.Create(TestImages.Heif.Av1Restoration8BitAvif).Bytes); + restorationCoverage |= GetRestorationCoverageFromAvif(TestFile.Create(TestImages.Heif.Av1Restoration10BitAvif).Bytes); + restorationCoverage |= GetRestorationCoverageFromAvif(TestFile.Create(TestImages.Heif.Av1Restoration12BitAvif).Bytes); + + int requiredCoverage = WienerRestorationCoverage | SelfGuidedRestorationCoverage; + Assert.Equal(requiredCoverage, restorationCoverage & requiredCoverage); + } + /// /// Validates every active-CDEF fixture under the hardware configuration selected by . /// @@ -183,6 +257,109 @@ public class Av1ReconstructionConformanceTests Av1ColorFormat.Yuv444); } + /// + /// Validates every active loop-restoration fixture under the hardware configuration selected by + /// . + /// + private static void ValidateLoopRestorationFixtures() + { + int restorationCoverage = ValidateLoopRestorationFixture( + TestImages.Heif.Av1Restoration8BitPayload, + TestImages.Heif.Av1Restoration8BitReference, + 768, + 512, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv420); + + restorationCoverage |= ValidateLoopRestorationFixture( + TestImages.Heif.Av1Restoration10BitPayload, + TestImages.Heif.Av1Restoration10BitReference, + 1024, + 428, + Av1BitDepth.TenBit, + Av1ColorFormat.Yuv444); + + restorationCoverage |= ValidateLoopRestorationFixture( + TestImages.Heif.Av1Restoration12BitPayload, + TestImages.Heif.Av1Restoration12BitReference, + 1024, + 428, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv444); + + // Exact output only proves both restoration algorithms when the independent fixture set + // actually selects at least one unit of each type during every feature-runner invocation. + int requiredCoverage = WienerRestorationCoverage | SelfGuidedRestorationCoverage; + Assert.Equal(requiredCoverage, restorationCoverage & requiredCoverage); + } + + /// + /// Validates active restoration after super-resolution for 8-bit 4:2:0 content. + /// + private static void ValidateLoopRestorationAndSuperResolution8Bit420() + => ValidateLoopRestorationFixture( + TestImages.Heif.Av1RestorationSuperResolution8BitPayload, + TestImages.Heif.Av1RestorationSuperResolution8BitReference, + 768, + 512, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv420, + requireSuperResolution: true); + + /// + /// Validates active restoration after super-resolution for 10-bit 4:2:2 content. + /// + private static void ValidateLoopRestorationAndSuperResolution10Bit422() + => ValidateLoopRestorationFixture( + TestImages.Heif.Av1RestorationSuperResolution10BitPayload, + TestImages.Heif.Av1RestorationSuperResolution10BitReference, + 512, + 256, + Av1BitDepth.TenBit, + Av1ColorFormat.Yuv422, + requireSuperResolution: true); + + /// + /// Validates active restoration after super-resolution for 12-bit 4:4:4 content. + /// + private static void ValidateLoopRestorationAndSuperResolution12Bit444() + => ValidateLoopRestorationFixture( + TestImages.Heif.Av1RestorationSuperResolution12BitPayload, + TestImages.Heif.Av1RestorationSuperResolution12BitReference, + 1024, + 428, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv444, + requireSuperResolution: true); + + /// + /// Validates every active-restoration presentation fixture under the hardware configuration selected by + /// . + /// + private static void ValidateRestorationPresentedFixtures() + { + ValidatePresentedFixture( + TestImages.Heif.Av1Restoration8BitAvif, + TestImages.Heif.Av1Restoration8BitPresentationReference, + 768, + 512, + HeifBitDepth.Bit8); + + ValidatePresentedFixture( + TestImages.Heif.Av1Restoration10BitAvif, + TestImages.Heif.Av1Restoration10BitPresentationReference, + 1024, + 428, + HeifBitDepth.Bit10); + + ValidatePresentedFixture( + TestImages.Heif.Av1Restoration12BitAvif, + TestImages.Heif.Av1Restoration12BitPresentationReference, + 1024, + 428, + HeifBitDepth.Bit12); + } + /// /// Validates one elementary-stream sample and its containing AVIF image. /// @@ -219,7 +396,9 @@ public class Av1ReconstructionConformanceTests /// The expected native chroma-sampling layout. /// Indicates whether the stream must signal and select nonzero CDEF strengths. /// Indicates whether the stream must use normative horizontal upscaling. - private static void ValidateNativeFixture( + /// Indicates whether the stream must select at least one loop-restoration unit. + /// A bit mask containing every selected loop-restoration filter type. + private static int ValidateNativeFixture( string payloadPath, string referencePath, int width, @@ -227,8 +406,10 @@ public class Av1ReconstructionConformanceTests Av1BitDepth bitDepth, Av1ColorFormat colorFormat, bool requireActiveCdef, - bool requireSuperResolution = false) + bool requireSuperResolution = false, + bool requireLoopRestoration = false) { + int restorationCoverage = 0; byte[] payload = TestFile.Create(payloadPath).Bytes; byte[] reference = TestFile.Create(referencePath).Bytes; using Av1Decoder decoder = new(Configuration.Default); @@ -245,7 +426,11 @@ public class Av1ReconstructionConformanceTests ObuFrameSize frameSize = decoder.FrameHeader.FrameSize; Assert.True(frameSize.FrameWidth < frameSize.SuperResolutionUpscaledWidth); Assert.Equal(width, frameSize.SuperResolutionUpscaledWidth); - Assert.False(decoder.FrameHeader.LoopRestorationParameters.UsesLoopRestoration); + if (!requireLoopRestoration) + { + // The original super-resolution fixtures isolate upscaling by disabling restoration. + Assert.False(decoder.FrameHeader.LoopRestorationParameters.UsesLoopRestoration); + } } ObuLoopFilterParameters filterParameters = decoder.FrameHeader.LoopFilterParameters; @@ -290,7 +475,16 @@ public class Av1ReconstructionConformanceTests Assert.True(hasActiveStrength); } + if (requireLoopRestoration) + { + Assert.True(decoder.FrameHeader.LoopRestorationParameters.UsesLoopRestoration); + Assert.NotNull(decoder.FrameInfo); + restorationCoverage = GetRestorationCoverage(decoder); + Assert.NotEqual(0, restorationCoverage); + } + AssertNativePlanesEqual(frameBuffer, reference); + return restorationCoverage; } /// @@ -337,6 +531,36 @@ public class Av1ReconstructionConformanceTests requireActiveCdef: false, requireSuperResolution: true); + /// + /// Validates one independently encoded stream that activates normative loop restoration. + /// + /// The AV1 elementary-stream sample. + /// The native planar output produced by the pinned scalar libaom decoder. + /// The expected reconstructed width. + /// The expected reconstructed height. + /// The expected AV1 sample precision. + /// The expected native chroma-sampling layout. + /// Whether the stream must upscale from a narrower coded frame. + /// A bit mask containing every selected loop-restoration filter type. + private static int ValidateLoopRestorationFixture( + string payloadPath, + string referencePath, + int width, + int height, + Av1BitDepth bitDepth, + Av1ColorFormat colorFormat, + bool requireSuperResolution = false) + => ValidateNativeFixture( + payloadPath, + referencePath, + width, + height, + bitDepth, + colorFormat, + requireActiveCdef: false, + requireSuperResolution: requireSuperResolution, + requireLoopRestoration: true); + /// /// Validates the public presentation and metadata produced from one complete AVIF container. /// @@ -388,6 +612,71 @@ public class Av1ReconstructionConformanceTests ImageComparer.Exact.VerifySimilarity(reference, image); } + /// + /// Decodes the sole image item in an independently generated AVIF fixture and returns its restoration coverage. + /// + /// The complete AVIF file. + /// A bit mask containing every selected loop-restoration filter type. + private static int GetRestorationCoverageFromAvif(Span imageBytes) + { + int offset = 0; + while (offset < imageBytes.Length) + { + int headerLength = HeifBoxReader.ParseHeader(imageBytes[offset..], out long payloadLength, out Heif4CharCode boxType); + Assert.InRange(payloadLength, 0, int.MaxValue); + int payloadLength32 = (int)payloadLength; + + if (boxType == Heif4CharCode.Mdat) + { + // These single-item fixtures deliberately make the complete mdat payload the AV1 item. Decoding + // those exact bytes proves the container used for pixel comparison actually selects restoration. + Span payload = imageBytes.Slice(offset + headerLength, payloadLength32); + using Av1Decoder decoder = new(Configuration.Default); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer(payload, null, null, out _); + + Assert.NotNull(decoder.FrameHeader); + Assert.True(decoder.FrameHeader.LoopRestorationParameters.UsesLoopRestoration); + Assert.NotNull(decoder.FrameInfo); + int restorationCoverage = GetRestorationCoverage(decoder); + Assert.NotEqual(0, restorationCoverage); + return restorationCoverage; + } + + offset = checked(offset + headerLength + payloadLength32); + } + + Assert.Fail("The AVIF fixture does not contain a media-data box."); + return 0; + } + + /// + /// Returns the restoration algorithms selected by the decoded frame's unit grids. + /// + /// The decoder after tile parsing and reconstruction. + /// A bit mask containing every selected loop-restoration filter type. + private static int GetRestorationCoverage(Av1Decoder decoder) + { + int restorationCoverage = 0; + for (int plane = 0; plane < decoder.SequenceHeader!.ColorConfig.PlaneCount; plane++) + { + int rowCount = decoder.FrameInfo!.GetLoopRestorationUnitRowCount(plane); + int columnCount = decoder.FrameInfo.GetLoopRestorationUnitColumnCount(plane); + for (int row = 0; row < rowCount; row++) + { + for (int column = 0; column < columnCount; column++) + { + Av1RestorationFilterType filterType = decoder.FrameInfo.GetLoopRestorationUnit(plane, row, column).FilterType; + if (filterType != Av1RestorationFilterType.None) + { + restorationCoverage |= 1 << (int)filterType; + } + } + } + } + + return restorationCoverage; + } + /// /// Compares every visible native component sample with the independent planar reference. /// diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SelfGuidedFilterTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SelfGuidedFilterTests.cs new file mode 100644 index 000000000..b6cab2849 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1SelfGuidedFilterTests.cs @@ -0,0 +1,426 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.LoopRestoration; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 self-guided restoration against a direct-window definition across the supported hardware-intrinsic configurations. +/// +[Trait("Format", "Heif")] +public class Av1SelfGuidedFilterTests +{ + /// + /// The number of source samples required on every side of a filtered processing unit. + /// + private const int Border = 3; + + /// + /// The number of fractional bits retained by each self-guided filter result. + /// + private const int RestorationBits = 4; + + /// + /// The number of fractional bits used by the projection coefficients. + /// + private const int ProjectionBits = 7; + + /// + /// The hardware configurations required to exercise the AVX2 path, portable 128-bit path, and scalar fallback. + /// + private const HwIntrinsics Configurations = + HwIntrinsics.AllowAll | HwIntrinsics.DisableAVX | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Gets the radii selected by each of the sixteen normative parameter sets. + /// + private static ReadOnlySpan ParameterRadii => + [ + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 2, 0, 2, 0, + ]; + + /// + /// Gets the variance scales selected by each of the sixteen normative parameter sets. + /// + private static ReadOnlySpan ParameterScales => + [ + 140, 3236, 112, 2158, 93, 1618, 80, 1438, + 70, 1295, 58, 1177, 47, 1079, 37, 996, + 30, 925, 25, 863, -1, 2589, -1, 1618, + -1, 1177, -1, 925, 56, -1, 22, -1, + ]; + + /// + /// Gets processing-unit dimensions covering narrow chroma units, odd frame edges, and both vector remainder widths. + /// + private static ReadOnlySpan ProcessingUnitDimensions => + [ + 1, 1, + 3, 5, + 7, 4, + 13, 9, + 29, 6, + ]; + + /// + /// Verifies every normative parameter set, sample precision, and processing-unit tail against the direct-window definition. + /// + [Fact] + public void FilterMatchesDirectWindowDefinitionAcrossIntrinsicWidths() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateFilters, Configurations); + + /// + /// Validates the complete self-guided parameter matrix in the active hardware-intrinsic configuration. + /// + private static void ValidateFilters() + { + foreach (int bitDepth in new[] { 8, 10, 12 }) + { + int maximumSample = (1 << bitDepth) - 1; + ReadOnlySpan processingUnitDimensions = ProcessingUnitDimensions; + for (int dimensionIndex = 0; dimensionIndex < processingUnitDimensions.Length; dimensionIndex += 2) + { + int width = processingUnitDimensions[dimensionIndex]; + int height = processingUnitDimensions[dimensionIndex + 1]; + int sourceStride = width + (Border * 2) + 5; + int destinationStride = width + 7; + ushort[] source = new ushort[sourceStride * (height + (Border * 2))]; + ushort[] expected = new ushort[destinationStride * height]; + ushort[] actual = new ushort[destinationStride * height]; + int[] scratch = new int[Av1SelfGuidedFilter.GetScratchLength(width, height)]; + int[] projectionCoefficients = new int[2]; + + FillSource(source, sourceStride, maximumSample); + + for (int parameterSetIndex = 0; parameterSetIndex < 16; parameterSetIndex++) + { + expected.AsSpan().Fill(ushort.MaxValue); + actual.AsSpan().Fill(ushort.MaxValue); + projectionCoefficients[0] = -96 + ((parameterSetIndex * 17) & 127); + projectionCoefficients[1] = -32 + ((parameterSetIndex * 29) & 127); + + FilterReference( + source, + sourceStride, + expected, + destinationStride, + width, + height, + bitDepth, + parameterSetIndex, + projectionCoefficients); + + Av1SelfGuidedFilter.FilterBlock( + source, + sourceStride, + actual, + destinationStride, + width, + height, + bitDepth, + parameterSetIndex, + projectionCoefficients, + scratch); + + AssertBlockEqual(expected, actual, destinationStride, width, height, bitDepth, parameterSetIndex); + } + } + } + } + + /// + /// Populates the bordered source with deterministic values spanning the selected sample precision. + /// + /// The complete bordered source storage. + /// The number of samples between source rows. + /// The largest encoded sample value. + private static void FillSource(Span source, int sourceStride, int maximumSample) + { + int rowCount = source.Length / sourceStride; + for (int row = 0; row < rowCount; row++) + { + for (int column = 0; column < sourceStride; column++) + { + int value = (row * 239) + (column * 101) + (row * column * 17) + (((row + column) & 3) * (maximumSample / 3)); + source[(row * sourceStride) + column] = (ushort)(value & maximumSample); + } + } + + // Exact endpoints make clipping and the full local-variance range observable without depending on random input. + source[0] = 0; + source[^1] = (ushort)maximumSample; + } + + /// + /// Applies the normative projection to direct-window self-guided results. + /// + /// The source rectangle beginning three samples above and left of the processing unit. + /// The number of samples between source rows. + /// The destination storage beginning at the restored processing-unit origin. + /// The number of samples between destination rows. + /// The processing-unit width. + /// The processing-unit height. + /// The encoded sample bit depth. + /// The self-guided parameter-set index. + /// The two transmitted projection coefficients. + private static void FilterReference( + ReadOnlySpan source, + int sourceStride, + Span destination, + int destinationStride, + int width, + int height, + int bitDepth, + int parameterSetIndex, + ReadOnlySpan projectionCoefficients) + { + int parameterOffset = parameterSetIndex * 2; + int radius0 = ParameterRadii[parameterOffset]; + int radius1 = ParameterRadii[parameterOffset + 1]; + int scale0 = ParameterScales[parameterOffset]; + int scale1 = ParameterScales[parameterOffset + 1]; + int projection0; + int projection1; + + if (radius0 == 0) + { + projection0 = 0; + projection1 = (1 << ProjectionBits) - projectionCoefficients[1]; + } + else if (radius1 == 0) + { + projection0 = projectionCoefficients[0]; + projection1 = 0; + } + else + { + projection0 = projectionCoefficients[0]; + projection1 = (1 << ProjectionBits) - projection0 - projectionCoefficients[1]; + } + + int maximumSample = (1 << bitDepth) - 1; + for (int row = 0; row < height; row++) + { + int sourceRowOffset = (row + Border) * sourceStride; + int destinationRowOffset = row * destinationStride; + for (int column = 0; column < width; column++) + { + int unfiltered = source[sourceRowOffset + column + Border] << RestorationBits; + int projected = unfiltered << ProjectionBits; + + if (radius0 > 0) + { + int filtered0 = CalculateFilteredSample(source, sourceStride, column, row, bitDepth, radius0, scale0); + projected += projection0 * (filtered0 - unfiltered); + } + + if (radius1 > 0) + { + int filtered1 = CalculateFilteredSample(source, sourceStride, column, row, bitDepth, radius1, scale1); + projected += projection1 * (filtered1 - unfiltered); + } + + destination[destinationRowOffset + column] = + (ushort)Math.Clamp(RoundPowerOfTwo(projected, ProjectionBits + RestorationBits), 0, maximumSample); + } + } + } + + /// + /// Calculates one fixed-point filtered sample directly from its local coefficient windows. + /// + /// The bordered source rectangle. + /// The number of samples between source rows. + /// The processing-unit column. + /// The processing-unit row. + /// The encoded sample bit depth. + /// The selected filter radius. + /// The selected variance scale. + /// The filtered sample with four fractional bits. + private static int CalculateFilteredSample( + ReadOnlySpan source, + int sourceStride, + int column, + int row, + int bitDepth, + int radius, + int scale) + { + int blendFactor = 0; + int localMean = 0; + int roundingBits; + + if (radius == 2 && (row & 1) == 0) + { + for (int coefficientRow = row - 1; coefficientRow <= row + 1; coefficientRow += 2) + { + for (int coefficientColumn = column - 1; coefficientColumn <= column + 1; coefficientColumn++) + { + int weight = coefficientColumn == column ? 6 : 5; + (int localBlendFactor, int localMeanValue) = + CalculateCoefficient(source, sourceStride, coefficientColumn, coefficientRow, bitDepth, radius, scale); + + blendFactor += weight * localBlendFactor; + localMean += weight * localMeanValue; + } + } + + roundingBits = 9; + } + else if (radius == 2) + { + for (int coefficientColumn = column - 1; coefficientColumn <= column + 1; coefficientColumn++) + { + int weight = coefficientColumn == column ? 6 : 5; + (int localBlendFactor, int localMeanValue) = + CalculateCoefficient(source, sourceStride, coefficientColumn, row, bitDepth, radius, scale); + + blendFactor += weight * localBlendFactor; + localMean += weight * localMeanValue; + } + + roundingBits = 8; + } + else + { + for (int coefficientRow = row - 1; coefficientRow <= row + 1; coefficientRow++) + { + for (int coefficientColumn = column - 1; coefficientColumn <= column + 1; coefficientColumn++) + { + int weight = coefficientRow == row || coefficientColumn == column ? 4 : 3; + (int localBlendFactor, int localMeanValue) = + CalculateCoefficient(source, sourceStride, coefficientColumn, coefficientRow, bitDepth, radius, scale); + + blendFactor += weight * localBlendFactor; + localMean += weight * localMeanValue; + } + } + + roundingBits = 9; + } + + int sample = source[((row + Border) * sourceStride) + column + Border]; + return RoundPowerOfTwo((blendFactor * sample) + localMean, roundingBits); + } + + /// + /// Calculates the blend factor and scaled local mean for one coefficient location by visiting every window sample. + /// + /// The bordered source rectangle. + /// The number of samples between source rows. + /// The coefficient column relative to the processing unit. + /// The coefficient row relative to the processing unit. + /// The encoded sample bit depth. + /// The square-window radius. + /// The variance scale. + /// The local blend factor and scaled mean. + private static (int BlendFactor, int LocalMean) CalculateCoefficient( + ReadOnlySpan source, + int sourceStride, + int column, + int row, + int bitDepth, + int radius, + int scale) + { + int centerX = column + Border; + int centerY = row + Border; + int sum = 0; + int squareSum = 0; + + for (int windowY = centerY - radius; windowY <= centerY + radius; windowY++) + { + int sourceRowOffset = windowY * sourceStride; + for (int windowX = centerX - radius; windowX <= centerX + radius; windowX++) + { + int sample = source[sourceRowOffset + windowX]; + sum += sample; + squareSum += sample * sample; + } + } + + int diameter = (radius * 2) + 1; + int windowArea = diameter * diameter; + int normalizedSquareSum = RoundPowerOfTwo(squareSum, 2 * (bitDepth - 8)); + int normalizedSum = RoundPowerOfTwo(sum, bitDepth - 8); + uint squareOfSum = (uint)normalizedSum * (uint)normalizedSum; + uint scaledSquareSum = (uint)normalizedSquareSum * (uint)windowArea; + uint variance = scaledSquareSum < squareOfSum ? 0 : scaledSquareSum - squareOfSum; + uint varianceIndex = Math.Min(RoundPowerOfTwo(variance * (uint)scale, 20), 255U); + + // The endpoint exceptions are part of the normative table. The middle values are the rounded x / (x + 1) ratio in Q8. + int blendFactor = varianceIndex switch + { + 0 => 1, + 255 => 256, + _ => (int)(((varianceIndex << 8) + ((varianceIndex + 1) >> 1)) / (varianceIndex + 1)), + }; + + uint reciprocal = radius == 1 ? 455U : 164U; + uint meanProduct = (uint)(256 - blendFactor) * (uint)sum * reciprocal; + int localMean = (int)RoundPowerOfTwo(meanProduct, 12); + return (blendFactor, localMean); + } + + /// + /// Verifies visible samples and confirms that the filter does not overwrite destination-row padding. + /// + /// The direct-window output. + /// The production output. + /// The number of samples between destination rows. + /// The processing-unit width. + /// The processing-unit height. + /// The encoded sample bit depth. + /// The self-guided parameter-set index. + private static void AssertBlockEqual( + ReadOnlySpan expected, + ReadOnlySpan actual, + int stride, + int width, + int height, + int bitDepth, + int parameterSetIndex) + { + for (int row = 0; row < height; row++) + { + int rowOffset = row * stride; + for (int column = 0; column < width; column++) + { + if (expected[rowOffset + column] != actual[rowOffset + column]) + { + Assert.Fail( + $"Self-guided parameter {parameterSetIndex}, {bitDepth}-bit block differs at ({column}, {row}): " + + $"expected {expected[rowOffset + column]}, actual {actual[rowOffset + column]}."); + } + } + + for (int column = width; column < stride; column++) + { + Assert.Equal(ushort.MaxValue, actual[rowOffset + column]); + } + } + } + + /// + /// Rounds a signed fixed-point value to the requested lower precision. + /// + /// The signed fixed-point value. + /// The number of low bits to discard. + /// The rounded signed value. + private static int RoundPowerOfTwo(int value, int bitCount) + => bitCount == 0 ? value : (value + (1 << (bitCount - 1))) >> bitCount; + + /// + /// Rounds an unsigned fixed-point value to the requested lower precision. + /// + /// The unsigned fixed-point value. + /// The number of low bits to discard. + /// The rounded unsigned value. + private static uint RoundPowerOfTwo(uint value, int bitCount) + => bitCount == 0 ? value : (value + (1U << (bitCount - 1))) >> bitCount; +} diff --git a/tests/ImageSharp.Tests/TestImages.cs b/tests/ImageSharp.Tests/TestImages.cs index 3679e7bc0..b1674b42b 100644 --- a/tests/ImageSharp.Tests/TestImages.cs +++ b/tests/ImageSharp.Tests/TestImages.cs @@ -1371,6 +1371,24 @@ public static class TestImages public const string Av1SuperResolution10BitReference = "Heif/Av1/Conformance/libaom-superres-cosmos-10b-libaom.yuv"; public const string Av1SuperResolution12BitPayload = "Heif/Av1/Conformance/libaom-superres-cosmos-12b.bit"; public const string Av1SuperResolution12BitReference = "Heif/Av1/Conformance/libaom-superres-cosmos-12b-libaom.yuv"; + public const string Av1Restoration8BitPayload = "Heif/Av1/Conformance/libaom-restoration-kodim23-8b.bit"; + public const string Av1Restoration8BitReference = "Heif/Av1/Conformance/libaom-restoration-kodim23-8b-libaom.yuv"; + public const string Av1Restoration8BitAvif = "Heif/Av1/Conformance/libavif-restoration-kodim23-8b.avif"; + public const string Av1Restoration8BitPresentationReference = "Heif/Av1/Conformance/libavif-restoration-kodim23-8b.png"; + public const string Av1Restoration10BitPayload = "Heif/Av1/Conformance/libaom-restoration-cosmos-10b.bit"; + public const string Av1Restoration10BitReference = "Heif/Av1/Conformance/libaom-restoration-cosmos-10b-libaom.yuv"; + public const string Av1Restoration10BitAvif = "Heif/Av1/Conformance/libavif-restoration-cosmos-10b.avif"; + public const string Av1Restoration10BitPresentationReference = "Heif/Av1/Conformance/libavif-restoration-cosmos-10b.png"; + public const string Av1Restoration12BitPayload = "Heif/Av1/Conformance/libaom-restoration-cosmos-12b.bit"; + public const string Av1Restoration12BitReference = "Heif/Av1/Conformance/libaom-restoration-cosmos-12b-libaom.yuv"; + public const string Av1Restoration12BitAvif = "Heif/Av1/Conformance/libavif-restoration-cosmos-12b.avif"; + public const string Av1Restoration12BitPresentationReference = "Heif/Av1/Conformance/libavif-restoration-cosmos-12b.png"; + public const string Av1RestorationSuperResolution8BitPayload = "Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b.bit"; + public const string Av1RestorationSuperResolution8BitReference = "Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b-libaom.yuv"; + public const string Av1RestorationSuperResolution10BitPayload = "Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422.bit"; + public const string Av1RestorationSuperResolution10BitReference = "Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422-libaom.yuv"; + public const string Av1RestorationSuperResolution12BitPayload = "Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b.bit"; + public const string Av1RestorationSuperResolution12BitReference = "Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b-libaom.yuv"; } public static class Ico diff --git a/tests/Images/Input/Heif/Av1/Conformance/README.md b/tests/Images/Input/Heif/Av1/Conformance/README.md index c3351146e..8916e92ff 100644 --- a/tests/Images/Input/Heif/Av1/Conformance/README.md +++ b/tests/Images/Input/Heif/Av1/Conformance/README.md @@ -30,3 +30,13 @@ The conformance tests compare every visible reconstructed sample with these file The `libaom-superres-*` streams were encoded from the same Kodak and Cosmos sources with the pinned generic libaom build. Their material options were `--usage=2 --passes=1 --limit=1 --obu --end-usage=q --cq-level=30 --cpu-used=4 --threads=1 --lag-in-frames=0 --full-still-picture-hdr --enable-cdef=0 --enable-restoration=0 --superres-mode=1 --superres-denominator=12 --superres-kf-denominator=12`, together with the matching input depth, output depth, and profile. Disabling CDEF and restoration isolates the normative horizontal upscaling result, while the tests separately require a coded width smaller than the displayed width so an unscaled stream cannot satisfy the reference comparison. The matching `libaom-superres-*-libaom.yuv` files were decoded by `aomdec --rawvideo` from that exact generic build. They retain the displayed 768x512 8-bit YUV 4:2:0 and 1024x428 10/12-bit YUV 4:4:4 layouts described above. + +The `libaom-restoration-*` streams were encoded from the same Kodak and Cosmos sources with the pinned generic libaom build. Their material options were `--usage=2 --passes=1 --limit=1 --obu --end-usage=q --cq-level=30 --cpu-used=4 --threads=1 --lag-in-frames=0 --full-still-picture-hdr --enable-cdef=0 --enable-restoration=1 --superres-mode=0`, together with the matching input depth, output depth, and profile. The matching `*-libaom.yuv` files were decoded by that build's `aomdec --rawvideo` and retain the 768x512 8-bit YUV 4:2:0 and 1024x428 10/12-bit YUV 4:4:4 layouts. The tests require at least one signaled restoration unit and compare every resulting native sample exactly. + +The `libavif-restoration-*` container templates were encoded from the same sources with the pinned generic libavif build. Pinned libavif forcibly disables restoration for 12-bit libaom encoding, and its default all-intra settings did not select active restoration for the other templates. Each template's sole AV1 item was therefore replaced mechanically with the matching active-restoration payload above. Only the single `iloc` extent length and terminal `mdat` box size changed; the libavif-generated codec configuration, dimensions, CICP properties, item relationships, and remaining container layout were retained. + +The matching `libavif-restoration-*.png` files were decoded from those exact AVIF containers with the pinned generic `avifdec -j 1 -d 8`; the 8-bit 4:2:0 reference additionally selected bilinear chroma upsampling. The tests first decode each container's actual `mdat` payload to require both Wiener and self-guided unit selection, then compare every presented RGBA byte with the scalar-libavif PNG exactly and without a tolerance. + +The `libaom-restoration-superres-*` streams combine active restoration with a coded width reduced by super-resolution denominator 12. They use the same pinned generic libaom build and material encoder options as the restoration streams, with `--superres-mode=1 --superres-denominator=12 --superres-kf-denominator=12`. The 8-bit fixture is 768x512 YUV 4:2:0, the 10-bit fixture is 512x256 YUV 4:2:2, and the 12-bit fixture is 1024x428 YUV 4:4:4. Their matching `*-libaom.yuv` files were decoded from the exact payloads by the pinned generic `aomdec --rawvideo` build. + +The 10-bit 4:2:2 source was produced from libavif's `abc.png` with pinned generic `avifenc` using `-j 1 -s 8 -q 100 -d 10 -y 422`, then decoded to Y4M before the combined libaom encode. Its clipped rightmost 128x128 coding block crosses a second 64x64 residual region. This independently exercises the required conversion of the luma-region cursor to the subsampled chroma transform grid instead of relying only on full-width 4:4:4 blocks. diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-10b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-10b-libaom.yuv new file mode 100644 index 000000000..fc9bf3c73 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-10b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:609d2dbdee3660b466b846b2269a8c061665ae7d52737bcf6c0004094c88dc80 +size 2629632 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-10b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-10b.bit new file mode 100644 index 000000000..d7eff8ae7 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-10b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cbdd03657e8e21ae26769a1e1e59920dba779c47cd1d03b18425148310f10a0f +size 26746 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-12b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-12b-libaom.yuv new file mode 100644 index 000000000..beb3e324e --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-12b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:61971cbe96f1cfc86b42b0608b91bebc7cc67bf758acf7cfdf7e17cd36f66beb +size 2629632 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-12b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-12b.bit new file mode 100644 index 000000000..05d854575 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-cosmos-12b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7444a3f87497ed68cad2f5af2363b80381523f976a982de0fa1f58db8533b23b +size 23182 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-kodim23-8b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-kodim23-8b-libaom.yuv new file mode 100644 index 000000000..80b0c7080 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-kodim23-8b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6305858ad4204af4df42ac4bb207e94e1a7f155d97241e2cb335cb46db8d6925 +size 589824 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-kodim23-8b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-kodim23-8b.bit new file mode 100644 index 000000000..bbf554513 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-kodim23-8b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0e605f0783daf8f6c530fe73572468b20f9834e7a8699a3e4e0f846741306711 +size 16460 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422-libaom.yuv new file mode 100644 index 000000000..cd0cb4319 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:86771fc87add9f179f327c0c9d44ae9ee0b373f977c23585e8bd0c322a95a215 +size 524288 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422.bit new file mode 100644 index 000000000..d93077ee2 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f3c3d6ed68a69a0a689b333ac71f016b5f60cbd48941faf0dcec707f8e2d6c9c +size 1131 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b-libaom.yuv new file mode 100644 index 000000000..c8b8de7fd --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:84666f72dd19ae549b7c5b77b07063e3a52c6a7e7d8798be126c0cdfcc8af5e4 +size 2629632 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b.bit new file mode 100644 index 000000000..35af9e9d9 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bed0130a163d9ad7c3068f6964c3944871d66da5efd2f37e435467183dba2402 +size 16125 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b-libaom.yuv new file mode 100644 index 000000000..1b3517226 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5c6e3642a399f59b0aea75261a69f0f31e6fae07998631831b17b7338efa3d9e +size 589824 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b.bit new file mode 100644 index 000000000..222d23582 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-restoration-superres-kodim23-8b.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f7b5ec685eb17237354dd7c7d08dddbe41bf408e5eb1c0fb697ceb4a0dec81e4 +size 12322 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-10b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-10b.avif new file mode 100644 index 000000000..d1f0acbbf --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-10b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d3215c84d33f1cb37f9c8998ed4c1982837b9207c7a3018cac0df1659691ed8b +size 27021 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-10b.png b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-10b.png new file mode 100644 index 000000000..309de9c1a --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-10b.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:24ff2fe2ab28f044945df7b35d4ed9213ae7e054e33b45fb92e4924566f3e944 +size 454939 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-12b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-12b.avif new file mode 100644 index 000000000..ecf55471e --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-12b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc446537fb7954656379384232857a23c5eab25187e63352097d276d8034869e +size 23453 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-12b.png b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-12b.png new file mode 100644 index 000000000..5cdc002e5 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-cosmos-12b.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4835eabb73ca6e51db7f6779a76499a9f2d07274116b816487681a51883f074f +size 436999 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-kodim23-8b.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-kodim23-8b.avif new file mode 100644 index 000000000..927b6c8fb --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-kodim23-8b.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fb86fc23e29c19566600fa8e5230038598c37836971feff35976b3f720fd5df0 +size 16735 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-kodim23-8b.png b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-kodim23-8b.png new file mode 100644 index 000000000..f457ec6db --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-restoration-kodim23-8b.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2098b78efb22b27473802e4e9e074d4948eda6d6bb06510069c9497e63f37bf5 +size 383051