From 49b348c5ddb752b093c36badf3a3ce54ca6dd4f5 Mon Sep 17 00:00:00 2001 From: James Jackson-South Date: Thu, 27 Aug 2026 07:58:06 +1000 Subject: [PATCH] Implement SIMD-first AV1 film grain synthesis --- HEIF_IMPLEMENTATION_PLAN.md | 11 +- .../Pipeline/FilmGrain/Av1FilmGrainDecoder.cs | 396 +------ .../FilmGrain/Av1FilmGrainGaussianSequence.cs | 9 +- .../Pipeline/FilmGrain/Av1FilmGrainNoise.cs | 985 ++++++++++++++++++ .../Pipeline/FilmGrain/Av1FilmGrainOverlap.cs | 313 ++++++ .../FilmGrain/Av1FilmGrainSampleOperator.cs | 189 ++++ .../Codecs/Heif/Av1FilmGrainBenchmarks.cs | 272 +++++ .../Av1/Av1ReconstructionConformanceTests.cs | 155 ++- tests/ImageSharp.Tests/TestImages.cs | 14 + .../Input/Heif/Av1/Conformance/README.md | 33 + ...ibaom-film-grain-circle-10b-422-libaom.yuv | 3 + .../libaom-film-grain-circle-10b-422.bit | 3 + ...circle-12b-444-identity-limited-libaom.yuv | 3 + ...-grain-circle-12b-444-identity-limited.bit | 3 + ...ibaom-film-grain-circle-12b-444-libaom.yuv | 3 + .../libaom-film-grain-circle-12b-444.bit | 3 + ...ilm-grain-circle-8b-400-limited-libaom.yuv | 3 + ...ibaom-film-grain-circle-8b-400-limited.bit | 3 + ...libaom-film-grain-circle-8b-420-libaom.yuv | 3 + ...ilm-grain-circle-8b-420-limited-libaom.yuv | 3 + ...ibaom-film-grain-circle-8b-420-limited.bit | 3 + .../libaom-film-grain-circle-8b-420.bit | 3 + ...lm-grain-draw-points-8b-420-odd-libaom.yuv | 3 + ...baom-film-grain-draw-points-8b-420-odd.bit | 3 + 24 files changed, 2017 insertions(+), 402 deletions(-) create mode 100644 src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainNoise.cs create mode 100644 src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainOverlap.cs create mode 100644 src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainSampleOperator.cs create mode 100644 tests/ImageSharp.Benchmarks/Codecs/Heif/Av1FilmGrainBenchmarks.cs create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-10b-422-libaom.yuv create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-10b-422.bit create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited-libaom.yuv create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited.bit create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-libaom.yuv create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444.bit create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited-libaom.yuv create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited.bit create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-libaom.yuv create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited-libaom.yuv create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited.bit create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420.bit create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd-libaom.yuv create mode 100644 tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd.bit diff --git a/HEIF_IMPLEMENTATION_PLAN.md b/HEIF_IMPLEMENTATION_PLAN.md index baf7727a4..9c671958d 100644 --- a/HEIF_IMPLEMENTATION_PLAN.md +++ b/HEIF_IMPLEMENTATION_PLAN.md @@ -275,7 +275,7 @@ This snapshot pins or classifies the available references and failures; it does | `Av1WienerFilter` | AV1 sections 7.17.4 and 7.17.5 Wiener restoration filtering and coefficient derivation | libaom `av1/common/restoration.c`, `av1/common/restoration.h`, `av1/common/convolve.c`, and `av1/common/convolve.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Preserve the implicit center-sample contribution, separable horizontal/vertical rounding, bit-depth-dependent 16-bit intermediate range, and final 8/10/12-bit clipping. Reuse `Vector128_.MultiplyAddAdjacent` for the contiguous horizontal eight-tap product with an exact scalar fallback. Keep the restoration stage disabled until stripe boundaries and self-guided filtering are both complete. | | `Av1SelfGuidedFilter` | AV1 sections 7.17.2 and 7.17.3 self-guided and box-filter processes | libaom `av1/common/restoration.c` and `av1/common/restoration.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Design the window layout, caller-owned scratch, local statistics, and projection traversal from libaom's architecture-specific kernels so the production path is SIMD-first. Implement the equivalent scalar fallback through the same contract. Preserve the sixteen normative radius/variance parameter sets, local mean and variance normalization, alternating-row radius-two optimization, decoded projection-coefficient behavior, signed rounding, and 8/10/12-bit clipping. Keep this image-reconstruction stage disabled until restoration stripe boundaries are complete. | | `Av1LoopRestorationBoundary`, `Av1LoopRestorationDecoder`, and `Av1FrameDecoder` restoration-stage ordering | AV1 section 7.17 loop restoration, including striped boundary semantics | libaom `av1/common/restoration.c`, `av1/common/restoration.h`, `av1/common/resize.c`, and `av1/decoder/decodeframe.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Preserve two deblocked rows at internal 64-luma stripe boundaries before CDEF, apply the existing normative SIMD-backed super-resolution kernel to saved rows when scaled, use post-CDEF/super-resolution samples at frame edges, extend the final restoration unit up to 150 percent of nominal size, and filter from immutable plane snapshots into separate output planes. This is bounded still-image reconstruction state, not retained reference-frame, track, timing, or playback state. | -| `Av1FilmGrainDecoder` and `Av1FilmGrainGaussianSequence` | AV1 section 7.18 film-grain synthesis | libaom `av1/decoder/grain_synthesis.c`, `av1/decoder/grain_synthesis.h`, and `aom_dsp/grain_params.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Preserve the normative 2,048-sample Gaussian sequence, linear-feedback shift register, luma/chroma autoregressive templates, scaling lookup interpolation, 32x32 block selection, boundary overlap, restricted-range clipping, monochrome and 4:2:0/4:2:2/4:4:4 layouts, and 8/10/12-bit arithmetic. Use allocator-owned scratch and runtime-optimized span copies. Apply grain only to the displayed still-image samples after all in-loop filters; reference-frame parameter inheritance remains sequence-playback state and is outside this codec scope. | +| `Av1FilmGrainDecoder`, `Av1FilmGrainGaussianSequence`, `Av1FilmGrainNoise`, `Av1FilmGrainOverlap`, and `Av1FilmGrainSampleOperator` | AV1 section 7.18 film-grain synthesis | libaom `av1/decoder/grain_synthesis.c`, `av1/decoder/grain_synthesis.h`, and `aom_dsp/grain_params.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Preserve the normative 2,048-sample Gaussian sequence as compile-time span data, the serial linear-feedback shift register and luma/chroma autoregressive templates, scaling lookup interpolation, 32x32 block selection, boundary overlap, restricted-range clipping, monochrome and 4:2:0/4:2:2/4:4:4 layouts, and 8/10/12-bit arithmetic. Use allocator-owned scratch. Apply scaling and noise through AVX2 gather arithmetic at every bit depth, retain the measured cross-platform 128-bit high-bit-depth path, and use the exact scalar path for 8-bit machines without AVX2 and for vector tails. Process horizontal overlap through preferred-native 512-bit, then 256-bit, 128-bit, and scalar tiers; retain scalar vertical overlap because each output row exposes only one or two strided samples. Keep SIMD width, ISA, sample storage, and bit depth out of folders, namespaces, files, and type names. Apply grain only to displayed still-image samples after all in-loop filters; reference-frame parameter inheritance remains sequence-playback state and is outside this codec scope. | | `Av1FrameInfo`, `Av1TileReader`, and `Av1BlockDecoder` transform/coefficient storage | AV1 section 5.11.39 coefficient syntax and section 7.11.2 reconstruction | libaom `av1/decoder/decodetxb.c` and `av1/decoder/decoder.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Preserve separate luma and chroma transform coefficients at monotonically advancing per-plane offsets within each superblock so reconstruction consumes the same transform-block order produced by tile parsing. | | `Av1InverseQuantizer` and `Av1InverseQuantizationLookup` | AV1 section 7.12.3 inverse quantization | libaom `aom_dsp/aom_dsp_common.h`, `av1/common/quant_common.c`, and `av1/decoder/decodetxb.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Select the per-segment matrix level, alias 64-pixel transform dimensions to their adjusted matrices, retain a flat level-15 matrix, and apply the five-bit inverse-matrix weight scale. The large managed lookup remains a single process-wide table. | | `Av1ForwardTransformer`, `Av1Inverse2dTransformer`, `Av1Transform2dFlipConfiguration`, the forward/inverse 1-D operator structs, and `Av1Transform1dMath` | AV1 forward transform definitions and section 7.11.2 inverse transform and reconstruction | libaom `av1/encoder/av1_fwd_txfm2d_hwy.h`, its AVX-512 and AVX2 instantiations, `av1/encoder/av1_fwd_txfm1d.c`, `av1/common/av1_inv_txfm1d_cfg.h`, `av1/common/av1_inv_txfm1d.c`, `av1/common/av1_inv_txfm2d.c`, the x86 AVX2/SSE4 implementations, and the corresponding Neon implementations at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Preserve the normative staged DCT, ADST, and identity arithmetic, direction-specific stage ranges and shifts, transposition, clipping, and high-bit-depth sample addition. Stateless static-generic operators follow ImageSharp's JPEG color-transform pattern. Forward operators share one behavior model across scalar, `Vector128`, `Vector256`, and `Vector512`, selecting packed or expanded lane storage at the 2-D boundary. Inverse production traversal retains the verified scalar, `Vector128`, and `Vector256` tiers until a wider upstream shape and complete-block benefit are both established. | @@ -352,7 +352,8 @@ This assessment was reconciled with the source tree on 2026-08-26. Unless a resu - The single-still `Av1Decoder` path now parses tile state before allocating and reconstructing one independently decodable frame, and it disposes the reconstruction planes after pixel conversion. It deliberately does not retain animation/video reference frames or implement `show_existing_frame` playback state. - Transform coefficient entropy derivation and updates now address the above contexts relative to the tile column and the left contexts relative to the current superblock row, preserve luma coordinates independently of chroma subsampling, and test every packed context entry for the libaom any-nonzero rule. Extended vertical partition updates advance the mode-information column rather than the row. The existing multi-superblock 4:4:4 AVIF fixture now completes tile parsing; independent coefficient-context vectors across tile boundaries, chroma layouts, bit depths, and edge-clipped transforms remain required. -- The reconstruction pipeline now records plane-relative transform geometry, preserves tile-local delta-Q and delta-LF predictors, derives segmentation and reference-adjusted filter levels, and runs the exact AV1 4-, 6-, 8-, and 14-tap deblocking kernels in normative vertical-then-horizontal order. Deblocking uses the same closed edge-operator architecture as the HEVC filter, with operators specialized by sample storage and orientation, `Vector128` lanes representing the four rows or columns along an edge, and an allocation-free scalar fallback for disabled intrinsics. Exact native-plane comparison with pinned scalar libaom output now verifies active deblocking and complete reconstruction for real 8-bit 4:2:0, 10-bit 4:4:4, and 12-bit 4:4:4 content; genuine AVIF containers separately verify presentation and public bit-depth metadata. The pipeline then applies CDEF through one semantic filter architecture: paired AVX2 and single-block `Vector128` direction analysis, closed primary/secondary strength operators, packed 4x4/4x8/8x4/8x8 constrained filtering, byte/16-bit output operators, and an exact allocation-free scalar fallback. Decoder orchestration now owns the immutable plane snapshots and clean direction/variance maps through ImageSharp's memory allocator, widens 8-bit source rows with the same AVX2/128-bit/scalar tiers as libaom, lists each unit's non-skipped blocks in fixed inline storage, analyzes listed blocks in pairs, and writes filtered bytes or 16-bit samples directly to the frame planes. Independently encoded active-CDEF 8-bit 4:2:0 and 10/12-bit 4:4:4 streams match every visible native sample produced by pinned scalar libaom with restoration disabled. Independently encoded AVIF containers at the same three bit depths also match pinned scalar-libavif presentation exactly under normal, 256-bit, 128-bit, and scalar color-conversion dispatch. Active super-resolution derives the Appendix A bounded coded width and applies the exact 64-phase, 8-tap horizontal filter with aligned reconstruction-edge input, 8/10/12-bit clipping, and the existing cross-platform `Vector128_.MultiplyAddAdjacent` helper. Independently encoded active-super-resolution AV1 streams at 8, 10, and 12 bits match every visible native sample produced by pinned scalar libaom under normal and forced-scalar dispatch; equivalent AVIF presentation fixtures remain required. Loop restoration follows super-resolution, preserves the required pre-CDEF deblocked context at internal stripes, and applies decoded Wiener or self-guided units from immutable plane snapshots. Independently encoded active-restoration streams at 8, 10, and 12 bits now match every native sample from pinned scalar libaom across AVX2, 128-bit, and scalar dispatch, with the fixture matrix proving both Wiener and self-guided unit selection. The matching AVIF containers also match pinned scalar-libavif presentation exactly, and an independent direct-window definition verifies all sixteen self-guided parameter sets at each supported bit depth across vector and scalar dispatch. Combined 8-bit 4:2:0, 10-bit 4:2:2, and 12-bit 4:4:4 streams additionally verify restoration-unit boundaries after super-resolution, including clipped chroma transform traversal at a coded-frame edge. The visible still-image path then applies the complete self-contained film-grain parameter set after all in-loop filters; independent film-grain vectors remain required before that stage has external pixel-level verification. +- The reconstruction pipeline now records plane-relative transform geometry, preserves tile-local delta-Q and delta-LF predictors, derives segmentation and reference-adjusted filter levels, and runs the exact AV1 4-, 6-, 8-, and 14-tap deblocking kernels in normative vertical-then-horizontal order. Deblocking uses the same closed edge-operator architecture as the HEVC filter, with operators specialized by sample storage and orientation, `Vector128` lanes representing the four rows or columns along an edge, and an allocation-free scalar fallback for disabled intrinsics. Exact native-plane comparison with pinned scalar libaom output now verifies active deblocking and complete reconstruction for real 8-bit 4:2:0, 10-bit 4:4:4, and 12-bit 4:4:4 content; genuine AVIF containers separately verify presentation and public bit-depth metadata. The pipeline then applies CDEF through one semantic filter architecture: paired AVX2 and single-block `Vector128` direction analysis, closed primary/secondary strength operators, packed 4x4/4x8/8x4/8x8 constrained filtering, byte/16-bit output operators, and an exact allocation-free scalar fallback. Decoder orchestration now owns the immutable plane snapshots and clean direction/variance maps through ImageSharp's memory allocator, widens 8-bit source rows with the same AVX2/128-bit/scalar tiers as libaom, lists each unit's non-skipped blocks in fixed inline storage, analyzes listed blocks in pairs, and writes filtered bytes or 16-bit samples directly to the frame planes. Independently encoded active-CDEF 8-bit 4:2:0 and 10/12-bit 4:4:4 streams match every visible native sample produced by pinned scalar libaom with restoration disabled. Independently encoded AVIF containers at the same three bit depths also match pinned scalar-libavif presentation exactly under normal, 256-bit, 128-bit, and scalar color-conversion dispatch. Active super-resolution derives the Appendix A bounded coded width and applies the exact 64-phase, 8-tap horizontal filter with aligned reconstruction-edge input, 8/10/12-bit clipping, and the existing cross-platform `Vector128_.MultiplyAddAdjacent` helper. Independently encoded active-super-resolution AV1 streams at 8, 10, and 12 bits match every visible native sample produced by pinned scalar libaom under normal and forced-scalar dispatch; equivalent AVIF presentation fixtures remain required. Loop restoration follows super-resolution, preserves the required pre-CDEF deblocked context at internal stripes, and applies decoded Wiener or self-guided units from immutable plane snapshots. Independently encoded active-restoration streams at 8, 10, and 12 bits now match every native sample from pinned scalar libaom across AVX2, 128-bit, and scalar dispatch, with the fixture matrix proving both Wiener and self-guided unit selection. The matching AVIF containers also match pinned scalar-libavif presentation exactly, and an independent direct-window definition verifies all sixteen self-guided parameter sets at each supported bit depth across vector and scalar dispatch. Combined 8-bit 4:2:0, 10-bit 4:2:2, and 12-bit 4:4:4 streams additionally verify restoration-unit boundaries after super-resolution, including clipped chroma transform traversal at a coded-frame edge. +- The visible still-image path applies the complete self-contained film-grain parameter set after all in-loop filters. Independently encoded pinned-libaom vectors match every native sample at 8, 10, and 12 bits across monochrome, 4:2:0, 4:2:2, and 4:4:4 layouts, full and restricted ranges, identity-matrix signaling, overlap, and odd 33x11 frame extension. `FeatureTestRunner` verifies normal, AVX-disabled, and fully scalar dispatch. A full-HD-equivalent 4:2:0 benchmark reports zero allocation: 8-bit AVX2 is 2.335 ms versus 5.806 ms scalar, while 12-bit AVX2 is 3.195 ms, cross-platform 128-bit is 7.382 ms, and scalar is 8.614 ms on the measured Ryzen platform. The slower 8-bit 128-bit lookup construction is deliberately not dispatched. - Palette mode now reads the normative luma/chroma mode and size CDFs, neighbor color caches, high-bit-depth color syntax, diagonal color-index maps, and clipped-edge padding. Reconstruction retains libaom's exact indexed-palette contract while using repeated eight-entry tables and native lane-local shuffles across runtime-selected `Vector512`, `Vector256`, and `Vector128` paths before an exact scalar fallback. `FeatureTestRunner` verifies palette sizes 2-8, transform widths 4-64, padded rows, and 8/10/12-bit samples. This path is not production-complete until its syntax and reconstruction are verified with an independently encoded palette AVIF fixture. - Non-reduced still-image sequence parsing now consumes decoder-model operating parameters, temporal presentation fields, and OBU-layer-applicable buffer-removal fields only to preserve AV1 bit alignment. The scheduling values are not retained, and no movie, track, timing, playback, or generic ISO BMFF surface has been introduced. Existing focused sequence-header coverage exercises only reduced-still syntax, so an independent non-reduced still AVIF vector remains required. - Loop-restoration unit parsing records tile-local switchable/Wiener/self-guided filter selections and coefficients in frame-owned plane grids, including super-resolution-adjusted unit corners and the corrected conditional 64x64-superblock unit-size bit. The active restoration stage implements the normative unit geometry, striped deblocked boundaries, Wiener filtering, self-guided projection, and 8/10/12-bit clipping. Self-guided filtering now follows libaom's summed-area-table, coefficient-grid, alternating-row radius-two, full radius-one, and projection stages through AVX2 and cross-platform 128-bit traversals with one exact scalar fallback and caller-owned scratch. Independently encoded fixtures cover active Wiener and self-guided reconstruction and exact AVIF presentation at every supported bit depth. An independent direct-window definition covers all sixteen self-guided parameter sets, narrow and odd processing units, both vector-tail widths, padded strides, and the scalar fallback. Combined active-restoration and super-resolution fixtures cover 4:2:0, 4:2:2, and 4:4:4 at 8, 10, and 12 bits, including restoration-unit boundaries and clipped chroma transform traversal. Other normative independently decodable still-image syntax paths still contain `NotImplementedException` or equivalent unsupported branches. Tile-local palette CDF adaptation is present; the remaining still-image frame-context behavior requires a separate source audit without introducing sequence playback state. @@ -522,7 +523,11 @@ Implement and verify in dependency order: - [x] Verify exact presented reconstruction with independently packaged active-restoration AVIF containers across 8/10/12-bit samples. - [x] Verify all sixteen self-guided parameter sets at 8, 10, and 12 bits against an independent direct-window definition, including narrow, odd, AVX2-tail, 128-bit-tail, and padded-stride processing units through `FeatureTestRunner`. - [x] Extend independent integration coverage across restoration-unit boundary shapes, chroma subsampling layouts, and super-resolution-adjusted unit geometry. -- [ ] Frame padding and film-grain synthesis when signaled. +- [x] Frame padding and film-grain synthesis when signaled. + - [x] Preserve the normative Gaussian sequence, LFSR, autoregressive templates, scaling interpolation, block selection, overlap, clipping, and odd-dimension extension with allocator-owned scratch. + - [x] Apply noise and horizontal overlap through measured SIMD-first semantic operators, with exact scalar fallbacks and scalar serial dependencies where the algorithm prevents useful vectorization. + - [x] Verify exact independent pinned-libaom fixtures across 8/10/12-bit monochrome, 4:2:0, 4:2:2, and 4:4:4 layouts, full and restricted ranges, identity signaling, overlap, and odd dimensions through `FeatureTestRunner`. + - [x] Benchmark full-HD-equivalent 8/12-bit 4:2:0 application across the supported dispatch tiers and verify zero allocation. For each SIMD-suitable item, design the data layout, operator contract, scratch ownership, traversal, and descending-width dispatch around the production SIMD path first. Implement the behaviorally equivalent scalar fallback through the same contract, then verify every supported vector width and the scalar path against normative or independent results before enabling the stage. Serial adaptive work such as entropy decoding remains scalar where data dependencies prevent meaningful vectorization. Remove constant feature-disable flags and unsupported branches only when their replacement is verified. Unsupported syntax must produce a codec-specific invalid-image error; it must never silently skip a normative reconstruction stage. diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainDecoder.cs index 69ae59ae2..1b4dc5d70 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainDecoder.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainDecoder.cs @@ -37,26 +37,6 @@ internal sealed class Av1FilmGrainDecoder /// private const int GaussianIndexBits = 11; - /// - /// The lower restricted-range luma value at eight-bit precision. - /// - private const int RestrictedLumaMinimum = 16; - - /// - /// The upper restricted-range luma value at eight-bit precision. - /// - private const int RestrictedLumaMaximum = 235; - - /// - /// The lower restricted-range chroma value at eight-bit precision. - /// - private const int RestrictedChromaMinimum = 16; - - /// - /// The upper restricted-range chroma value at eight-bit precision. - /// - private const int RestrictedChromaMaximum = 240; - /// /// The sequence-level bit-depth and chroma-sampling configuration. /// @@ -473,7 +453,7 @@ internal sealed class Av1FilmGrainDecoder { // Blend the incoming template columns with the outgoing columns saved by the block on the left. // Writing back to the column buffers produces the exact grain region applied at this boundary. - VerticalOverlap( + Av1FilmGrainOverlap.Vertical( yColumnBuffer, 2, lumaGrain[((lumaOffsetY * lumaBlockWidth) + lumaOffsetX)..], @@ -492,7 +472,7 @@ internal sealed class Av1FilmGrainDecoder chromaSubblockHeight + (2 >> subsamplingY), (height - (halfY << 1)) >> subsamplingY); - VerticalOverlap( + Av1FilmGrainOverlap.Vertical( cbColumnBuffer, chromaOverlapWidth, cbGrain[((chromaOffsetY * chromaBlockWidth) + chromaOffsetX)..], @@ -504,7 +484,7 @@ internal sealed class Av1FilmGrainDecoder grainMinimum, grainMaximum); - VerticalOverlap( + Av1FilmGrainOverlap.Vertical( crColumnBuffer, chromaOverlapWidth, crGrain[((chromaOffsetY * chromaBlockWidth) + chromaOffsetX)..], @@ -542,7 +522,7 @@ internal sealed class Av1FilmGrainDecoder ? Span.Empty : crColumnBuffer[columnGrainOffset..]; - AddNoiseToBlock( + Av1FilmGrainNoise.Apply( parameters, scalingY, scalingCb, @@ -572,7 +552,7 @@ internal sealed class Av1FilmGrainDecoder { // At an interior corner, first combine the saved top boundary with the already blended left // boundary. The resulting corner is then part of the horizontal boundary applied below. - HorizontalOverlap( + Av1FilmGrainOverlap.Horizontal( yLineBuffer[(halfX << 1)..], lumaStride, yColumnBuffer, @@ -589,7 +569,7 @@ internal sealed class Av1FilmGrainDecoder int chromaOverlapWidth = 2 >> subsamplingX; int chromaOverlapHeight = 2 >> subsamplingY; int chromaLineOffset = halfX * chromaOverlapWidth; - HorizontalOverlap( + Av1FilmGrainOverlap.Horizontal( cbLineBuffer[chromaLineOffset..], chromaStride, cbColumnBuffer, @@ -601,7 +581,7 @@ internal sealed class Av1FilmGrainDecoder grainMinimum, grainMaximum); - HorizontalOverlap( + Av1FilmGrainOverlap.Horizontal( crLineBuffer[chromaLineOffset..], chromaStride, crColumnBuffer, @@ -624,7 +604,7 @@ internal sealed class Av1FilmGrainDecoder LumaSubblockSize - templateColumnAdjustment, width - (overlappedColumn << 1)); - HorizontalOverlap( + Av1FilmGrainOverlap.Horizontal( yLineBuffer[(overlappedColumn << 1)..], lumaStride, lumaGrain[((lumaOffsetY * lumaBlockWidth) + lumaOffsetX + templateColumnAdjustment)..], @@ -644,7 +624,7 @@ internal sealed class Av1FilmGrainDecoder chromaSubblockWidth - chromaColumnAdjustment, (width - (overlappedColumn << 1)) >> subsamplingX); - HorizontalOverlap( + Av1FilmGrainOverlap.Horizontal( cbLineBuffer[chromaDestinationOffset..], chromaStride, cbGrain[((chromaOffsetY * chromaBlockWidth) + chromaOffsetX + chromaColumnAdjustment)..], @@ -656,7 +636,7 @@ internal sealed class Av1FilmGrainDecoder grainMinimum, grainMaximum); - HorizontalOverlap( + Av1FilmGrainOverlap.Horizontal( crLineBuffer[chromaDestinationOffset..], chromaStride, crGrain[((chromaOffsetY * chromaBlockWidth) + chromaOffsetX + chromaColumnAdjustment)..], @@ -691,7 +671,7 @@ internal sealed class Av1FilmGrainDecoder // Apply the completed top boundary as a one-unit half-height strip, which is two luma rows and // one or two chroma rows depending on vertical subsampling. - AddNoiseToBlock( + Av1FilmGrainNoise.Apply( parameters, scalingY, scalingCb, @@ -738,7 +718,7 @@ internal sealed class Av1FilmGrainDecoder Span interiorCbGrain = isMonochrome ? Span.Empty : cbGrain[chromaGrainOffset..]; Span interiorCrGrain = isMonochrome ? Span.Empty : crGrain[chromaGrainOffset..]; - AddNoiseToBlock( + Av1FilmGrainNoise.Apply( parameters, scalingY, scalingCb, @@ -1194,315 +1174,6 @@ internal sealed class Av1FilmGrainDecoder lookup[(int)values[pointCount - 1]..].Fill((int)scalings[pointCount - 1]); } - /// - /// Adds a selected luma/chroma grain region to one rectangular frame region. - /// - /// The native eight-bit or high-bit-depth sample type. - /// The frame grain parameters. - /// The luma scaling lookup table. - /// The first chroma scaling lookup table. - /// The second chroma scaling lookup table. - /// The destination luma region. - /// The destination first-chroma region. - /// The destination second-chroma region. - /// The destination luma row stride. - /// The destination chroma row stride. - /// The selected luma grain region. - /// The selected first-chroma grain region. - /// The selected second-chroma grain region. - /// The luma grain row stride. - /// The chroma grain row stride. - /// Half the destination luma height. - /// Half the destination luma width. - /// The decoded sample bit depth. - /// The horizontal chroma subsampling shift. - /// The vertical chroma subsampling shift. - /// Whether chroma processing is absent. - /// Whether chroma planes carry direct RGB components. - private static void AddNoiseToBlock( - ObuFilmGrainParameters parameters, - ReadOnlySpan scalingY, - ReadOnlySpan scalingCb, - ReadOnlySpan scalingCr, - Span luma, - Span cb, - Span cr, - int lumaStride, - int chromaStride, - ReadOnlySpan lumaGrain, - ReadOnlySpan cbGrain, - ReadOnlySpan crGrain, - int lumaGrainStride, - int chromaGrainStride, - int halfLumaHeight, - int halfLumaWidth, - int bitDepth, - int subsamplingX, - int subsamplingY, - bool isMonochrome, - bool isIdentityMatrix) - where TSample : unmanaged - { - // GrainScalingMinus8 stores a shift in the range eight through eleven. The half-unit bias makes the - // signed scaled-grain contribution round before it is added to the restored sample. - int scalingShift = (int)parameters.GrainScalingMinus8 + 8; - int roundingOffset = 1 << (scalingShift - 1); - int depthScale = 1 << (bitDepth - 8); - int sampleMaximum = (256 * depthScale) - 1; - int lumaMinimum = 0; - int lumaMaximum = sampleMaximum; - int chromaMinimum = 0; - int chromaMaximum = sampleMaximum; - if (parameters.ClipToRestrictedRange) - { - // Legal-range constants are specified at eight-bit precision and scale exactly for 10- and 12-bit data. - // Identity matrices carry RGB-like planes, so every plane uses the luma legal range rather than YUV chroma. - lumaMinimum = RestrictedLumaMinimum * depthScale; - lumaMaximum = RestrictedLumaMaximum * depthScale; - chromaMinimum = (isIdentityMatrix ? RestrictedLumaMinimum : RestrictedChromaMinimum) * depthScale; - chromaMaximum = (isIdentityMatrix ? RestrictedLumaMaximum : RestrictedChromaMaximum) * depthScale; - } - - if (!isMonochrome) - { - // Chroma multipliers are biased by 128 in the bitstream, and offsets are biased by 256 after conversion - // to the active bit depth. Restoring those signed values keeps the scaling-index equation entirely integral. - int cbMultiplier = (int)parameters.CbMult - 128; - int cbLumaMultiplier = (int)parameters.CbLumaMult - 128; - int cbOffset = ((int)parameters.CbOffset * depthScale) - (256 * depthScale); - int crMultiplier = (int)parameters.CrMult - 128; - int crLumaMultiplier = (int)parameters.CrLumaMult - 128; - int crOffset = ((int)parameters.CrOffset * depthScale) - (256 * depthScale); - if (parameters.ChromaScalingFromLuma) - { - // A luma-derived chroma function selects the luma coordinate directly: 64 is unity in the Q6 - // multiplier domain, while the chroma sample multiplier and both offsets are forced to zero. - cbMultiplier = 0; - cbLumaMultiplier = 64; - cbOffset = 0; - crMultiplier = 0; - crLumaMultiplier = 64; - crOffset = 0; - } - - bool applyCb = parameters.NumCbPoints != 0 || parameters.ChromaScalingFromLuma; - bool applyCr = parameters.NumCrPoints != 0 || parameters.ChromaScalingFromLuma; - int chromaHeight = halfLumaHeight << (1 - subsamplingY); - int chromaWidth = halfLumaWidth << (1 - subsamplingX); - - // Chroma is processed before luma so its scaling coordinate observes restored luma, not luma after grain. - // This ordering also makes Cb and Cr independent of whether a luma scaling function is present. - for (int row = 0; row < chromaHeight; row++) - { - for (int column = 0; column < chromaWidth; column++) - { - int lumaIndex = ((row << subsamplingY) * lumaStride) + (column << subsamplingX); - int averageLuma = GetSample(luma, lumaIndex); - if (subsamplingX != 0) - { - // Horizontally subsampled chroma is centered over two luma columns. Vertical subsampling - // changes which luma row is selected but does not introduce a second-row average here. - averageLuma = (averageLuma + GetSample(luma, lumaIndex + 1) + 1) >> 1; - } - - int chromaIndex = (row * chromaStride) + column; - int grainIndex = (row * chromaGrainStride) + column; - if (applyCb) - { - int cbSample = GetSample(cb, chromaIndex); - int scalingIndex = Av1Math.Clamp( - (((averageLuma * cbLumaMultiplier) + (cbMultiplier * cbSample)) >> 6) + cbOffset, - 0, - sampleMaximum); - - int grainScale = ScaleLookup(scalingCb, scalingIndex, bitDepth); - SetSample( - cb, - chromaIndex, - Av1Math.Clamp( - cbSample + (((grainScale * cbGrain[grainIndex]) + roundingOffset) >> scalingShift), - chromaMinimum, - chromaMaximum)); - } - - if (applyCr) - { - int crSample = GetSample(cr, chromaIndex); - int scalingIndex = Av1Math.Clamp( - (((averageLuma * crLumaMultiplier) + (crMultiplier * crSample)) >> 6) + crOffset, - 0, - sampleMaximum); - - int grainScale = ScaleLookup(scalingCr, scalingIndex, bitDepth); - SetSample( - cr, - chromaIndex, - Av1Math.Clamp( - crSample + (((grainScale * crGrain[grainIndex]) + roundingOffset) >> scalingShift), - chromaMinimum, - chromaMaximum)); - } - } - } - } - - if (parameters.NumYPoints != 0) - { - // The half-dimension contract expands back to the exact luma rectangle owned by this boundary or interior - // pass. Each sample uses its restored value as the scaling coordinate before grain is added in place. - int lumaHeight = halfLumaHeight << 1; - int lumaWidth = halfLumaWidth << 1; - for (int row = 0; row < lumaHeight; row++) - { - for (int column = 0; column < lumaWidth; column++) - { - int sampleIndex = (row * lumaStride) + column; - int sample = GetSample(luma, sampleIndex); - int grainScale = ScaleLookup(scalingY, sample, bitDepth); - int grainIndex = (row * lumaGrainStride) + column; - SetSample( - luma, - sampleIndex, - Av1Math.Clamp( - sample + (((grainScale * lumaGrain[grainIndex]) + roundingOffset) >> scalingShift), - lumaMinimum, - lumaMaximum)); - } - } - } - } - - /// - /// Reads a bit-depth scaling value, interpolating between eight-bit entries when required. - /// - /// The 256-entry scaling lookup table. - /// The sample-scale lookup coordinate. - /// The decoded sample bit depth. - /// The interpolated grain scaling value. - [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static int ScaleLookup(ReadOnlySpan lookup, int index, int bitDepth) - { - int depthShift = bitDepth - 8; - int lookupIndex = index >> depthShift; - if (depthShift == 0 || lookupIndex == 255) - { - // Eight-bit coordinates address the table directly. The last high-bit-depth interval has no following - // entry, so endpoint extension returns entry 255 without attempting interpolation. - return lookup[lookupIndex]; - } - - // The low depthShift bits are the fractional position between adjacent eight-bit lookup coordinates. - int fraction = index & ((1 << depthShift) - 1); - return lookup[lookupIndex] + - ((((lookup[lookupIndex + 1] - lookup[lookupIndex]) * fraction) + (1 << (depthShift - 1))) >> depthShift); - } - - /// - /// Blends the two grain columns on a vertical block boundary. - /// - /// The saved grain columns from the block on the left. - /// The saved-column row stride. - /// The grain columns selected for the block on the right. - /// The right-block row stride. - /// The overlap destination. - /// The destination row stride. - /// The one- or two-sample overlap width. - /// The overlap height. - /// The minimum grain value. - /// The maximum grain value. - private static void VerticalOverlap( - ReadOnlySpan left, - int leftStride, - ReadOnlySpan right, - int rightStride, - Span destination, - int destinationStride, - int width, - int height, - int minimum, - int maximum) - { - for (int row = 0; row < height; row++) - { - int leftOffset = row * leftStride; - int rightOffset = row * rightStride; - int destinationOffset = row * destinationStride; - if (width == 1) - { - // A subsampled one-column boundary uses the dedicated 23:22 overlap weights. - destination[destinationOffset] = Av1Math.Clamp( - ((left[leftOffset] * 23) + (right[rightOffset] * 22) + 16) >> 5, - minimum, - maximum); - } - else - { - // The two-column kernel biases the outer samples toward their originating block and crosses the - // 27:17 weights for the inner samples. These fixed weights are part of AV1 grain synthesis. - destination[destinationOffset] = Av1Math.Clamp( - ((left[leftOffset] * 27) + (right[rightOffset] * 17) + 16) >> 5, - minimum, - maximum); - - destination[destinationOffset + 1] = Av1Math.Clamp( - ((left[leftOffset + 1] * 17) + (right[rightOffset + 1] * 27) + 16) >> 5, - minimum, - maximum); - } - } - } - - /// - /// Blends the one or two grain rows on a horizontal block boundary. - /// - /// The saved grain rows from the block above. - /// The saved-row stride. - /// The grain rows selected for the block below. - /// The lower-block row stride. - /// The overlap destination. - /// The destination row stride. - /// The overlap width. - /// The one- or two-sample overlap height. - /// The minimum grain value. - /// The maximum grain value. - private static void HorizontalOverlap( - ReadOnlySpan top, - int topStride, - ReadOnlySpan bottom, - int bottomStride, - Span destination, - int destinationStride, - int width, - int height, - int minimum, - int maximum) - { - for (int column = 0; column < width; column++) - { - if (height == 1) - { - // Vertically subsampled chroma collapses the overlap to the single-row 23:22 kernel. - destination[column] = Av1Math.Clamp( - ((top[column] * 23) + (bottom[column] * 22) + 16) >> 5, - minimum, - maximum); - } - else - { - // Luma and full-height chroma use the crossed two-row 27:17 overlap kernel. - destination[column] = Av1Math.Clamp( - ((top[column] * 27) + (bottom[column] * 17) + 16) >> 5, - minimum, - maximum); - - destination[destinationStride + column] = Av1Math.Clamp( - ((top[topStride + column] * 17) + (bottom[bottomStride + column] * 27) + 16) >> 5, - minimum, - maximum); - } - } - } - /// /// Copies a rectangular grain region while preserving independent source and destination strides. /// @@ -1561,47 +1232,4 @@ internal sealed class Av1FilmGrainDecoder randomRegister = (ushort)((randomRegister >> 1) | (feedback << 15)); return (randomRegister >> (16 - bitCount)) & ((1 << bitCount) - 1); } - - /// - /// Reads an eight-bit or high-bit-depth sample without a format-conversion buffer. - /// - /// The native sample type. - /// The sample span. - /// The sample index. - /// The unsigned sample value. - [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static int GetSample(Span samples, int index) - where TSample : unmanaged - { - ref TSample sample = ref samples[index]; - - // Callers close TSample over byte or ushort, so the JIT removes this branch and emits a native unsigned load. - return typeof(TSample) == typeof(byte) - ? Unsafe.As(ref sample) - : Unsafe.As(ref sample); - } - - /// - /// Writes an eight-bit or high-bit-depth sample without a format-conversion buffer. - /// - /// The native sample type. - /// The sample span. - /// The sample index. - /// The already clipped unsigned value. - [MethodImpl(MethodImplOptions.AggressiveInlining)] - private static void SetSample(Span samples, int index, int value) - where TSample : unmanaged - { - // As in GetSample, the closed generic type leaves only the matching native store in generated code. - if (typeof(TSample) == typeof(byte)) - { - byte byteValue = (byte)value; - samples[index] = Unsafe.As(ref byteValue); - } - else - { - ushort ushortValue = (ushort)value; - samples[index] = Unsafe.As(ref ushortValue); - } - } } diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainGaussianSequence.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainGaussianSequence.cs index a8d1df485..509d0b78f 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainGaussianSequence.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainGaussianSequence.cs @@ -9,9 +9,9 @@ namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.FilmGrain; internal static class Av1FilmGrainGaussianSequence { /// - /// The signed 12-bit Gaussian samples indexed by the film-grain pseudo-random generator. + /// Gets the signed 12-bit Gaussian samples in their normative index order. /// - private static readonly short[] Values = + public static ReadOnlySpan Samples => [ 56, 568, -180, 172, 124, -84, 172, -64, -900, 24, 820, 224, 1248, 996, 272, -8, -916, -388, -732, -104, -188, 800, 112, -652, -320, -376, 140, -252, 492, -168, 44, -788, @@ -142,9 +142,4 @@ internal static class Av1FilmGrainGaussianSequence 364, 100, -744, -1056, -32, 540, 280, 144, -676, -32, -232, -280, -224, 96, 568, -76, 172, 148, 148, 104, 32, -296, -32, 788, -80, 32, -16, 280, 288, 944, 428, -484, ]; - - /// - /// Gets the Gaussian samples in their normative index order. - /// - public static ReadOnlySpan Samples => Values; } diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainNoise.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainNoise.cs new file mode 100644 index 000000000..73668f5ec --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainNoise.cs @@ -0,0 +1,985 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.FilmGrain; + +/// +/// Applies selected AV1 grain blocks to restored luma and chroma samples. +/// +internal static class Av1FilmGrainNoise +{ + /// + /// The lower restricted-range luma value at eight-bit precision. + /// + private const int RestrictedLumaMinimum = 16; + + /// + /// The upper restricted-range luma value at eight-bit precision. + /// + private const int RestrictedLumaMaximum = 235; + + /// + /// The lower restricted-range chroma value at eight-bit precision. + /// + private const int RestrictedChromaMinimum = 16; + + /// + /// The upper restricted-range chroma value at eight-bit precision. + /// + private const int RestrictedChromaMaximum = 240; + + /// + /// Adds a selected grain rectangle to its corresponding restored samples. + /// + /// The native sample type. + /// The complete frame grain parameters. + /// The luma scaling lookup table. + /// The first chroma scaling lookup table. + /// The second chroma scaling lookup table. + /// The restored luma rectangle. + /// The restored first chroma rectangle. + /// The restored second chroma rectangle. + /// The luma row stride in samples. + /// The chroma row stride in samples. + /// The selected luma grain rectangle. + /// The selected first chroma grain rectangle. + /// The selected second chroma grain rectangle. + /// The luma grain row stride. + /// The chroma grain row stride. + /// Half the luma rectangle height. + /// Half the luma rectangle width. + /// The decoded sample bit depth. + /// The horizontal chroma subsampling shift. + /// The vertical chroma subsampling shift. + /// Whether the frame has no chroma planes. + /// Whether every plane uses the luma restricted range. + public static void Apply( + ObuFilmGrainParameters parameters, + ReadOnlySpan scalingY, + ReadOnlySpan scalingCb, + ReadOnlySpan scalingCr, + Span luma, + Span cb, + Span cr, + int lumaStride, + int chromaStride, + ReadOnlySpan lumaGrain, + ReadOnlySpan cbGrain, + ReadOnlySpan crGrain, + int lumaGrainStride, + int chromaGrainStride, + int halfLumaHeight, + int halfLumaWidth, + int bitDepth, + int subsamplingX, + int subsamplingY, + bool isMonochrome, + bool isIdentityMatrix) + where TSample : unmanaged + { + int scalingShift = (int)parameters.GrainScalingMinus8 + 8; + int roundingOffset = 1 << (scalingShift - 1); + int depthScale = 1 << (bitDepth - 8); + int sampleMaximum = (256 * depthScale) - 1; + int lumaMinimum = 0; + int lumaMaximum = sampleMaximum; + int chromaMinimum = 0; + int chromaMaximum = sampleMaximum; + if (parameters.ClipToRestrictedRange) + { + // Restricted-range endpoints are signaled at eight-bit precision and scale exactly at higher depths. + lumaMinimum = RestrictedLumaMinimum * depthScale; + lumaMaximum = RestrictedLumaMaximum * depthScale; + chromaMinimum = (isIdentityMatrix ? RestrictedLumaMinimum : RestrictedChromaMinimum) * depthScale; + chromaMaximum = (isIdentityMatrix ? RestrictedLumaMaximum : RestrictedChromaMaximum) * depthScale; + } + + if (!isMonochrome) + { + int cbMultiplier = (int)parameters.CbMult - 128; + int cbLumaMultiplier = (int)parameters.CbLumaMult - 128; + int cbOffset = ((int)parameters.CbOffset * depthScale) - (256 * depthScale); + int crMultiplier = (int)parameters.CrMult - 128; + int crLumaMultiplier = (int)parameters.CrLumaMult - 128; + int crOffset = ((int)parameters.CrOffset * depthScale) - (256 * depthScale); + if (parameters.ChromaScalingFromLuma) + { + // Unity in the Q6 luma-multiplier domain selects luma directly and removes the chroma contribution. + cbMultiplier = 0; + cbLumaMultiplier = 64; + cbOffset = 0; + crMultiplier = 0; + crLumaMultiplier = 64; + crOffset = 0; + } + + ApplyChroma( + scalingCb, + scalingCr, + luma, + cb, + cr, + lumaStride, + chromaStride, + cbGrain, + crGrain, + chromaGrainStride, + halfLumaHeight << (1 - subsamplingY), + halfLumaWidth << (1 - subsamplingX), + bitDepth, + subsamplingX, + subsamplingY, + parameters.NumCbPoints != 0 || parameters.ChromaScalingFromLuma, + parameters.NumCrPoints != 0 || parameters.ChromaScalingFromLuma, + cbMultiplier, + cbLumaMultiplier, + cbOffset, + crMultiplier, + crLumaMultiplier, + crOffset, + roundingOffset, + scalingShift, + sampleMaximum, + chromaMinimum, + chromaMaximum); + } + + if (parameters.NumYPoints != 0) + { + ApplyLuma( + scalingY, + luma, + lumaStride, + lumaGrain, + lumaGrainStride, + halfLumaHeight << 1, + halfLumaWidth << 1, + bitDepth, + roundingOffset, + scalingShift, + lumaMinimum, + lumaMaximum); + } + } + + /// + /// Selects the widest available luma traversal. + /// + private static void ApplyLuma( + ReadOnlySpan scaling, + Span samples, + int sampleStride, + ReadOnlySpan grain, + int grainStride, + int height, + int width, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum) + where TSample : unmanaged + { + if (Avx2.IsSupported) + { + ApplyLuma( + scaling, + samples, + sampleStride, + grain, + grainStride, + height, + width, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum, + Vector256.Zero); + + return; + } + + if (CanVectorizeWithoutGather(bitDepth)) + { + ApplyLuma( + scaling, + samples, + sampleStride, + grain, + grainStride, + height, + width, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum, + Vector128.Zero); + + return; + } + + ApplyLumaScalar(scaling, samples, sampleStride, grain, grainStride, height, width, bitDepth, roundingOffset, scalingShift, minimum, maximum); + } + + /// + /// Applies luma grain eight samples at a time. + /// + private static void ApplyLuma( + ReadOnlySpan scaling, + Span samples, + int sampleStride, + ReadOnlySpan grain, + int grainStride, + int height, + int width, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum, + Vector256 vector) + where TSample : unmanaged + { + ref TSample sampleBase = ref MemoryMarshal.GetReference(samples); + ref int grainBase = ref MemoryMarshal.GetReference(grain); + for (int row = 0; row < height; row++) + { + int sampleRowOffset = row * sampleStride; + int grainRowOffset = row * grainStride; + int column = 0; + int vectorEnd = width - Vector256.Count; + for (; column <= vectorEnd; column += Vector256.Count) + { + ref TSample destination = ref Unsafe.Add(ref sampleBase, sampleRowOffset + column); + Vector256 source = Av1FilmGrainSampleOperator.Load8(ref destination); + Vector256 grainValues = Vector256.LoadUnsafe(ref grainBase, (nuint)(grainRowOffset + column)); + Vector256 result = AddNoise(source, grainValues, scaling, bitDepth, roundingOffset, scalingShift, minimum, maximum); + Av1FilmGrainSampleOperator.Store8(ref destination, result); + } + + ApplyLumaScalar( + scaling, + samples.Slice(sampleRowOffset + column), + sampleStride, + grain.Slice(grainRowOffset + column), + grainStride, + 1, + width - column, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum); + } + } + + /// + /// Applies luma grain four samples at a time. + /// + private static void ApplyLuma( + ReadOnlySpan scaling, + Span samples, + int sampleStride, + ReadOnlySpan grain, + int grainStride, + int height, + int width, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum, + Vector128 vector) + where TSample : unmanaged + { + ref TSample sampleBase = ref MemoryMarshal.GetReference(samples); + ref int grainBase = ref MemoryMarshal.GetReference(grain); + for (int row = 0; row < height; row++) + { + int sampleRowOffset = row * sampleStride; + int grainRowOffset = row * grainStride; + int column = 0; + int vectorEnd = width - Vector128.Count; + for (; column <= vectorEnd; column += Vector128.Count) + { + ref TSample destination = ref Unsafe.Add(ref sampleBase, sampleRowOffset + column); + Vector128 source = Av1FilmGrainSampleOperator.Load4(ref destination); + Vector128 grainValues = Vector128.LoadUnsafe(ref grainBase, (nuint)(grainRowOffset + column)); + Vector128 result = AddNoise(source, grainValues, scaling, bitDepth, roundingOffset, scalingShift, minimum, maximum); + Av1FilmGrainSampleOperator.Store4(ref destination, result); + } + + ApplyLumaScalar( + scaling, + samples.Slice(sampleRowOffset + column), + sampleStride, + grain.Slice(grainRowOffset + column), + grainStride, + 1, + width - column, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum); + } + } + + /// + /// Applies the luma scalar remainder or complete scalar fallback. + /// + private static void ApplyLumaScalar( + ReadOnlySpan scaling, + Span samples, + int sampleStride, + ReadOnlySpan grain, + int grainStride, + int height, + int width, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum) + where TSample : unmanaged + { + ref TSample sampleBase = ref MemoryMarshal.GetReference(samples); + ref int grainBase = ref MemoryMarshal.GetReference(grain); + for (int row = 0; row < height; row++) + { + int sampleRowOffset = row * sampleStride; + int grainRowOffset = row * grainStride; + for (int column = 0; column < width; column++) + { + ref TSample destination = ref Unsafe.Add(ref sampleBase, sampleRowOffset + column); + int source = Av1FilmGrainSampleOperator.Load(ref destination); + int scale = ScaleLookup(scaling, source, bitDepth); + int value = source + (((scale * Unsafe.Add(ref grainBase, grainRowOffset + column)) + roundingOffset) >> scalingShift); + Av1FilmGrainSampleOperator.Store(ref destination, Av1Math.Clamp(value, minimum, maximum)); + } + } + } + + /// + /// Selects the widest available chroma traversal. + /// + private static void ApplyChroma( + ReadOnlySpan scalingCb, + ReadOnlySpan scalingCr, + Span luma, + Span cb, + Span cr, + int lumaStride, + int chromaStride, + ReadOnlySpan cbGrain, + ReadOnlySpan crGrain, + int grainStride, + int height, + int width, + int bitDepth, + int subsamplingX, + int subsamplingY, + bool applyCb, + bool applyCr, + int cbMultiplier, + int cbLumaMultiplier, + int cbOffset, + int crMultiplier, + int crLumaMultiplier, + int crOffset, + int roundingOffset, + int scalingShift, + int sampleMaximum, + int minimum, + int maximum) + where TSample : unmanaged + { + if (Avx2.IsSupported) + { + ApplyChroma( + scalingCb, + scalingCr, + luma, + cb, + cr, + lumaStride, + chromaStride, + cbGrain, + crGrain, + grainStride, + height, + width, + bitDepth, + subsamplingX, + subsamplingY, + applyCb, + applyCr, + cbMultiplier, + cbLumaMultiplier, + cbOffset, + crMultiplier, + crLumaMultiplier, + crOffset, + roundingOffset, + scalingShift, + sampleMaximum, + minimum, + maximum, + Vector256.Zero); + + return; + } + + if (CanVectorizeWithoutGather(bitDepth)) + { + ApplyChroma( + scalingCb, + scalingCr, + luma, + cb, + cr, + lumaStride, + chromaStride, + cbGrain, + crGrain, + grainStride, + height, + width, + bitDepth, + subsamplingX, + subsamplingY, + applyCb, + applyCr, + cbMultiplier, + cbLumaMultiplier, + cbOffset, + crMultiplier, + crLumaMultiplier, + crOffset, + roundingOffset, + scalingShift, + sampleMaximum, + minimum, + maximum, + Vector128.Zero); + + return; + } + + ApplyChromaScalar( + scalingCb, + scalingCr, + luma, + cb, + cr, + lumaStride, + chromaStride, + cbGrain, + crGrain, + grainStride, + height, + width, + bitDepth, + subsamplingX, + subsamplingY, + applyCb, + applyCr, + cbMultiplier, + cbLumaMultiplier, + cbOffset, + crMultiplier, + crLumaMultiplier, + crOffset, + roundingOffset, + scalingShift, + sampleMaximum, + minimum, + maximum); + } + + /// + /// Applies chroma grain eight samples at a time. + /// + private static void ApplyChroma( + ReadOnlySpan scalingCb, + ReadOnlySpan scalingCr, + Span luma, + Span cb, + Span cr, + int lumaStride, + int chromaStride, + ReadOnlySpan cbGrain, + ReadOnlySpan crGrain, + int grainStride, + int height, + int width, + int bitDepth, + int subsamplingX, + int subsamplingY, + bool applyCb, + bool applyCr, + int cbMultiplier, + int cbLumaMultiplier, + int cbOffset, + int crMultiplier, + int crLumaMultiplier, + int crOffset, + int roundingOffset, + int scalingShift, + int sampleMaximum, + int minimum, + int maximum, + Vector256 vector) + where TSample : unmanaged + { + ref TSample lumaBase = ref MemoryMarshal.GetReference(luma); + ref TSample cbBase = ref MemoryMarshal.GetReference(cb); + ref TSample crBase = ref MemoryMarshal.GetReference(cr); + ref int cbGrainBase = ref MemoryMarshal.GetReference(cbGrain); + ref int crGrainBase = ref MemoryMarshal.GetReference(crGrain); + Vector256 zero = Vector256.Zero; + Vector256 maximumIndex = Vector256.Create(sampleMaximum); + for (int row = 0; row < height; row++) + { + ref TSample lumaRow = ref Unsafe.Add(ref lumaBase, (row << subsamplingY) * lumaStride); + int chromaRowOffset = row * chromaStride; + int grainRowOffset = row * grainStride; + int column = 0; + int vectorEnd = width - Vector256.Count; + for (; column <= vectorEnd; column += Vector256.Count) + { + ref TSample lumaSource = ref Unsafe.Add(ref lumaRow, column << subsamplingX); + Vector256 averageLuma = Av1FilmGrainSampleOperator.LoadChromaLuma8(ref lumaSource, subsamplingX); + if (applyCb) + { + ref TSample destination = ref Unsafe.Add(ref cbBase, chromaRowOffset + column); + Vector256 source = Av1FilmGrainSampleOperator.Load8(ref destination); + Vector256 scalingIndex = ((averageLuma * cbLumaMultiplier) + (source * cbMultiplier)) >> 6; + scalingIndex = Vector256.Min(Vector256.Max(scalingIndex + Vector256.Create(cbOffset), zero), maximumIndex); + Vector256 grainValues = Vector256.LoadUnsafe(ref cbGrainBase, (nuint)(grainRowOffset + column)); + Vector256 result = AddNoise( + source, + grainValues, + scalingCb, + scalingIndex, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum); + + Av1FilmGrainSampleOperator.Store8(ref destination, result); + } + + if (applyCr) + { + ref TSample destination = ref Unsafe.Add(ref crBase, chromaRowOffset + column); + Vector256 source = Av1FilmGrainSampleOperator.Load8(ref destination); + Vector256 scalingIndex = ((averageLuma * crLumaMultiplier) + (source * crMultiplier)) >> 6; + scalingIndex = Vector256.Min(Vector256.Max(scalingIndex + Vector256.Create(crOffset), zero), maximumIndex); + Vector256 grainValues = Vector256.LoadUnsafe(ref crGrainBase, (nuint)(grainRowOffset + column)); + Vector256 result = AddNoise( + source, + grainValues, + scalingCr, + scalingIndex, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum); + + Av1FilmGrainSampleOperator.Store8(ref destination, result); + } + } + + ApplyChromaScalar( + scalingCb, + scalingCr, + luma.Slice(((row << subsamplingY) * lumaStride) + (column << subsamplingX)), + cb.Slice(chromaRowOffset + column), + cr.Slice(chromaRowOffset + column), + lumaStride, + chromaStride, + cbGrain.Slice(grainRowOffset + column), + crGrain.Slice(grainRowOffset + column), + grainStride, + 1, + width - column, + bitDepth, + subsamplingX, + subsamplingY, + applyCb, + applyCr, + cbMultiplier, + cbLumaMultiplier, + cbOffset, + crMultiplier, + crLumaMultiplier, + crOffset, + roundingOffset, + scalingShift, + sampleMaximum, + minimum, + maximum); + } + } + + /// + /// Applies chroma grain four samples at a time. + /// + private static void ApplyChroma( + ReadOnlySpan scalingCb, + ReadOnlySpan scalingCr, + Span luma, + Span cb, + Span cr, + int lumaStride, + int chromaStride, + ReadOnlySpan cbGrain, + ReadOnlySpan crGrain, + int grainStride, + int height, + int width, + int bitDepth, + int subsamplingX, + int subsamplingY, + bool applyCb, + bool applyCr, + int cbMultiplier, + int cbLumaMultiplier, + int cbOffset, + int crMultiplier, + int crLumaMultiplier, + int crOffset, + int roundingOffset, + int scalingShift, + int sampleMaximum, + int minimum, + int maximum, + Vector128 vector) + where TSample : unmanaged + { + ref TSample lumaBase = ref MemoryMarshal.GetReference(luma); + ref TSample cbBase = ref MemoryMarshal.GetReference(cb); + ref TSample crBase = ref MemoryMarshal.GetReference(cr); + ref int cbGrainBase = ref MemoryMarshal.GetReference(cbGrain); + ref int crGrainBase = ref MemoryMarshal.GetReference(crGrain); + Vector128 zero = Vector128.Zero; + Vector128 maximumIndex = Vector128.Create(sampleMaximum); + for (int row = 0; row < height; row++) + { + ref TSample lumaRow = ref Unsafe.Add(ref lumaBase, (row << subsamplingY) * lumaStride); + int chromaRowOffset = row * chromaStride; + int grainRowOffset = row * grainStride; + int column = 0; + int vectorEnd = width - Vector128.Count; + for (; column <= vectorEnd; column += Vector128.Count) + { + ref TSample lumaSource = ref Unsafe.Add(ref lumaRow, column << subsamplingX); + Vector128 averageLuma = Av1FilmGrainSampleOperator.LoadChromaLuma4(ref lumaSource, subsamplingX); + if (applyCb) + { + ref TSample destination = ref Unsafe.Add(ref cbBase, chromaRowOffset + column); + Vector128 source = Av1FilmGrainSampleOperator.Load4(ref destination); + Vector128 scalingIndex = ((averageLuma * cbLumaMultiplier) + (source * cbMultiplier)) >> 6; + scalingIndex = Vector128.Min(Vector128.Max(scalingIndex + Vector128.Create(cbOffset), zero), maximumIndex); + Vector128 grainValues = Vector128.LoadUnsafe(ref cbGrainBase, (nuint)(grainRowOffset + column)); + Vector128 result = AddNoise( + source, + grainValues, + scalingCb, + scalingIndex, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum); + + Av1FilmGrainSampleOperator.Store4(ref destination, result); + } + + if (applyCr) + { + ref TSample destination = ref Unsafe.Add(ref crBase, chromaRowOffset + column); + Vector128 source = Av1FilmGrainSampleOperator.Load4(ref destination); + Vector128 scalingIndex = ((averageLuma * crLumaMultiplier) + (source * crMultiplier)) >> 6; + scalingIndex = Vector128.Min(Vector128.Max(scalingIndex + Vector128.Create(crOffset), zero), maximumIndex); + Vector128 grainValues = Vector128.LoadUnsafe(ref crGrainBase, (nuint)(grainRowOffset + column)); + Vector128 result = AddNoise( + source, + grainValues, + scalingCr, + scalingIndex, + bitDepth, + roundingOffset, + scalingShift, + minimum, + maximum); + + Av1FilmGrainSampleOperator.Store4(ref destination, result); + } + } + + ApplyChromaScalar( + scalingCb, + scalingCr, + luma.Slice(((row << subsamplingY) * lumaStride) + (column << subsamplingX)), + cb.Slice(chromaRowOffset + column), + cr.Slice(chromaRowOffset + column), + lumaStride, + chromaStride, + cbGrain.Slice(grainRowOffset + column), + crGrain.Slice(grainRowOffset + column), + grainStride, + 1, + width - column, + bitDepth, + subsamplingX, + subsamplingY, + applyCb, + applyCr, + cbMultiplier, + cbLumaMultiplier, + cbOffset, + crMultiplier, + crLumaMultiplier, + crOffset, + roundingOffset, + scalingShift, + sampleMaximum, + minimum, + maximum); + } + } + + /// + /// Applies the chroma scalar remainder or complete scalar fallback. + /// + private static void ApplyChromaScalar( + ReadOnlySpan scalingCb, + ReadOnlySpan scalingCr, + Span luma, + Span cb, + Span cr, + int lumaStride, + int chromaStride, + ReadOnlySpan cbGrain, + ReadOnlySpan crGrain, + int grainStride, + int height, + int width, + int bitDepth, + int subsamplingX, + int subsamplingY, + bool applyCb, + bool applyCr, + int cbMultiplier, + int cbLumaMultiplier, + int cbOffset, + int crMultiplier, + int crLumaMultiplier, + int crOffset, + int roundingOffset, + int scalingShift, + int sampleMaximum, + int minimum, + int maximum) + where TSample : unmanaged + { + ref TSample lumaBase = ref MemoryMarshal.GetReference(luma); + ref TSample cbBase = ref MemoryMarshal.GetReference(cb); + ref TSample crBase = ref MemoryMarshal.GetReference(cr); + ref int cbGrainBase = ref MemoryMarshal.GetReference(cbGrain); + ref int crGrainBase = ref MemoryMarshal.GetReference(crGrain); + for (int row = 0; row < height; row++) + { + int lumaRowOffset = (row << subsamplingY) * lumaStride; + int chromaRowOffset = row * chromaStride; + int grainRowOffset = row * grainStride; + for (int column = 0; column < width; column++) + { + int lumaOffset = lumaRowOffset + (column << subsamplingX); + int averageLuma = Av1FilmGrainSampleOperator.Load(ref Unsafe.Add(ref lumaBase, lumaOffset)); + if (subsamplingX != 0) + { + averageLuma = (averageLuma + Av1FilmGrainSampleOperator.Load(ref Unsafe.Add(ref lumaBase, lumaOffset + 1)) + 1) >> 1; + } + + int chromaOffset = chromaRowOffset + column; + int grainOffset = grainRowOffset + column; + if (applyCb) + { + ref TSample destination = ref Unsafe.Add(ref cbBase, chromaOffset); + int source = Av1FilmGrainSampleOperator.Load(ref destination); + int scalingIndex = Av1Math.Clamp( + (((averageLuma * cbLumaMultiplier) + (source * cbMultiplier)) >> 6) + cbOffset, + 0, + sampleMaximum); + + int scale = ScaleLookup(scalingCb, scalingIndex, bitDepth); + int value = source + (((scale * Unsafe.Add(ref cbGrainBase, grainOffset)) + roundingOffset) >> scalingShift); + Av1FilmGrainSampleOperator.Store(ref destination, Av1Math.Clamp(value, minimum, maximum)); + } + + if (applyCr) + { + ref TSample destination = ref Unsafe.Add(ref crBase, chromaOffset); + int source = Av1FilmGrainSampleOperator.Load(ref destination); + int scalingIndex = Av1Math.Clamp( + (((averageLuma * crLumaMultiplier) + (source * crMultiplier)) >> 6) + crOffset, + 0, + sampleMaximum); + + int scale = ScaleLookup(scalingCr, scalingIndex, bitDepth); + int value = source + (((scale * Unsafe.Add(ref crGrainBase, grainOffset)) + roundingOffset) >> scalingShift); + Av1FilmGrainSampleOperator.Store(ref destination, Av1Math.Clamp(value, minimum, maximum)); + } + } + } + } + + /// + /// Adds scaled grain to eight source samples and clips the result. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 AddNoise( + Vector256 source, + Vector256 grain, + ReadOnlySpan scaling, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum) + => AddNoise(source, grain, scaling, source, bitDepth, roundingOffset, scalingShift, minimum, maximum); + + /// + /// Adds scaled grain to eight source samples using independent scaling coordinates. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector256 AddNoise( + Vector256 source, + Vector256 grain, + ReadOnlySpan scaling, + Vector256 scalingIndex, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum) + { + Vector256 scale = ScaleLookup(scaling, scalingIndex, bitDepth); + Vector256 result = source + (((scale * grain) + Vector256.Create(roundingOffset)) >> scalingShift); + return Vector256.Min(Vector256.Max(result, Vector256.Create(minimum)), Vector256.Create(maximum)); + } + + /// + /// Adds scaled grain to four source samples and clips the result. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 AddNoise( + Vector128 source, + Vector128 grain, + ReadOnlySpan scaling, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum) + => AddNoise(source, grain, scaling, source, bitDepth, roundingOffset, scalingShift, minimum, maximum); + + /// + /// Adds scaled grain to four source samples using independent scaling coordinates. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 AddNoise( + Vector128 source, + Vector128 grain, + ReadOnlySpan scaling, + Vector128 scalingIndex, + int bitDepth, + int roundingOffset, + int scalingShift, + int minimum, + int maximum) + { + Vector128 scale = ScaleLookup(scaling, scalingIndex, bitDepth); + Vector128 result = source + (((scale * grain) + Vector128.Create(roundingOffset)) >> scalingShift); + return Vector128.Min(Vector128.Max(result, Vector128.Create(minimum)), Vector128.Create(maximum)); + } + + /// + /// Determines whether portable vector arithmetic repays the cost of scalar scaling-table reads. + /// + /// The decoded sample bit depth. + /// Whether to use the portable vector traversal. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static bool CanVectorizeWithoutGather(int bitDepth) + { + // Portable Vector128 has no indexed table load. At eight bits, assembling each scaling vector from four + // scalar reads is slower than the complete scalar loop; high-depth interpolation contains enough arithmetic + // to amortize those reads. The AVX2 path uses native gather and remains the primary traversal at every depth. + return bitDepth > 8 && Vector128.IsHardwareAccelerated; + } + + /// + /// Gathers eight scaling values and interpolates high-bit-depth coordinates. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static unsafe Vector256 ScaleLookup(ReadOnlySpan scaling, Vector256 index, int bitDepth) + { + int depthShift = bitDepth - 8; + Vector256 tableIndex = index >> depthShift; + fixed (int* table = scaling) + { + Vector256 current = Avx2.GatherVector256(table, tableIndex, sizeof(int)); + if (depthShift == 0) + { + return current; + } + + // Clamping the following index extends entry 255 across the final interpolation interval. + Vector256 nextIndex = Vector256.Min(tableIndex + Vector256.One, Vector256.Create(255)); + Vector256 next = Avx2.GatherVector256(table, nextIndex, sizeof(int)); + Vector256 fraction = index & Vector256.Create((1 << depthShift) - 1); + return current + ((((next - current) * fraction) + Vector256.Create(1 << (depthShift - 1))) >> depthShift); + } + } + + /// + /// Reads four scaling values and interpolates high-bit-depth coordinates. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static Vector128 ScaleLookup(ReadOnlySpan scaling, Vector128 index, int bitDepth) + => Vector128.Create( + ScaleLookup(scaling, index.GetElement(0), bitDepth), + ScaleLookup(scaling, index.GetElement(1), bitDepth), + ScaleLookup(scaling, index.GetElement(2), bitDepth), + ScaleLookup(scaling, index.GetElement(3), bitDepth)); + + /// + /// Reads one scaling value, interpolating between eight-bit entries when required. + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int ScaleLookup(ReadOnlySpan scaling, int index, int bitDepth) + { + int depthShift = bitDepth - 8; + int tableIndex = index >> depthShift; + if (depthShift == 0 || tableIndex == 255) + { + return scaling[tableIndex]; + } + + int fraction = index & ((1 << depthShift) - 1); + return scaling[tableIndex] + ((((scaling[tableIndex + 1] - scaling[tableIndex]) * fraction) + (1 << (depthShift - 1))) >> depthShift); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainOverlap.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainOverlap.cs new file mode 100644 index 000000000..8e18f43d6 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainOverlap.cs @@ -0,0 +1,313 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Numerics; +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.FilmGrain; + +/// +/// Blends grain samples across adjacent synthesis blocks. +/// +internal static class Av1FilmGrainOverlap +{ + /// + /// Blends the two grain columns on a vertical block boundary. + /// + /// The saved grain columns from the block on the left. + /// The saved-column row stride. + /// The grain columns selected for the block on the right. + /// The right-block row stride. + /// The overlap destination. + /// The destination row stride. + /// The one- or two-sample overlap width. + /// The overlap height. + /// The minimum grain value. + /// The maximum grain value. + public static void Vertical( + ReadOnlySpan left, + int leftStride, + ReadOnlySpan right, + int rightStride, + Span destination, + int destinationStride, + int width, + int height, + int minimum, + int maximum) + { + // Each row contributes only one or two strided samples. Gather plus scalar scatter would do more work than + // the fixed scalar kernel, while the horizontally contiguous boundary below benefits directly from SIMD. + if (width == 1) + { + for (int row = 0; row < height; row++) + { + int leftOffset = row * leftStride; + int rightOffset = row * rightStride; + int destinationOffset = row * destinationStride; + + // A subsampled one-column boundary uses the dedicated 23:22 overlap weights. + destination[destinationOffset] = Av1Math.Clamp( + ((left[leftOffset] * 23) + (right[rightOffset] * 22) + 16) >> 5, + minimum, + maximum); + } + + return; + } + + for (int row = 0; row < height; row++) + { + int leftOffset = row * leftStride; + int rightOffset = row * rightStride; + int destinationOffset = row * destinationStride; + + // The two-column kernel biases the outer samples toward their originating block and crosses the 27:17 + // weights for the inner samples. These fixed weights are part of AV1 grain synthesis. + destination[destinationOffset] = Av1Math.Clamp( + ((left[leftOffset] * 27) + (right[rightOffset] * 17) + 16) >> 5, + minimum, + maximum); + + destination[destinationOffset + 1] = Av1Math.Clamp( + ((left[leftOffset + 1] * 17) + (right[rightOffset + 1] * 27) + 16) >> 5, + minimum, + maximum); + } + } + + /// + /// Blends the one or two grain rows on a horizontal block boundary. + /// + /// The saved grain rows from the block above. + /// The saved-row stride. + /// The grain rows selected for the block below. + /// The lower-block row stride. + /// The overlap destination. + /// The destination row stride. + /// The overlap width. + /// The one- or two-sample overlap height. + /// The minimum grain value. + /// The maximum grain value. + public static void Horizontal( + ReadOnlySpan top, + int topStride, + ReadOnlySpan bottom, + int bottomStride, + Span destination, + int destinationStride, + int width, + int height, + int minimum, + int maximum) + { + if (height == 1) + { + // Vertically subsampled chroma collapses the overlap to the single-row 23:22 kernel. + BlendRow(top, bottom, destination, width, 23, 22, minimum, maximum); + return; + } + + // Luma and full-height chroma use the crossed two-row 27:17 overlap kernel. + BlendRow(top, bottom, destination, width, 27, 17, minimum, maximum); + BlendRow( + top[topStride..], + bottom[bottomStride..], + destination[destinationStride..], + width, + 17, + 27, + minimum, + maximum); + } + + /// + /// Blends one contiguous overlap row using the widest useful vector width and a scalar remainder. + /// + /// The samples from the preceding block. + /// The samples from the following block. + /// The blended samples. + /// The number of samples to blend. + /// The preceding-block weight. + /// The following-block weight. + /// The minimum grain value. + /// The maximum grain value. + private static void BlendRow( + ReadOnlySpan left, + ReadOnlySpan right, + Span destination, + int width, + int leftWeight, + int rightWeight, + int minimum, + int maximum) + { + int column = 0; + + // Vector exposes the runtime's preferred native width. This avoids selecting split 512-bit operations on + // machines whose execution resources are 256 bits wide while retaining a native 512-bit traversal elsewhere. + if (Vector512.IsHardwareAccelerated && Vector.Count == Vector512.Count) + { + column = Blend(left, right, destination, width, column, leftWeight, rightWeight, minimum, maximum, Vector512.Zero); + } + + if (Vector256.IsHardwareAccelerated) + { + column = Blend(left, right, destination, width, column, leftWeight, rightWeight, minimum, maximum, Vector256.Zero); + } + + if (Vector128.IsHardwareAccelerated) + { + column = Blend(left, right, destination, width, column, leftWeight, rightWeight, minimum, maximum, Vector128.Zero); + } + + for (; column < width; column++) + { + int value = ((left[column] * leftWeight) + (right[column] * rightWeight) + 16) >> 5; + destination[column] = Av1Math.Clamp(value, minimum, maximum); + } + } + + /// + /// Blends complete 512-bit groups from one overlap row. + /// + /// The samples from the preceding block. + /// The samples from the following block. + /// The blended samples. + /// The number of samples to blend. + /// The first unprocessed sample. + /// The preceding-block weight. + /// The following-block weight. + /// The minimum grain value. + /// The maximum grain value. + /// The overload-selection value. + /// The first sample not processed by this vector width. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int Blend( + ReadOnlySpan left, + ReadOnlySpan right, + Span destination, + int width, + int column, + int leftWeight, + int rightWeight, + int minimum, + int maximum, + Vector512 vector) + { + ref int leftBase = ref MemoryMarshal.GetReference(left); + ref int rightBase = ref MemoryMarshal.GetReference(right); + ref int destinationBase = ref MemoryMarshal.GetReference(destination); + Vector512 leftWeights = Vector512.Create(leftWeight); + Vector512 rightWeights = Vector512.Create(rightWeight); + Vector512 rounding = Vector512.Create(16); + Vector512 minima = Vector512.Create(minimum); + Vector512 maxima = Vector512.Create(maximum); + int vectorEnd = width - Vector512.Count; + for (; column <= vectorEnd; column += Vector512.Count) + { + Vector512 leftValues = Vector512.LoadUnsafe(ref leftBase, (nuint)column); + Vector512 rightValues = Vector512.LoadUnsafe(ref rightBase, (nuint)column); + Vector512 result = ((leftValues * leftWeights) + (rightValues * rightWeights) + rounding) >> 5; + Vector512.Min(Vector512.Max(result, minima), maxima).StoreUnsafe(ref destinationBase, (nuint)column); + } + + return column; + } + + /// + /// Blends complete 256-bit groups from one overlap row. + /// + /// The samples from the preceding block. + /// The samples from the following block. + /// The blended samples. + /// The number of samples to blend. + /// The first unprocessed sample. + /// The preceding-block weight. + /// The following-block weight. + /// The minimum grain value. + /// The maximum grain value. + /// The overload-selection value. + /// The first sample not processed by this vector width. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int Blend( + ReadOnlySpan left, + ReadOnlySpan right, + Span destination, + int width, + int column, + int leftWeight, + int rightWeight, + int minimum, + int maximum, + Vector256 vector) + { + ref int leftBase = ref MemoryMarshal.GetReference(left); + ref int rightBase = ref MemoryMarshal.GetReference(right); + ref int destinationBase = ref MemoryMarshal.GetReference(destination); + Vector256 leftWeights = Vector256.Create(leftWeight); + Vector256 rightWeights = Vector256.Create(rightWeight); + Vector256 rounding = Vector256.Create(16); + Vector256 minima = Vector256.Create(minimum); + Vector256 maxima = Vector256.Create(maximum); + int vectorEnd = width - Vector256.Count; + for (; column <= vectorEnd; column += Vector256.Count) + { + Vector256 leftValues = Vector256.LoadUnsafe(ref leftBase, (nuint)column); + Vector256 rightValues = Vector256.LoadUnsafe(ref rightBase, (nuint)column); + Vector256 result = ((leftValues * leftWeights) + (rightValues * rightWeights) + rounding) >> 5; + Vector256.Min(Vector256.Max(result, minima), maxima).StoreUnsafe(ref destinationBase, (nuint)column); + } + + return column; + } + + /// + /// Blends complete 128-bit groups from one overlap row. + /// + /// The samples from the preceding block. + /// The samples from the following block. + /// The blended samples. + /// The number of samples to blend. + /// The first unprocessed sample. + /// The preceding-block weight. + /// The following-block weight. + /// The minimum grain value. + /// The maximum grain value. + /// The overload-selection value. + /// The first sample not processed by this vector width. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + private static int Blend( + ReadOnlySpan left, + ReadOnlySpan right, + Span destination, + int width, + int column, + int leftWeight, + int rightWeight, + int minimum, + int maximum, + Vector128 vector) + { + ref int leftBase = ref MemoryMarshal.GetReference(left); + ref int rightBase = ref MemoryMarshal.GetReference(right); + ref int destinationBase = ref MemoryMarshal.GetReference(destination); + Vector128 leftWeights = Vector128.Create(leftWeight); + Vector128 rightWeights = Vector128.Create(rightWeight); + Vector128 rounding = Vector128.Create(16); + Vector128 minima = Vector128.Create(minimum); + Vector128 maxima = Vector128.Create(maximum); + int vectorEnd = width - Vector128.Count; + for (; column <= vectorEnd; column += Vector128.Count) + { + Vector128 leftValues = Vector128.LoadUnsafe(ref leftBase, (nuint)column); + Vector128 rightValues = Vector128.LoadUnsafe(ref rightBase, (nuint)column); + Vector128 result = ((leftValues * leftWeights) + (rightValues * rightWeights) + rounding) >> 5; + Vector128.Min(Vector128.Max(result, minima), maxima).StoreUnsafe(ref destinationBase, (nuint)column); + } + + return column; + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainSampleOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainSampleOperator.cs new file mode 100644 index 000000000..4fae3c7ed --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/FilmGrain/Av1FilmGrainSampleOperator.cs @@ -0,0 +1,189 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; +using System.Runtime.Intrinsics.X86; +using SixLabors.ImageSharp.Common.Helpers; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.FilmGrain; + +/// +/// Loads and stores native AV1 samples for the film-grain arithmetic pipeline. +/// +/// The native sample type. +internal readonly struct Av1FilmGrainSampleOperator + where TSample : unmanaged +{ + /// + /// Loads eight consecutive samples into 32-bit lanes. + /// + /// The first source sample. + /// The widened samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 Load8(ref TSample source) + { + if (typeof(TSample) == typeof(byte)) + { + ref byte sourceBytes = ref Unsafe.As(ref source); + ulong packed = Unsafe.ReadUnaligned(ref sourceBytes); + return Avx2.ConvertToVector256Int32(Vector128.CreateScalarUnsafe(packed).AsByte()); + } + + ref ushort sourceValues = ref Unsafe.As(ref source); + return Avx2.ConvertToVector256Int32(Vector128.LoadUnsafe(ref sourceValues)); + } + + /// + /// Loads four consecutive samples into 32-bit lanes. + /// + /// The first source sample. + /// The widened samples. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 Load4(ref TSample source) + { + if (typeof(TSample) == typeof(byte)) + { + ref byte sourceBytes = ref Unsafe.As(ref source); + uint packedBytes = Unsafe.ReadUnaligned(ref sourceBytes); + Vector128 widened = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packedBytes).AsByte()); + return Vector128.WidenLower(widened).AsInt32(); + } + + ref ushort sourceValues = ref Unsafe.As(ref source); + ulong packedValues = Unsafe.ReadUnaligned(ref Unsafe.As(ref sourceValues)); + Vector64 packedSamples = Vector64.CreateScalarUnsafe(packedValues).AsUInt16(); + return Vector128.WidenLower(Vector128.Create(packedSamples, Vector64.Zero)).AsInt32(); + } + + /// + /// Loads the luma coordinates corresponding to eight chroma samples. + /// + /// The first luma sample. + /// The horizontal chroma subsampling shift. + /// The luma values used by chroma scaling. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector256 LoadChromaLuma8(ref TSample source, int subsamplingX) + { + if (subsamplingX == 0) + { + return Load8(ref source); + } + + Vector256 lumaPairs; + if (typeof(TSample) == typeof(byte)) + { + ref byte sourceBytes = ref Unsafe.As(ref source); + Vector128 packed = Vector128.LoadUnsafe(ref sourceBytes); + lumaPairs = Vector256.Create(Vector128.WidenLower(packed).AsInt16(), Vector128.WidenUpper(packed).AsInt16()); + } + else + { + ref ushort sourceValues = ref Unsafe.As(ref source); + lumaPairs = Vector256.LoadUnsafe(ref sourceValues).AsInt16(); + } + + // Horizontal 4:2:x chroma uses the rounded mean of each adjacent luma pair. + return (Avx2.MultiplyAddAdjacent(lumaPairs, Vector256.Create((short)1)) + Vector256.One) >> 1; + } + + /// + /// Loads the luma coordinates corresponding to four chroma samples. + /// + /// The first luma sample. + /// The horizontal chroma subsampling shift. + /// The luma values used by chroma scaling. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static Vector128 LoadChromaLuma4(ref TSample source, int subsamplingX) + { + if (subsamplingX == 0) + { + return Load4(ref source); + } + + Vector128 lumaPairs; + if (typeof(TSample) == typeof(byte)) + { + ref byte sourceBytes = ref Unsafe.As(ref source); + ulong packed = Unsafe.ReadUnaligned(ref sourceBytes); + lumaPairs = Vector128.WidenLower(Vector128.CreateScalarUnsafe(packed).AsByte()).AsInt16(); + } + else + { + ref ushort sourceValues = ref Unsafe.As(ref source); + lumaPairs = Vector128.LoadUnsafe(ref sourceValues).AsInt16(); + } + + // Multiply-add with unity coefficients collapses four adjacent pairs without scalar deinterleaving. + return (Vector128_.MultiplyAddAdjacent(lumaPairs, Vector128.Create((short)1)) + Vector128.One) >> 1; + } + + /// + /// Stores eight already clipped 32-bit samples in their native representation. + /// + /// The first destination sample. + /// The samples to store. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Store8(ref TSample destination, Vector256 values) + { + Vector128 packed = Vector128.Narrow(values.GetLower().AsUInt32(), values.GetUpper().AsUInt32()); + if (typeof(TSample) == typeof(byte)) + { + Vector64 bytes = Vector128.Narrow(packed, Vector128.Zero).GetLower(); + bytes.StoreUnsafe(ref Unsafe.As(ref destination)); + } + else + { + packed.StoreUnsafe(ref Unsafe.As(ref destination)); + } + } + + /// + /// Stores four already clipped 32-bit samples in their native representation. + /// + /// The first destination sample. + /// The samples to store. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Store4(ref TSample destination, Vector128 values) + { + Vector64 packed = Vector128.Narrow(values.AsUInt32(), Vector128.Zero).GetLower(); + if (typeof(TSample) == typeof(byte)) + { + Vector64 bytes = Vector128.Narrow(Vector128.Create(packed, Vector64.Zero), Vector128.Zero).GetLower(); + Unsafe.WriteUnaligned(ref Unsafe.As(ref destination), bytes.AsUInt32().GetElement(0)); + } + else + { + packed.StoreUnsafe(ref Unsafe.As(ref destination)); + } + } + + /// + /// Reads one native sample as an integer. + /// + /// The source sample. + /// The unsigned sample value. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Load(ref TSample source) + => typeof(TSample) == typeof(byte) + ? Unsafe.As(ref source) + : Unsafe.As(ref source); + + /// + /// Stores one already clipped sample in its native representation. + /// + /// The destination sample. + /// The sample value. + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void Store(ref TSample destination, int value) + { + if (typeof(TSample) == typeof(byte)) + { + Unsafe.As(ref destination) = (byte)value; + } + else + { + Unsafe.As(ref destination) = (ushort)value; + } + } +} diff --git a/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1FilmGrainBenchmarks.cs b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1FilmGrainBenchmarks.cs new file mode 100644 index 000000000..3c20be940 --- /dev/null +++ b/tests/ImageSharp.Benchmarks/Codecs/Heif/Av1FilmGrainBenchmarks.cs @@ -0,0 +1,272 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using BenchmarkDotNet.Attributes; +using BenchmarkDotNet.Columns; +using BenchmarkDotNet.Configs; +using BenchmarkDotNet.Jobs; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline.FilmGrain; + +namespace SixLabors.ImageSharp.Benchmarks.Codecs.Heif; + +/// +/// Measures AV1 film-grain application across full-HD-equivalent 4:2:0 component planes. +/// +[Config(typeof(Configuration))] +[MemoryDiagnoser(displayGenColumns: false)] +[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] +[CategoriesColumn] +public class Av1FilmGrainBenchmarks +{ + /// + /// The aligned full-HD luma width. + /// + private const int Width = 1920; + + /// + /// The aligned full-HD luma height. + /// + private const int Height = 1088; + + /// + /// The luma width and height of one selected grain block. + /// + private const int BlockSize = 32; + + /// + /// The 4:2:0 chroma-plane width. + /// + private const int ChromaWidth = Width / 2; + + /// + /// The 4:2:0 chroma-plane height. + /// + private const int ChromaHeight = Height / 2; + + /// + /// The 4:2:0 chroma width and height of one selected grain block. + /// + private const int ChromaBlockSize = BlockSize / 2; + + /// + /// The deterministic eight-bit luma plane. + /// + private readonly byte[] luma8 = new byte[Width * Height]; + + /// + /// The deterministic eight-bit first chroma plane. + /// + private readonly byte[] cb8 = new byte[ChromaWidth * ChromaHeight]; + + /// + /// The deterministic eight-bit second chroma plane. + /// + private readonly byte[] cr8 = new byte[ChromaWidth * ChromaHeight]; + + /// + /// The deterministic twelve-bit luma plane. + /// + private readonly ushort[] luma12 = new ushort[Width * Height]; + + /// + /// The deterministic twelve-bit first chroma plane. + /// + private readonly ushort[] cb12 = new ushort[ChromaWidth * ChromaHeight]; + + /// + /// The deterministic twelve-bit second chroma plane. + /// + private readonly ushort[] cr12 = new ushort[ChromaWidth * ChromaHeight]; + + /// + /// The expanded luma scaling function. + /// + private readonly int[] scalingY = new int[256]; + + /// + /// The expanded first chroma scaling function. + /// + private readonly int[] scalingCb = new int[256]; + + /// + /// The expanded second chroma scaling function. + /// + private readonly int[] scalingCr = new int[256]; + + /// + /// The selected luma grain block. + /// + private readonly int[] lumaGrain = new int[BlockSize * BlockSize]; + + /// + /// The selected first chroma grain block. + /// + private readonly int[] cbGrain = new int[ChromaBlockSize * ChromaBlockSize]; + + /// + /// The selected second chroma grain block. + /// + private readonly int[] crGrain = new int[ChromaBlockSize * ChromaBlockSize]; + + /// + /// The active grain parameters shared by both measured sample precisions. + /// + private readonly ObuFilmGrainParameters parameters = new() + { + NumYPoints = 2, + ChromaScalingFromLuma = true, + GrainScalingMinus8 = 3 + }; + + /// + /// Populates deterministic source planes and grain blocks outside the measured traversal. + /// + [GlobalSetup] + public void Setup() + { + for (int index = 0; index < this.luma8.Length; index++) + { + int value = ((index * 37) + 113) & byte.MaxValue; + this.luma8[index] = (byte)value; + this.luma12[index] = (ushort)(value << 4); + } + + for (int index = 0; index < this.cb8.Length; index++) + { + int cb = ((index * 53) + 97) & byte.MaxValue; + int cr = ((index * 71) + 41) & byte.MaxValue; + this.cb8[index] = (byte)cb; + this.cr8[index] = (byte)cr; + this.cb12[index] = (ushort)(cb << 4); + this.cr12[index] = (ushort)(cr << 4); + } + + for (int index = 0; index < this.lumaGrain.Length; index++) + { + this.lumaGrain[index] = ((index * 29) & byte.MaxValue) - 128; + } + + for (int index = 0; index < this.cbGrain.Length; index++) + { + this.cbGrain[index] = ((index * 43) & byte.MaxValue) - 128; + this.crGrain[index] = ((index * 61) & byte.MaxValue) - 128; + } + + // A zero scaling function keeps every invocation's source planes stable. The measured code still performs the + // production lookup, interpolation, grain multiplication, clipping, and native sample packing for every lane. + this.scalingY.AsSpan().Clear(); + this.scalingCb.AsSpan().Clear(); + this.scalingCr.AsSpan().Clear(); + } + + /// + /// Applies eight-bit grain blocks across full-HD-equivalent 4:2:0 planes. + /// + /// The final luma sample, keeping the output observable. + [Benchmark] + [BenchmarkCategory("8Bit")] + public byte Apply8Bit() + { + for (int y = 0; y < Height; y += BlockSize) + { + for (int x = 0; x < Width; x += BlockSize) + { + int lumaOffset = (y * Width) + x; + int chromaOffset = ((y / 2) * ChromaWidth) + (x / 2); + + Av1FilmGrainNoise.Apply( + this.parameters, + this.scalingY, + this.scalingCb, + this.scalingCr, + this.luma8.AsSpan(lumaOffset), + this.cb8.AsSpan(chromaOffset), + this.cr8.AsSpan(chromaOffset), + Width, + ChromaWidth, + this.lumaGrain, + this.cbGrain, + this.crGrain, + BlockSize, + ChromaBlockSize, + BlockSize / 2, + BlockSize / 2, + 8, + 1, + 1, + isMonochrome: false, + isIdentityMatrix: false); + } + } + + return this.luma8[^1]; + } + + /// + /// Applies twelve-bit grain blocks across full-HD-equivalent 4:2:0 planes. + /// + /// The final luma sample, keeping the output observable. + [Benchmark] + [BenchmarkCategory("12Bit")] + public ushort Apply12Bit() + { + for (int y = 0; y < Height; y += BlockSize) + { + for (int x = 0; x < Width; x += BlockSize) + { + int lumaOffset = (y * Width) + x; + int chromaOffset = ((y / 2) * ChromaWidth) + (x / 2); + + Av1FilmGrainNoise.Apply( + this.parameters, + this.scalingY, + this.scalingCb, + this.scalingCr, + this.luma12.AsSpan(lumaOffset), + this.cb12.AsSpan(chromaOffset), + this.cr12.AsSpan(chromaOffset), + Width, + ChromaWidth, + this.lumaGrain, + this.cbGrain, + this.crGrain, + BlockSize, + ChromaBlockSize, + BlockSize / 2, + BlockSize / 2, + 12, + 1, + 1, + isMonochrome: false, + isIdentityMatrix: false); + } + } + + return this.luma12[^1]; + } + + /// + /// Configures production-process measurements for hardware, no-AVX, and scalar grain application. + /// + public sealed class Configuration : ManualConfig + { + /// + /// Initializes a new instance of the class. + /// + public Configuration() + { + this.AddJob(Job.ShortRun.WithId("Hardware").AsBaseline()); + + this.AddJob( + Job.ShortRun + .WithId("NoAvx") + .WithEnvironmentVariable("DOTNET_EnableAVX", "0")); + + this.AddJob( + Job.ShortRun + .WithId("Scalar") + .WithEnvironmentVariable("DOTNET_EnableHWIntrinsic", "0")); + } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs index 50694fe0d..00fc945b6 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs @@ -168,6 +168,14 @@ public class Av1ReconstructionConformanceTests Assert.Equal(requiredCoverage, restorationCoverage & requiredCoverage); } + /// + /// Verifies film-grain template generation, block selection, overlap, chroma scaling, subsampling, high-bit-depth + /// arithmetic, and exact native presentation samples against scalar libaom. + /// + [Fact] + public void DecodeWithFilmGrainMatchesPinnedLibaomReference() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateFilmGrainFixtures, LoopRestorationConfigurations); + /// /// Validates every active-CDEF fixture under the hardware configuration selected by . /// @@ -360,6 +368,73 @@ public class Av1ReconstructionConformanceTests HeifBitDepth.Bit12); } + /// + /// Validates every active film-grain fixture under the hardware configuration selected by + /// . + /// + private static void ValidateFilmGrainFixtures() + { + ValidateFilmGrainFixture( + TestImages.Heif.Av1FilmGrain8BitPayload, + TestImages.Heif.Av1FilmGrain8BitReference, + 100, + 60, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv420); + + ValidateFilmGrainFixture( + TestImages.Heif.Av1FilmGrain10BitPayload, + TestImages.Heif.Av1FilmGrain10BitReference, + 100, + 60, + Av1BitDepth.TenBit, + Av1ColorFormat.Yuv422); + + ValidateFilmGrainFixture( + TestImages.Heif.Av1FilmGrain12BitPayload, + TestImages.Heif.Av1FilmGrain12BitReference, + 100, + 60, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv444); + + ValidateFilmGrainFixture( + TestImages.Heif.Av1FilmGrain8BitRestrictedPayload, + TestImages.Heif.Av1FilmGrain8BitRestrictedReference, + 100, + 60, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv420, + requireRestrictedRange: true); + + ValidateFilmGrainFixture( + TestImages.Heif.Av1FilmGrain8BitMonochromePayload, + TestImages.Heif.Av1FilmGrain8BitMonochromeReference, + 100, + 60, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv400, + requireRestrictedRange: true); + + ValidateFilmGrainFixture( + TestImages.Heif.Av1FilmGrain12BitIdentityPayload, + TestImages.Heif.Av1FilmGrain12BitIdentityReference, + 100, + 60, + Av1BitDepth.TwelveBit, + Av1ColorFormat.Yuv444, + requireRestrictedRange: true, + requireIdentityMatrix: true); + + ValidateFilmGrainFixture( + TestImages.Heif.Av1FilmGrainOddDimensionsPayload, + TestImages.Heif.Av1FilmGrainOddDimensionsReference, + 33, + 11, + Av1BitDepth.EightBit, + Av1ColorFormat.Yuv420); + } + /// /// Validates one elementary-stream sample and its containing AVIF image. /// @@ -397,6 +472,10 @@ public class Av1ReconstructionConformanceTests /// Indicates whether the stream must signal and select nonzero CDEF strengths. /// Indicates whether the stream must use normative horizontal upscaling. /// Indicates whether the stream must select at least one loop-restoration unit. + /// Indicates whether the displayed frame must synthesize signaled film grain. + /// Indicates whether film grain must clip every plane to its restricted range. + /// Indicates whether restricted chroma clipping must use the luma endpoints. + /// Indicates whether the stream must signal a nonzero deblocking strength. /// A bit mask containing every selected loop-restoration filter type. private static int ValidateNativeFixture( string payloadPath, @@ -407,7 +486,11 @@ public class Av1ReconstructionConformanceTests Av1ColorFormat colorFormat, bool requireActiveCdef, bool requireSuperResolution = false, - bool requireLoopRestoration = false) + bool requireLoopRestoration = false, + bool requireFilmGrain = false, + bool requireRestrictedRange = false, + bool requireIdentityMatrix = false, + bool requireActiveLoopFilter = true) { int restorationCoverage = 0; byte[] payload = TestFile.Create(payloadPath).Bytes; @@ -433,12 +516,15 @@ public class Av1ReconstructionConformanceTests } } - ObuLoopFilterParameters filterParameters = decoder.FrameHeader.LoopFilterParameters; - Assert.True( - filterParameters.FilterLevel[0] != 0 - || filterParameters.FilterLevel[1] != 0 - || filterParameters.FilterLevelU != 0 - || filterParameters.FilterLevelV != 0); + if (requireActiveLoopFilter) + { + ObuLoopFilterParameters filterParameters = decoder.FrameHeader.LoopFilterParameters; + Assert.True( + filterParameters.FilterLevel[0] != 0 + || filterParameters.FilterLevel[1] != 0 + || filterParameters.FilterLevelU != 0 + || filterParameters.FilterLevelV != 0); + } if (requireActiveCdef) { @@ -483,6 +569,22 @@ public class Av1ReconstructionConformanceTests Assert.NotEqual(0, restorationCoverage); } + if (requireFilmGrain) + { + Assert.True(decoder.FrameHeader.FilmGrainParameters.ApplyGrain); + } + + if (requireRestrictedRange) + { + Assert.True(decoder.FrameHeader.FilmGrainParameters.ClipToRestrictedRange); + } + + if (requireIdentityMatrix) + { + Assert.NotNull(decoder.SequenceHeader); + Assert.Equal(ObuMatrixCoefficients.Identity, decoder.SequenceHeader.ColorConfig.MatrixCoefficients); + } + AssertNativePlanesEqual(frameBuffer, reference); return restorationCoverage; } @@ -561,6 +663,39 @@ public class Av1ReconstructionConformanceTests requireSuperResolution: requireSuperResolution, requireLoopRestoration: true); + /// + /// Validates one independently encoded stream that applies film grain to the displayed samples. + /// + /// The AV1 elementary-stream sample. + /// The native planar output produced by the pinned scalar libaom decoder. + /// The expected displayed width. + /// The expected displayed height. + /// The expected AV1 sample precision. + /// The expected native chroma-sampling layout. + /// Whether film grain must clip every plane to its restricted range. + /// Whether restricted chroma clipping must use the luma endpoints. + private static void ValidateFilmGrainFixture( + string payloadPath, + string referencePath, + int width, + int height, + Av1BitDepth bitDepth, + Av1ColorFormat colorFormat, + bool requireRestrictedRange = false, + bool requireIdentityMatrix = false) + => ValidateNativeFixture( + payloadPath, + referencePath, + width, + height, + bitDepth, + colorFormat, + requireActiveCdef: false, + requireFilmGrain: true, + requireRestrictedRange: requireRestrictedRange, + requireIdentityMatrix: requireIdentityMatrix, + requireActiveLoopFilter: false); + /// /// Validates the public presentation and metadata produced from one complete AVIF container. /// @@ -692,7 +827,11 @@ public class Av1ReconstructionConformanceTests }; int referenceOffset = 0; - foreach (Av1Plane plane in new[] { Av1Plane.Y, Av1Plane.U, Av1Plane.V }) + ReadOnlySpan planes = frameBuffer.ColorFormat == Av1ColorFormat.Yuv400 + ? [Av1Plane.Y] + : [Av1Plane.Y, Av1Plane.U, Av1Plane.V]; + + foreach (Av1Plane plane in planes) { int subsamplingX = plane == Av1Plane.Y ? 0 : chromaSubsamplingX; int subsamplingY = plane == Av1Plane.Y ? 0 : chromaSubsamplingY; diff --git a/tests/ImageSharp.Tests/TestImages.cs b/tests/ImageSharp.Tests/TestImages.cs index b1674b42b..e796f6885 100644 --- a/tests/ImageSharp.Tests/TestImages.cs +++ b/tests/ImageSharp.Tests/TestImages.cs @@ -1389,6 +1389,20 @@ public static class TestImages public const string Av1RestorationSuperResolution10BitReference = "Heif/Av1/Conformance/libaom-restoration-superres-abc-10b-422-libaom.yuv"; public const string Av1RestorationSuperResolution12BitPayload = "Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b.bit"; public const string Av1RestorationSuperResolution12BitReference = "Heif/Av1/Conformance/libaom-restoration-superres-cosmos-12b-libaom.yuv"; + public const string Av1FilmGrain8BitPayload = "Heif/Av1/Conformance/libaom-film-grain-circle-8b-420.bit"; + public const string Av1FilmGrain8BitReference = "Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-libaom.yuv"; + public const string Av1FilmGrain10BitPayload = "Heif/Av1/Conformance/libaom-film-grain-circle-10b-422.bit"; + public const string Av1FilmGrain10BitReference = "Heif/Av1/Conformance/libaom-film-grain-circle-10b-422-libaom.yuv"; + public const string Av1FilmGrain12BitPayload = "Heif/Av1/Conformance/libaom-film-grain-circle-12b-444.bit"; + public const string Av1FilmGrain12BitReference = "Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-libaom.yuv"; + public const string Av1FilmGrain8BitRestrictedPayload = "Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited.bit"; + public const string Av1FilmGrain8BitRestrictedReference = "Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited-libaom.yuv"; + public const string Av1FilmGrain8BitMonochromePayload = "Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited.bit"; + public const string Av1FilmGrain8BitMonochromeReference = "Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited-libaom.yuv"; + public const string Av1FilmGrain12BitIdentityPayload = "Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited.bit"; + public const string Av1FilmGrain12BitIdentityReference = "Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited-libaom.yuv"; + public const string Av1FilmGrainOddDimensionsPayload = "Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd.bit"; + public const string Av1FilmGrainOddDimensionsReference = "Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd-libaom.yuv"; } public static class Ico diff --git a/tests/Images/Input/Heif/Av1/Conformance/README.md b/tests/Images/Input/Heif/Av1/Conformance/README.md index 8916e92ff..ce35f77ba 100644 --- a/tests/Images/Input/Heif/Av1/Conformance/README.md +++ b/tests/Images/Input/Heif/Av1/Conformance/README.md @@ -40,3 +40,36 @@ The matching `libavif-restoration-*.png` files were decoded from those exact AVI The `libaom-restoration-superres-*` streams combine active restoration with a coded width reduced by super-resolution denominator 12. They use the same pinned generic libaom build and material encoder options as the restoration streams, with `--superres-mode=1 --superres-denominator=12 --superres-kf-denominator=12`. The 8-bit fixture is 768x512 YUV 4:2:0, the 10-bit fixture is 512x256 YUV 4:2:2, and the 12-bit fixture is 1024x428 YUV 4:4:4. Their matching `*-libaom.yuv` files were decoded from the exact payloads by the pinned generic `aomdec --rawvideo` build. The 10-bit 4:2:2 source was produced from libavif's `abc.png` with pinned generic `avifenc` using `-j 1 -s 8 -q 100 -d 10 -y 422`, then decoded to Y4M before the combined libaom encode. Its clipped rightmost 128x128 coding block crosses a second 64x64 residual region. This independently exercises the required conversion of the luma-region cursor to the subsampled chroma transform grid instead of relying only on full-width 4:4:4 blocks. + +## Film-grain coverage + +The film-grain pairs were generated independently from ImageSharp. Each `.bit` file is an AV1 still-picture OBU stream, and the matching `-libaom.yuv` file is the exact visible planar output from the pinned scalar libaom decoder. + +The source images are `tests/data/circle-trns-after-plte.png` and `tests/data/draw_points.png` from the pinned libavif revision above. The streams and native references use the same pinned libaom revision. Intermediate Y4M inputs were produced with libavif 1.4.2 linked to that libaom revision. + +| Stream | libaom vector | Native layout | Range | Covered behavior | +| --- | ---: | --- | --- | --- | +| `libaom-film-grain-circle-8b-420.bit` | 2 | 8-bit 4:2:0 | Full | Lag-three templates, boundary overlap, and independent luma and chroma scaling | +| `libaom-film-grain-circle-10b-422.bit` | 15 | 10-bit 4:2:2 | Full | Lag-two templates, boundary overlap, and chroma scaling derived from luma | +| `libaom-film-grain-circle-12b-444.bit` | 16 | 12-bit 4:4:4 | Full | Lag-three templates, boundary overlap, high-depth interpolation, and grain scale shift two | +| `libaom-film-grain-circle-8b-420-limited.bit` | 1 | 8-bit 4:2:0 | Restricted | Independent restricted luma and chroma endpoints | +| `libaom-film-grain-circle-8b-400-limited.bit` | 3 | 8-bit monochrome | Restricted | Monochrome synthesis, overlap, and restricted luma clipping | +| `libaom-film-grain-circle-12b-444-identity-limited.bit` | 14 | 12-bit 4:4:4 identity | Restricted | High-depth identity-matrix clipping, including luma endpoints for all three planes | +| `libaom-film-grain-draw-points-8b-420-odd.bit` | 2 | 8-bit 4:2:0, 33×11 | Full | Odd-width and odd-height extension, a partial final block, and overlap at the visible frame edge | + +The common libaom encoder options were: + +```text +--usage=2 --passes=1 --limit=1 --obu --end-usage=q --cq-level=30 --cpu-used=4 +--threads=1 --lag-in-frames=0 --full-still-picture-hdr --enable-cdef=0 --enable-restoration=0 +``` + +Each stream adds the bit depth, input bit depth, profile, monochrome or identity-matrix flag where applicable, and the `--film-grain-test` value shown above. The twelve-bit streams use a ten-bit Y4M input and `--bit-depth=12 --input-bit-depth=10`; this is the supported high-depth promotion path in the pinned generic aomenc build. + +References were decoded with: + +```text +aomdec --rawvideo --output=.yuv .bit +``` + +Tests compare every visible native Y, U, and V sample exactly. No tolerant image comparison is used. diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-10b-422-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-10b-422-libaom.yuv new file mode 100644 index 000000000..d42e8992c --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-10b-422-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2bcf2a42e9f01044a32557e45ac19d0077c4bb3f2791cb3d01e22d86d9d9c7c5 +size 24000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-10b-422.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-10b-422.bit new file mode 100644 index 000000000..24ee79ebb --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-10b-422.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9fa3da33ba476f569c7825e134f7727571463416814486e95656806756d91b69 +size 407 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited-libaom.yuv new file mode 100644 index 000000000..166ab6fa8 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7970ad0150245766dddc2b6ce480e1b0be68c734d0d15fd1f38a0d7b87ec42c9 +size 36000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited.bit new file mode 100644 index 000000000..8077dd619 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-identity-limited.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d295643fb78a8705a2ff2de6c9b40db9d94fb92a510038b00d5fc9ebee83d9a5 +size 364 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-libaom.yuv new file mode 100644 index 000000000..d42a192cd --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:583077d4dc530c57271ab4c990708dbe5765622978878f4305b5f64a6512fc37 +size 36000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444.bit new file mode 100644 index 000000000..cd7dc04fe --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-12b-444.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:48fe85c152a0ad35adfe963a62164f8f105cbc88e88af6e7e0a6c6dc4e124f79 +size 475 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited-libaom.yuv new file mode 100644 index 000000000..91854a712 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bb0afbff82dc67937b7c3411b11eb8ed73033bf68194c01bf947750258ea00b1 +size 6000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited.bit new file mode 100644 index 000000000..2c8cf0cc9 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-400-limited.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e16477e1ce47f0b4c30c21168c0e33e1bfb3c48d67e37ea4cddee8a2c0478457 +size 229 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-libaom.yuv new file mode 100644 index 000000000..c40a3b31f --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6433d14c6b863eca1b244d1e3834832de1cc231e2f3733633c88aef3cb326132 +size 9000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited-libaom.yuv new file mode 100644 index 000000000..554223c41 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1b6dc8bba3a944a9abd041e3f8f253806f4689412ffab8fb71b74bc99281480 +size 9000 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited.bit new file mode 100644 index 000000000..3ffccd10e --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420-limited.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aba642e2258e7da69ce54f333bd9a30037e22cbca191ad7674e5fc341bb154df +size 379 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420.bit new file mode 100644 index 000000000..ee7313881 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-circle-8b-420.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0f082a3cbbe9f4d16ba8735ae48d95fa9a06fdca56658dd9e937a1c661d0e45b +size 381 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd-libaom.yuv new file mode 100644 index 000000000..0a0e99a7c --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b86740f4707d30d3835504b5b39622ed8dd88418bb052a88e3edd84258a6230a +size 567 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd.bit b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd.bit new file mode 100644 index 000000000..5aee2e672 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libaom-film-grain-draw-points-8b-420-odd.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ae77f08e0c30ada187e7a630b4b502cd8349227f9128d449b5d1e3badcc5dd1a +size 153