From 87a11a4c28452ef6a9fed4feac18f6a792e5c27e Mon Sep 17 00:00:00 2001 From: James Jackson-South Date: Thu, 3 Sep 2026 08:29:19 +1000 Subject: [PATCH] Add AV1 intra-block-copy mode selection --- HEIF_IMPLEMENTATION_PLAN.md | 6 +- ...traSuperblockEncoder.ChromaModeDecision.cs | 4 +- ...blockEncoder.IntraBlockCopyModeDecision.cs | 528 ++++++++++++++++++ .../Av1IntraSuperblockEncoder.ModeDecision.cs | 74 ++- .../Av1IntraSuperblockEncoder.Operator.cs | 94 ++++ .../Av1/Av1IntraSuperblockEncoderTests.cs | 287 ++++++++++ 6 files changed, 971 insertions(+), 22 deletions(-) create mode 100644 src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.IntraBlockCopyModeDecision.cs diff --git a/HEIF_IMPLEMENTATION_PLAN.md b/HEIF_IMPLEMENTATION_PLAN.md index dc38ec01ce..1b686ff87b 100644 --- a/HEIF_IMPLEMENTATION_PLAN.md +++ b/HEIF_IMPLEMENTATION_PLAN.md @@ -825,10 +825,10 @@ Encoder verification contract: - [~] A non-owning encoder-frame view now separates visible conversion regions from coded regions and performs complete left, top, right, bottom, and corner extension across each bordered plane. Current libaom uses 8-sample-aligned coded dimensions, a 32-sample-aligned luma stride with chroma stride derived from it, and a 64-pixel luma border for non-resized all-intra encoding. One operation-ready frame owner now rents the aligned Y, U, and V storage contiguously, exposes non-owning `Buffer2D` plane views, and returns the rent exactly once. A 4K 4:2:0 frame occupies about 13.0 MiB at 8-bit or 26.0 MiB at 10/12-bit; source and reconstruction therefore remain distinct frame owners rather than adding a full-frame copy. The corrected tests use this real ownership path and verify the exact 54 KiB 64x64 4:2:0 rent. The frame-encoder operation now instantiates matching source and reconstruction owners with ordinary `using` lifetimes and converts packed pixels directly into the source owner before extension. - [~] Temporal delimiter, sequence header, frame header, combined-frame tile-group writing, and an internal reduced-still-picture frame operation now exist locally. The remaining required metadata, padding, multi-tile, option, and public encoder paths are not complete. - [~] Implement superblock and partition analysis for every permitted block size and partition. The current baseline deliberately splits every in-frame node to 8x8 blocks and records decisions in current-libaom writer preorder; block-size selection and non-split partition analysis remain. -- [~] Implement intra mode search, palette, filter intra, chroma-from-luma, and intra-block copy decisions. Live luma search now covers all 13 zero-angle base modes and all six nonzero adjustments for each of the eight directional modes. Joint spatial chroma search covers the same 61 candidates, combines both chroma planes in one rate-distortion decision, and preserves the winning shared angle adjustment. Chroma-from-luma now searches the complete signed alpha alphabet from reconstructed luma and retains its joint U/V syntax. Filter-intra now searches all five predictors after ordinary luma modes. Palette entropy, retained state, production syntax, exhaustive luma and paired chroma palette selection, and adaptive production activation are complete; intra-block-copy mode decisions remain. +- [~] Implement intra mode search, palette, filter intra, chroma-from-luma, and intra-block copy decisions. Live luma search now covers all 13 zero-angle base modes and all six nonzero adjustments for each of the eight directional modes. Joint spatial chroma search covers the same 61 candidates, combines both chroma planes in one rate-distortion decision, and preserves the winning shared angle adjustment. Chroma-from-luma now searches the complete signed alpha alphabet from reconstructed luma and retains its joint U/V syntax. Filter-intra now searches all five predictors after ordinary luma modes. Palette entropy, retained state, production syntax, exhaustive luma and paired chroma palette selection, adaptive palette activation, and joint intra-block-copy mode selection are complete; adaptive intra-block-copy frame activation remains. - [ ] Implement inter mode search for bounded sequences, including reference selection and the decoder-supported inter tools. - [~] Current-libaom `av1_quantize_fp_no_qmatrix` arithmetic is implemented as a closed generic forward-quantizer family with Vector512, Vector256, Vector128, and scalar paths, raster-order output, coded 64-point coefficient limits, and scan-order EOB selection. Transform search, coefficient optimization, and lossless behavior remain. -- [~] Implement real rate-distortion selection and make quality and effort change work, size, and output quality. The complete luma and joint chroma candidate sets, including chroma-from-luma and filter-intra, now perform live rate-distortion selection; quality mapping, effort-dependent pruning, and the remaining searches are not implemented. +- [~] Implement real rate-distortion selection and make quality and effort change work, size, and output quality. The complete luma and joint chroma candidate sets, including chroma-from-luma, filter-intra, palette, and intra-block copy, now perform live rate-distortion selection; quality mapping, effort-dependent pruning, and the remaining searches are not implemented. - [~] Encoder rate accounting converts the entropy writer's live inverse cumulative distributions into current-libaom fixed-point symbol costs without allocating or duplicating probability state. Read-only luma-mode, directional-delta, filter-intra, chroma-mode, block-skip, transform-size, transform-block-skip, and complete transform-coefficient queries share the exact distributions mutated by the subsequent entropy write. Complete coefficient costing follows current libaom's optimized shape: it returns immediately for an empty transform, uses the EOB-specific base-range context, fuses magnitude, sign, base-range, and Golomb accounting into one reverse traversal, and combines repeated full base-range chunks instead of replaying each emitted symbol. Tile-lifetime level and context scratch is reused, the one-coefficient path neither clears nor initializes the forward-neighbor level map, and steady-state queries allocate nothing. Transform-size writing and costing share one subdivision-depth calculation, while shared closed symbol operations keep the writer and cost mappings for transform skip, transform type, and EOB syntax identical without forcing the estimator through the writer's slower two-pass coefficient traversal. The current-libaom fixed-point RD combiner preserves 64-bit distortion and rounds the weighted 1/512-bit rate at the required boundary. Its key-frame multiplier follows libaom's squared DC-quantizer formula and exact 10/12-bit normalization. Live final-block selection evaluates all 61 legal 8x8 luma candidates: the 13 zero-angle base modes in current-libaom order, followed by six nonzero adjustments for each directional mode. Joint chroma selection evaluates the equivalent 61 spatial candidates, combines U and V distortion plus coefficient rate, and charges one live chroma-mode and shared-angle symbol over the actual subsampled 4x4, 4x8, or 8x8 geometry. Chroma-from-luma subsamples the reconstructed luma block once into fixed-stride Q3 stack scratch, subtracts the rounded mean, evaluates all 33 signed alpha values independently for each plane with complete transform RD, and combines the cached plane results across all 1,088 valid joint pairs with one live sign cost and the conditional U/V magnitude costs. This is the allocation-free equivalent of current libaom's exhaustive 33-value path: it requires 66 evaluation transforms rather than transforming every joint pair, preserves DC-before-CfL-before-spatial tie order, and fixes the implicit chroma transform to DCT-DCT. Filter-intra follows ordinary luma candidates, searches all five predictors in syntax order, and evaluates every legal transform while reusing one prepared prediction and source residual per filter mode. Every candidate includes its live mode, angle, filter mode, alpha, and coefficient rate plus normalized pixel-domain distortion. Each prepared reference edge retains the common-corner prefix and twice the transform dimension required by directional prediction. A shared encoder/decoder availability calculation selects reconstructed top-right and bottom-left extensions according to tile, frame, superblock, and block reconstruction order; unavailable extensions repeat the nearest coded endpoint. Missing top or left edges retain current libaom's perpendicular-sample and bit-depth-midpoint rules. Directional prediction applies the AV1 three-degree adjustment step and reuses transform workspace for zone-three transposition before the transform overwrites it, keeping candidate evaluation allocation-free. The winning luma and chroma signed adjustments are retained in the packed final-block state consumed by the tile writer. The tile writer invokes these stack-only selectors after mapping current neighbors and immediately before writing each block, so later decisions see reconstructed samples, coefficient contexts, and CDF updates from every preceding block. Block skip is read only after the callback has combined every coded plane. Luma candidate scratch remains one 8x8 reconstruction and one 8x8 coefficient span on the stack; chroma uses one transform-sized reconstruction and coefficient span for each of U and V. Only a newly winning candidate is copied into retained frame storage. Production fixtures force every luma base predictor, both extreme adjustments in all three directional zones, available top-right and bottom-left extensions, high-bit-depth adjustment propagation, exact signed luma and chroma angle-rate terms, joint U/V decisions, packed chroma state, and 4:2:0, 4:2:2, and 4:4:4 transform geometry. The CfL fixtures derive target chroma from a pilot production encode's actual reconstructed luma through an independent scalar Q3 oracle and prove exact positive/negative alpha syntax plus zero-residual DCT-DCT reconstruction for all three subsampling geometries at 8, 10, and 12 bits. The stable fixed-DC traversal comparison uses neutral samples for which both the baseline and live search are contractually DC and skipped, instead of relying on textured content to happen to select the baseline mode. Luma palette selection now evaluates dominant-color and one-dimensional K-means candidates for every legal size, snaps near-cache colors with the reference threshold and tie order, removes duplicate snapped colors, extends boundary maps from active samples, and performs complete transform rate-distortion search. Ordinary DC and filter-intra candidates pay the palette-disabled symbol whenever screen-content syntax is enabled. The exact net11 Release rebuild reports 1,992 test-project warnings and zero errors, all 58 intra-superblock cases pass, all 8,935 AVIF cases pass, and all 230 HEIF cases pass. Remaining mode decision work includes transform-size search, broader joint mode/transform refinement, partition search, and effort-dependent pruning. Non-empty intra blocks deliberately remain non-skipped, matching current libaom; later inter mode selection owns its distinct skip-transform RD decision. - [~] The tile writer now publishes one packed coefficient context per covered 4x4 edge unit and derives luma/chroma skip plus DC-sign contexts from the complete transform edges using current-libaom units. Partition, transform, and coefficient neighbor state retains only the above and left context regions used by current libaom; the unused third top-left region, its granularity state, and its unused sentinel are removed. One picture owner now packs segmentation plus every tile's partition, luma, chroma, and transform edges into one clean byte allocation with typed non-owning views; together with the separately typed packed mode-information owner, the complete picture state uses two allocator rents rather than seven. Exact aligned lengths, clean initialization, and balanced exactly-once returns are covered in Release. Multi-tile payload ownership and verified CDF update behavior remain. - [~] Encoder mode information now uses a frame-owned integer alias grid over a packed 8-byte value allocation, matching current libaom's `mi_grid_base` and `mi_alloc` relationship without a managed object or reference per 4x4 entry. The visible dimensions are aligned to eight luma samples, the grid stride and allocated row count are aligned to 32 mode-information units, and optional 8x8 allocation granularity reduces the value store in both dimensions exactly as current libaom does. One clean ImageSharp byte owner contains both independently typed regions, reducing libaom's two allocation lifetimes to one without a copy. At 4K, the 4x4 layout occupies about 6.0 MiB in total; the 8x8 layout occupies about 3.0 MiB. Exact geometry, clean allocation, typed lengths, aligned mapping, untouched row padding, and exactly-once return pass 4 of 4 direct net11 VSTest cases in Release. Every coded 4x4 cell covered by square, rectangular, or clipped edge blocks maps to its owning allocation entry before context-dependent symbols are written. Packed syntax, relative neighbor lookup, full block mapping, writer traversal, entropy, and OBU coverage pass 1,947 of 1,947 direct net11 VSTest cases in Release; complete mode decision still remains. @@ -859,7 +859,7 @@ Encoder verification contract: - [~] Paired chroma palette clustering now preserves current libaom's squared two-component distance, first-centroid tie order, independently rounded U/V means, paired deterministic empty-cluster replacement, preceding-state retention on increased distortion, and 50-iteration limit. Keeping the source planes separate avoids interleave/deinterleave copies and improves on libaom's AVX2 ceiling with Vector512, Vector256, Vector128, then scalar dispatch through ImageSharp's shared vector-count helpers. Three independent tests cover exact paired convergence, midpoint initialization, 12-bit distance and index parity, untouched destination bounds, and every intrinsic tier. The exact Release test-project build reports 1,992 baseline warnings and zero errors; the focused three-case set, complete 8,934-case AVIF set, and complete 230-case HEIF set pass direct foreground net11 Release VSTest. Roslynk reports zero compiler errors and no touched-file analyzer warnings. Candidate integration and production activation remain in the open chroma-palette checkpoint. - [~] Live paired chroma palette selection now follows current libaom's complete 2-through-8 color-size search, U-plane neighbor-cache snapping, stable U-ordered color pairs, shared U/V index map, implicit DCT-DCT transform, and strict rate-distortion winner replacement. It improves on speed-configured libaom by applying no early header-cost pruning, keeps planar U/V source data separate, and reuses the SIMD-first prediction, residual, transform, quantization, and reconstruction operators without allocator-backed candidate storage. The production tile regression proves both palette-mode probability branches, exact paired colors and indices, coefficient-free reconstruction, and nonempty syntax. The complete 58-case intra-superblock set, 8,935-case AVIF set, and 230-case HEIF set pass direct foreground net11 Release VSTest. The exact Release test-project build reports 1,992 baseline warnings and zero errors; Roslynk reports zero compiler errors and no touched-file analyzer warnings. Production frame activation remains the next checkpoint. - [~] Production palette activation now matches current libaom's default good-quality screen detector: it scans only complete 16x16 luma blocks, normalizes high-bit-depth samples to eight bits, admits 2-through-4-color blocks, and uses the reference's strict greater-than-ten-percent frame-area threshold. A 256-bit stack bitset and a fifth-color early exit replace libaom's larger per-block histogram without changing the decision, allocation, or source precision. The adaptive sequence flag remains enabled, the frame flag is set before picture-state allocation, and intra-block copy remains disabled. Focused regressions prove strict-threshold equality, high-bit-depth normalization, five-color rejection, emitted frame-header activation, production decode, and generated payload retention. The exact Release test-project build reports 1,992 baseline warnings and zero errors; all 8,935 AVIF cases and all 230 HEIF cases pass direct foreground net11 Release VSTest. Current-main `aomdec` at `a40ed1ea9e4ecc3df58a5bccb76623f2c94ae727` accepts all 30 regenerated production payloads, including the 54-byte palette case. Roslynk reports zero compiler errors and no touched-file analyzer warnings. -- [~] Intra-block-copy rate accounting now uses the live frame-local flag and displacement-vector distributions without copying or adapting either context during candidate measurement. Displacement-vector costing and writing share one closed symbol operation over the exact current-libaom joint, sign, magnitude-class, class-zero, and integer-offset syntax; final mode evaluation applies libaom's 120/128 displacement-rate weight with nearest-integer rounding. Independent fixed costs cover all four joint states, both signs, class zero, and large offset classes before adaptive writes, followed by an encoder/decoder round trip through the same sequence. Encoder and decoder reference-vector derivation now share the exact eight-candidate spatial scan, independent nearest and outer-region ranking, top-right partition geometry, clamping, and tile-relative fallback. Selected vectors use a naturally aligned pair of signed 16-bit components packed into the existing picture-state owner only when intra-block copy is permitted; a 3840x2160 frame retains 130,560 vectors in 510 KiB while leaving the compact 8-byte mode allocation unchanged. The tile writer derives the same reference and emits the retained vector without another allocation or copy. Coefficient costing and writing now select the inter transform sets and frame-local probability tables required by intra-block copy; independent tests verify every legal symbol against the exact default inter distribution and round-trip full and reduced sets from 4x4 through 32x32. Legal 8x8 hash discovery now indexes every visible source origin, including unaligned origins, in libaom's coarse-to-fine insertion order with the same 256-candidate bucket cap. A separable rolling hash fills one packed picture-lifetime workspace before reconstruction, then reuses that workspace for integer candidate links; exact wide or SIMD block comparison rejects hash collisions, and SIMD variance uses libaom's eight-bit normalization at 8, 10, and 12 bits. Power-of-two bucket arrays scale down with small images and stop at the reference's 16-bit limit, avoiding libaom's fixed six-size pointer table; the 3840x2160 search index occupies about 32.2 MiB and introduces no additional owner or frame copy. Above and left search rectangles, integer displacement legality, strict tie order, and live raw displacement rate follow current libaom. Motion-candidate ranking uses libaom's undiscounted probability cost and exact variance-domain error-per-bit scaling, separately from the later 120/128 final-mode discount. The allocation-free full-pixel core now follows current libaom's NSTEP search: it clamps the spatial reference to each legal region, traverses the fixed 15-stage radii and site order, skips equivalent centered 210-pixel stages, repeats progressively shorter paths, and compares their winners in the normalized variance domain. Paths above the speed-zero screen-content threshold continue through libaom's 256-pixel, one-pixel-step exhaustive mesh. Four adjacent byte or high-bit-depth candidates share each SIMD source load, strict row-major tie ordering is retained, and the final legal tail column remains searchable where libaom's current four-wide remainder loop omits it. Byte and high-bit-depth operators compute each 8x8 absolute difference with Vector128 before scalar fallback; high-bit-depth SAD remains in its native sample scale while its quantizer-derived rate multiplier uses libaom's normalized AC step. The exact net11 Release build reports 1,992 baseline warnings and zero errors; all 1,970 focused entropy and intra-block-copy cases, all 9,238 non-HEVC HEIF, AV1, and AVIF cases pass through direct foreground VSTest with tiered compilation disabled so runtime promotion bookkeeping cannot enter exact allocation-counter windows, and Roslynk reports zero compiler errors with no touched-file analyzer diagnostics. Joint luma/chroma rate-distortion selection, production activation, and adaptive frame-flag clearing remain before intra-block copy can be enabled. +- [~] Intra-block-copy rate accounting now uses the live frame-local flag and displacement-vector distributions without copying or adapting either context during candidate measurement. Displacement-vector costing and writing share one closed symbol operation over the exact current-libaom joint, sign, magnitude-class, class-zero, and integer-offset syntax; final mode evaluation applies libaom's 120/128 displacement-rate weight with nearest-integer rounding. Independent fixed costs cover all four joint states, both signs, class zero, and large offset classes before adaptive writes, followed by an encoder/decoder round trip through the same sequence. Encoder and decoder reference-vector derivation now share the exact eight-candidate spatial scan, independent nearest and outer-region ranking, top-right partition geometry, clamping, and tile-relative fallback. Selected vectors use a naturally aligned pair of signed 16-bit components packed into the existing picture-state owner only when intra-block copy is permitted; a 3840x2160 frame retains 130,560 vectors in 510 KiB while leaving the compact 8-byte mode allocation unchanged. The tile writer derives the same reference and emits the retained vector without another allocation or copy. Coefficient costing and writing now select the inter transform sets and frame-local probability tables required by intra-block copy; independent tests verify every legal symbol against the exact default inter distribution and round-trip full and reduced sets from 4x4 through 32x32. Legal 8x8 hash discovery now indexes every visible source origin, including unaligned origins, in libaom's coarse-to-fine insertion order with the same 256-candidate bucket cap. A separable rolling hash fills one packed picture-lifetime workspace before reconstruction, then reuses that workspace for integer candidate links; exact wide or SIMD block comparison rejects hash collisions, and SIMD variance uses libaom's eight-bit normalization at 8, 10, and 12 bits. Power-of-two bucket arrays scale down with small images and stop at the reference's 16-bit limit, avoiding libaom's fixed six-size pointer table; the 3840x2160 search index occupies about 32.2 MiB and introduces no additional owner or frame copy. Above and left search rectangles, integer displacement legality, strict tie order, and live raw displacement rate follow current libaom. Motion-candidate ranking uses libaom's undiscounted probability cost and exact variance-domain error-per-bit scaling, separately from the later 120/128 final-mode discount. The allocation-free full-pixel core now follows current libaom's NSTEP search: it clamps the spatial reference to each legal region, traverses the fixed 15-stage radii and site order, skips equivalent centered 210-pixel stages, repeats progressively shorter paths, and compares their winners in the normalized variance domain. Paths above the speed-zero screen-content threshold continue through libaom's 256-pixel, one-pixel-step exhaustive mesh. Four adjacent byte or high-bit-depth candidates share each SIMD source load, strict row-major tie ordering is retained, and the final legal tail column remains searchable where libaom's current four-wide remainder loop omits it. Byte and high-bit-depth operators compute each 8x8 absolute difference with Vector128 before scalar fallback; high-bit-depth SAD remains in its native sample scale while its quantizer-derived rate multiplier uses libaom's normalized AC step. Production mode decision now derives the same spatial displacement reference used by the writer, deduplicates hash and full-pixel finalists in search order, and evaluates every surviving vector through complete luma and chroma transform RD. This intentionally improves on libaom's preliminary-error pruning by permitting a hash and pixel finalist from the same search region to compete using their final syntax and reconstruction costs. Prediction is prepared once per plane and vector, including integer or half-sample chroma phase, then reused across every legal inter transform without an allocator rent or frame copy. The joint comparison includes the live intra-block-copy flag, discounted displacement rate, skip flag, coefficient syntax, and normalized Y/U/V distortion; an empty transform alternative can win only when its complete skip cost is strictly lower, while conventional intra and earlier vectors retain tie precedence. Winning reconstruction, coefficients, transform state, DC modes, cleared palette/filter/CfL state, and displacement are copied once into the existing retained stores. Production regressions force the path at 8 and 12 bits and force 4:2:0 horizontal half-sample chroma with an unaligned reference. The net11 Release solution build reports zero errors; all 2,030 focused entropy, intra-block-copy, and intra-superblock cases and all 9,240 non-HEVC HEIF, AV1, and AVIF cases pass through direct foreground VSTest, with tiered compilation disabled only for the full allocation-sensitive suite. Adaptive production activation and frame-flag clearing remain before intra-block copy can be enabled by the public encoder. - [x] The expanded checkpoint exposed a pre-existing transform-block test that asserted uninitialized pooled padding was zero. The test now initializes the complete physical luma plane with a sentinel and proves the block operation leaves both adjacent padding samples unchanged. The exact net11 Release rebuild remains at 1,005 baseline warnings and zero errors, the focused allocator-order set passes 30 of 30 cases, and the complete HEIF/AV1 namespace passes 8,859 of 8,859 direct VSTest cases with zero failures or skips. - [x] Combined-frame OBU output now counts the byte-aligned frame and tile-group headers, non-final tile-size fields, and owned tile payloads before emitting the OBU size. It retains only the small allocator-owned header scratch and writes each entropy-coded tile span directly from its detached owner, removing the second file-sized allocator rent and complete-payload copy. A 64 KiB regression proves exactly one sub-payload-sized byte rent with a balanced return and verifies the exact streamed tile tail; the existing two-tile round trip proves size-prefix and ordering parity. The focused writer and production-frame set passes 32 of 32 direct net11 VSTest cases, current-main `aomdec` accepts all 29 generated native-format payloads, and the complete HEIF/AV1 namespace passes 8,860 of 8,860 cases with zero failures or skips. - [x] Finalized fixed-block decisions now set the block-level transform-skip flag only when every retained luma and coded chroma transform has zero EOB, matching current libaom's conjunction of per-plane skip state. The previous always-false flag produced legal but redundant non-skip and zero-coefficient syntax. Monochrome and 4:2:0 regressions prove both branches from actual coefficient state; the focused decision and production-frame set passes 32 of 32 direct net11 VSTest cases. Current-main `aomdec` accepts all 29 regenerated payloads, the recorded decoded-frame MD5s are unchanged, and affected 16x16 constant 8-bit and 10-bit payloads are one byte smaller. The complete HEIF/AV1 namespace passes 8,862 of 8,862 cases with zero failures or skips. diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ChromaModeDecision.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ChromaModeDecision.cs index b7991d7540..c28c89872a 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ChromaModeDecision.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ChromaModeDecision.cs @@ -56,7 +56,8 @@ internal static partial class Av1IntraSuperblockEncoder ref Av1EncoderPaletteInfo paletteInfo, out int selectedAngleDelta, out byte selectedChromaFromLumaIndex, - out sbyte selectedChromaFromLumaSigns) + out sbyte selectedChromaFromLumaSigns, + out long selectedCost) { const Av1BlockSize BlockSize = Av1BlockSize.Block8x8; const int MaximumSampleCount = 8 * 8; @@ -469,6 +470,7 @@ internal static partial class Av1IntraSuperblockEncoder selectedChromaFromLumaSigns = 0; } + selectedCost = bestCost; return bestMode; } diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.IntraBlockCopyModeDecision.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.IntraBlockCopyModeDecision.cs new file mode 100644 index 0000000000..88c5676238 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.IntraBlockCopyModeDecision.cs @@ -0,0 +1,528 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy; +using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; +using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; +using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; +using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; + +/// +/// Provides full rate-distortion selection for intra-block-copy candidates. +/// +internal static partial class Av1IntraSuperblockEncoder +{ + internal partial struct ModeDecision + where TSample : unmanaged + where TOperator : struct, IBlockEncodingOperator + { + private void SelectIntraBlockCopy( + Av1SymbolEncoder writer, + Av1MacroBlockD macroBlock, + Point blockOrigin, + ushort tileIndex, + long regularModeCost, + int regularEmptyTransformRate, + ref Av1MacroBlockModeInfo modeInfo, + ref Av1EncoderBlockStruct block, + ref Av1EncoderPaletteInfo paletteInfo) + { + const Av1BlockSize BlockSize = Av1BlockSize.Block8x8; + const Av1TransformSize LumaTransformSize = Av1TransformSize.Size8x8; + const int MaximumSampleCount = 8 * 8; + Buffer2DRegion lumaSource = this.source.GetPlane(Av1Plane.Y); + Buffer2DRegion lumaReconstruction = this.reconstruction.GetPlane(Av1Plane.Y); + Point modeInfoPosition = new( + blockOrigin.X >> Av1Constants.ModeInfoSizeLog2, + blockOrigin.Y >> Av1Constants.ModeInfoSizeLog2); + + Span referenceCandidates = stackalloc Av1MotionVector[8]; + Span referenceWeights = stackalloc int[8]; + Av1MotionVector reference = Av1IntraBlockCopy.FindReference( + this.picture, + macroBlock, + modeInfoPosition, + BlockSize, + Av1PartitionType.None, + referenceCandidates, + referenceWeights); + + Span candidates = stackalloc Av1MotionVector[4]; + Av1IntraBlockCopySearchIndex search = this.picture.IntraBlockCopySearch; + int candidateCount = search.FindCandidates( + lumaSource, + lumaReconstruction, + blockOrigin, + macroBlock.Tile, + this.picture.Sequence.SequenceHeader, + writer, + reference, + this.rateMultiplier, + candidates); + + candidateCount += search.FindPixelCandidates( + lumaSource, + lumaReconstruction, + blockOrigin, + macroBlock.Tile, + this.picture.Sequence.SequenceHeader, + writer, + reference, + this.quantization.QIndex[0], + this.rateMultiplier, + candidates[candidateCount..]); + + // Hash and full-pixel searches can converge on the same vector. Preserve the first search-order + // occurrence so repeated vectors do not pay for duplicate transform searches or alter ties. + int uniqueCandidateCount = 0; + for (int candidateIndex = 0; candidateIndex < candidateCount; candidateIndex++) + { + Av1MotionVector candidate = candidates[candidateIndex]; + bool duplicate = false; + for (int uniqueIndex = 0; uniqueIndex < uniqueCandidateCount; uniqueIndex++) + { + if (candidate == candidates[uniqueIndex]) + { + duplicate = true; + break; + } + } + + if (!duplicate) + { + candidates[uniqueCandidateCount++] = candidate; + } + } + + if (uniqueCandidateCount == 0) + { + return; + } + + int skipContext = Av1TileWriter.GetSkipContext(macroBlock); + int regularRateAdjustment = writer.GetUseIntraBlockCopyCost(false) + + writer.GetSkipCost(modeInfo.Block.Skip, skipContext); + + if (modeInfo.Block.Skip) + { + regularRateAdjustment -= regularEmptyTransformRate; + } + + long bestCost = regularModeCost + Av1RateDistortion.GetCost(this.rateMultiplier, regularRateAdjustment, 0); + bool hasSelectedCandidate = false; + bool selectedSkip = false; + Av1MotionVector selectedVector = default; + Av1EncoderTransformBlockState selectedLumaState = default; + Av1EncoderTransformBlockState selectedBlueState = default; + Av1EncoderTransformBlockState selectedRedState = default; + Span selectedLumaReconstruction = stackalloc TSample[MaximumSampleCount]; + Span selectedBlueReconstruction = stackalloc TSample[MaximumSampleCount]; + Span selectedRedReconstruction = stackalloc TSample[MaximumSampleCount]; + Span selectedLumaCoefficients = stackalloc int[MaximumSampleCount]; + Span selectedBlueCoefficients = stackalloc int[MaximumSampleCount]; + Span selectedRedCoefficients = stackalloc int[MaximumSampleCount]; + + Span lumaPrediction = stackalloc TSample[MaximumSampleCount]; + Span lumaResidual = stackalloc short[MaximumSampleCount]; + Span lumaCandidateReconstruction = stackalloc TSample[MaximumSampleCount]; + Span lumaCandidateCoefficients = stackalloc int[MaximumSampleCount]; + Span bluePrediction = stackalloc TSample[MaximumSampleCount]; + Span blueResidual = stackalloc short[MaximumSampleCount]; + Span blueCandidateReconstruction = stackalloc TSample[MaximumSampleCount]; + Span blueCandidateCoefficients = stackalloc int[MaximumSampleCount]; + Span redPrediction = stackalloc TSample[MaximumSampleCount]; + Span redResidual = stackalloc short[MaximumSampleCount]; + Span redCandidateReconstruction = stackalloc TSample[MaximumSampleCount]; + Span redCandidateCoefficients = stackalloc int[MaximumSampleCount]; + Span transformReconstruction = stackalloc TSample[MaximumSampleCount]; + Span transformCoefficients = stackalloc int[MaximumSampleCount]; + + Av1TransformBlockContext lumaContext = Av1TileWriter.GetTransformBlockContexts( + Av1ComponentType.Luminance, + this.picture.LuminanceDcSignLevelCoefficientNeighbors[tileIndex], + blockOrigin, + BlockSize, + LumaTransformSize); + + ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig; + int subsamplingX = colorConfig.SubSamplingX ? 1 : 0; + int subsamplingY = colorConfig.SubSamplingY ? 1 : 0; + Point chromaOrigin = new(blockOrigin.X >> subsamplingX, blockOrigin.Y >> subsamplingY); + Av1TransformSize chromaTransformSize = BlockSize.GetMaxUvTransformSize( + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + Av1TransformBlockContext blueContext = default; + Av1TransformBlockContext redContext = default; + if (!this.source.IsMonochrome) + { + Av1BlockSize chromaBlockSize = BlockSize.GetSubsampled( + colorConfig.SubSamplingX, + colorConfig.SubSamplingY); + + blueContext = Av1TileWriter.GetTransformBlockContexts( + Av1ComponentType.Chroma, + this.picture.CbDcSignLevelCoefficientNeighbors[tileIndex], + chromaOrigin, + chromaBlockSize, + chromaTransformSize); + + redContext = Av1TileWriter.GetTransformBlockContexts( + Av1ComponentType.Chroma, + this.picture.CrDcSignLevelCoefficientNeighbors[tileIndex], + chromaOrigin, + chromaBlockSize, + chromaTransformSize); + } + + for (int candidateIndex = 0; candidateIndex < uniqueCandidateCount; candidateIndex++) + { + Av1MotionVector candidate = candidates[candidateIndex]; + this.EvaluateIntraBlockCopyPlane( + writer, + candidate, + Av1Plane.Y, + Av1ComponentType.Luminance, + blockOrigin, + 0, + 0, + LumaTransformSize, + lumaContext, + lumaPrediction, + lumaResidual, + transformReconstruction, + transformCoefficients, + lumaCandidateReconstruction, + lumaCandidateCoefficients, + out Av1EncoderTransformBlockState lumaCandidateState, + out int lumaRate, + out long lumaDistortion, + out bool hasEmptyLuma, + out Av1EncoderTransformBlockState emptyLumaState, + out long emptyLumaDistortion); + + int blueRate = 0; + int redRate = 0; + long blueDistortion = 0; + long redDistortion = 0; + long emptyBlueDistortion = 0; + long emptyRedDistortion = 0; + bool hasEmptyBlue = true; + bool hasEmptyRed = true; + Av1EncoderTransformBlockState blueCandidateState = default; + Av1EncoderTransformBlockState redCandidateState = default; + Av1EncoderTransformBlockState emptyBlueState = default; + Av1EncoderTransformBlockState emptyRedState = default; + if (!this.source.IsMonochrome) + { + this.EvaluateIntraBlockCopyPlane( + writer, + candidate, + Av1Plane.U, + Av1ComponentType.Chroma, + blockOrigin, + subsamplingX, + subsamplingY, + chromaTransformSize, + blueContext, + bluePrediction, + blueResidual, + transformReconstruction, + transformCoefficients, + blueCandidateReconstruction, + blueCandidateCoefficients, + out blueCandidateState, + out blueRate, + out blueDistortion, + out hasEmptyBlue, + out emptyBlueState, + out emptyBlueDistortion); + + this.EvaluateIntraBlockCopyPlane( + writer, + candidate, + Av1Plane.V, + Av1ComponentType.Chroma, + blockOrigin, + subsamplingX, + subsamplingY, + chromaTransformSize, + redContext, + redPrediction, + redResidual, + transformReconstruction, + transformCoefficients, + redCandidateReconstruction, + redCandidateCoefficients, + out redCandidateState, + out redRate, + out redDistortion, + out hasEmptyRed, + out emptyRedState, + out emptyRedDistortion); + } + + int displacementRate = writer.GetDisplacementVectorCost(candidate, reference); + int candidateRate = writer.GetUseIntraBlockCopyCost(true) + + displacementRate + + writer.GetSkipCost(false, skipContext) + + lumaRate + + blueRate + + redRate; + + long candidateDistortion = lumaDistortion + blueDistortion + redDistortion; + long candidateCost = Av1RateDistortion.GetCost(this.rateMultiplier, candidateRate, candidateDistortion); + bool candidateSkip = false; + if (hasEmptyLuma && hasEmptyBlue && hasEmptyRed) + { + int skipRate = writer.GetUseIntraBlockCopyCost(true) + + displacementRate + + writer.GetSkipCost(true, skipContext); + + long skipDistortion = emptyLumaDistortion + emptyBlueDistortion + emptyRedDistortion; + long skipCost = Av1RateDistortion.GetCost(this.rateMultiplier, skipRate, skipDistortion); + if (skipCost < candidateCost) + { + candidateCost = skipCost; + candidateSkip = true; + } + } + + // Conventional intra and earlier IBC vectors retain strict search-order precedence on equal RD. + if (candidateCost >= bestCost) + { + continue; + } + + bestCost = candidateCost; + hasSelectedCandidate = true; + selectedSkip = candidateSkip; + selectedVector = candidate; + if (candidateSkip) + { + lumaPrediction.CopyTo(selectedLumaReconstruction); + selectedLumaCoefficients.Clear(); + selectedLumaState = emptyLumaState; + if (!this.source.IsMonochrome) + { + int chromaSampleCount = chromaTransformSize.GetSize2d(); + bluePrediction[..chromaSampleCount].CopyTo(selectedBlueReconstruction); + redPrediction[..chromaSampleCount].CopyTo(selectedRedReconstruction); + selectedBlueCoefficients[..chromaSampleCount].Clear(); + selectedRedCoefficients[..chromaSampleCount].Clear(); + selectedBlueState = emptyBlueState; + selectedRedState = emptyRedState; + } + } + else + { + lumaCandidateReconstruction.CopyTo(selectedLumaReconstruction); + lumaCandidateCoefficients.CopyTo(selectedLumaCoefficients); + selectedLumaState = lumaCandidateState; + if (!this.source.IsMonochrome) + { + int chromaSampleCount = chromaTransformSize.GetSize2d(); + blueCandidateReconstruction[..chromaSampleCount].CopyTo(selectedBlueReconstruction); + redCandidateReconstruction[..chromaSampleCount].CopyTo(selectedRedReconstruction); + blueCandidateCoefficients[..chromaSampleCount].CopyTo(selectedBlueCoefficients); + redCandidateCoefficients[..chromaSampleCount].CopyTo(selectedRedCoefficients); + selectedBlueState = blueCandidateState; + selectedRedState = redCandidateState; + } + } + } + + if (!hasSelectedCandidate) + { + return; + } + + Span retainedLumaCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.Y); + Span retainedLumaTransformBlocks = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.Y); + + int lumaTransformIndex = this.codedAreaLuma / + Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + ref Av1EncoderTransformBlockState retainedLumaState = ref retainedLumaTransformBlocks[lumaTransformIndex]; + CopyCandidate( + selectedLumaReconstruction, + selectedLumaCoefficients, + lumaReconstruction, + blockOrigin, + retainedLumaCoefficients[this.codedAreaLuma..], + LumaTransformSize, + selectedLumaState, + ref retainedLumaState); + + if (!this.source.IsMonochrome) + { + Span retainedBlueCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.U); + Span retainedRedCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.V); + Span retainedBlueTransformBlocks = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.U); + + Span retainedRedTransformBlocks = + this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.V); + + int chromaTransformIndex = this.codedAreaChroma / + Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount; + + ref Av1EncoderTransformBlockState retainedBlueState = ref retainedBlueTransformBlocks[chromaTransformIndex]; + ref Av1EncoderTransformBlockState retainedRedState = ref retainedRedTransformBlocks[chromaTransformIndex]; + CopyCandidate( + selectedBlueReconstruction, + selectedBlueCoefficients, + this.reconstruction.GetPlane(Av1Plane.U), + chromaOrigin, + retainedBlueCoefficients[this.codedAreaChroma..], + chromaTransformSize, + selectedBlueState, + ref retainedBlueState); + + CopyCandidate( + selectedRedReconstruction, + selectedRedCoefficients, + this.reconstruction.GetPlane(Av1Plane.V), + chromaOrigin, + retainedRedCoefficients[this.codedAreaChroma..], + chromaTransformSize, + selectedRedState, + ref retainedRedState); + } + + modeInfo.Block.Mode = Av1PredictionMode.DC; + modeInfo.Block.UvMode = Av1ChromaPredictionMode.DC; + modeInfo.Block.Skip = selectedSkip; + modeInfo.Block.UseIntraBlockCopy = true; + block.FilterIntraMode = Av1FilterIntraMode.AllFilterIntraModes; + block.PredictionUnit.AngleDelta[(int)Av1PlaneType.Y] = 0; + block.PredictionUnit.AngleDelta[(int)Av1PlaneType.Uv] = 0; + block.PredictionUnit.ChromaFromLumaIndex = 0; + block.PredictionUnit.ChromaFromLumaSigns = 0; + paletteInfo = default; + this.picture.SetDisplacementVector(modeInfoPosition, selectedVector); + } + + private void EvaluateIntraBlockCopyPlane( + Av1SymbolEncoder writer, + Av1MotionVector vector, + Av1Plane plane, + Av1ComponentType componentType, + Point lumaOrigin, + int subsamplingX, + int subsamplingY, + Av1TransformSize transformSize, + Av1TransformBlockContext blockContext, + Span prediction, + Span residual, + Span transformReconstruction, + Span transformCoefficients, + Span selectedReconstruction, + Span selectedCoefficients, + out Av1EncoderTransformBlockState selectedState, + out int selectedRate, + out long selectedDistortion, + out bool hasEmptyTransform, + out Av1EncoderTransformBlockState emptyState, + out long emptyDistortion) + { + Point planeOrigin = new(lumaOrigin.X >> subsamplingX, lumaOrigin.Y >> subsamplingY); + int sourceColumnQ4 = (planeOrigin.X << 4) + (vector.Column << (1 - subsamplingX)); + int sourceRowQ4 = (planeOrigin.Y << 4) + (vector.Row << (1 - subsamplingY)); + Point predictionOrigin = new(sourceColumnQ4 >> 4, sourceRowQ4 >> 4); + int sampleCount = transformSize.GetSize2d(); + Buffer2DRegion sourcePlane = this.source.GetPlane(plane); + Buffer2DRegion reconstructionPlane = this.reconstruction.GetPlane(plane); + TOperator.PrepareIntraBlockCopy( + sourcePlane, + planeOrigin, + reconstructionPlane, + predictionOrigin, + (sourceColumnQ4 & 15) != 0, + (sourceRowQ4 & 15) != 0, + prediction[..sampleCount], + residual[..sampleCount], + transformSize); + + Av1TransformSetType transformSetType = Av1SymbolContextHelper.GetExtendedTransformSetType( + transformSize, + isInter: true, + this.picture.Parent.FrameHeader.UseReducedTransformSet); + + long bestCost = long.MaxValue; + selectedState = default; + selectedRate = 0; + selectedDistortion = 0; + hasEmptyTransform = false; + emptyState = default; + emptyDistortion = 0; + for (Av1TransformType transformType = Av1TransformType.DctDct; + transformType < Av1TransformType.AllTransformTypes; + transformType++) + { + if (!transformType.IsExtendedSetUsed(transformSetType)) + { + continue; + } + + Av1EncoderTransformBlockState candidateState = default; + long candidateDistortion = TOperator.EncodePredictionCandidate( + this.blockWorkspace, + sourcePlane, + planeOrigin, + prediction[..sampleCount], + residual[..sampleCount], + transformReconstruction[..sampleCount], + transformCoefficients[..sampleCount], + transformSize, + transformType, + plane, + this.quantization.QIndex[0], + this.quantization.DeltaQDc[(int)plane], + this.quantization.DeltaQAc[(int)plane], + this.bitDepth, + ref candidateState); + + int candidateRate = writer.GetCoefficientCost( + transformSize, + transformType, + Av1PredictionMode.DC, + transformCoefficients[..sampleCount], + componentType, + blockContext, + candidateState.EndOfBlock, + this.picture.Parent.FrameHeader.UseReducedTransformSet, + Av1FilterIntraMode.AllFilterIntraModes, + usesInterTransformSet: true); + + long candidateCost = Av1RateDistortion.GetCost( + this.rateMultiplier, + candidateRate, + candidateDistortion); + + if (candidateCost < bestCost) + { + transformReconstruction[..sampleCount].CopyTo(selectedReconstruction); + transformCoefficients[..sampleCount].CopyTo(selectedCoefficients); + bestCost = candidateCost; + selectedState = candidateState; + selectedRate = candidateRate; + selectedDistortion = candidateDistortion; + } + + if (candidateState.EndOfBlock == 0 && + (!hasEmptyTransform || candidateDistortion < emptyDistortion)) + { + hasEmptyTransform = true; + emptyState = candidateState; + emptyDistortion = candidateDistortion; + } + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ModeDecision.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ModeDecision.cs index 5cfa4acbde..6ade0c1715 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ModeDecision.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ModeDecision.cs @@ -195,30 +195,49 @@ internal static partial class Av1IntraSuperblockEncoder ref lumaState, ref paletteInfo, out int lumaAngleDelta, - out Av1FilterIntraMode filterIntraMode); + out Av1FilterIntraMode filterIntraMode, + out long lumaCost); block.PredictionUnit.AngleDelta[(int)Av1PlaneType.Y] = (sbyte)lumaAngleDelta; block.FilterIntraMode = filterIntraMode; - this.codedAreaLuma += LumaTransformSize.GetSize2d(); bool lumaTransformEmpty = lumaState.EndOfBlock == 0; if (this.source.IsMonochrome) { + int emptyTransformRate = lumaTransformEmpty + ? this.GetEmptyTransformRate( + writer, + this.picture.LuminanceDcSignLevelCoefficientNeighbors[tileIndex], + Av1ComponentType.Luminance, + blockOrigin, + BlockSize, + LumaTransformSize, + lumaState.TransformType, + modeInfo.Block.Mode, + block.FilterIntraMode) + : 0; + modeInfo.Block.Skip = lumaTransformEmpty && Av1TileWriter.ShouldSkipCoefficients( writer, Av1TileWriter.GetSkipContext(macroBlock), - this.GetEmptyTransformRate( - writer, - this.picture.LuminanceDcSignLevelCoefficientNeighbors[tileIndex], - Av1ComponentType.Luminance, - blockOrigin, - BlockSize, - LumaTransformSize, - lumaState.TransformType, - modeInfo.Block.Mode, - block.FilterIntraMode)); + emptyTransformRate); + if (this.picture.Parent.FrameHeader.AllowIntraBlockCopy) + { + this.SelectIntraBlockCopy( + writer, + macroBlock, + blockOrigin, + tileIndex, + lumaCost, + emptyTransformRate, + ref modeInfo, + ref block, + ref paletteInfo); + } + + this.codedAreaLuma += LumaTransformSize.GetSize2d(); return; } @@ -258,20 +277,22 @@ internal static partial class Av1IntraSuperblockEncoder ref paletteInfo, out int chromaAngleDelta, out byte chromaFromLumaIndex, - out sbyte chromaFromLumaSigns); + out sbyte chromaFromLumaSigns, + out long chromaCost); block.PredictionUnit.AngleDelta[(int)Av1PlaneType.Uv] = (sbyte)chromaAngleDelta; block.PredictionUnit.ChromaFromLumaIndex = chromaFromLumaIndex; block.PredictionUnit.ChromaFromLumaSigns = chromaFromLumaSigns; bool allTransformsEmpty = lumaTransformEmpty && blueState.EndOfBlock == 0 && redState.EndOfBlock == 0; + int regularEmptyTransformRate = 0; if (allTransformsEmpty) { Av1BlockSize chromaBlockSize = BlockSize.GetSubsampled( colorConfig.SubSamplingX, colorConfig.SubSamplingY); - int emptyTransformRate = this.GetEmptyTransformRate( + regularEmptyTransformRate = this.GetEmptyTransformRate( writer, this.picture.LuminanceDcSignLevelCoefficientNeighbors[tileIndex], Av1ComponentType.Luminance, @@ -282,7 +303,7 @@ internal static partial class Av1IntraSuperblockEncoder modeInfo.Block.Mode, block.FilterIntraMode); - emptyTransformRate += this.GetEmptyTransformRate( + regularEmptyTransformRate += this.GetEmptyTransformRate( writer, this.picture.CbDcSignLevelCoefficientNeighbors[tileIndex], Av1ComponentType.Chroma, @@ -293,7 +314,7 @@ internal static partial class Av1IntraSuperblockEncoder modeInfo.Block.Mode, Av1FilterIntraMode.AllFilterIntraModes); - emptyTransformRate += this.GetEmptyTransformRate( + regularEmptyTransformRate += this.GetEmptyTransformRate( writer, this.picture.CrDcSignLevelCoefficientNeighbors[tileIndex], Av1ComponentType.Chroma, @@ -307,9 +328,24 @@ internal static partial class Av1IntraSuperblockEncoder modeInfo.Block.Skip = Av1TileWriter.ShouldSkipCoefficients( writer, Av1TileWriter.GetSkipContext(macroBlock), - emptyTransformRate); + regularEmptyTransformRate); } + if (this.picture.Parent.FrameHeader.AllowIntraBlockCopy) + { + this.SelectIntraBlockCopy( + writer, + macroBlock, + blockOrigin, + tileIndex, + lumaCost + chromaCost, + regularEmptyTransformRate, + ref modeInfo, + ref block, + ref paletteInfo); + } + + this.codedAreaLuma += LumaTransformSize.GetSize2d(); this.codedAreaChroma += chromaTransformSize.GetSize2d(); } @@ -352,7 +388,8 @@ internal static partial class Av1IntraSuperblockEncoder ref Av1EncoderTransformBlockState retainedState, ref Av1EncoderPaletteInfo paletteInfo, out int selectedAngleDelta, - out Av1FilterIntraMode selectedFilterIntraMode) + out Av1FilterIntraMode selectedFilterIntraMode, + out long selectedCost) { const Av1BlockSize BlockSize = Av1BlockSize.Block8x8; const Av1TransformSize TransformSize = Av1TransformSize.Size8x8; @@ -695,6 +732,7 @@ internal static partial class Av1IntraSuperblockEncoder selectedFilterIntraMode = Av1FilterIntraMode.AllFilterIntraModes; } + selectedCost = bestTransformCost; return bestMode; } diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.Operator.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.Operator.cs index 1e8c6d6705..80ddb3ca35 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.Operator.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.Operator.cs @@ -6,6 +6,7 @@ using System.Runtime.Intrinsics; using SixLabors.ImageSharp.Formats.Heif.Av1.Motion; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.ChromaFromLuma; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.IntraBlockCopy; using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; using SixLabors.ImageSharp.Memory; @@ -215,6 +216,29 @@ internal static partial class Av1IntraSuperblockEncoder Av1TransformSize transformSize, Av1BitDepth bitDepth); + /// + /// Builds an intra-block-copy prediction and the matching source residual. + /// + /// The coded source plane. + /// The destination block origin in plane samples. + /// The reconstructed plane containing the reference samples. + /// The integer reference origin preceding any half-sample phase. + /// Indicates whether the horizontal source phase is one half-sample. + /// Indicates whether the vertical source phase is one half-sample. + /// The contiguous prediction destination. + /// The contiguous source-minus-prediction destination. + /// The prediction dimensions. + public static abstract void PrepareIntraBlockCopy( + Buffer2DRegion source, + Point blockOrigin, + Buffer2DRegion reconstruction, + Point predictionOrigin, + bool halfX, + bool halfY, + Span prediction, + Span residual, + Av1TransformSize transformSize); + /// /// Encodes one prepared prediction with the selected transform into decision scratch. /// @@ -715,6 +739,41 @@ internal static partial class Av1IntraSuperblockEncoder height); } + /// + public static void PrepareIntraBlockCopy( + Buffer2DRegion source, + Point blockOrigin, + Buffer2DRegion reconstruction, + Point predictionOrigin, + bool halfX, + bool halfY, + Span prediction, + Span residual, + Av1TransformSize transformSize) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + Av1IntraBlockCopyPredictor.Predict( + Av1TransformBlockEncoder.GetPlaneSpan(reconstruction, predictionOrigin), + reconstruction.Stride, + prediction, + width, + width, + height, + halfX, + halfY); + + Av1ResidualBuilder.Subtract( + Av1TransformBlockEncoder.GetPlaneSpan(source, blockOrigin), + source.Stride, + prediction, + width, + residual, + width, + width, + height); + } + /// public static long EncodePredictionCandidate( Av1EncoderBlockWorkspace workspace, @@ -1197,6 +1256,41 @@ internal static partial class Av1IntraSuperblockEncoder height); } + /// + public static void PrepareIntraBlockCopy( + Buffer2DRegion source, + Point blockOrigin, + Buffer2DRegion reconstruction, + Point predictionOrigin, + bool halfX, + bool halfY, + Span prediction, + Span residual, + Av1TransformSize transformSize) + { + int width = transformSize.GetWidth(); + int height = transformSize.GetHeight(); + Av1IntraBlockCopyPredictor.Predict( + MemoryMarshal.Cast(Av1TransformBlockEncoder.GetPlaneSpan(reconstruction, predictionOrigin)), + reconstruction.Stride, + MemoryMarshal.Cast(prediction), + width, + width, + height, + halfX, + halfY); + + Av1ResidualBuilder.Subtract( + Av1TransformBlockEncoder.GetPlaneSpan(source, blockOrigin), + source.Stride, + prediction, + width, + residual, + width, + width, + height); + } + /// public static long EncodePredictionCandidate( Av1EncoderBlockWorkspace workspace, diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraSuperblockEncoderTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraSuperblockEncoderTests.cs index 4b02892b9c..cefefc60d4 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraSuperblockEncoderTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraSuperblockEncoderTests.cs @@ -1864,6 +1864,290 @@ public class Av1IntraSuperblockEncoderTests Assert.NotEqual(0, tileWriter.GetTileData(0).Length); } + [Fact] + public void ProductionTileSelectsIntraBlockCopyByFullRateDistortion() + { + VerifyProductionTileSelectsIntraBlockCopy( + Av1BitDepth.EightBit, + 8, + static value => (byte)value, + static (source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new Av1IntraTileWriter( + Configuration.Default, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + initialSize: 4096)); + + VerifyProductionTileSelectsIntraBlockCopy( + Av1BitDepth.TwelveBit, + 12, + static value => (ushort)(value << 4), + static (source, reconstruction, picture, coefficients, superblockWorkspace, blockWorkspace) => + new Av1IntraTileWriter( + Configuration.Default, + source, + reconstruction, + picture, + coefficients, + superblockWorkspace, + blockWorkspace, + initialSize: 4096)); + } + + private static void VerifyProductionTileSelectsIntraBlockCopy( + Av1BitDepth bitDepth, + int bitDepthValue, + SampleFactory createSample, + TileWriterFactory createTileWriter) + where TSample : unmanaged + { + const int Width = 328; + const int Height = 8; + const int QIndex = 1; + const int ReferenceColumn = 0; + const int TargetColumn = 320; + ObuColorConfig colorConfig = new() + { + IsMonochrome = true, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = bitDepth + }; + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + bitDepthValue, + Av1ColorFormat.Yuv400, + 0, + 0); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + bitDepthValue, + Av1ColorFormat.Yuv400, + 0, + 0); + + Buffer2DRegion sourcePlane = source.Frame.CodedView.GetPlane(Av1Plane.Y); + for (int row = 0; row < Height; row++) + { + Span sourceRow = sourcePlane.DangerousGetRowSpan(row); + for (int column = 0; column < Width; column++) + { + sourceRow[column] = createSample(17 + (((column * 29) + (row * 43)) % 211)); + } + + for (int column = 0; column < 8; column++) + { + // The repeated high-contrast block has one legal hash match five completed 64-pixel regions earlier. + TSample sample = createSample(((column * 73) + (row * 109) + (((column + row) & 1) * 127)) & 255); + sourceRow[ReferenceColumn + column] = sample; + sourceRow[TargetColumn + column] = sample; + } + } + + ClearPlane(reconstruction.Luma); + using Av1EncoderModeInfoBuffer modeInfo = new( + Configuration.Default, + Width, + Height, + disallow4x4AllFrames: true); + + Av1PictureControlSet pictureTemplate = CreatePicture( + modeInfo, + colorConfig, + use128x128Superblock: false, + QIndex); + + pictureTemplate.Parent.FrameHeader.AllowScreenContentTools = true; + pictureTemplate.Parent.FrameHeader.AllowIntraBlockCopy = true; + pictureTemplate.Parent.FrameHeader.FrameSize.FrameWidth = Width; + pictureTemplate.Parent.FrameHeader.FrameSize.FrameHeight = Height; + using Av1EncoderPictureBuffer picture = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + pictureTemplate.Parent.FrameHeader, + Width, + Height); + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + using Av1IntraTileWriter tileWriter = createTileWriter( + source.Frame, + reconstruction.Frame, + picture.Picture, + coefficients, + superblockWorkspace, + blockWorkspace); + + Point targetModeInfoPosition = new(TargetColumn >> Av1Constants.ModeInfoSizeLog2, 0); + ref Av1MacroBlockModeInfo targetMode = ref picture.Picture.GetMacroBlockModeInfo(targetModeInfoPosition); + Assert.True(targetMode.Block.UseIntraBlockCopy); + Assert.Equal(Av1PredictionMode.DC, targetMode.Block.Mode); + Assert.Equal(Av1ChromaPredictionMode.DC, targetMode.Block.UvMode); + var displacementVector = picture.Picture.GetDisplacementVector(targetModeInfoPosition); + Assert.Equal(0, displacementVector.Row); + Assert.Equal((ReferenceColumn - TargetColumn) * 8, displacementVector.Column); + Assert.Equal(Av1FilterIntraMode.AllFilterIntraModes, superblockWorkspace.FinalBlocks[0].FilterIntraMode); + Assert.Equal(0, superblockWorkspace.PaletteInfo.PaletteSizes[0]); + Assert.NotEqual(0, tileWriter.GetTileData(0).Length); + } + + [Fact] + public void ProductionTileRetainsHalfSampleChromaIntraBlockCopy() + { + const int Width = 328; + const int Height = 8; + const int QIndex = 1; + const int ReferenceColumn = 1; + const int TargetColumn = 320; + ObuColorConfig colorConfig = new() + { + IsMonochrome = false, + SubSamplingX = true, + SubSamplingY = true, + BitDepth = Av1BitDepth.EightBit + }; + + using Av1EncoderFrameBuffer source = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv420, + 1, + 1); + + using Av1EncoderFrameBuffer reconstruction = new( + Configuration.Default, + Width, + Height, + 8, + Av1ColorFormat.Yuv420, + 1, + 1); + + Buffer2DRegion lumaSource = source.Frame.CodedView.GetPlane(Av1Plane.Y); + for (int row = 0; row < Height; row++) + { + Span lumaRow = lumaSource.DangerousGetRowSpan(row); + for (int column = 0; column < Width; column++) + { + lumaRow[column] = (byte)(23 + (((column * 31) + (row * 47)) % 197)); + } + + for (int column = 0; column < 8; column++) + { + byte sample = (byte)(((column * 79) + (row * 113) + (((column + row) & 1) * 127)) & 255); + lumaRow[ReferenceColumn + column] = sample; + lumaRow[TargetColumn + column] = sample; + } + } + + int chromaTargetColumn = TargetColumn >> 1; + Buffer2DRegion blueSource = source.Frame.CodedView.GetPlane(Av1Plane.U); + Buffer2DRegion redSource = source.Frame.CodedView.GetPlane(Av1Plane.V); + for (int row = 0; row < Height >> 1; row++) + { + Span blueRow = blueSource.DangerousGetRowSpan(row); + Span redRow = redSource.DangerousGetRowSpan(row); + for (int column = 0; column < Width >> 1; column++) + { + blueRow[column] = (byte)(32 + (((column * 17) + (row * 29)) % 160)); + redRow[column] = (byte)(40 + (((column * 23) + (row * 37)) % 152)); + } + + for (int column = 0; column < 4; column++) + { + // An odd luma displacement maps 4:2:0 chroma between adjacent reference samples. + blueRow[chromaTargetColumn + column] = (byte)((blueRow[column] + blueRow[column + 1] + 1) >> 1); + redRow[chromaTargetColumn + column] = (byte)((redRow[column] + redRow[column + 1] + 1) >> 1); + } + } + + ClearPlane(reconstruction.Luma); + ClearPlane(Assert.IsType>(reconstruction.ChromaBlue)); + ClearPlane(Assert.IsType>(reconstruction.ChromaRed)); + using Av1EncoderModeInfoBuffer modeInfo = new( + Configuration.Default, + Width, + Height, + disallow4x4AllFrames: true); + + Av1PictureControlSet pictureTemplate = CreatePicture( + modeInfo, + colorConfig, + use128x128Superblock: false, + QIndex); + + pictureTemplate.Parent.FrameHeader.AllowScreenContentTools = true; + pictureTemplate.Parent.FrameHeader.AllowIntraBlockCopy = true; + pictureTemplate.Parent.FrameHeader.FrameSize.FrameWidth = Width; + pictureTemplate.Parent.FrameHeader.FrameSize.FrameHeight = Height; + using Av1EncoderPictureBuffer picture = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + pictureTemplate.Parent.FrameHeader, + Width, + Height); + + using Av1EncoderCoefficientBuffer coefficients = new( + Configuration.Default, + pictureTemplate.Sequence.SequenceHeader, + Width, + Height); + + using Av1EncoderSuperblockWorkspace superblockWorkspace = new(Configuration.Default); + using Av1EncoderBlockWorkspace blockWorkspace = new(Configuration.Default); + using Av1IntraTileWriter tileWriter = new( + Configuration.Default, + source.Frame, + reconstruction.Frame, + picture.Picture, + coefficients, + superblockWorkspace, + blockWorkspace, + initialSize: 4096); + + Point targetModeInfoPosition = new(TargetColumn >> Av1Constants.ModeInfoSizeLog2, 0); + ref Av1MacroBlockModeInfo targetMode = ref picture.Picture.GetMacroBlockModeInfo(targetModeInfoPosition); + Assert.True(targetMode.Block.UseIntraBlockCopy); + var displacementVector = picture.Picture.GetDisplacementVector(targetModeInfoPosition); + Assert.Equal(0, displacementVector.Row); + Assert.Equal((ReferenceColumn - TargetColumn) * 8, displacementVector.Column); + Assert.Equal(8, displacementVector.Column & 15); + Av1TransformSetType interTransformSet = Av1SymbolContextHelper.GetExtendedTransformSetType( + Av1TransformSize.Size4x4, + isInter: true, + useReducedSet: false); + + Assert.True(coefficients.GetTransformBlockSpan(5, Av1Plane.U)[0].TransformType.IsExtendedSetUsed(interTransformSet)); + Assert.True(coefficients.GetTransformBlockSpan(5, Av1Plane.V)[0].TransformType.IsExtendedSetUsed(interTransformSet)); + Assert.NotEqual( + (byte)0, + reconstruction.Frame.CodedView.GetPlane(Av1Plane.U).DangerousGetRowSpan(0)[chromaTargetColumn]); + + Assert.NotEqual( + (byte)0, + reconstruction.Frame.CodedView.GetPlane(Av1Plane.V).DangerousGetRowSpan(0)[chromaTargetColumn]); + + Assert.NotEqual(0, tileWriter.GetTileData(0).Length); + } + [Fact] public void TileWriterMapsClippedRasterTraversalToEverySuperblockCoefficientSegment() { @@ -2263,6 +2547,9 @@ public class Av1IntraSuperblockEncoderTests Av1EncoderBlockWorkspace blockWorkspace) where TSample : unmanaged; + private delegate TSample SampleFactory(int value) + where TSample : unmanaged; + private delegate void FilterPrediction( Av1FilterIntraMode mode, Span destination,