diff --git a/HEIF_IMPLEMENTATION_PLAN.md b/HEIF_IMPLEMENTATION_PLAN.md index 2b086a603..36f31f7ef 100644 --- a/HEIF_IMPLEMENTATION_PLAN.md +++ b/HEIF_IMPLEMENTATION_PLAN.md @@ -29,7 +29,7 @@ Checkboxes may be marked complete only when the implementation and the verificat ## Delivery dashboard -Last reconciled with the source tree on 2026-08-29 against production checkpoint `864b11ad9791d3254a38691246270c569001b5a1`. Committed checkpoints include the AV1 transform architecture, OBU framing, intra-block copy, 12-profile reconstruction matrix, layered-item properties, layered reference/header/CDF/motion-field state, inter-frame intra blocks, SIMD-first translational prediction, complete single-reference inter reconstruction, compound reference trees and modes, paired reference-MV derivation, reference-dependent bounded sequence decoding, allocation-free SIMD-first equal averaging, selected inter-intra prediction, and selectable distance-weighted, wedge, and difference-weighted compound blending. Four pinned bounded sequences pass exact native-plane and presentation comparison, normative syntax and production reconstruction pass directly at 8/10/12 bit, SIMD-first mask and blend paths pass through `FeatureTestRunner`, and constrained allocation returns every tracked buffer exactly once. Both source targets build with zero warnings and errors; the `net10.0` test project builds with zero errors and its 1,013 pre-existing warnings; 25 direct focused cases and six `FeatureTestRunner` cases pass without failures or skips; the documentation review is complete; and `git diff --check` is clean. This dashboard is the authoritative delivery order. The detailed phase checklists below provide subsystem evidence; they do not override the current-stage marker or permit work to skip ahead. +Last reconciled with the source tree on 2026-08-29 against committed production checkpoint `59a4fbad5ed3bc7c34331c4bdc65acc7acde3f04` plus the locally complete scaled-reference checkpoint awaiting commit. Committed checkpoints include the AV1 transform architecture, OBU framing, intra-block copy, 12-profile reconstruction matrix, layered-item properties, layered reference/header/CDF/motion-field state, inter-frame intra blocks, SIMD-first translational prediction, complete single-reference inter reconstruction, compound reference trees and modes, paired reference-MV derivation, reference-dependent bounded sequence decoding, allocation-free SIMD-first equal averaging, selected inter-intra prediction, selectable compound blending, and OBMC. The local checkpoint adds exact Q14 reference scaling, variable-phase Q10 separable convolution, size-changing production reconstruction, scaled OBMC neighbors, the normative 288-sample retained border, and per-block inter deblocking decisions. Its pinned 40x40-to-80x80 bounded AVIF passes exact base and final native-plane comparison, exact final presentation comparison, normal and scalar `FeatureTestRunner` execution, constrained allocation, and collision-free exactly-once ownership tracking. Both source targets build with zero warnings and errors; the `net10.0` test project builds with zero errors and its 1,013 pre-existing warnings; 87 focused `net10.0` cases pass without failures or skips; and the documentation and diff checks are complete. This dashboard is the authoritative delivery order. The detailed phase checklists below provide subsystem evidence; they do not override the current-stage marker or permit work to skip ahead. Status meanings: @@ -39,15 +39,15 @@ Status meanings: - **Not started:** supporting primitives may exist, but the production format path is absent. - **Current:** the only work item that should be advanced before taking the next queued item. -Current development stage: **Stage 3 — complete AV1 still-image decoding.** The decoder retains reference/header/CDF/motion-field state, derives frame-level skip-mode references, consumes temporal segment prediction, decodes intra-coded blocks inside inter frames, reconstructs single-reference NEAREST/NEAR/NEW/GLOBAL translational blocks, and reconstructs equal-average, distance-weighted, wedge, difference-weighted, and inter-intra prediction before residual traversal. Commit `864b11ad9791d3254a38691246270c569001b5a1` records the selectable-compound checkpoint. OBMC is current; warped motion, scaled references, and non-translational global prediction remain explicitly unsupported. Neither AV1 nor HEVC production encoding is implemented. +Current development stage: **Stage 3 — complete AV1 still-image decoding.** The decoder retains reference/header/CDF/motion-field state, derives frame-level skip-mode references, consumes temporal segment prediction, decodes intra-coded blocks inside inter frames, and reconstructs translational single-reference, compound, inter-intra, OBMC, and scaled-reference prediction before residual traversal. The scaled-reference implementation and independent evidence are complete locally; its checkpoint commit is the only remaining gate before warped motion becomes current. Warped motion and non-translational global prediction remain explicitly unsupported. Neither AV1 nor HEVC production encoding is implemented. -Immediate checkpoint: **implement and independently verify OBMC before advancing to scaled references.** Preserve the committed translational and compound-prediction contracts, keep predictor workspaces allocator-owned and allocation-free per block, and require independently encoded bounded AVIF content with exact native-plane comparison for the production OBMC branch. This work remains inside the existing image-item and bounded image-sequence surfaces and must not expand into a general ISO BMFF/video model. +Immediate checkpoint: **commit the complete scaled-reference implementation and its verified evidence before advancing to warped motion.** The production branch, independent native and presentation comparisons, constrained allocator coverage, Release builds, focused tests, documentation review, and diff validation are complete. This work remains inside the existing image-item and bounded image-sequence surfaces and does not expand into a general ISO BMFF/video model. | Order | Delivery stage | State | Delivered state | Gate that remains open | | --- | --- | --- | --- | --- | | 1 | Baseline, provenance, documentation, and public contract | In progress | Pinned codec references, a bounded image-only scope, encoder options, typed bit depth, decoder-option propagation, and extensive HEIF documentation exist. | Complete the all-file documentation audit, record a fresh Release baseline, finish distinct public HEIC/AVIF save boundaries, and close API review. | | 2 | Bounded HEIF item and image-sequence container | In progress | Still-item parsing, grids, auxiliary alpha, metadata properties, bounded image-sequence tracks, Identify, and all-sync AV1 sequence presentation are connected. | Complete adversarial boundary coverage, remaining item/property behavior, reference-dependent sequence reconstruction, and the bounded sequence writer. | -| 3 | Still-image AV1 and HEVC decoding | **Current** | HEVC reconstruction reaches exact HM/libheif fixtures across the recorded 8/10/12-bit and chroma cases. AV1 work includes bounded OBU framing, reconstruction, filters, grain, color, transforms, intra-block copy, an exact independent 12-profile bit-depth/chroma matrix through every dispatch tier, retained layered reference/header/CDF/motion-field state, temporal segment prediction, inter-frame intra blocks, SIMD-first translational prediction, exact single-reference inter reconstruction, exact equal-average compound reconstruction, and committed selectable compound and inter-intra reconstruction through reference-dependent bounded sequences. | Implement OBMC, scaled-reference, warped, and non-translational global prediction; remove every other valid AV1 still-image unsupported branch with independent vectors; then complete the remaining HEVC profile and Range Extensions matrix. | +| 3 | Still-image AV1 and HEVC decoding | **Current** | HEVC reconstruction reaches exact HM/libheif fixtures across the recorded 8/10/12-bit and chroma cases. AV1 work includes bounded OBU framing, reconstruction, filters, grain, color, transforms, intra-block copy, an exact independent 12-profile bit-depth/chroma matrix through every dispatch tier, retained layered reference/header/CDF/motion-field state, temporal segment prediction, inter-frame intra blocks, SIMD-first translational prediction, exact single-reference inter reconstruction, exact equal-average and selectable compound reconstruction, inter-intra prediction, committed OBMC, and a locally complete independently verified scaled-reference branch. | Commit scaled-reference, then implement warped and non-translational global prediction; remove every other valid AV1 still-image unsupported branch with independent vectors; then complete the remaining HEVC profile and Range Extensions matrix. | | 4 | Complete decoded presentation and animation | In progress | Shared SIMD-first AV1/HEVC color conversion, ICC application, grids, transforms, direct planar alpha composition, frame metadata, repetition, and independently decodable AV1 sequence samples exist. | Close the full color/ICC cross-product, HEVC sequence decoding, AV1/HEVC reference-dependent samples, frame-local metadata/alpha behavior, and independent animated decode vectors. | | 5 | AV1/AVIF encoding | Not started | RGB-to-planar conversion, forward transforms, OBU writer foundations, options, and container-writing infrastructure exist. | `HeifEncoderCore` still rejects AV1. Implement a real independently decodable lossy/lossless AV1 payload and the complete AVIF item/metadata matrix. | | 6 | HEVC/HEIC encoding | Not started | Shared input color conversion, options, and HEIF writer infrastructure exist. | `HeifEncoderCore` still rejects HEVC. Implement a real independently decodable lossy/lossless HEVC payload and the complete HEIC item/metadata matrix. | @@ -90,7 +90,9 @@ Immediate checkpoint: **implement and independently verify OBMC before advancing - [x] Decode all compound reference-pair trees and inter modes, retain paired spatial/temporal reference-MV candidates through DRL selection, and reconstruct two translational predictors with allocation-free equal-weight averaging across 8/10/12-bit samples without changing the single-reference predictor contract or rounding model. - [x] Verify the equal-average production branch with the pinned 19-frame libavif YUV444 sequence. The final native Y, U, and V planes match pinned libaom exactly, final RGBA presentation matches pinned libavif exactly, production `DecodeBlock()` is covered at 8/10/12 bit, constrained plane allocation passes, and allocator tracking returns every retained and temporal motion-field buffer exactly once. - [x] Decode selected inter-intra prediction and selectable distance-weighted, wedge, and difference-weighted compound blending. Normative-order syntax, direct 8/10/12-bit production reconstruction, SIMD/scalar arithmetic, four pinned bounded sequences, exact final native planes and presentation, constrained contiguous planes, and balanced allocator returns pass. Both source targets build with zero warnings and errors; the test project builds with zero errors and pre-existing repository warnings; 25 direct focused cases and six `FeatureTestRunner` cases pass without failures or skips; and `git diff --check` is clean. - - [ ] Decode OBMC, scaled references, warped motion, and non-translational global-motion prediction. + - [x] Decode and reconstruct above and left OBMC through allocator-owned prediction workspaces, exact pinned masks, and the existing SIMD-first masked blender. Commit `59a4fbad5ed3bc7c34331c4bdc65acc7acde3f04` records direct 8/10/12-bit production reconstruction, 4:2:0 and 4:2:2 overlap geometry, a real 19-frame YUV444 sequence containing more than one hundred traced OBMC blocks, exact final pinned-libaom native planes, exact pinned-libavif presentation through normal and scalar `FeatureTestRunner` configurations, constrained tracked allocation, zero-warning `net10.0` and `net11.0` source builds, a zero-error analyzer build with 1,013 pre-existing warnings, 14 direct focused cases plus the feature-isolated comparison, and clean staged `git diff --check`. + - [x] Decode and independently verify scaled-reference prediction through exact Q14/Q10 arithmetic, allocation-free SIMD/scalar convolution, 8/10/12-bit production reconstruction, scaled OBMC, active inter deblocking, constrained allocation, and a genuine size-changing bounded AVIF. + - [ ] **Queued until the scaled-reference commit:** decode warped motion, then non-translational global-motion prediction. - [ ] Verify every remaining connected mode and filter with independently encoded bounded AV1 content and exact native-plane comparisons. - [ ] Return the explicitly selected spatial layer or the final displayed layer, keeping reference reconstruction separate from display-only film grain. - [ ] Verify color and auxiliary-alpha output exactly against both pinned libavif progressive fixtures under normal SIMD dispatch and all required `FeatureTestRunner` fallbacks. @@ -306,8 +308,9 @@ This snapshot pins or classifies the available references and failures; it does | `ObuReader.ReadSequenceHeader`, `ReadUncompressedFrameHeader`, decoder-model parsing, and operating-parameter consumption | AV1 sections 5.5.2 through 5.5.4 sequence timing and decoder-model syntax, section 5.9.2 uncompressed frame-header syntax, and section 5.9.31 temporal-point syntax | libaom `av1/decoder/decodeframe.c` functions `av1_read_decoder_model_info`, `av1_read_op_parameters_info`, `read_temporal_point_info`, and `read_uncompressed_header`, plus `common/av1_config.c`, at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Read the normative 32-bit decoding-tick field; consume operating-point buffer delays and the layer-applicable frame removal times needed to keep a non-reduced still-image sequence aligned; and read presentation time only under the normative decoder-model condition. Retain no scheduling or playback behavior from those values and introduce no ISO BMFF timing, track, sample-table, or sequence surface. | | `Av1FrameBuffer` high-bit-depth sample layout and `Av1YuvConverter` 10/12-bit packed-pixel conversion | AV1 section 6.4.1 bit depth and H.273 sample-range scaling | libaom `aom_scale/yv12config.h`, `av1/common/idct.c`, and `av1/common/reconintra.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f`; libavif `src/avif.c` and `src/reformat.c` at `092276ce89098ead06db80975173191e5fee1826` | Establish two-byte native sample storage with sample-unit strides for 10/12-bit reconstruction and use ImageSharp's existing `Rgb48` pixel-operation paths in both directions so packed-pixel staging does not reduce high-bit-depth samples to eight bits. | | `Av1PredictionDecoder`, `Av1IntraPredictor`, `Av1ChromaFromLumaContext`, `Av1ChromaFromLumaPredictor`, `Av1PartitionInfo`, and the DC, directional, Paeth, smooth, filter-intra, and chroma-from-luma predictor operators | AV1 sections 7.11.2 and 7.11.2.3 intra prediction | libaom `aom_dsp/intrapred.c`, `av1/common/reconintra.c`, `av1/common/av1_common_int.h`, `av1/common/blockd.h`, `av1/common/cfl.c`, and `av1/common/cfl.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Design the shared predictor contract, sample layout, reference preparation, and scratch ownership for widest-to-narrowest SIMD traversal, then provide the equivalent scalar fallback through that contract. Preserve luma/chroma mode-neighbor addressing, directional upsampling, Paeth selection, smooth normalization, filter-intra taps, high-bit-depth clipping, chroma-from-luma storage/subsampling, and chroma-from-luma row strides. The WIP rectangular byte-pipeline smooth digest expectations encode width/height-swapped weights and must be replaced only from an independently generated oracle, not regenerated from this implementation. | +| `Av1ReferenceScale`, `Av1InterPredictor` scaled operators, `Av1BlockDecoder` scaled-reference traversal, and `Av1FrameBuffer` prediction border | AV1 sections 7.11.3.3 reference scaling and 7.11.3.4 block inter prediction | libaom `av1/common/scale.c`, `av1/common/scale.h`, `av1/common/reconinter.h`, `av1/decoder/decodeframe.c`, `av1/common/convolve.c`, `av1/common/filter.h`, and `aom_scale/yv12config.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Preserve Q14 reference scale factors, Q10 variable phases and steps, signed coordinate rounding, scaled UMV margins, the 288-sample reference border, reduced filter selection, bit-depth-dependent two-pass rounding, and byte/high-bit-depth clipping. A four-output `Vector128` gather-shaped traversal and exact scalar tail share one behavior model and caller-owned scratch. The production path covers ordinary, compound, and OBMC prediction without per-block allocation. | | `Av1TileReader` palette mode/color-map parsing, `Av1SymbolDecoder` palette distributions, `Av1BlockModeInfo` palette state, and `Av1PredictionDecoder` palette reconstruction | AV1 sections 5.11.46, 5.11.49, and 7.11.2 palette prediction | libaom `av1/decoder/decodemv.c`, `av1/decoder/detokenize.c`, `av1/decoder/decoder.h`, `av1/common/pred_common.c`, `av1/common/pred_common.h`, and `av1/common/entropymode.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Reuse the existing tile range decoder and frame-owned mode map while porting the normative palette cache merge, high-bit-depth color deltas, tile-adaptive mode/size/index distributions, diagonal color-map traversal, edge padding, and direct palette-sample reconstruction. This is AV1 still-image compression syntax and does not add retained video reference state or any ISO BMFF surface. | -| `Av1DeblockingFilter`, `Av1LoopFilterContext`, and `Av1LoopFilterDecoder` | AV1 section 7.14 deblocking loop filter | libaom `aom_dsp/loopfilter.c` and `av1/common/av1_loopfilter.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Design the 8- and high-bit-depth edge layout and 4-, 6-, 8-, and 14-tap kernels for SIMD-first traversal, with sharpness thresholds, still-frame intra filter-level derivation, transform-edge selection, and plane traversal shared by the exact scalar fallback. Implement and verify both through the same contract before enabling the stage. This is normative AV1 image reconstruction and adds neither generic ISO BMFF models nor retained video reference state. | +| `Av1DeblockingFilter`, `Av1LoopFilterContext`, and `Av1LoopFilterDecoder` | AV1 section 7.14 deblocking loop filter | libaom `aom_dsp/loopfilter.c` and `av1/common/av1_loopfilter.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Preserve the 8- and high-bit-depth edge layout and 4-, 6-, 8-, and 14-tap kernels for SIMD-first traversal, with sharpness thresholds, per-block intra/inter reference and mode deltas, current and previous skipped-transform decisions, transform and prediction-unit edge selection, and plane traversal shared by the exact scalar fallback. This is normative AV1 image reconstruction and adds neither generic ISO BMFF models nor retained video reference state. | | `Av1CdefDecoder`, `Av1CdefFilter`, and CDEF-unit strength storage | AV1 sections 7.15.2 through 7.15.4 constrained directional enhancement filtering | libaom `av1/common/cdef.c`, `av1/common/cdef_block.c`, `av1/common/cdef.h`, and `av1/common/cdef_block.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Design direction search and constrained primary/secondary filtering around vector-friendly block storage and SIMD-first kernels, with variance adjustment, subsampling direction conversion, skipped-8x8 selection, and frame-edge sentinel behavior shared by the scalar fallback. Use a frame-owned source snapshot so filtering never consumes already modified samples. This is normative AV1 still-image reconstruction and introduces no ISO BMFF, track, timing, or sequence-playback surface. | | `Av1SuperResolutionDecoder`, `Av1SuperResolutionFilter`, frame-size derivation, and decoded-image dimensions | AV1 section 7.16 normative super-resolution upscaling | libaom `av1/common/resize.c`, `av1/common/resize.h`, `av1/common/convolve.c`, `av1/common/x86/av1_convolve_horiz_rs_sse4.c`, and `aom_dsp/aom_filter.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Port the fixed 64-phase, 8-tap horizontal filter, phase/step derivation, replicated frame edges, chroma width rounding, signed rounding, and 8/10/12-bit clipping. Follow libaom's four-output multiply/add, two-stage horizontal reduction, packed-store traversal with exact scalar fallback. Generic image resizing is not normative AV1 super-resolution. This adds no track, timing, fragment, animation, or generic ISO BMFF model. | | `Av1TileReader` loop-restoration unit syntax, `Av1SymbolDecoder` restoration distributions/subexponential codes, and `Av1FrameInfo` unit storage | AV1 section 5.11.57 `read_lr` and `read_lr_unit` syntax | libaom `av1/decoder/decodeframe.c`, `av1/common/restoration.c`, `av1/common/restoration.h`, `av1/common/entropymode.c`, `aom_dsp/binary_codes_reader.c`, and `aom_dsp/recenter.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Decode tile-local switchable/Wiener/self-guided selections, finite reference-subexponential coefficients, chroma Wiener windows, self-guided parameter sets, super-resolution-adjusted unit corners, and the AV1 nearest-unit-count rule into frame-owned per-plane grids. This is compressed still-image syntax and adds no movie, track, timing, fragment, audio, or sequence surface. | @@ -392,6 +395,8 @@ This assessment was reconciled with the source tree on 2026-08-29, including pro - The bounded `Av1Decoder` session parses all tile state before allocating and reconstructing each coded image layer or sequence sample. After successful completion it retains the ungrained reference planes, frame header, frame information, and published entropy snapshot in the refreshed slots, while presentation-only ownership remains separate. A new accepted sequence header resets both parser and retained-owner state. Bounded image-sequence decoding accepts `show_existing_frame`, including its key-frame reference reset and separate film-grain presentation ownership, without exposing a general animation/video reference model. - Committed inter-frame support parses single-reference selection, builds the fixed-capacity spatial and temporal reference-MV stack, decodes NEAREST/NEAR/NEW/GLOBAL and DRL syntax, decodes or infers interpolation filters, checks inter-intra and motion-mode eligibility, and invokes simple translational prediction before residual reconstruction. Commit `8b9aa4b2cfe59ff0f8c348d9bb06678a5ff12f2a` records exact corrected dependent-frame native-plane and final-presentation comparisons plus the complete focused Release matrix and ownership/allocation evidence. - Commit `e1a764b8f2e8a8db98a003fb3b913213f40084db` owns every compound reference-selection and inter-mode distribution through the retained CDF lifecycle, derives primary and secondary candidate vectors as inseparable pairs, reconstructs both retained references through the existing translational predictor, and combines them through one SIMD-first byte/ushort averaging operator with scalar parity. The pinned 19-frame libavif sequence exercises the complete production block branch and matches pinned libaom native planes and pinned libavif presentation exactly after decoding all preceding reference samples. +- Commit `59a4fbad5ed3bc7c34331c4bdc65acc7acde3f04` reconstructs eligible above and left OBMC neighbors through the retained primary reference, exact libaom masks, and the existing SIMD-first masked blender without per-block allocation. Direct production tests cover 8/10/12-bit storage plus 4:2:0 and 4:2:2 plane geometry. A pinned 19-frame sequence contains more than one hundred traced OBMC blocks and passes exact final native-plane, final-presentation, constrained-allocation, normal-dispatch, and scalar-dispatch comparison. +- The locally complete scaled-reference checkpoint maps current-frame Q4 positions into retained-frame Q10 coordinates with pinned-libaom Q14 scale factors and signed rounding, then applies allocation-free variable-phase two-pass convolution across byte and high-bit-depth storage. The production block path selects scaling from actual retained/current dimensions, covers both primary and secondary workspaces plus scaled OBMC neighbors, and retains libaom's 288-sample prediction border. The real layered fixture also exposed and now verifies per-block inter loop-filter reference/mode deltas and skipped-transform edge decisions that the earlier intra-only deblocking integration did not model. - Source inspection against pinned libaom `03087864cf4bea6abb0d28f95cf7843511413d8f` resolves two disputed audit claims in favor of the current predicates. The interpolation call path uses `is_nontrans_global_motion()`, which returns false only for `TRANSLATION`, so an identity `GLOBALMV` block omits filter symbols. The single-reference spatial extension loops use `MAX_MV_REF_CANDIDATES`, which is two, while the full reference-MV stack capacity is eight. These loops are spatial extension, not temporal extension. - Every padded AV1 frame plane requests ImageSharp's contiguous allocation contract, and constrained-allocator reconstruction passes without copying. Motion fields use allocator-owned storage and counted leases; allocation tracking passes for initialization, retained aliases, failure unwinding, presentation ownership, decoder-result ownership, and exactly-once disposal. The corrected logical progressive color payload reaches the production inter branch and its final native planes match pinned libaom exactly. - Transform coefficient entropy derivation and updates now address the above contexts relative to the tile column and the left contexts relative to the current superblock row, preserve luma coordinates independently of chroma subsampling, and test every packed context entry for the libaom any-nonzero rule. Extended vertical partition updates advance the mode-information column rather than the row. The existing multi-superblock 4:4:4 AVIF fixture now completes tile parsing; independent coefficient-context vectors across tile boundaries, chroma layouts, bit depths, and edge-clipped transforms remain required. @@ -422,7 +427,8 @@ This assessment was reconciled with the source tree on 2026-08-29, including pro ### Tests - HEVC coverage includes exact native-plane comparison with HM output, exact complete-image comparison with pinned libheif/libde265 references, and the 10 official Sony GENERAL Range Extensions first-picture fixtures across 8/10/12-bit monochrome, 4:2:0, 4:2:2, and 4:4:4 reconstruction. The remaining exposed profiles and individual Range Extensions tools still need exact independent vectors. -- AV1 has focused bitstream, prediction, entropy, reconstruction, filter, film-grain, color, and transform coverage, plus real libavif inputs. A real two-layer libavif-derived OBU stream verifies the bounded frame lifecycle, retained-slot occupancy, resolved inter references, and `frame_size_with_refs` dimensions through a fake tile lifecycle. The corrected progressive color item then verifies its production single-reference inter branch against exact pinned-libaom native planes and pinned-libavif presentation. The pinned 19-frame YUV444 sequence separately reaches production equal-average compound reconstruction after every preceding reference sample and matches its final native and presented references exactly. A real palette stream truncated inside its tile entropy payload verifies libaom-equivalent overflow/trailing-bit rejection and decoder-session recovery. The committed equal-average checkpoint passes 119 direct focused cases plus three `FeatureTestRunner` cases, zero-warning `net10.0` and `net11.0` source builds, a zero-error test-project analyzer build with pre-existing repository warnings, allocator lifetime validation, and `git diff --check`. Four additional pinned sequences reach distance-weighted, both wedge signs, both difference-mask orientations, and smooth and wedge inter-intra reconstruction with exact final native and presented output. Commit `864b11ad9791d3254a38691246270c569001b5a1` records the selectable-compound checkpoint after both source targets, the test-project analyzer build, 25 direct focused cases, six `FeatureTestRunner` cases, constrained allocation, exactly-once tracked returns, and `git diff --check` passed. Valid still-image syntax paths still contain explicit unsupported branches, so the independent AV1 decode matrix is not complete. +- AV1 has focused bitstream, prediction, entropy, reconstruction, filter, film-grain, color, and transform coverage, plus real libavif inputs. A real two-layer libavif-derived OBU stream verifies the bounded frame lifecycle, retained-slot occupancy, resolved inter references, and `frame_size_with_refs` dimensions through a fake tile lifecycle. The corrected progressive color item then verifies its production single-reference inter branch against exact pinned-libaom native planes and pinned-libavif presentation. The pinned 19-frame YUV444 sequence separately reaches production equal-average compound reconstruction after every preceding reference sample and matches its final native and presented references exactly. A real palette stream truncated inside its tile entropy payload verifies libaom-equivalent overflow/trailing-bit rejection and decoder-session recovery. The committed equal-average checkpoint passes 119 direct focused cases plus three `FeatureTestRunner` cases, zero-warning `net10.0` and `net11.0` source builds, a zero-error test-project analyzer build with pre-existing repository warnings, allocator lifetime validation, and `git diff --check`. Four additional pinned sequences reach distance-weighted, both wedge signs, both difference-mask orientations, and smooth and wedge inter-intra reconstruction with exact final native and presented output. Commit `864b11ad9791d3254a38691246270c569001b5a1` records the selectable-compound checkpoint after both source targets, the test-project analyzer build, 25 direct focused cases, six `FeatureTestRunner` cases, constrained allocation, exactly-once tracked returns, and `git diff --check` passed. Commit `59a4fbad5ed3bc7c34331c4bdc65acc7acde3f04` adds exact traced OBMC sequence evidence, subsampled direct reconstruction, constrained allocation, both source builds, the analyzer build, 14 direct cases, the normal/scalar `FeatureTestRunner` comparison, and clean staged `git diff --check`. The local scaled-reference checkpoint adds a genuine size-changing layered AVIF, exact retained-base and final native planes, exact final presentation, active inter deblocking, constrained allocation, normal/scalar dispatch, direct 8/10/12-bit block reconstruction, and 87 passing focused cases. Valid still-image syntax paths still contain explicit unsupported branches, so the independent AV1 decode matrix is not complete. +- Scaled-reference fixture provenance is pinned to libavif `062e582e8afda88e6baf988fdcf046a801efa0f5` and libaom `03087864cf4bea6abb0d28f95cf7843511413d8f`. The source is libavif `tests/data/webp_logo_animated.y4m`, encoded as two layered YUV444 inputs with `avifenc -j 1 -q 90 --layered --scaling-mode:u 1/2 --scaling-mode:u 1/1 `. The resulting item contains a 701-byte 40x40 base layer followed by a 1,494-byte 80x80 dependent layer. SHA-256 values are `EB239F31EC8DBF5E97AD6F52670FCA6497AE2A933822CFE724C75F66AAA2520B` for the AVIF, `B7E30E04A935414A517BAA2DF06AB756DA18BA7C291220D9D7C063297761AE82` for the 2,195-byte logical AV1 payload, `873DC1AB5623910FBF8053CBF684399F4F3BDDBA4110492E3E11DE86E01FEC1B` for the 4,800-byte pinned-libaom base YUV, `964F4D85A79FC10EEB3722E2C9DA3415083E1EB8C2611FE154946FA55E2171ED` for the pinned-libaom final Y4M, and `DC4C6DBE6BD92C5FCE1E3E23700AFA603EF04ED02EDD336213EBBA1E3BD84BA0` for the pinned-libavif final PNG. - The AV1 transform matrix verifies scalar, `Vector128`, `Vector256`, and `Vector512` forward representations plus the production inverse tiers across every valid size/type combination and supported bit depth. All 511 focused forward and inverse cases pass in Release; `FeatureTestRunner` isolates hardware tiers, every two-dimensional configuration exercises production dispatch, and the complete-block benchmark records zero managed allocation. - Independent libavif fixtures cover primary, grid, auxiliary-alpha, ICC, metadata-skipping, and all-sync image-sequence presentation. Reference-dependent AV1 and HEVC sequence reconstruction and independent HEVC ICC sequence coverage remain open. - Focused decoder-option tests cover strict, ancillary-only, image-data, and metadata-skipping behavior for still items and sequence samples. Complete adversarial dimension, allocation, malformed-container, and resource-limit coverage remains open. @@ -570,14 +576,16 @@ Implement and verify in dependency order: - [x] Decode all compound reference trees and inter modes, derive and retain paired reference-MV candidates, reconstruct both unscaled translational references, and combine them with equal-weight averaging through `Vector512`, `Vector256`, `Vector128`, and scalar paths without per-block allocation. - [x] Verify equal-average compound reconstruction with the pinned 19-frame libavif sequence, exact pinned-libaom final native planes, exact pinned-libavif final presentation, direct 8/10/12-bit production-block tests, constrained allocation, and exactly-once motion-field disposal. Both Release source targets pass with zero warnings and errors; the test-project analyzer build passes with zero errors and pre-existing repository warnings; 119 direct focused cases and three `FeatureTestRunner` cases pass without failures or skips; and `git diff --check` is clean. - [x] Decode and reconstruct selected inter-intra prediction and selectable distance-weighted, wedge, and difference-weighted compound blending through reconstructed reference planes. Direct syntax and 8/10/12-bit production tests, SIMD/scalar mask and blend tests, exact four-sequence native/presentation comparisons, constrained contiguous allocation, and exactly-once tracked returns pass. Both source targets build with zero warnings and errors; the test project builds with zero errors and pre-existing repository warnings; 25 direct focused cases and six `FeatureTestRunner` cases pass without failures or skips; and `git diff --check` is clean. Commit `864b11ad9791d3254a38691246270c569001b5a1` records the checkpoint. - - [ ] Decode and reconstruct OBMC, scaled references, warped motion, and non-translational global motion through reconstructed reference planes. + - [x] Decode and reconstruct OBMC through above and left neighboring prediction, exact libaom masks, allocator-owned workspaces, and the existing SIMD-first masked blender. Commit `59a4fbad5ed3bc7c34331c4bdc65acc7acde3f04` records exact pinned native/presentation evidence, direct 8/10/12-bit and subsampled reconstruction, constrained allocation, both Release source builds, and normal/scalar dispatch. + - [x] Decode scaled references through exact Q14 scale factors, signed Q4-to-Q10 coordinate rounding, variable per-output phases and steps, regular/smooth/sharp/bilinear reduced and full filters, bit-depth-dependent two-pass rounding, scaled OBMC neighbors, and the normative 288-sample retained border without per-block allocation. The independently generated 40x40-to-80x80 layered fixture proves the base reference before retention and in the production retained session, then matches the final pinned-libaom Y/U/V planes and pinned-libavif RGBA presentation exactly. Active inter deblocking additionally verifies per-block reference/mode deltas and skipped inter/intra boundary decisions. Normal and scalar `FeatureTestRunner` runs, constrained contiguous planes, unique allocation identities, 8/10/12-bit production-block coverage, zero-warning `net10.0` and `net11.0` source builds, the zero-error test-project build with 1,013 pre-existing warnings, 87 focused cases, documentation review, and `git diff --check` pass. + - [ ] **Queued until the scaled-reference commit:** decode warped motion and non-translational global motion through reconstructed reference planes. - [ ] Verify every remaining connected inter mode and filter with independently encoded bounded AV1 content and exact native-plane comparisons. - [ ] Lossless and high-bit-depth reconstruction with correct clipping and intermediate precision. - [x] Route lossless 4x4 blocks through allocation-free reversible inverse Walsh-Hadamard reconstruction for 8/10/12-bit samples, including the DC-only specialization, `Vector128` production traversal, scalar fallback, exact clipping, and `FeatureTestRunner` parity. - [x] Verify lossless syntax, inverse quantization, prediction, and presented reconstruction with independently encoded 8/10/12-bit AVIF fixtures. The tests require coded residuals with palette and intra-block copy disabled, compare every native YUV sample with the pinned generic libaom-backed decoder, and compare every presented RGBA byte with pinned generic libavif exactly under normal hardware dispatch and the scalar fallback. - [x] Deblocking loop filter. - [x] Implement allocation-free SIMD-first 4-, 6-, 8-, and 14-tap filtering for vertical and horizontal edges in 8/10/12-bit storage through closed edge operators, with exact scalar fallback and `FeatureTestRunner` parity against an independent definition. - - [x] Verify deblocking syntax, filter-level derivation, and boundary traversal with independently encoded 8/10/12-bit AV1 samples and exact scalar-libaom planes; verify presented reconstruction and public precision with genuine AVIF containers at every supported bit depth. + - [x] Verify deblocking syntax, per-block intra/inter reference and mode delta derivation, skipped-transform boundary decisions on both sides of each edge, and boundary traversal with independently encoded 8/10/12-bit AV1 samples and exact scalar-libaom planes; verify presented reconstruction and public precision with genuine AVIF containers at every supported bit depth. - [x] CDEF. - [x] Replace the scalar helper bucket with an allocation-free SIMD-first `Av1CdefFilter`: vector direction analysis, 4x4/4x8/8x4/8x8 constrained filtering, byte/16-bit output operators, and one exact scalar fallback verified through `FeatureTestRunner`. - [x] Move the immutable deblocked-plane snapshot and frame-wide direction/variance maps to allocator-owned decoder memory, preserving luma-first paired direction derivation, fixed-storage skipped-block selection, asymmetric chroma direction conversion, frame-edge sentinels, SIMD source widening, and direct frame-plane output. @@ -810,7 +818,9 @@ The dashboard and immediate execution queue define the remaining critical path. - [x] Finish the complete focused Release matrix, final source/documentation review, and checkpoint implementation for the simple single-reference inter slice. Commit `8b9aa4b2cfe59ff0f8c348d9bb06678a5ff12f2a` records the corrected real dependent-frame AVIF, exact pinned-libaom native-plane and pinned-libavif presentation comparisons, and 293 focused cases passing with zero failures or skips. - [x] Implement compound reference selection and equal averaging with paired reference-MV derivation, allocation-free SIMD/scalar parity, and exact independent native/presentation evidence through a reference-dependent bounded AVIF sequence. Commit `e1a764b8f2e8a8db98a003fb3b913213f40084db` records the checkpoint. - [x] Implement and independently verify selected inter-intra prediction and selectable distance-weighted, wedge, and difference-weighted compound blending through exact pinned native-plane and presentation comparisons, direct 8/10/12-bit production coverage, constrained allocation, and SIMD/scalar parity. -- [ ] **Current:** implement OBMC, then continue the remaining Phase 3 inter modes in dependency order: scaled references, warped motion, and non-translational global prediction. Each mode requires independent exact evidence before the next begins. +- [x] Implement and independently verify OBMC through exact traced native-plane and presentation evidence, direct 8/10/12-bit and subsampled production coverage, constrained allocation, and normal/scalar dispatch. Commit `59a4fbad5ed3bc7c34331c4bdc65acc7acde3f04` records the checkpoint. +- [x] Implement and independently verify scaled references through pinned fixed-point oracles, direct 8/10/12-bit production reconstruction, a genuine 40x40-to-80x80 bounded AVIF, exact retained-base/final-native/final-presentation comparisons, active inter deblocking, constrained allocation, and normal/scalar dispatch. The checkpoint is complete locally and awaiting its commit. +- [ ] **Queued until that commit:** implement warped motion, then non-translational global prediction. Each mode requires independent exact evidence before the next begins. - [ ] Remove every other unsupported valid AV1 still-image syntax path, correct the audited 12-bit inverse-transform arithmetic, and prove the complete AVIF decode matrix with independent inputs and scalar/SIMD parity. - [ ] Close Phase 4 by completing the remaining HEVC profile and Range Extensions matrix with exact native-plane and presented-image evidence. - [ ] Close Phase 5 and the decode portion of the bounded sequence ledger: color, ICC, alpha, grids, presentation transforms, reference-dependent samples, and complete animated AVIF/HEIC decode. diff --git a/src/ImageSharp/Formats/Heif/Av1/Av1FrameBuffer.cs b/src/ImageSharp/Formats/Heif/Av1/Av1FrameBuffer.cs index eccc0228d..1af68049a 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Av1FrameBuffer.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Av1FrameBuffer.cs @@ -18,9 +18,9 @@ internal class Av1FrameBuffer : IDisposable /// /// The number of luma border samples reserved for prediction and in-loop filtering. /// - // A 128-sample UMV block plus filter support reaches 135 luma samples beyond an edge. The 144-sample value also - // leaves 72 samples on a horizontally subsampled plane, exceeding its corresponding 71-sample maximum. - private const int DecoderPaddingValue = 144; + // Scaled prediction may start 284 luma samples outside a retained frame and then consume three preceding filter + // taps. The normative 288-sample border keeps that entire source window directly addressable without block copies. + public const int DecoderPaddingValue = 288; /// /// The allocation-mask bit for the luma plane. diff --git a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs index 9c8ec96a3..cd7b68b39 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Pipeline/LoopFilter/Av1LoopFilterDecoder.cs @@ -3,6 +3,7 @@ using System.Runtime.InteropServices; using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction; using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; using SixLabors.ImageSharp.Formats.Heif.Av1.Transform; @@ -222,22 +223,24 @@ internal class Av1LoopFilterDecoder Point planeTransformPosition = new(adjustedColumn >> subX, adjustedRow >> subY); Point previousPlaneTransformPosition = new(previousColumn >> subX, previousRow >> subY); Av1BlockModeInfo modeInfo = this.frameInfo.GetModeInfoAt(modeInfoPosition); + Av1BlockModeInfo previousModeInfo = this.frameInfo.GetModeInfoAt(previousModeInfoPosition); Av1TransformSize transformSize = this.loopFilterContext.GetTransformSize(plane, planeTransformPosition); Av1TransformSize previousTransformSize = this.loopFilterContext.GetTransformSize(plane, previousPlaneTransformPosition); - Av1BlockSize planeBlockSize = modeInfo.BlockSize.GetSubsampled(subX, subY); int planeX = x >> subX; int planeY = y >> subY; - bool isBlockEdge = verticalBoundary - ? planeX % planeBlockSize.GetWidth() == 0 - : planeY % planeBlockSize.GetHeight() == 0; - bool isTransformEdge = verticalBoundary ? planeX % transformSize.GetWidth() == 0 : planeY % transformSize.GetHeight() == 0; - // The still-image decoder accepts key and intra-only frames, so every decoded block satisfies the AV1 - // isIntra condition. Retaining the other predicates mirrors the normative edge decision without inter state. - bool applyFilter = isTransformEdge && (isBlockEdge || !modeInfo.Skip || this.frameHeader.IsIntra); + // A skipped intra block still has reconstructed prediction samples and is not a skipped inter transform. + // libaom therefore applies the skip predicate only when the corresponding primary reference is inter. + bool currentSkippedTransform = modeInfo.Skip && modeInfo.ReferenceFrames[0] > Av1ReferenceFrameType.Intra; + bool previousSkippedTransform = previousModeInfo.Skip && previousModeInfo.ReferenceFrames[0] > Av1ReferenceFrameType.Intra; + + // The mode-info map stores one object for every covered position, so object identity is the exact equivalent + // of libaom's current-versus-previous MB_MODE_INFO pointer comparison at a prediction-unit boundary. + bool isBlockEdge = !ReferenceEquals(modeInfo, previousModeInfo); + bool applyFilter = isTransformEdge && (isBlockEdge || !currentSkippedTransform || !previousSkippedTransform); if (!applyFilter) { return; @@ -246,7 +249,7 @@ internal class Av1LoopFilterDecoder int currentLevel = this.GetFilterLevel(modeInfo, modeInfoPosition, plane, pass); int filterLevel = currentLevel != 0 ? currentLevel - : this.GetFilterLevel(this.frameInfo.GetModeInfoAt(previousModeInfoPosition), previousModeInfoPosition, plane, pass); + : this.GetFilterLevel(previousModeInfo, previousModeInfoPosition, plane, pass); if (filterLevel == 0) { @@ -367,13 +370,25 @@ internal class Av1LoopFilterDecoder if (parameters.ReferenceDeltaModeEnabled) { - // Every supported AVIF still-picture block uses INTRA_FRAME, whose reference delta is index zero and - // whose prediction mode does not consume either inter mode delta. int referenceScale = 1 << (level >> 5); + Av1ReferenceFrameType referenceFrame = modeInfo.ReferenceFrames[0]; level = Av1Math.Clip3( 0, Av1Constants.MaxLoopFilter, - level + (parameters.ReferenceDeltas[0] * referenceScale)); + level + (parameters.ReferenceDeltas[(int)referenceFrame] * referenceScale)); + + if (referenceFrame > Av1ReferenceFrameType.Intra) + { + // AV1's second mode-delta class contains every inter mode except the two global-motion modes. + // Keeping this classification next to the level arithmetic mirrors libaom's mode_lf_lut lookup. + int modeDeltaIndex = modeInfo.YMode is Av1PredictionMode.GlobalMotionVector or + Av1PredictionMode.GlobalGlobalMotionVector ? 0 : 1; + + level = Av1Math.Clip3( + 0, + Av1Constants.MaxLoopFilter, + level + (parameters.ModeDeltas[modeDeltaIndex] * referenceScale)); + } } return level; diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1InterPredictor.Scaled.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1InterPredictor.Scaled.cs new file mode 100644 index 000000000..4fefd2a0a --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1InterPredictor.Scaled.cs @@ -0,0 +1,449 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.InteropServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Reconstructs reference-scaled inter prediction through variable-phase separable convolution. +/// +internal static partial class Av1InterPredictor +{ + /// + /// Gets the scratch capacity required by one scaled prediction block. + /// + public static int GetScaledScratchLength(int width, int height, int verticalPhase, int verticalStep) + { + int intermediateHeight = ((((height - 1) * verticalStep) + verticalPhase) >> Av1ReferenceScale.SubpixelBits) + FilterCoefficientCount; + return Math.Max(width, Vector128.Count) * intermediateHeight; + } + + /// + /// Gets a dimension-only upper bound for one scaled prediction block's scratch capacity. + /// + public static int GetMaximumScaledScratchLength(int width, int height) + => Math.Max(width, Vector128.Count) * ((height * 2) + FilterCoefficientCount); + + /// + /// Reconstructs an 8-bit scaled prediction using variable source positions and phases. + /// + public static void PredictScaled( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int horizontalStep, + int verticalPhase, + int verticalStep, + Span scratch) + => DispatchScaled( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + 8, + scratch); + + /// + /// Reconstructs an 8-, 10-, or 12-bit scaled prediction using variable source positions and phases. + /// + public static void PredictScaled( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int horizontalStep, + int verticalPhase, + int verticalStep, + int bitDepth, + Span scratch) + => DispatchScaled( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalFilter, + verticalFilter, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + /// + /// Selects the horizontal filter family for scaled prediction. + /// + private static void DispatchScaled( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int horizontalStep, + int verticalPhase, + int verticalStep, + int bitDepth, + Span scratch) + where T : unmanaged + where TSample : struct, IScaledSampleOperator + { + switch (horizontalFilter) + { + case Av1InterpolationFilter.Regular: + DispatchScaledVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Smooth: + DispatchScaledVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Sharp: + DispatchScaledVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + default: + DispatchScaledVertical( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + verticalFilter, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + } + } + + /// + /// Selects the vertical filter family for a closed horizontal scaled-prediction operator. + /// + private static void DispatchScaledVertical( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int horizontalStep, + int verticalPhase, + int verticalStep, + int bitDepth, + Span scratch) + where T : unmanaged + where TSample : struct, IScaledSampleOperator + where THorizontal : struct, IAv1InterPredictorOperator + { + switch (verticalFilter) + { + case Av1InterpolationFilter.Regular: + PredictScaled( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Smooth: + PredictScaled( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + case Av1InterpolationFilter.Sharp: + PredictScaled( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + default: + PredictScaled( + source, + sourceStride, + sourceOrigin, + destination, + destinationStride, + width, + height, + horizontalPhase, + horizontalStep, + verticalPhase, + verticalStep, + bitDepth, + scratch); + + break; + } + } + + /// + /// Applies variable-phase horizontal filtering followed by variable-phase vertical filtering. + /// + private static void PredictScaled( + ReadOnlySpan source, + int sourceStride, + int sourceOrigin, + Span destination, + int destinationStride, + int width, + int height, + int horizontalPhase, + int horizontalStep, + int verticalPhase, + int verticalStep, + int bitDepth, + Span scratch) + where T : unmanaged + where TSample : struct, IScaledSampleOperator + where THorizontal : struct, IAv1InterPredictorOperator + where TVertical : struct, IAv1InterPredictorOperator + { + ref T sourceBase = ref Unsafe.Add(ref MemoryMarshal.GetReference(source), sourceOrigin); + ref T destinationBase = ref MemoryMarshal.GetReference(destination); + ref short scratchBase = ref MemoryMarshal.GetReference(scratch); + int scratchStride = Math.Max(width, Vector128.Count); + int intermediateHeight = ((((height - 1) * verticalStep) + verticalPhase) >> Av1ReferenceScale.SubpixelBits) + FilterCoefficientCount; + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + int intermediateRange = bitDepth + FilterBits - Round0Bits + 2; + int round0 = Round0Bits + Math.Max(intermediateRange - 16, 0); + bool useReducedHorizontalFilter = width <= 4; + bool useReducedVerticalFilter = height <= 4; + + // Scaled positions change both the integer source sample and filter phase at each output column. Four-lane + // vectors gather those independent positions into one multiply-accumulate chain without allocating an index map. + for (int row = 0; row < intermediateHeight; row++) + { + ref T sourceRow = ref Unsafe.Add(ref sourceBase, (row - 3) * sourceStride); + ref short scratchRow = ref Unsafe.Add(ref scratchBase, row * scratchStride); + int column = 0; + if (Vector128.IsHardwareAccelerated) + { + for (; column <= width - Vector128.Count; column += Vector128.Count) + { + int position0 = horizontalPhase + (column * horizontalStep); + int position1 = position0 + horizontalStep; + int position2 = position1 + horizontalStep; + int position3 = position2 + horizontalStep; + int source0 = (position0 >> Av1ReferenceScale.SubpixelBits) - 3; + int source1 = (position1 >> Av1ReferenceScale.SubpixelBits) - 3; + int source2 = (position2 >> Av1ReferenceScale.SubpixelBits) - 3; + int source3 = (position3 >> Av1ReferenceScale.SubpixelBits) - 3; + ReadOnlySpan coefficients0 = THorizontal.GetCoefficients((position0 & Av1ReferenceScale.SubpixelMask) >> 6, useReducedHorizontalFilter); + ReadOnlySpan coefficients1 = THorizontal.GetCoefficients((position1 & Av1ReferenceScale.SubpixelMask) >> 6, useReducedHorizontalFilter); + ReadOnlySpan coefficients2 = THorizontal.GetCoefficients((position2 & Av1ReferenceScale.SubpixelMask) >> 6, useReducedHorizontalFilter); + ReadOnlySpan coefficients3 = THorizontal.GetCoefficients((position3 & Av1ReferenceScale.SubpixelMask) >> 6, useReducedHorizontalFilter); + Vector128 result = Vector128.Create(horizontalBias); + for (int tap = 0; tap < FilterCoefficientCount; tap++) + { + Vector128 samples = Vector128.Create( + TSample.Load(ref sourceRow, source0 + tap), + TSample.Load(ref sourceRow, source1 + tap), + TSample.Load(ref sourceRow, source2 + tap), + TSample.Load(ref sourceRow, source3 + tap)); + + Vector128 coefficients = Vector128.Create( + (int)coefficients0[tap], + coefficients1[tap], + coefficients2[tap], + coefficients3[tap]); + + result += samples * coefficients; + } + + Vector64 intermediate = Av1IntraPredictorBase.Narrow( + RoundPowerOfTwo(result, round0), + Vector128.Zero).GetLower(); + + intermediate.StoreUnsafe(ref scratchRow, (nuint)column); + } + } + + for (; column < width; column++) + { + int position = horizontalPhase + (column * horizontalStep); + int sourceColumn = (position >> Av1ReferenceScale.SubpixelBits) - 3; + ReadOnlySpan coefficients = THorizontal.GetCoefficients( + (position & Av1ReferenceScale.SubpixelMask) >> 6, + useReducedHorizontalFilter); + + int sum = horizontalBias; + for (int tap = 0; tap < FilterCoefficientCount; tap++) + { + sum += coefficients[tap] * TSample.Load(ref sourceRow, sourceColumn + tap); + } + + Unsafe.Add(ref scratchRow, column) = (short)RoundPowerOfTwo(sum, round0); + } + } + + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + int verticalBias = 1 << offsetBits; + int roundOffset = (1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1)); + for (int row = 0; row < height; row++) + { + int position = verticalPhase + (row * verticalStep); + int sourceRowIndex = position >> Av1ReferenceScale.SubpixelBits; + ReadOnlySpan coefficients = TVertical.GetCoefficients( + (position & Av1ReferenceScale.SubpixelMask) >> 6, + useReducedVerticalFilter); + + ref short scratchRow = ref Unsafe.Add(ref scratchBase, sourceRowIndex * scratchStride); + ref short coefficientBase = ref MemoryMarshal.GetReference(coefficients); + ref T destinationRow = ref Unsafe.Add(ref destinationBase, row * destinationStride); + int column = 0; + if (Vector128.IsHardwareAccelerated) + { + Vector128 initial = Vector128.Create(verticalBias); + Vector128 offset = Vector128.Create(roundOffset); + for (; column <= width - Vector128.Count; column += Vector128.Count) + { + Convolve( + ref scratchRow, + scratchStride, + (nuint)column, + ref coefficientBase, + FilterCoefficientCount, + initial, + out Vector128 result0, + out Vector128 result1); + + result0 = RoundPowerOfTwo(result0, round1) - offset; + result1 = RoundPowerOfTwo(result1, round1) - offset; + TSample.StoreVector(ref destinationRow, column, result0, result1, bitDepth); + } + } + + for (; column < width; column++) + { + int sum = verticalBias + ConvolveScalar( + ref Unsafe.Add(ref scratchRow, column), + scratchStride, + ref coefficientBase, + FilterCoefficientCount); + + TSample.StoreScalar( + ref destinationRow, + column, + RoundPowerOfTwo(sum, round1) - roundOffset, + bitDepth); + } + } + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1InterPredictor.ScaledOperator.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1InterPredictor.ScaledOperator.cs new file mode 100644 index 000000000..593621499 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1InterPredictor.ScaledOperator.cs @@ -0,0 +1,105 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Runtime.CompilerServices; +using System.Runtime.Intrinsics; + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Defines sample-storage operators for reference-scaled prediction. +/// +internal static partial class Av1InterPredictor +{ + /// + /// Supplies sample loading, clipping, and storage for a scaled predictor pipeline. + /// + /// The native sample storage type. + private interface IScaledSampleOperator + where T : unmanaged + { + /// + /// Loads one source sample as a signed accumulator value. + /// + /// The first source sample. + /// The sample offset. + /// The widened sample value. + public static abstract int Load(ref T source, int index); + + /// + /// Clips and stores eight completed vector lanes. + /// + /// The first destination sample. + /// The output offset. + /// The first four completed lanes. + /// The second four completed lanes. + /// The decoded sample precision. + public static abstract void StoreVector( + ref T destination, + int index, + Vector128 result0, + Vector128 result1, + int bitDepth); + + /// + /// Clips and stores one completed scalar value. + /// + /// The first destination sample. + /// The output offset. + /// The completed sample value. + /// The decoded sample precision. + public static abstract void StoreScalar(ref T destination, int index, int value, int bitDepth); + } + + /// + /// Implements scaled prediction storage for 8-bit samples. + /// + private readonly struct ScaledByteOperator : IScaledSampleOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Load(ref byte source, int index) => Unsafe.Add(ref source, index); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void StoreVector( + ref byte destination, + int index, + Vector128 result0, + Vector128 result1, + int bitDepth) + => PackBytes(result0, result1, Vector128.Zero, Vector128.Zero) + .GetLower() + .StoreUnsafe(ref destination, (nuint)index); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void StoreScalar(ref byte destination, int index, int value, int bitDepth) + => Unsafe.Add(ref destination, index) = (byte)Math.Clamp(value, byte.MinValue, byte.MaxValue); + } + + /// + /// Implements scaled prediction storage for 8-, 10-, and 12-bit samples. + /// + private readonly struct ScaledUInt16Operator : IScaledSampleOperator + { + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static int Load(ref ushort source, int index) => Unsafe.Add(ref source, index); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void StoreVector( + ref ushort destination, + int index, + Vector128 result0, + Vector128 result1, + int bitDepth) + => PackHighBitDepth(result0, result1, (1 << bitDepth) - 1).StoreUnsafe(ref destination, (nuint)index); + + /// + [MethodImpl(MethodImplOptions.AggressiveInlining)] + public static void StoreScalar(ref ushort destination, int index, int value, int bitDepth) + => Unsafe.Add(ref destination, index) = (ushort)Math.Clamp(value, 0, (1 << bitDepth) - 1); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ReferenceScale.cs b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ReferenceScale.cs new file mode 100644 index 000000000..0ac3e11bb --- /dev/null +++ b/src/ImageSharp/Formats/Heif/Av1/Prediction/Inter/Av1ReferenceScale.cs @@ -0,0 +1,94 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; + +/// +/// Converts current-frame prediction coordinates into a retained reference frame's sample grid. +/// +internal readonly struct Av1ReferenceScale +{ + /// + /// The identity scale in the normative Q14 representation. + /// + private const int IdentityScale = 1 << 14; + + /// + /// The number of fractional bits carried by scaled prediction positions and steps. + /// + public const int SubpixelBits = 10; + + /// + /// The mask selecting one scaled sample's fractional position. + /// + public const int SubpixelMask = (1 << SubpixelBits) - 1; + + /// + /// The half-unit offset that centers Q4 input coordinates on the Q10 reference grid. + /// + public const int ExtraOffset = 1 << (SubpixelBits - 4 - 1); + + /// + /// Initializes a new instance of the struct. + /// + /// The retained reference width. + /// The retained reference height. + /// The current coded-frame width. + /// The current coded-frame height. + public Av1ReferenceScale(int referenceWidth, int referenceHeight, int currentWidth, int currentHeight) + { + this.HorizontalScale = ((referenceWidth << 14) + (currentWidth >> 1)) / currentWidth; + this.VerticalScale = ((referenceHeight << 14) + (currentHeight >> 1)) / currentHeight; + this.HorizontalStep = (this.HorizontalScale + 8) >> 4; + this.VerticalStep = (this.VerticalScale + 8) >> 4; + } + + /// + /// Gets the horizontal Q14 scale factor. + /// + public int HorizontalScale { get; } + + /// + /// Gets the vertical Q14 scale factor. + /// + public int VerticalScale { get; } + + /// + /// Gets the horizontal per-output-sample step in Q10 reference samples. + /// + public int HorizontalStep { get; } + + /// + /// Gets the vertical per-output-sample step in Q10 reference samples. + /// + public int VerticalStep { get; } + + /// + /// Gets a value indicating whether either reference dimension differs from the current frame. + /// + public bool IsScaled => this.HorizontalScale != IdentityScale || this.VerticalScale != IdentityScale; + + /// + /// Scales one horizontal Q4 current-frame coordinate into the Q10 reference grid. + /// + public int ScaleHorizontal(int value) => Scale(value, this.HorizontalScale); + + /// + /// Scales one vertical Q4 current-frame coordinate into the Q10 reference grid. + /// + public int ScaleVertical(int value) => Scale(value, this.VerticalScale); + + /// + /// Applies libaom's signed fixed-point rounding without relying on implementation-defined negative shifts. + /// + private static int Scale(int value, int scale) + { + long offset = (scale - IdentityScale) * 8L; + long scaled = ((long)value * scale) + offset; + const int shift = 8; + const long rounding = 1L << (shift - 1); + return scaled < 0 + ? (int)-((-scaled + rounding) >> shift) + : (int)((scaled + rounding) >> shift); + } +} diff --git a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs index 0af7f4fef..da123952f 100644 --- a/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs +++ b/src/ImageSharp/Formats/Heif/Av1/Transform/Av1BlockDecoder.cs @@ -129,7 +129,9 @@ internal sealed class Av1BlockDecoder : IDisposable int maximumBlockArea = maximumBlockLength * maximumBlockLength; int predictorWorkingLength = Math.Max( Av1PredictionDecoder.ScratchLength, - Av1InterPredictor.GetScratchLength(maximumBlockLength, maximumBlockLength)); + Math.Max( + Av1InterPredictor.GetScratchLength(maximumBlockLength, maximumBlockLength), + Av1InterPredictor.GetMaximumScaledScratchLength(maximumBlockLength, maximumBlockLength))); // Compound prediction retains the complete second predictor until both references have been sampled. // Reserve it once beside the convolution workspace so block traversal never rents temporary memory. @@ -364,6 +366,45 @@ internal sealed class Av1BlockDecoder : IDisposable : secondaryReferenceFrameBuffer!; Av1MotionVector motionVector = modeInfo.MotionVectors[referenceIndex]; + int destinationStride = referenceIndex == 0 ? reconstructionStride : predictionWidth; + bool isScaledReference = activeReferenceFrameBuffer.Width != this.frameHeader.FrameSize.FrameWidth || + activeReferenceFrameBuffer.Height != this.frameHeader.FrameSize.FrameHeight; + + if (isScaledReference) + { + Span scaledDestination = default; + Span scaledHighBitDepthDestination = default; + if (highBitDepth) + { + scaledHighBitDepthDestination = referenceIndex == 0 + ? MemoryMarshal.Cast(highBitDepthBlockReconstructionBuffer[reconstructionStride..]) + : highBitDepthSecondPrediction; + } + else + { + scaledDestination = referenceIndex == 0 + ? blockReconstructionBuffer[reconstructionStride..] + : secondPrediction; + } + + this.PredictScaledReference( + activeReferenceFrameBuffer, + motionVector, + plane, + subX, + subY, + pixelPosition, + predictionWidth, + predictionHeight, + modeInfo.InterpolationFilters[1], + modeInfo.InterpolationFilters[0], + scaledDestination, + scaledHighBitDepthDestination, + destinationStride, + predictionScratch); + + continue; + } // AV1 predicts the complete declared plane block even when its luma extent crosses the frame boundary. // Subsampled dimensions retain the mandatory four-sample minimum used by set_plane_n4 in libaom. @@ -376,7 +417,7 @@ internal sealed class Av1BlockDecoder : IDisposable // The UMV clamp is expressed in one-sixteenth plane-sample units. A 128-sample block can legally // address 135 samples beyond an edge once its prediction extent and eight-tap filter support are - // included; the frame-owned 144-sample luma border keeps that source directly addressable. + // included; the frame-owned prediction border keeps that source directly addressable. horizontalMotionQ4 = Av1Math.Clip3( (partitionInfo.ModeBlockToLeftEdge * horizontalEdgeScale) - horizontalExtensionQ4, (partitionInfo.ModeBlockToRightEdge * horizontalEdgeScale) + horizontalExtensionQ4 - 16, @@ -395,7 +436,6 @@ internal sealed class Av1BlockDecoder : IDisposable // floor used to select the integer source sample. int horizontalPhase = sourceColumnQ4 & 15; int verticalPhase = sourceRowQ4 & 15; - int destinationStride = referenceIndex == 0 ? reconstructionStride : predictionWidth; if (highBitDepth) { @@ -950,7 +990,7 @@ internal sealed class Av1BlockDecoder : IDisposable } /// - /// Resolves one canonical reference and verifies that unit-step inter prediction can sample it. + /// Resolves one canonical retained reference frame. /// private Av1FrameBuffer ResolveReferenceFrame(Av1ReferenceFrameType referenceFrame) { @@ -959,16 +999,118 @@ internal sealed class Av1BlockDecoder : IDisposable // The uncompressed-header parser validates each selected slot and the reference store remains unchanged // until frame reconstruction completes, so every parsed inter block resolves the same retained owner. - Av1FrameBuffer referenceFrameBuffer = this.referenceFrames!.Resolve((int)referenceSlot)!.FrameBuffer; - if (referenceFrameBuffer.Width != this.frameHeader.FrameSize.FrameWidth || - referenceFrameBuffer.Height != this.frameHeader.FrameSize.FrameHeight) + return this.referenceFrames!.Resolve((int)referenceSlot)!.FrameBuffer; + } + + /// + /// Predicts one block from a retained reference whose visible dimensions differ from the current coded frame. + /// + private void PredictScaledReference( + Av1FrameBuffer referenceFrameBuffer, + Av1MotionVector motionVector, + int plane, + int subX, + int subY, + Point predictionOrigin, + int predictionWidth, + int predictionHeight, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + Span destination, + Span highBitDepthDestination, + int destinationStride, + Span predictionScratch) + { + Av1ReferenceScale scale = new( + referenceFrameBuffer.Width, + referenceFrameBuffer.Height, + this.frameHeader.FrameSize.FrameWidth, + this.frameHeader.FrameSize.FrameHeight); + + int currentColumnQ4 = (predictionOrigin.X << 4) + (motionVector.Column << (1 - subX)); + int currentRowQ4 = (predictionOrigin.Y << 4) + (motionVector.Row << (1 - subY)); + int sourceColumnQ10 = scale.ScaleHorizontal(currentColumnQ4) + Av1ReferenceScale.ExtraOffset; + int sourceRowQ10 = scale.ScaleVertical(currentRowQ4) + Av1ReferenceScale.ExtraOffset; + int referencePlaneWidth = Av1Math.DivideLog2Ceiling(referenceFrameBuffer.Width, subX); + int referencePlaneHeight = Av1Math.DivideLog2Ceiling(referenceFrameBuffer.Height, subY); + int horizontalMargin = (Av1FrameBuffer.DecoderPaddingValue >> subX) - 4; + int verticalMargin = (Av1FrameBuffer.DecoderPaddingValue >> subY) - 4; + + // The scaled coordinate clamp is intentionally wider than the ordinary block-relative UMV clamp. The retained + // frame owns the normative border, so every variable-phase eight-tap source remains directly addressable. + sourceColumnQ10 = Av1Math.Clip3( + -horizontalMargin << Av1ReferenceScale.SubpixelBits, + (referencePlaneWidth + 4) << Av1ReferenceScale.SubpixelBits, + sourceColumnQ10); + + sourceRowQ10 = Av1Math.Clip3( + -verticalMargin << Av1ReferenceScale.SubpixelBits, + (referencePlaneHeight + 4) << Av1ReferenceScale.SubpixelBits, + sourceRowQ10); + + int horizontalPhase = sourceColumnQ10 & Av1ReferenceScale.SubpixelMask; + int verticalPhase = sourceRowQ10 & Av1ReferenceScale.SubpixelMask; + if (this.frameBuffer.BytesPerSample == 2) { - // Scaled prediction changes both the source coordinate and the per-output-sample step. Running the - // unit-step predictor here would silently reconstruct valid scaled-reference streams incorrectly. - throw new NotSupportedException("AV1 scaled-reference inter prediction is not implemented."); + Span source = referenceFrameBuffer.GetPaddedPlaneSpan16( + (Av1Plane)plane, + subX, + subY, + out int sourceStride, + out Point sourceOrigin); + + int sourceIndex = + ((sourceOrigin.Y + (sourceRowQ10 >> Av1ReferenceScale.SubpixelBits)) * sourceStride) + + sourceOrigin.X + + (sourceColumnQ10 >> Av1ReferenceScale.SubpixelBits); + + Av1InterPredictor.PredictScaled( + source, + sourceStride, + sourceIndex, + highBitDepthDestination, + destinationStride, + predictionWidth, + predictionHeight, + horizontalFilter, + verticalFilter, + horizontalPhase, + scale.HorizontalStep, + verticalPhase, + scale.VerticalStep, + this.frameBuffer.BitDepth.GetBitCount(), + predictionScratch); } + else + { + Span source = referenceFrameBuffer.GetPaddedPlaneSpan( + (Av1Plane)plane, + subX, + subY, + out int sourceStride, + out Point sourceOrigin); - return referenceFrameBuffer; + int sourceIndex = + ((sourceOrigin.Y + (sourceRowQ10 >> Av1ReferenceScale.SubpixelBits)) * sourceStride) + + sourceOrigin.X + + (sourceColumnQ10 >> Av1ReferenceScale.SubpixelBits); + + Av1InterPredictor.PredictScaled( + source, + sourceStride, + sourceIndex, + destination, + destinationStride, + predictionWidth, + predictionHeight, + horizontalFilter, + verticalFilter, + horizontalPhase, + scale.HorizontalStep, + verticalPhase, + scale.VerticalStep, + predictionScratch); + } } /// @@ -1184,6 +1326,30 @@ internal sealed class Av1BlockDecoder : IDisposable { Av1FrameBuffer referenceFrameBuffer = this.ResolveReferenceFrame(neighbor.ReferenceFrames[0]); Av1MotionVector motionVector = neighbor.MotionVectors[0]; + bool isScaledReference = referenceFrameBuffer.Width != this.frameHeader.FrameSize.FrameWidth || + referenceFrameBuffer.Height != this.frameHeader.FrameSize.FrameHeight; + + if (isScaledReference) + { + this.PredictScaledReference( + referenceFrameBuffer, + motionVector, + plane, + subX, + subY, + predictionOrigin, + predictionWidth, + predictionHeight, + neighbor.InterpolationFilters[1], + neighbor.InterpolationFilters[0], + destination, + highBitDepthDestination, + predictionWidth, + predictionScratch); + + return; + } + int sourceColumnQ4 = (predictionOrigin.X << 4) + (motionVector.Column << (1 - subX)); int sourceRowQ4 = (predictionOrigin.Y << 4) + (motionVector.Row << (1 - subY)); int horizontalExtensionQ4 = (4 + predictionWidth) << 4; diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs index dde92ff76..888ae0b2f 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1CompoundBlockDecoderTests.cs @@ -111,6 +111,77 @@ public class Av1CompoundBlockDecoderTests } } + /// + /// Verifies that the production block branch maps a smaller current frame into a larger retained reference. + /// + /// The native sample depth. + [Theory] + [InlineData((int)Av1BitDepth.EightBit)] + [InlineData((int)Av1BitDepth.TenBit)] + [InlineData((int)Av1BitDepth.TwelveBit)] + public void DecodeBlockReconstructsScaledSingleReferencePrediction(int bitDepthValue) + { + Av1BitDepth bitDepth = (Av1BitDepth)bitDepthValue; + ObuSequenceHeader sequenceHeader = CreateSequenceHeader(bitDepth, 16); + ObuFrameHeader frameHeader = CreateFrameHeader(); + frameHeader.GetReferenceFrameIndices()[0] = 0; + + using Av1ReferenceFrameStore referenceFrames = new(); + Assert.True(referenceFrames.Commit(1, CreateScaledPatternReferenceFrame(sequenceHeader), showFrame: false)); + + using Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + Av1ColorFormat.Yuv400, + false); + + frameBuffer.Width = 8; + frameBuffer.Height = 8; + using Av1FrameInfo frameInfo = new(sequenceHeader); + Av1SuperblockInfo superblockInfo = frameInfo.GetSuperblock(Point.Empty); + superblockInfo.GetTransformInfoY()[0] = new Av1TransformInfo(Av1TransformSize.Size8x8, 0, 0); + + Av1BlockModeInfo modeInfo = CreateSingleReferenceModeInfo(Av1BlockSize.Block8x8, Point.Empty); + modeInfo.InterpolationFilters.Fill(Av1InterpolationFilter.Bilinear); + modeInfo.SetTransformUnitCount(Av1PlaneType.Y, 1); + + Av1LoopFilterContext loopFilterContext = new(sequenceHeader); + Av1InverseQuantizer inverseQuantizer = new(sequenceHeader, frameHeader); + using Av1BlockDecoder decoder = new( + sequenceHeader, + frameHeader, + frameBuffer, + loopFilterContext, + inverseQuantizer, + referenceFrames); + + decoder.UpdateSuperblock(superblockInfo); + decoder.DecodeBlock( + modeInfo, + Point.Empty, + Av1BlockSize.Block8x8, + superblockInfo, + new Av1TileInfo(0, 0, frameHeader)); + + for (int row = 0; row < 8; row++) + { + for (int column = 0; column < 8; column++) + { + ushort expected = (ushort)(5 + (column * 4) + (row * 16)); + if (bitDepth == Av1BitDepth.EightBit) + { + Span samples = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(row); + Assert.Equal((byte)expected, samples[column]); + } + else + { + Span samples = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, row, 0, 0); + Assert.Equal(expected, samples[column]); + } + } + } + } + /// /// Verifies selectable compound reconstruction through the production block branch at every supported bit depth. /// @@ -575,6 +646,44 @@ public class Av1CompoundBlockDecoderTests return new Av1ReferenceFrame(frameBuffer, frameHeader, frameInfo); } + /// + /// Creates a 16x16 retained frame whose linear pattern has an exact half-sample bilinear result. + /// + private static Av1ReferenceFrame CreateScaledPatternReferenceFrame(ObuSequenceHeader sequenceHeader) + { + Av1FrameBuffer frameBuffer = new( + Configuration.Default, + sequenceHeader, + Av1ColorFormat.Yuv400, + false); + + frameBuffer.Width = 16; + frameBuffer.Height = 16; + for (int row = 0; row < 16; row++) + { + if (sequenceHeader.ColorConfig.BitDepth == Av1BitDepth.EightBit) + { + Span samples = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0).DangerousGetRowSpan(row); + for (int column = 0; column < 16; column++) + { + samples[column] = (byte)((column * 2) + (row * 8)); + } + } + else + { + Span samples = frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, row, 0, 0); + for (int column = 0; column < 16; column++) + { + samples[column] = (ushort)((column * 2) + (row * 8)); + } + } + } + + Av1ReferenceFrameBorder.Extend(frameBuffer); + using Av1FrameInfo frameInfo = new(sequenceHeader); + return new Av1ReferenceFrame(frameBuffer, CreateFrameHeader(16), frameInfo); + } + /// /// Gets the deterministic luma value stored at one reference-frame coordinate. /// diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs index 191f313b9..43d544681 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReconstructionConformanceTests.cs @@ -128,6 +128,21 @@ public class Av1ReconstructionConformanceTests /// private const int ProgressiveFirstLayerSize = 55; + /// + /// The displayed width and height of the independent scaled-reference fixture. + /// + private const int ScaledReferenceFixtureSize = 80; + + /// + /// The retained base-layer width and height of the independent scaled-reference fixture. + /// + private const int ScaledReferenceBaseLayerSize = 40; + + /// + /// The byte length of the scaled-reference fixture's base layer as declared by its a1lx property. + /// + private const int ScaledReferenceFirstLayerSize = 701; + /// /// The displayed width and height of the independent compound image sequence. /// @@ -415,6 +430,40 @@ public class Av1ReconstructionConformanceTests ValidateProgressiveSingleReferenceFixture(configuration, verifyPresentation: false); } + /// + /// Verifies an independently encoded 40x40 retained layer scaled into an 80x80 dependent layer against exact + /// pinned-libaom native planes and pinned-libavif presentation. + /// + [Fact] + public void DecodeScaledReferenceMatchesPinnedReferences() + => FeatureTestRunner.RunWithHwIntrinsicsFeature( + ValidateScaledReferenceFixtureWithDefaultConfiguration, + ReconstructionConfigurations); + + /// + /// Verifies scaled-reference reconstruction with constrained tracked allocation and contiguous frame planes. + /// + [Fact] + [ValidateDisposedMemoryAllocations] + public void DecodeScaledReferenceWithConstrainedAllocator() + { + TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 1_024 }; + allocator.EnableNonThreadSafeLogging(); + Configuration configuration = Configuration.Default.Clone(); + configuration.MemoryAllocator = allocator; + + ValidateScaledReferenceFixture(configuration, verifyPresentation: false); + + Assert.Contains(allocator.AllocationLog, request => request.ElementType.Name == "RetainedMotionFieldEntry"); + Assert.Contains(allocator.AllocationLog, request => request.ElementType.Name == "TemporalMotionFieldEntry"); + Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); + Assert.All( + allocator.AllocationLog, + allocation => Assert.Single( + allocator.ReturnLog, + returned => returned.AllocationId == allocation.AllocationId)); + } + /// /// Verifies that the production dependent-frame result owns its motion-field storage until decoder disposal. /// @@ -447,30 +496,30 @@ public class Av1ReconstructionConformanceTests // The decoder's inspectable FrameInfo result remains the final motion-field owner until decoder disposal. Assert.DoesNotContain( allocator.ReturnLog, - returned => returned.HashCodeOfBuffer == retainedMotionField.HashCodeOfBuffer); + returned => returned.AllocationId == retainedMotionField.AllocationId); Assert.DoesNotContain( allocator.ReturnLog, - returned => returned.HashCodeOfBuffer == temporalMotionField.HashCodeOfBuffer); + returned => returned.AllocationId == temporalMotionField.AllocationId); frameBuffer.Dispose(); Assert.DoesNotContain( allocator.ReturnLog, - returned => returned.HashCodeOfBuffer == retainedMotionField.HashCodeOfBuffer); + returned => returned.AllocationId == retainedMotionField.AllocationId); Assert.DoesNotContain( allocator.ReturnLog, - returned => returned.HashCodeOfBuffer == temporalMotionField.HashCodeOfBuffer); + returned => returned.AllocationId == temporalMotionField.AllocationId); decoder.Dispose(); decoder.Dispose(); Assert.Single( allocator.ReturnLog, - returned => returned.HashCodeOfBuffer == retainedMotionField.HashCodeOfBuffer); + returned => returned.AllocationId == retainedMotionField.AllocationId); - Assert.Single(allocator.ReturnLog, returned => returned.HashCodeOfBuffer == temporalMotionField.HashCodeOfBuffer); + Assert.Single(allocator.ReturnLog, returned => returned.AllocationId == temporalMotionField.AllocationId); } /// @@ -504,7 +553,7 @@ public class Av1ReconstructionConformanceTests allocator.AllocationLog, allocation => Assert.Single( allocator.ReturnLog, - returned => returned.HashCodeOfBuffer == allocation.HashCodeOfBuffer)); + returned => returned.AllocationId == allocation.AllocationId)); } /// @@ -730,7 +779,7 @@ public class Av1ReconstructionConformanceTests allocator.AllocationLog, allocation => Assert.Single( allocator.ReturnLog, - returned => returned.HashCodeOfBuffer == allocation.HashCodeOfBuffer)); + returned => returned.AllocationId == allocation.AllocationId)); } /// @@ -1500,6 +1549,146 @@ public class Av1ReconstructionConformanceTests private static void ValidateProgressiveSingleReferenceFixtureWithDefaultConfiguration() => ValidateProgressiveSingleReferenceFixture(Configuration.Default, verifyPresentation: true); + /// + /// Runs the exact scaled-reference native and presentation comparison with the default configuration. + /// + private static void ValidateScaledReferenceFixtureWithDefaultConfiguration() + => ValidateScaledReferenceFixture(Configuration.Default, verifyPresentation: true); + + /// + /// Verifies the genuine size-changing layered fixture with the requested allocator. + /// + /// The decoder configuration. + /// Whether to verify the public RGBA presentation. + private static void ValidateScaledReferenceFixture(Configuration configuration, bool verifyPresentation) + { + byte[] payload = TestFile.Create(TestImages.Heif.Av1ScaledReferencePayload).Bytes; + byte[] baseReferenceBytes = TestFile.Create(TestImages.Heif.Av1ScaledReferenceBaseNativeReference).Bytes; + byte[] referenceBytes = TestFile.Create(TestImages.Heif.Av1ScaledReferenceNativeReference).Bytes; + ReadOnlySpan fileHeader = + "YUV4MPEG2 W80 H80 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=LIMITED\n"u8; + + ReadOnlySpan frameHeader = "FRAME\n"u8; + ReadOnlySpan nativeReference = referenceBytes; + Assert.True(nativeReference.StartsWith(fileHeader)); + nativeReference = nativeReference[fileHeader.Length..]; + Assert.True(nativeReference.StartsWith(frameHeader)); + nativeReference = nativeReference[frameHeader.Length..]; + Assert.Equal(ScaledReferenceFixtureSize * ScaledReferenceFixtureSize * 3, nativeReference.Length); + + // Decode the independently declared base extent alone to prove that the retained reference is 40x40 rather + // than relying on the 80x80 item presentation dimensions recorded by the container. + using (Av1Decoder baseDecoder = new(configuration)) + using (Av1FrameBuffer baseFrameBuffer = baseDecoder.DecodeFrameBuffer( + payload.AsSpan(0, ScaledReferenceFirstLayerSize), + null, + null, + out _)) + { + Assert.Equal(ScaledReferenceBaseLayerSize, baseFrameBuffer.Width); + Assert.Equal(ScaledReferenceBaseLayerSize, baseFrameBuffer.Height); + Assert.Equal(ScaledReferenceBaseLayerSize * ScaledReferenceBaseLayerSize * 3, baseReferenceBytes.Length); + AssertNativePlanesEqual(baseDecoder, baseFrameBuffer, baseReferenceBytes); + } + + // Exercise the same retained owner across two calls so the independently verified base samples are checked + // in the exact decoder session that supplies the size-changing reference to the dependent frame. + using (Av1Decoder sequenceDecoder = new(configuration)) + { + sequenceDecoder.DecodeSequenceReference( + payload.AsSpan(0, ScaledReferenceFirstLayerSize), + null, + null); + + Av1FrameBuffer retainedBaseFrameBuffer = Assert.IsType>(sequenceDecoder.FrameBuffer); + AssertNativePlanesEqual(sequenceDecoder, retainedBaseFrameBuffer, baseReferenceBytes); + using Av1FrameBuffer sequenceFrameBuffer = sequenceDecoder.DecodeFrameBuffer( + payload.AsSpan(ScaledReferenceFirstLayerSize), + null, + null, + out _); + + AssertNativePlanesEqual(sequenceDecoder, sequenceFrameBuffer, nativeReference); + } + + using Av1Decoder decoder = new(configuration); + using Av1FrameBuffer frameBuffer = decoder.DecodeFrameBuffer( + payload, + null, + null, + out _, + new Av1LayeredImageIndex(ScaledReferenceFirstLayerSize, 0, 0)); + + Assert.Equal(ScaledReferenceFixtureSize, frameBuffer.Width); + Assert.Equal(ScaledReferenceFixtureSize, frameBuffer.Height); + Assert.Equal(Av1BitDepth.EightBit, frameBuffer.BitDepth); + Assert.Equal(Av1ColorFormat.Yuv444, frameBuffer.ColorFormat); + Assert.Equal(1, frameBuffer.BufferY!.FastMemoryGroup.Count); + Assert.Equal(1, frameBuffer.BufferCb!.FastMemoryGroup.Count); + Assert.Equal(1, frameBuffer.BufferCr!.FastMemoryGroup.Count); + + ObuSequenceHeader sequenceHeader = Assert.IsType(decoder.SequenceHeader); + ObuFrameHeader finalFrameHeader = Assert.IsType(decoder.FrameHeader); + Av1FrameInfo frameInfo = Assert.IsType(decoder.FrameInfo); + + Assert.Equal(ScaledReferenceFixtureSize, sequenceHeader.MaxFrameWidth); + Assert.Equal(ScaledReferenceFixtureSize, sequenceHeader.MaxFrameHeight); + Assert.Equal(ObuFrameType.InterFrame, finalFrameHeader.FrameType); + Assert.True(finalFrameHeader.LoopFilterParameters.ReferenceDeltaModeEnabled); + Assert.NotEqual(0, finalFrameHeader.LoopFilterParameters.FilterLevelU); + Assert.NotEqual(0, finalFrameHeader.LoopFilterParameters.FilterLevelV); + int interBlockCount = 0; + int intraBlockCount = 0; + int skippedInterBlockCount = 0; + int superblockSizeLog2 = sequenceHeader.SuperblockSizeLog2; + int superblockColumnCount = Av1Math.AlignPowerOf2(sequenceHeader.MaxFrameWidth, superblockSizeLog2) >> superblockSizeLog2; + int superblockRowCount = Av1Math.AlignPowerOf2(sequenceHeader.MaxFrameHeight, superblockSizeLog2) >> superblockSizeLog2; + for (int superblockRow = 0; superblockRow < superblockRowCount; superblockRow++) + { + for (int superblockColumn = 0; superblockColumn < superblockColumnCount; superblockColumn++) + { + Av1SuperblockInfo superblock = frameInfo.GetSuperblock(new Point(superblockColumn, superblockRow)); + foreach (Av1BlockModeInfo modeInfo in superblock.GetModeInfos()) + { + if (modeInfo.ReferenceFrames[0] >= Av1ReferenceFrameType.Last) + { + interBlockCount++; + if (modeInfo.Skip) + { + skippedInterBlockCount++; + } + } + else + { + intraBlockCount++; + } + } + } + } + + Assert.NotEqual(0, interBlockCount); + Assert.NotEqual(0, intraBlockCount); + Assert.NotEqual(0, skippedInterBlockCount); + AssertNativePlanesEqual(decoder, frameBuffer, nativeReference); + + if (!verifyPresentation) + { + return; + } + + DecoderOptions options = new() { Configuration = configuration, MaxFrames = 1 }; + byte[] imageBytes = TestFile.Create(TestImages.Heif.Av1ScaledReferenceAvif).Bytes; + byte[] presentationBytes = TestFile.Create(TestImages.Heif.Av1ScaledReferencePresentationReference).Bytes; + using Image image = Image.Load(options, imageBytes); + using Image presentationReference = Image.Load(presentationBytes); + + Assert.Equal(ScaledReferenceFixtureSize, image.Width); + Assert.Equal(ScaledReferenceFixtureSize, image.Height); + Assert.Single(image.Frames); + Assert.Equal(HeifBitDepth.Bit8, image.Metadata.GetHeifMetadata().BitDepth); + ImageComparer.Exact.VerifySimilarity(presentationReference, image); + } + /// /// Verifies the final dependent layer with the requested allocator. /// diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceFrameStoreTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceFrameStoreTests.cs index 75d122098..7eaa25e72 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceFrameStoreTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ReferenceFrameStoreTests.cs @@ -275,7 +275,7 @@ public class Av1ReferenceFrameStoreTests allocator.AllocationLog, allocation => Assert.Single( allocator.ReturnLog, - returned => returned.HashCodeOfBuffer == allocation.HashCodeOfBuffer)); + returned => returned.AllocationId == allocation.AllocationId)); Assert.Equal(2, allocator.ReturnLog.Count); } @@ -335,7 +335,7 @@ public class Av1ReferenceFrameStoreTests allocator.AllocationLog, allocation => Assert.Single( allocator.ReturnLog, - returned => returned.HashCodeOfBuffer == allocation.HashCodeOfBuffer)); + returned => returned.AllocationId == allocation.AllocationId)); Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count); } diff --git a/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ScaledInterPredictorTests.cs b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ScaledInterPredictorTests.cs new file mode 100644 index 000000000..3fc6ae274 --- /dev/null +++ b/tests/ImageSharp.Tests/Formats/Heif/Av1/Av1ScaledInterPredictorTests.cs @@ -0,0 +1,612 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction.Inter; +using SixLabors.ImageSharp.Tests.TestUtilities; + +namespace SixLabors.ImageSharp.Tests.Formats.Heif.Av1; + +/// +/// Verifies AV1 reference scaling and variable-phase inter convolution against an independent libaom-shaped oracle. +/// +[Trait("Format", "Avif")] +public class Av1ScaledInterPredictorTests +{ + /// + /// The number of fractional bits in each interpolation coefficient. + /// + private const int FilterBits = 7; + + /// + /// The ordinary first-pass rounding distance. + /// + private const int Round0Bits = 3; + + /// + /// The number of samples in every stored interpolation row. + /// + private const int FilterTapCount = 8; + + /// + /// The source border retained around the independently generated active coordinates. + /// + private const int SourcePadding = 16; + + /// + /// The guarded destination elements before the active block. + /// + private const int DestinationPrefix = 11; + + /// + /// The guarded destination elements after each active row. + /// + private const int DestinationRowPadding = 9; + + /// + /// The guarded destination elements after the final row. + /// + private const int DestinationSuffix = 17; + + /// + /// The byte value used to detect writes outside the active destination block. + /// + private const byte ByteSentinel = 0xD3; + + /// + /// The ushort value used to detect writes outside the active destination block. + /// + private const ushort UInt16Sentinel = 0xDEAD; + + /// + /// Exercises the native vector path and the complete scalar fallback in separate processes. + /// + private const HwIntrinsics PredictorConfigurations = HwIntrinsics.AllowAll | HwIntrinsics.DisableHWIntrinsic; + + /// + /// Verifies the pinned Q14 scale factors, Q10 steps, and signed coordinate rounding. + /// + [Fact] + public void ReferenceScaleMatchesPinnedLibaomFixedPointRules() + { + Av1ReferenceScale downscaledReference = new(40, 24, 64, 48); + + Assert.True(downscaledReference.IsScaled); + Assert.Equal(10240, downscaledReference.HorizontalScale); + Assert.Equal(8192, downscaledReference.VerticalScale); + Assert.Equal(640, downscaledReference.HorizontalStep); + Assert.Equal(512, downscaledReference.VerticalStep); + Assert.Equal(ScaleCoordinate(37, 10240), downscaledReference.ScaleHorizontal(37)); + Assert.Equal(ScaleCoordinate(-37, 10240), downscaledReference.ScaleHorizontal(-37)); + + Av1ReferenceScale enlargedReference = new(96, 72, 64, 48); + + Assert.Equal(24576, enlargedReference.HorizontalScale); + Assert.Equal(24576, enlargedReference.VerticalScale); + Assert.Equal(1536, enlargedReference.HorizontalStep); + Assert.Equal(1536, enlargedReference.VerticalStep); + + Av1ReferenceScale identity = new(64, 48, 64, 48); + + Assert.False(identity.IsScaled); + Assert.Equal(1024, identity.HorizontalStep); + Assert.Equal(1024, identity.VerticalStep); + } + + /// + /// Verifies exact scaled 8-bit output, variable filter phases, vector tails, and untouched destination padding. + /// + [Fact] + public void BytePredictionMatchesLibaomOracleAcrossIntrinsicConfigurations() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateBytePredictions, PredictorConfigurations); + + /// + /// Verifies exact scaled 8-, 10-, and 12-bit output under the native vector and scalar configurations. + /// + [Fact] + public void HighBitDepthPredictionMatchesLibaomOracleAcrossIntrinsicConfigurations() + => FeatureTestRunner.RunWithHwIntrinsicsFeature(ValidateHighBitDepthPredictions, PredictorConfigurations); + + /// + /// Applies each scaled-prediction scenario to byte storage. + /// + private static void ValidateBytePredictions() + { + foreach (ScaledPredictionCase testCase in CreatePredictionCases()) + { + byte[] source = CreateByteSource(testCase, out int sourceStride, out int sourceOrigin); + int destinationStride = testCase.Width + DestinationRowPadding; + byte[] expected = CreateByteDestination(testCase, destinationStride); + byte[] actual = (byte[])expected.Clone(); + short[] scratch = new short[ + Av1InterPredictor.GetScaledScratchLength( + testCase.Width, + testCase.Height, + testCase.VerticalPhase, + testCase.VerticalStep)]; + + ApplyReference(source, sourceStride, sourceOrigin, expected, destinationStride, testCase, 8); + + Av1InterPredictor.PredictScaled( + source, + sourceStride, + sourceOrigin, + actual.AsSpan(DestinationPrefix), + destinationStride, + testCase.Width, + testCase.Height, + testCase.HorizontalFilter, + testCase.VerticalFilter, + testCase.HorizontalPhase, + testCase.HorizontalStep, + testCase.VerticalPhase, + testCase.VerticalStep, + scratch); + + Assert.Equal(expected, actual); + } + } + + /// + /// Applies each scaled-prediction scenario to every supported high-bit-depth precision. + /// + private static void ValidateHighBitDepthPredictions() + { + int[] bitDepths = [8, 10, 12]; + foreach (int bitDepth in bitDepths) + { + foreach (ScaledPredictionCase testCase in CreatePredictionCases()) + { + ushort[] source = CreateUInt16Source(testCase, bitDepth, out int sourceStride, out int sourceOrigin); + int destinationStride = testCase.Width + DestinationRowPadding; + ushort[] expected = CreateUInt16Destination(testCase, destinationStride); + ushort[] actual = (ushort[])expected.Clone(); + short[] scratch = new short[ + Av1InterPredictor.GetScaledScratchLength( + testCase.Width, + testCase.Height, + testCase.VerticalPhase, + testCase.VerticalStep)]; + + ApplyReference(source, sourceStride, sourceOrigin, expected, destinationStride, testCase, bitDepth); + + Av1InterPredictor.PredictScaled( + source, + sourceStride, + sourceOrigin, + actual.AsSpan(DestinationPrefix), + destinationStride, + testCase.Width, + testCase.Height, + testCase.HorizontalFilter, + testCase.VerticalFilter, + testCase.HorizontalPhase, + testCase.HorizontalStep, + testCase.VerticalPhase, + testCase.VerticalStep, + bitDepth, + scratch); + + Assert.Equal(expected, actual); + } + } + } + + /// + /// Creates cases covering variable phases, every filter family, reduced kernels, and vector tails. + /// + private static ScaledPredictionCase[] CreatePredictionCases() => + [ + new("fixture-regular-8x8", 8, 8, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Regular, 800, 512, 800, 512), + new("fixture-regular-4x8", 4, 8, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Regular, 800, 512, 800, 512), + new("fixture-regular-8x4", 8, 4, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Regular, 800, 512, 800, 512), + new("bilinear-variable-phase", 13, 9, Av1InterpolationFilter.Bilinear, Av1InterpolationFilter.Bilinear, 192, 1536, 512, 640), + new("regular-smooth-wide", 20, 8, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Smooth, 64, 2048, 448, 2048), + new("sharp-bilinear-tail", 12, 5, Av1InterpolationFilter.Sharp, Av1InterpolationFilter.Bilinear, 512, 2048, 192, 2048), + new("reduced-regular", 4, 8, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Smooth, 192, 2048, 448, 2048), + new("reduced-sharp-maps-to-regular", 4, 8, Av1InterpolationFilter.Sharp, Av1InterpolationFilter.Smooth, 192, 2048, 448, 2048), + new("reduced-smooth", 8, 4, Av1InterpolationFilter.Regular, Av1InterpolationFilter.Smooth, 64, 2048, 832, 2048) + ]; + + /// + /// Creates deterministic padded byte source storage for one prediction case. + /// + private static byte[] CreateByteSource(ScaledPredictionCase testCase, out int stride, out int origin) + { + GetSourceGeometry(testCase, out int width, out int height); + stride = width; + origin = (SourcePadding * stride) + SourcePadding; + byte[] source = new byte[width * height]; + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + source[(row * stride) + column] = (byte)(((row * 29) + (column * 47) + (row * column * 3)) & byte.MaxValue); + } + } + + return source; + } + + /// + /// Creates deterministic padded ushort source storage for one prediction case. + /// + private static ushort[] CreateUInt16Source(ScaledPredictionCase testCase, int bitDepth, out int stride, out int origin) + { + GetSourceGeometry(testCase, out int width, out int height); + stride = width; + origin = (SourcePadding * stride) + SourcePadding; + int maximum = (1 << bitDepth) - 1; + ushort[] source = new ushort[width * height]; + for (int row = 0; row < height; row++) + { + for (int column = 0; column < width; column++) + { + source[(row * stride) + column] = (ushort)(((row * 269) + (column * 443) + (row * column * 31)) & maximum); + } + } + + return source; + } + + /// + /// Computes storage dimensions that keep every requested eight-tap read inside the test source. + /// + private static void GetSourceGeometry(ScaledPredictionCase testCase, out int width, out int height) + { + int maximumHorizontalPosition = testCase.HorizontalPhase + ((testCase.Width - 1) * testCase.HorizontalStep); + int maximumVerticalPosition = testCase.VerticalPhase + ((testCase.Height - 1) * testCase.VerticalStep); + width = (2 * SourcePadding) + (maximumHorizontalPosition >> Av1ReferenceScale.SubpixelBits) + FilterTapCount; + height = (2 * SourcePadding) + (maximumVerticalPosition >> Av1ReferenceScale.SubpixelBits) + FilterTapCount; + } + + /// + /// Creates a guarded byte destination initialized to its sentinel. + /// + private static byte[] CreateByteDestination(ScaledPredictionCase testCase, int stride) + { + byte[] destination = new byte[DestinationPrefix + (stride * testCase.Height) + DestinationSuffix]; + Array.Fill(destination, ByteSentinel); + return destination; + } + + /// + /// Creates a guarded ushort destination initialized to its sentinel. + /// + private static ushort[] CreateUInt16Destination(ScaledPredictionCase testCase, int stride) + { + ushort[] destination = new ushort[DestinationPrefix + (stride * testCase.Height) + DestinationSuffix]; + Array.Fill(destination, UInt16Sentinel); + return destination; + } + + /// + /// Applies the independent variable-phase two-pass reference convolution to byte storage. + /// + private static void ApplyReference( + byte[] source, + int sourceStride, + int sourceOrigin, + byte[] destination, + int destinationStride, + ScaledPredictionCase testCase, + int bitDepth) + { + short[] intermediate = CreateIntermediate(testCase); + int intermediateStride = testCase.Width; + int round0 = GetRound0Bits(bitDepth); + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + Span coefficients = stackalloc short[FilterTapCount]; + + for (int row = 0; row < intermediate.Length / intermediateStride; row++) + { + for (int column = 0; column < testCase.Width; column++) + { + int position = testCase.HorizontalPhase + (column * testCase.HorizontalStep); + int sourceColumn = (position >> Av1ReferenceScale.SubpixelBits) - 3; + FillCoefficients(testCase.HorizontalFilter, (position & Av1ReferenceScale.SubpixelMask) >> 6, testCase.Width <= 4, coefficients); + int sourceIndex = sourceOrigin + ((row - 3) * sourceStride) + sourceColumn; + int sum = horizontalBias + Convolve(source, sourceIndex, coefficients); + intermediate[(row * intermediateStride) + column] = (short)RoundPowerOfTwo(sum, round0); + } + } + + WriteReference(intermediate, intermediateStride, destination, destinationStride, testCase, bitDepth); + } + + /// + /// Applies the independent variable-phase two-pass reference convolution to ushort storage. + /// + private static void ApplyReference( + ushort[] source, + int sourceStride, + int sourceOrigin, + ushort[] destination, + int destinationStride, + ScaledPredictionCase testCase, + int bitDepth) + { + short[] intermediate = CreateIntermediate(testCase); + int intermediateStride = testCase.Width; + int round0 = GetRound0Bits(bitDepth); + int horizontalBias = 1 << (bitDepth + FilterBits - 1); + Span coefficients = stackalloc short[FilterTapCount]; + + for (int row = 0; row < intermediate.Length / intermediateStride; row++) + { + for (int column = 0; column < testCase.Width; column++) + { + int position = testCase.HorizontalPhase + (column * testCase.HorizontalStep); + int sourceColumn = (position >> Av1ReferenceScale.SubpixelBits) - 3; + FillCoefficients(testCase.HorizontalFilter, (position & Av1ReferenceScale.SubpixelMask) >> 6, testCase.Width <= 4, coefficients); + int sourceIndex = sourceOrigin + ((row - 3) * sourceStride) + sourceColumn; + int sum = horizontalBias + Convolve(source, sourceIndex, coefficients); + intermediate[(row * intermediateStride) + column] = (short)RoundPowerOfTwo(sum, round0); + } + } + + WriteReference(intermediate, intermediateStride, destination, destinationStride, testCase, bitDepth); + } + + /// + /// Allocates the oracle's independently shaped intermediate block. + /// + private static short[] CreateIntermediate(ScaledPredictionCase testCase) + { + int height = ((((testCase.Height - 1) * testCase.VerticalStep) + testCase.VerticalPhase) >> Av1ReferenceScale.SubpixelBits) + FilterTapCount; + return new short[testCase.Width * height]; + } + + /// + /// Completes byte output from the horizontally filtered intermediate block. + /// + private static void WriteReference( + short[] intermediate, + int intermediateStride, + byte[] destination, + int destinationStride, + ScaledPredictionCase testCase, + int bitDepth) + { + int maximum = byte.MaxValue; + Span coefficients = stackalloc short[FilterTapCount]; + for (int row = 0; row < testCase.Height; row++) + { + int position = testCase.VerticalPhase + (row * testCase.VerticalStep); + int sourceRow = position >> Av1ReferenceScale.SubpixelBits; + FillCoefficients(testCase.VerticalFilter, (position & Av1ReferenceScale.SubpixelMask) >> 6, testCase.Height <= 4, coefficients); + for (int column = 0; column < testCase.Width; column++) + { + int value = FinishConvolution(intermediate, (sourceRow * intermediateStride) + column, intermediateStride, coefficients, bitDepth); + destination[DestinationPrefix + (row * destinationStride) + column] = (byte)Math.Clamp(value, 0, maximum); + } + } + } + + /// + /// Completes high-bit-depth output from the horizontally filtered intermediate block. + /// + private static void WriteReference( + short[] intermediate, + int intermediateStride, + ushort[] destination, + int destinationStride, + ScaledPredictionCase testCase, + int bitDepth) + { + int maximum = (1 << bitDepth) - 1; + Span coefficients = stackalloc short[FilterTapCount]; + for (int row = 0; row < testCase.Height; row++) + { + int position = testCase.VerticalPhase + (row * testCase.VerticalStep); + int sourceRow = position >> Av1ReferenceScale.SubpixelBits; + FillCoefficients(testCase.VerticalFilter, (position & Av1ReferenceScale.SubpixelMask) >> 6, testCase.Height <= 4, coefficients); + for (int column = 0; column < testCase.Width; column++) + { + int value = FinishConvolution(intermediate, (sourceRow * intermediateStride) + column, intermediateStride, coefficients, bitDepth); + destination[DestinationPrefix + (row * destinationStride) + column] = (ushort)Math.Clamp(value, 0, maximum); + } + } + } + + /// + /// Removes both normative convolution biases after the vertical pass. + /// + private static int FinishConvolution( + short[] intermediate, + int sourceIndex, + int sourceStride, + ReadOnlySpan coefficients, + int bitDepth) + { + int round0 = GetRound0Bits(bitDepth); + int round1 = (2 * FilterBits) - round0; + int offsetBits = bitDepth + (2 * FilterBits) - round0; + int verticalBias = 1 << offsetBits; + int roundOffset = (1 << (offsetBits - round1)) + (1 << (offsetBits - round1 - 1)); + int sum = verticalBias + Convolve(intermediate, sourceIndex, sourceStride, coefficients); + return RoundPowerOfTwo(sum, round1) - roundOffset; + } + + /// + /// Computes one byte convolution sum. + /// + private static int Convolve(byte[] source, int sourceIndex, ReadOnlySpan coefficients) + { + int sum = 0; + for (int tap = 0; tap < FilterTapCount; tap++) + { + sum += source[sourceIndex + tap] * coefficients[tap]; + } + + return sum; + } + + /// + /// Computes one ushort convolution sum. + /// + private static int Convolve(ushort[] source, int sourceIndex, ReadOnlySpan coefficients) + { + int sum = 0; + for (int tap = 0; tap < FilterTapCount; tap++) + { + sum += source[sourceIndex + tap] * coefficients[tap]; + } + + return sum; + } + + /// + /// Computes one vertical convolution sum from the biased intermediate block. + /// + private static int Convolve(short[] source, int sourceIndex, int sourceStride, ReadOnlySpan coefficients) + { + int sum = 0; + for (int tap = 0; tap < FilterTapCount; tap++) + { + sum += source[sourceIndex + (tap * sourceStride)] * coefficients[tap]; + } + + return sum; + } + + /// + /// Selects one pinned coefficient row without reading production filter storage. + /// + private static void FillCoefficients(Av1InterpolationFilter filter, int phase, bool reduced, Span destination) + { + destination.Clear(); + if (filter == Av1InterpolationFilter.Bilinear) + { + destination[3] = (short)(128 - (phase * 8)); + destination[4] = (short)(phase * 8); + return; + } + + if (reduced && filter == Av1InterpolationFilter.Sharp) + { + filter = Av1InterpolationFilter.Regular; + } + + ReadOnlySpan source = (filter, reduced, phase) switch + { + (Av1InterpolationFilter.Regular, false, 1) => [0, 2, -6, 126, 8, -2, 0, 0], + (Av1InterpolationFilter.Regular, false, 4) => [0, 2, -14, 110, 38, -10, 2, 0], + (Av1InterpolationFilter.Regular, false, 12) => [0, 2, -10, 38, 110, -14, 2, 0], + (Av1InterpolationFilter.Smooth, false, 7) => [0, -2, 16, 54, 48, 12, 0, 0], + (Av1InterpolationFilter.Sharp, false, 8) => [-4, 12, -24, 80, 80, -24, 12, -4], + (Av1InterpolationFilter.Regular, true, 3) => [0, 0, -10, 116, 28, -6, 0, 0], + (Av1InterpolationFilter.Regular, true, 4) => [0, 0, -12, 110, 38, -8, 0, 0], + (Av1InterpolationFilter.Regular, true, 12) => [0, 0, -8, 38, 110, -12, 0, 0], + (Av1InterpolationFilter.Smooth, true, 13) => [0, 0, 4, 40, 62, 22, 0, 0], + _ => throw new InvalidOperationException($"The scaled oracle has no row for {filter}, phase {phase}, reduced {reduced}.") + }; + + source.CopyTo(destination); + } + + /// + /// Computes libaom's bit-depth-dependent first-pass shift. + /// + private static int GetRound0Bits(int bitDepth) + { + int intermediateRange = bitDepth + FilterBits - Round0Bits + 2; + return Round0Bits + Math.Max(intermediateRange - 16, 0); + } + + /// + /// Applies integer power-of-two rounding. + /// + private static int RoundPowerOfTwo(int value, int bits) => (value + (1 << (bits - 1))) >> bits; + + /// + /// Independently applies libaom's signed Q14-to-Q10 scale conversion. + /// + private static int ScaleCoordinate(int value, int scale) + { + long scaled = ((long)value * scale) + ((scale - (1 << 14)) * 8L); + const int shift = 8; + const long rounding = 1L << (shift - 1); + return scaled < 0 + ? (int)-((-scaled + rounding) >> shift) + : (int)((scaled + rounding) >> shift); + } + + /// + /// Describes one scaled prediction case. + /// + private readonly struct ScaledPredictionCase + { + /// + /// Initializes a new instance of the struct. + /// + public ScaledPredictionCase( + string name, + int width, + int height, + Av1InterpolationFilter horizontalFilter, + Av1InterpolationFilter verticalFilter, + int horizontalPhase, + int horizontalStep, + int verticalPhase, + int verticalStep) + { + this.Name = name; + this.Width = width; + this.Height = height; + this.HorizontalFilter = horizontalFilter; + this.VerticalFilter = verticalFilter; + this.HorizontalPhase = horizontalPhase; + this.HorizontalStep = horizontalStep; + this.VerticalPhase = verticalPhase; + this.VerticalStep = verticalStep; + } + + /// + /// Gets the diagnostic case name. + /// + public string Name { get; } + + /// + /// Gets the output width. + /// + public int Width { get; } + + /// + /// Gets the output height. + /// + public int Height { get; } + + /// + /// Gets the horizontal interpolation filter. + /// + public Av1InterpolationFilter HorizontalFilter { get; } + + /// + /// Gets the vertical interpolation filter. + /// + public Av1InterpolationFilter VerticalFilter { get; } + + /// + /// Gets the initial horizontal Q10 position. + /// + public int HorizontalPhase { get; } + + /// + /// Gets the horizontal Q10 source step. + /// + public int HorizontalStep { get; } + + /// + /// Gets the initial vertical Q10 position. + /// + public int VerticalPhase { get; } + + /// + /// Gets the vertical Q10 source step. + /// + public int VerticalStep { get; } + + /// + public override string ToString() => this.Name; + } +} diff --git a/tests/ImageSharp.Tests/TestImages.cs b/tests/ImageSharp.Tests/TestImages.cs index f80d087f8..80e9f1ebd 100644 --- a/tests/ImageSharp.Tests/TestImages.cs +++ b/tests/ImageSharp.Tests/TestImages.cs @@ -1357,6 +1357,11 @@ public static class TestImages public const string Av1Progressive8BitPayload = "Heif/Av1/Conformance/libavif-progressive-draw-points-8b.bit"; public const string Av1Progressive8BitReference = "Heif/Av1/Conformance/libavif-progressive-draw-points-8b-libaom-y4m.yuv"; public const string Av1Progressive8BitPresentationReference = "Heif/Av1/Conformance/libavif-progressive-draw-points-8b.png"; + public const string Av1ScaledReferenceAvif = "Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.avif"; + public const string Av1ScaledReferencePayload = "Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.bit"; + public const string Av1ScaledReferenceBaseNativeReference = "Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-base-libaom.yuv"; + public const string Av1ScaledReferenceNativeReference = "Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-libaom.y4m"; + public const string Av1ScaledReferencePresentationReference = "Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.png"; public const string Av1AverageCompoundSequenceAvif = "Heif/Av1/Conformance/libavif-webp-logo-average-compound.avif"; public const string Av1AverageCompoundSequenceNativeReference = "Heif/Av1/Conformance/libavif-webp-logo-average-compound-libaom.y4m"; public const string Av1AverageCompoundSequencePresentationReference = "Heif/Av1/Conformance/libavif-webp-logo-average-compound-libavif.png"; diff --git a/tests/ImageSharp.Tests/TestUtilities/TestMemoryAllocator.cs b/tests/ImageSharp.Tests/TestUtilities/TestMemoryAllocator.cs index 667224912..1f5b68bb1 100644 --- a/tests/ImageSharp.Tests/TestUtilities/TestMemoryAllocator.cs +++ b/tests/ImageSharp.Tests/TestUtilities/TestMemoryAllocator.cs @@ -12,6 +12,7 @@ internal class TestMemoryAllocator : MemoryAllocator { private List allocationLog; private List returnLog; + private int nextAllocationId; public TestMemoryAllocator(byte dirtyValue = 42) { @@ -39,15 +40,16 @@ internal class TestMemoryAllocator : MemoryAllocator protected override AllocationTrackedMemoryManager AllocateCore(int length, AllocationOptions options = AllocationOptions.None) { - T[] array = this.AllocateArray(length, options); - return new BasicArrayBuffer(array, length, this); + int allocationId = ++this.nextAllocationId; + T[] array = this.AllocateArray(length, options, allocationId); + return new BasicArrayBuffer(array, length, this, allocationId); } - private T[] AllocateArray(int length, AllocationOptions options) + private T[] AllocateArray(int length, AllocationOptions options, int allocationId) where T : struct { T[] array = new T[length + 42]; - this.allocationLog?.Add(AllocationRequest.Create(options, length, array)); + this.allocationLog?.Add(AllocationRequest.Create(options, length, array, allocationId)); if (options == AllocationOptions.None) { @@ -61,17 +63,24 @@ internal class TestMemoryAllocator : MemoryAllocator private void Return(BasicArrayBuffer buffer) where T : struct { - this.returnLog?.Add(new ReturnRequest(buffer.Array.GetHashCode())); + this.returnLog?.Add(new ReturnRequest(buffer.AllocationId, buffer.Array.GetHashCode())); } public struct AllocationRequest { - private AllocationRequest(Type elementType, AllocationOptions allocationOptions, int length, int lengthInBytes, int hashCodeOfBuffer) + private AllocationRequest( + Type elementType, + AllocationOptions allocationOptions, + int length, + int lengthInBytes, + int allocationId, + int hashCodeOfBuffer) { this.ElementType = elementType; this.AllocationOptions = allocationOptions; this.Length = length; this.LengthInBytes = lengthInBytes; + this.AllocationId = allocationId; this.HashCodeOfBuffer = hashCodeOfBuffer; if (elementType == typeof(Vector4)) @@ -79,11 +88,11 @@ internal class TestMemoryAllocator : MemoryAllocator } } - public static AllocationRequest Create(AllocationOptions allocationOptions, int length, T[] buffer) + public static AllocationRequest Create(AllocationOptions allocationOptions, int length, T[] buffer, int allocationId) { Type type = typeof(T); int elementSize = Marshal.SizeOf(type); - return new AllocationRequest(type, allocationOptions, length, length * elementSize, buffer.GetHashCode()); + return new AllocationRequest(type, allocationOptions, length, length * elementSize, allocationId, buffer.GetHashCode()); } public Type ElementType { get; } @@ -94,16 +103,21 @@ internal class TestMemoryAllocator : MemoryAllocator public int LengthInBytes { get; } + public int AllocationId { get; } + public int HashCodeOfBuffer { get; } } public struct ReturnRequest { - public ReturnRequest(int hashCodeOfBuffer) + public ReturnRequest(int allocationId, int hashCodeOfBuffer) { + this.AllocationId = allocationId; this.HashCodeOfBuffer = hashCodeOfBuffer; } + public int AllocationId { get; } + public int HashCodeOfBuffer { get; } } @@ -116,16 +130,17 @@ internal class TestMemoryAllocator : MemoryAllocator private readonly TestMemoryAllocator allocator; private GCHandle pinHandle; - public BasicArrayBuffer(T[] array, int length, TestMemoryAllocator allocator) + public BasicArrayBuffer(T[] array, int length, TestMemoryAllocator allocator, int allocationId) { this.allocator = allocator; DebugGuard.MustBeLessThanOrEqualTo(length, array.Length, nameof(length)); this.Array = array; this.Length = length; + this.AllocationId = allocationId; } public BasicArrayBuffer(T[] array, TestMemoryAllocator allocator) - : this(array, array.Length, allocator) + : this(array, array.Length, allocator, 0) { } @@ -139,6 +154,11 @@ internal class TestMemoryAllocator : MemoryAllocator /// public int Length { get; } + /// + /// Gets the stable identity recorded for this allocation. + /// + public int AllocationId { get; } + /// public override Span GetSpan() => this.Array.AsSpan(0, this.Length); diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-base-libaom.yuv b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-base-libaom.yuv new file mode 100644 index 000000000..f1b5a5126 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-base-libaom.yuv @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:873dc1ab5623910fbf8053cbf684399f4f3bddba4110492e3e11de86e01fec1b +size 4800 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-libaom.y4m b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-libaom.y4m new file mode 100644 index 000000000..fa02030df --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference-libaom.y4m @@ -0,0 +1,3 @@ +YUV4MPEG2 W80 H80 F25:1 Ip A0:0 C444 XYSCSS=444 XCOLORRANGE=LIMITED +FRAME +??>>>>>>>>VUUTTTTTTTTuwUTuwU뤯???B=Vcuuuuvt`V=?ACBBBBBBBBBBBBBBBBDB]u©tZ;TTTTTTTTTTTTTTTTTTTTUUUUUUUUUUUUUUUUWSdqsfU>FTXyWSB;Tiž\S;FTĻUKTi˾]U,S˰½iV.,UʰjT-,WɲkT--WɱiT-/UɰkT-0UȲkT-.TůkT-,Tª{zjS--Tn\UUSVSXXljS--TfVUVzzVUVfjS--Tª{USeeSUz~jT--T©sTUURq~~jT--TzWVVRu~~~~~~dT--TUSTW|~~~~~~dT--TfUU^}~~~~~~~dU--TVcgRz}}~~~~~~~}dU--TfRTf~~}~}}|}cT--TXXWX~~}~~~~}}|~cU--TT|yTu~~~}}~}|}{}}|bU.-TwUTo}|}~~~~~}}}||||{z}bV.-TtRTl~{}}}~~~}|{|}zzyx}bV--TuSTm|~~}}|||}{z{yyyy}_T--T|UUp~~~~~|{{||zyzxyxwz}]S--TySy|Vt~}}||}|{{yxzzxutx~^T--TVVVU{~}|}|||yyyyyxustuwy^T--TeTR^|~|{||zxzyyxvuuutttz^T--TzSklTl|}{{zxyzyxvuuutttuty^T--T`QW[|{|{yyyyywuuttstsssru^T--TvySVVTt|{zxyyywuuuuuutssrqqs^T--TvmSgeUl{yyyzzyvttttttttrqrqqt^T--TwfVVVSe~{xwyzxuttttttsqqqrrqqr^T--TwnTTmlTTl{yyzzywvuttssssrrrqrrqqp^T--Tw~~t[QUXUSS]lyzyyyxvuuuutssrrrqqqqqqqp^T--Tw~~~laTUSUTRTU_jzzzzyxuutsssssrrrqqqqqqqqqp^T--Tx~}~~~}zmkfgmluwyyyxxwutssssrrrqqqqqqqqqqqqp^T--Tx~}~~~~~~~}}~}}}}y{yyywvustsssssrrrqqqqqqqqqqqqp^T--Tx~~~~~~~~~~}||}||zyyxxxxvtssssssrrrqqppppqqqqqqqqp^T--Tx~~~~~~}}}}|}{{{{{zzyywvuutssssrqqqqqqppppqqqqqqqqp^T--Tx~~~~}}}}}|}{{zzzzxwvvuttssrrrqqqqqqqppqqqqqqqqqqp^T--Tx~}~~~~}}}|}|{{zzyyxusqttttssrqqqqqqqqqppqqqqqqqqqqp^T-.Tm}}}}~~~~|}~|{{{{zzzzyyttttttssrrrqqqqqqqqqqqqqqqqqqqqqp^T,-Sk~~}}}~~~~}}}|||~{yxxyzyvtsttutsqprrqqqqqqqqqqqqqqqqqqqqqqp\T,V[~~~~~}{{{{|z{yzzxuutttttssrrqrqqqqqqqqqqqqqqqqqqqqqqqp]TST|z{||xzyzzxuutttttssrrqqqqqqqqqqqqqqqqqqqqqqqqqpTT=T]~~|zyyxusutrqppqrqqqqqqqqqqqqqqqqqqqqqtqoWV@LTWmuwwwvvvvvvvvvvvuoptooqjmloojkgggggggggggggggggggggggggggggggcVUKBSSUTTTTTTTTTTTTTTTTTUTTSTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTSSTSC>ABBBBBBBBBBBBBCAABBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB=뀀%&&&%%$%56665%%$45BCCCB55$ffffffffff$$4BQRRRQB5$!\\\\\\\\\\%$4AQQcccQQB4#\OCDDEFFS\"'5EP`mrl_RG8*#~\O@@AABCOX)%$,4"&/6@HPZj{f\\\\\\\]^_``````_^]\\\\\\\]]]]^^]]\\\\\TFC?7-*1=CKQRQKA7/*,29>FL]Zg}f\\\fffffedcbaaaaa``_________^^^^^]]\\\\\TKD@8,'*38BDED@80*',4;HMUS]Wl\\\f\OOCCCDEFFFFFFFGGGGGGGGGFFEEDDDEEEEEEE<;6/($%*,67665/)%+,+24>STS^[\\f\CCCCCCBBAAAAAAA@@@@@@@@@@@@AABCDEEEEEE<82,'$%&&+(')*'##+-57:CGVTW]o]\aNCCAABAAAA@@@@@@?????@@@A@??ABDEEDDDDDCC<82,,)'%('%$%')*/577;CHQTV[no]\aJ?AAAAAAAA@@@@@@@@@@AAAABBBBCDDCCCCDCCCCA<820.,*)(''')-/1579@EFNTV[no]\aJ@AAAAAAAA@@@@@@@@@@AAAABBBCDDDDDCDDCCCCCA<831/-.,++-/13879AFHGLTV[no]\aKABAAAAAAA@@@@@@@@@@AAAABBBCDEEDDCDDCCCCCCA<653021112467<9DFIIGLTV[no]\aKABAAAAAAA@@AAAABBBBBBBBCCCCDDDDDCDDCCCCBBBB=;:8766778:>=>>??@BDGGGGHGFMTV[no]\aK@AAAAAAAA@@AAABBCCCCCCCCCDDDEEEEDDDCBBBBBBBA@@ABBCDCFHJGGGGGFFMTV[no\]`K@AAAAAAAA@AAAABBCCCCDCCDFFDEGILLHFDCBBBBBBBBBBCDDDDEGHKGGGGGEFMTV[no\]_J@@@@@@AA@@ABBBBBBBCFE@CEEER[[[[[[[[REEDAAABBCDEEEEFFFGGGFFFFFFMTV[no[^^J@@@@@@AAAAABBBCCCCEEFEEER[[[hhhhhh[[[RECCDCDEGGFGGHHGGGFFFFFAALSU[no[^^J@@@@@@AABBCBBCCCDDEBFDI[[[pp[[[JFEFGHHHGGGHGGFFFFFAAAALRU[no[^^J@@@@@@AABCCBCCCCDDEECM^[pp[VNHHHHHHGGGGGFFEFFAAAAALRU[oo[]]J@@@@@@DCCCDDDEEEFFFCF[Zpp^\LGHHHGGFFFFEDCBAAAAAALQT[oo[]]J@@@@D@DCCDDDEEEEEEDG]\usY^LGGGGFFFEEDCCBAAAAAALQT[oo[]]J@@@DDDDDDDDEEEEEEDCS\o~oYUHGGFFEEDDCBBBAAAAAALQTZoo[]]J@D@DDDDDDDDEEEEDEFFWZ^]KGGFFDDCCCBBBAAAAAFKQTZoo[]]J@DDDDDDEEEDEDEECEDPZrq[SCEEJCBBBBBBBBBJJJJKPTZoo[][JDDDDDDDEEEDDDDDCECV\[YADCCBBBBBBBBBBJJJJJOSZoo[][JDDDDDDDDDDDDDDDBEH[ed[CDBBBBBBBBBBJJJJJJJNRZoo[][JDDDDDDDDDDDDDDDACD\kk\IDBBBBBBBBJIIIIIIIILRZoo[[[JDDDDDDDDDDDDDCCBBO\oj\KCCBBBBBJJJIIIIIIIILQZpo[[[JDDDDDDDDDDDCCCBBAL\pj\MBDBBBBJJJIIHHHHHHHKQZpo[[[JDDDDDDDDDCCCCCCB@O\jj[NAFBBJJJJJIIHHHHHHHJPZpo[[[JDDDDDDDCCCCCCCCB@J[d~i[M@FJJJJJJIIIHHGGGGGJPZpo[[ZJDDDDDDCCCBBCBBB@FE]^~}~[XKHGIIIIIIIHHHGGGGFFJPZpo[ZYJDDCCCCCBBBBCCCDCGEU]ssZTIHHIIIIIIHHHGGFFFFFJPZpp[YXJDCCCBBBBBBBCDDDFHGN\`|`ZOIIIIIHHHHHGGGFFFFFFIQ[op[YWJCCCBBBBBBBCDEEFFGGGSXo}n[\IIIIIHHHHHGGGGGGFFFGIQ[op[YVJCCBBBBBBCDEFFFFHFHEJY[vp]^KIIIHHHHHGGGGGGGGGHHHIQ[op[XUJCBBBAAACDEFGGGFGEGFFLYZxv]^KKIHHHHHGGGGGGGGGGHHHHIQ[op[XUJCBCABABDEFGIIHGGDGFFGQZ[qpZ^KKKIHHHHGGGGGGGGGHIIHHHIQ[op[XUICBDAC@@FGGHIIHGGFFFEHER[[ats`ZZKKJJHGGGGGGGGGGGHHHIIHHHIQ[op[XUHAAEAEGGGGGGGGGGFFFFFAAANY_[\hqopmg_]]WNIIIHHGGFGGGGGGHHHHHHHHHHHQ[op[XUGAAFGFGGGGGGGGGGFFFAAAAAABJT[\YW\[]]TKGHHHHGGFFFGGGGGGHHHHHHHHHHHR[op[XUGGGGGGGGGGGFEEGGFFAAAAAA@@@CHKORRNKHGHHHHHHGGGGGGGGGGGGGHHHHHHHHHR\op[XUGGGGGGGGGFFFFEEDCBAAAAAAABDEHIJJKJIHHHHHHGGGGGGGGHHHHGGGHHHHHHHHIR\op[XUGGGGGGGGFFEFGECBAAAAAAAA@ADGHJIHHGGGHHGGGGGGGGGHHHHHHHHHHHHHHHHHIR\op[XUGGGGGGGGGFEEDCBAAAAAAAAAEIKJHIIHHGGGHHHGGFGGGHHHHHHHHHHHHHHHHHHHIS\op[XTGGGGGGGGGEDCBAAAAAAAAJJJJJJIIIIHHGGFFFGGGFGGGHHHHHHHHHHHHHHHHHHHIS\op[XSGGGGGGGGECBA@ABBAAAAJJJJJJJIIIIHHGGFEEFFGGGGHHHHHHHHHHHHHHHHHHHHIS[op[XRIGGFFFFEDCBBAAAAAAAJJJIIIIIHHHHGGFFFFFFGGGGHHHHHHHHHHHHHHHHHHHHHHS[qp[XQJFFFEDBBBBBBAAAAAJJJJJIIIHHHGGGGFFFEFFGHHHHHHHHHHHHHHHHHHHHHHHHHIUZv[XPLGFGECAABBBBAAJJJJJJJJIIIHHGGFFFGGGGGHHHHHHHHHHHHHHHHHHHHHHHHHHHJX]|[[PNLJFFFFDCBKKJJJJJJJJJJIIIHHGGGGGGGGGGHHHHHHHHHHHHHHHHHHHHHHHHHHHKZ_c[XQPRPNMMMMLLLKKKKKKKKKKJJJJIHHGGGGHIIJJIIHHHHHHHHHHHHHHHHHHHHKIHIX]ec[WPNPOOOOTMMMLLLSSSQQLQPPPPPPPPPPOOOOOOOOOOOOOOOOOOOOOOOOOOOOONQX]acZY\^ZZ[[[[\\[[[[[[[[[[ZZZZ[[[[[ZZYYYYYYZZ[[[[[[[[[[[[[[[[[[\]]Y\edcgddddddddccccccccccbbbcccddddccccccccccccccccccccccccccccbcf~vvvvvvvvvvppppppppppqleeeeeeko~qleeeeeeko~~sssssqlkkkkkkmottttttttttttttttttttttt{{|uqqqqqqqppoooooooppppppppppppppppppppppt{}xryurqqqqqqqqqqqqqqqqrrrrrrrrrrrrrqqqppppppty~tsnqw~urrqssssssstttttttssrrrrrrrrrrrqqqqppppppt{|xxoolpyrrrsqlleeeffggggggggggggggggggfffffgggggggotz~~|solqnnurruqeeeeeeedddddddefggggggggggfffffgggggggosyzxolkmnos{yrqskeedddddddeeeeffgggghhhhggggggggggggggggoty{xtojiimpqyyrqsicddddddddeeeeffgggghhhhggggggggggggggggioty{{vtrkgghmppyyrqsicddddddeeeefffgghhhhhhhgggggggggggggggggiotxz}~}{rqqkgfgimppyyrqsicddddddeeeefffgghhhhhhhggggggggggggggggggiotux|{{{zyxwuoongeehkmppyyrqsicddddddeeffgggghhhhhhhhggggggggggggggggggggloprvvuussqphgggffikmppyyrqsicdddddeeffggggghhhhhhhhgggggffggggggggggggghjkmnonmmkjiggggghikmppyyrqsidddeeeeffgghhhhihhhhhhhggggffffffggggggggggghhijiihhfffghgghijkmppyyqqsieeeffffffgghhhhhhhhhhhhggghhiiiihhhggggggggfgggggffgfffghhhhhjkmppyzqrrifffffffgggghhhhhhgggghghhhlpppppppplhhhgggggfggffffggghhhiiiiikmppyzprrhggggggggggghhhhhggggghhhlpppvvvvvvppplhgghgggggffffggghhhiiijjklopyzprrhggggggggggggggggggggghkpppzzpppihghhfffffffggghhiijjjjkkopyzprrhgggggggggggggggggggghkopzzpnjhggfffgggghhhhiijjjjjkkopzzppphggggggghhgggggggggghkotzzqsjhggffgghhhhiijjjjjjjkjnpzzppphggggggghhgggggggggginsx|rnjhhgghiiiiijjjjjjjjjkjnpzzppphggggggghhggggggggghlqw|wrmihhhhiiiijjjjjjjjjjkjnozzppphggggggghhggggggggginuz~}wqjhhhhiiijjjjjjjijjijjnozzppphggggggggggggggggghkmxynmihhhijjjjjjjjjiiiijjnozzppphggggggggggggggggginmnpjiiijjjjjjjjjjhiiiikoozzppphgggggggggggggggggiorrpliiijjjjjjjiihhiiiiloozzppphgggggggggggggggggjpuvooiiijjjjjjhghhhiiiiloozzppphgggggggggggggggggipwvooiiijjjjhhhhhhhiiiimpoyzppphgggggggggggggggggipwvooiiijjjhhhhhhhhiiiimppyzppphggggggggggggggggghouvooiiijhhhhhhhhhhiiiimppyzppphggggggggggggggggfgnrvoohhhhhhhhhhhhhiiiiimppyzppphggggggghhgggggggeflq|nqhhghhhhiiiiiiiijjjjmppyzppphggggggghhgggggggdekr{{{nohghhhhhiiiiiiiijjjjlopyzppphggggggghhgggggggddjqtqnlhghhhhhiiiiiiiiiiiikopyzppphgggggggggffffffgeehkr{xnmhhhhhhhhiiiiiiiiiiiikopzzppphgggggggggggggffgeegfpp|yopkhhhhhhhhiiiiiiiihhhhjnpzzppphgggggggfffffffffffggkou|{opiihhhhhhhhiiiiiihhhhhhjnpzzppphgggggggffffffffggggjgkpnyxmpiiihhhhhhhhiiiiiihhhhhhjnozzppphgggggggfffffffgggghighmmnq{zpmkjiiihhhhhhhhiiiiihhhhhhhjnozzpppgggggggggggggggghhhiijjjlmnpsuvw|uyonomkiiiiiiiiiiiiiiiihhhhhhhhjnozzpooggggggggggggggghhiiijjjjjlmnppqppppoljhhiiiiiiiiiiiiiiiihhhhhhhhjoozzpooggggggggghhhhhhhiijjjjjjjjjkkkllkjjihhhhiiiiiiiihhhhiiiihhhhhhhhjopzzoongggggggghhhhhhhiijjjjjjjjihhighhhhihhhhhiiiiiiiihhhhiiiihhhhhhhhkopzzonnggggggghhiiiiiijjjjjjjjjjiiihhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhkopzzonnggggggghhiiiiijjjjjjjjjjiihhhhhhhhhhhhhhiiiiiiiihhhhhhhhhhhhhhhhkppzzonnggghhhhhiiiijjjjjjjjjhhhhhhhhhhhiihhiiiiiiiiiiiihhhhhhhhhhhhhhhhkppzzonlhhhhhhhhijkkkkkkjjjihhhhhhhhhhhhiiihiiiiiiiiiiiihhhhhhhhhhhhhhhhkppzzpolihhhhhhijjjjjjjjjjjihhhhhhhhiiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhjop{zpolihhhhhhijjjjjjjjjhhhhhhhhhiiiiijiiiiihiihhhhhhhhhhhhhhhhhhhhhhhhioq}poljijiiiiijjjjjjhhhhhhhhhhhhhiiiiiiiiihhhhhhhhhhhhhhhhhhhhhhhhhhhhinq~ppljjjiiiiijjhhhhhhhhhhhhhhhhhgggghhiihhhhhhhhhhhhhhhhhhhhhhhhhhhhhinq~tpohikkjjjklllllllllllllllllkkkjjjjiihiihhhhhhhhhhhhhhhhhhhhhhhiiikppvtpomklmmmmlnnnnnnlllmmnmmmmllllkkkkkkkjjjjiiiiiiiiiiiiiiiiiiiijenonstoprrqqqqpppppppppppppppppppppppppppoooooopppppppppppppppppppoonpu~}yuttttttttttttttttttttttttuuuuutttssssssttttttttttttttttttttux} \ No newline at end of file diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.avif b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.avif new file mode 100644 index 000000000..e883c8c61 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.avif @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eb239f31ec8dbf5e97ad6f52670fca6497ae2a933822cfe724c75f66aaa2520b +size 2494 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.bit b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.bit new file mode 100644 index 000000000..7824a8572 --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.bit @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b7e30e04a935414a517baa2df06ab756da18ba7c291220d9d7c063297761ae82 +size 2195 diff --git a/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.png b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.png new file mode 100644 index 000000000..5ac780b8a --- /dev/null +++ b/tests/Images/Input/Heif/Av1/Conformance/libavif-webp-logo-scaled-reference.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc4c6dbe6bd92c5fce1e3e23700afa603ef04ed02edd336213ebba1e3bd84ba0 +size 5759