📷 A modern, cross-platform, 2D Graphics library for .NET
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

153 KiB

HEIF family implementation plan

Goal

Complete a production-quality, fully managed HEIF family image-format implementation for ImageSharp. The implementation must support still and animated HEIC, HIF/HEIF, and AVIF image files, interoperate with independent encoders and decoders, follow the existing ImageSharp architecture and code style, reuse existing ImageSharp infrastructure wherever its semantics match, and use SIMD for measured hot paths without maintaining a separate behavior model.

HEIF is the shared ISO BMFF-derived container. HEIC carries HEVC image items, AVIF carries AV1 image items, and .hif/.heif are container extensions whose payload codec must be determined from brands and item types rather than the filename. The completed implementation will support HEVC, AV1, and legacy JPEG image items. Other registered HEIF payload codecs must not be advertised unless they are implemented and independently verified.

Completion boundary

This plan has one PR completion gate. The phases below are dependency order and internal verification points only; none is a separately releasable or merge-complete subset. The PR is not complete until the complete matrix is implemented and independently verified.

Full completion includes:

  • XML documentation covers every type and contract in the HEIF implementation, including the AV1 and HEVC codec internals, and inline comments explain non-obvious container layouts, bitstream rules, numerical algorithms, SIMD choices, and interoperability constraints.
  • HEIF, HEIC, and AVIF brands and item types are detected without relying on file extensions.
  • HEVC/HEIC and AV1/AVIF decode and encode are complete for every image behavior included in the final scope.
  • Legacy JPEG image-item decode and encode are standards compliant for HEIF/HIF files.
  • Every bit depth and chroma format permitted by the exposed HEVC and AV1 profiles is supported, including 8, 10, and 12-bit and monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 paths.
  • Full- and limited-range conversion supports every valid signaled color-primary, transfer-characteristic, matrix-coefficient, and chroma-sample-position combination, including identity RGB signaling.
  • Every normative AV1 compression tool permitted by the final image scope decodes without an unsupported branch or silently skipped reconstruction.
  • The AV1 encoder implements real lossy and lossless compression with working quality and effort controls, complete mode decision, prediction, transform, quantization, entropy coding, and legal in-loop filter decisions.
  • Every normative HEVC compression tool permitted by the final image scope decodes across the exposed profiles, including the range-extension tools required for high bit depth and 4:2:2/4:4:4.
  • The HEVC encoder implements real lossy and lossless compression with working quality and effort controls, complete coding-tree, prediction, transform, quantization, CABAC, deblocking, and sample-adaptive-offset decisions.
  • Primary images, alpha auxiliary images, image grids, ICC and CICP color information, Exif, XMP, pixel aspect ratio, clean aperture, rotation, and mirroring are complete.
  • AVIF interoperability is independently verified with libavif and libaom, and HEIC interoperability is independently verified with a separately selected HEVC/HEIF implementation.

Checkboxes may be marked complete only when the implementation and the verification required by the corresponding phase exit gate are both recorded. Source presence, compilation, self-roundtrip, or a temporary vertical slice is not completion evidence.

Delivery dashboard

Last reconciled with the source tree on 2026-08-27 against the worktree based on commit 2bd81bb7c, including the completed AV1 transform architecture checkpoint. This dashboard is the authoritative delivery order. The detailed phase checklists below provide subsystem evidence; they do not override the current-stage marker or permit work to skip ahead.

Status meanings:

  • Complete: the implementation and its phase exit evidence are recorded.
  • In progress: usable implementation exists, but one or more required behaviors or verification gates remain open.
  • Not started: supporting primitives may exist, but the production format path is absent.
  • Current: the only work item that should be advanced before taking the next queued item.

Current development stage: Stage 3 — complete AV1 still-image decoding. The transform checkpoint is closed: forward transforms use one libaom-shaped operator architecture across scalar, Vector128, Vector256, and Vector512, inverse production traversal uses the verified scalar, Vector128, and Vector256 tiers, and implementation-mechanic type and file suffixes have been removed. Neither AV1 nor HEVC production encoding is implemented.

Immediate checkpoint: remove every remaining valid AV1 still-image unsupported branch and prove the complete decode matrix. Each syntax tool must be implemented through the established SIMD-first architecture with scalar fallback and verified with independent AVIF/libaom evidence across supported bit depths, chroma layouts, filters, grain, and color signaling.

Order Delivery stage State Delivered state Gate that remains open
1 Baseline, provenance, documentation, and public contract In progress Pinned codec references, a bounded image-only scope, encoder options, typed bit depth, decoder-option propagation, and extensive HEIF documentation exist. Complete the all-file documentation audit, record a fresh Release baseline, finish distinct public HEIC/AVIF save boundaries, and close API review.
2 Bounded HEIF item and image-sequence container In progress Still-item parsing, grids, auxiliary alpha, metadata properties, bounded image-sequence tracks, Identify, and all-sync AV1 sequence presentation are connected. Complete adversarial boundary coverage, remaining item/property behavior, reference-dependent sequence reconstruction, and the bounded sequence writer.
3 Still-image AV1 and HEVC decoding Current HEVC reconstruction reaches exact HM/libheif fixtures across the recorded 8/10/12-bit and chroma cases. AV1 includes reconstruction, filter, grain, color, and SIMD-first transforms using one static-generic stage architecture across the supported vector widths. Remove every valid AV1 still-image unsupported branch with independent vectors, followed by the remaining HEVC profile and Range Extensions conformance matrix.
4 Complete decoded presentation and animation In progress Shared SIMD-first AV1/HEVC color conversion, ICC application, grids, transforms, direct planar alpha composition, frame metadata, repetition, and independently decodable AV1 sequence samples exist. Close the full color/ICC cross-product, HEVC sequence decoding, AV1/HEVC reference-dependent samples, frame-local metadata/alpha behavior, and independent animated decode vectors.
5 AV1/AVIF encoding Not started RGB-to-planar conversion, forward transforms, OBU writer foundations, options, and container-writing infrastructure exist. HeifEncoderCore still rejects AV1. Implement a real independently decodable lossy/lossless AV1 payload and the complete AVIF item/metadata matrix.
6 HEVC/HEIC encoding Not started Shared input color conversion, options, and HEIF writer infrastructure exist. HeifEncoderCore still rejects HEVC. Implement a real independently decodable lossy/lossless HEVC payload and the complete HEIC item/metadata matrix.
7 Animated AVIF and HEIC encoding Not started Image-level repetition and per-frame duration contracts exist; bounded sequence parsing defines the required container surface. Write the bounded sequence container and encode frame dependencies, timing, repetition, alpha, presentation, color, and metadata through both completed payload encoders.
8 Hardening and release readiness Not started Focused tests, independent fixtures, and representative SIMD benchmarks exist for completed slices. Close the complete verification matrix, fuzz and malformed-input coverage, all Release/API/package checks, documentation and notices, and remove every placeholder or unsupported advertised path.

Immediate execution queue

  • Finish the libaom-shaped AV1 forward-transform architecture, measured production dispatch, inverse-tier correction, suffix cleanup, FeatureTestRunner matrix, and focused Release verification recorded below.
  • Current: complete AV1 still-image decoding for every valid still syntax path and independently verify the full bit-depth, chroma, compression-tool, filter, grain, and color matrix.
  • Complete the remaining HEVC still-image profile and Range Extensions matrix with exact independent native-plane and presentation evidence.
  • Close shared decoded presentation, ICC, alpha, grid, transform, metadata, and animated AV1/HEVC decode gates.
  • Implement and independently verify real AV1/AVIF still encoding.
  • Implement and independently verify real HEVC/HEIC still encoding.
  • Implement and independently verify animated AVIF and HEIC encoding through the bounded image-sequence surface.
  • Complete the public API, documentation, provenance, hardening, performance, and release gates, then run the full Release matrix.

Performance, allocation, documentation, and independent test work are part of each delivery stage. They are not deferred cleanup phases, and the next queued stage must not begin while the current stage has unverified code or an open correctness gate.

Detailed implementation ledger

  • Implemented foundation; conformance gate open: complete HEVC still-picture reconstruction and connect its native planes to the verified shared SIMD-first HEIF color pipeline.
    • Correct AV1 clipped-edge partition entropy handling and verify every block-mode cell from a real libavif AVIF against libaom 3.14.1.
    • Preserve the exact ICC payload from an independently encoded AVIF primary item.
    • Prove that a genuine non-sRGB AVIF profile changes decoded pixels and matches the independently converted source image within the documented AV1 tolerance.
    • Verify canonical-sRGB compaction and metadata skipping independently from ICC preservation and color conversion.
    • Replace the AV1 RGB/YUV arithmetic layer with JPEG's closed-generic static operator pattern in both directions: frame-scoped operator selection, in-place planar decode, RGB inputs with out component planes for encode, Vector512/Vector256/Vector128 traversal, one scalar tail, and JPEG's optimized byte-plane pack/unpack contracts for eight-bit pixels.
    • Keep codec row scheduling sequential and reuse frame-scoped allocator-backed buffers. Reuse JPEG's optimized UnpackIntoRgbPlanes contract for 8-bit encoding and pooled Rgb48 staging for 10/12-bit encoding so high-precision input is not truncated through an eight-bit or Vector4 intermediate.
    • Implement SIMD range expansion, chroma reconstruction, matrix/transfer conversion, clamping, and RGB packing for coefficient YCbCr, identity GBR, YCgCo, constant luminance, SMPTE ST 2085, and PQ/HLG ICtCp, with focused SIMD-lane/scalar-tail parity coverage.
    • Implement the H.273 V4 IPT-C2 and reversible YCgCo-Re/YCgCo-Ro transforms in both directions as closed SIMD operators, including their reduced RGB precision, signed integer lifting, full- and limited-range behavior, and 4:4:4 constraint.
    • Verify IPT-C2 against independently calculated matrix values and verify YCgCo-Re/YCgCo-Ro against exact 8/10/12-bit code values, limited-range black/white values, and SIMD-lane/scalar-tail parity.
    • Implement SIMD 4:2:0 and 4:2:2 encoder downsampling with odd-width and odd-height tails, and verify the stored 8/12-bit chroma samples against independently encoded full-resolution planes.
    • Move the H.273 parameter resolver, closed color operators, transfer functions, sample loading/storage, chroma filtering, and RGB packing into one shared HEIF color pipeline used by both AV1 and HEVC rather than maintaining codec-specific arithmetic copies.
    • Drive AV1 and HEVC through the same IHeifPlanarSampleBuffer<TSample> and HeifPlanarColorConverter architecture. Codec namespaces now adapt only native rows, sample precision, subsampling, crop windows, chroma positions, and codec color signaling; pooled row scheduling, chroma reconstruction/downsampling, fixed-point specialization, SIMD traversal, scalar tails, and RGB packing are selected by the shared HEIF driver.
    • Implement sequential pooled HEVC conversion in both directions for monochrome, 4:2:0, 4:2:2, and 4:4:4; independent luma/chroma precision; 8/10/12-bit full/limited ranges; and all six progressive 4:2:0 chroma sample locations. Eight-bit paths use JPEG's optimized RGB plane contracts, while high-bit-depth paths retain 16-bit packed pixels.
    • Implement allocation-free SIMD-first HEVC intra prediction for all 35 modes, 4/8/16/32 blocks, and 8/10/12-bit samples, including three-tap and strong-bilinear reference filtering, negative-angle reference extension, luma boundary filters, and SIMD horizontal transposition. Verify the normal and forced-scalar paths against one scalar oracle.
      • Match HM's channel-specific smoothing rules: combined 4:4:4 chroma uses the normal three-tap filter when selected, while strong bilinear smoothing remains restricted to luma syntax. Exact official RExt reconstruction passes at 8, 10, and 12 bits.
    • Implement allocation-free reconstructed-plane reference collection and normative substitution for complete, empty, and partially available borders, including asymmetric 4:2:2 availability units and the HEVC mode/size/component filter-selection rules.
    • Implement allocation-free SIMD-first HEVC inverse quantization for flat and custom scaling lists, 4/8/16/32 transforms, transform-skip scaling rules, extended precision, signed normalization shifts, and transform-range clipping. Store compact and transform-sized scaling matrices in one contiguous parameter-set allocation so the hot path can widen consecutive weights without coordinate division.
    • Derive each reference-unit availability flag from coding-tree, slice, tile, constrained-intra, and picture-boundary state, then connect prediction to transform-unit traversal and the bounded item decoder lifecycle.
    • Register the bounded hvc1 still-image decoder, reconstruct the selected PPS/SPS picture, crop the conformance window directly in the pooled row converter, and pass the native planes through the shared SIMD-first H.273 pipeline.
    • Compare native HEVC planes byte-for-byte with HM 18 output for four item payloads extracted from independently produced HEIC files: two 8-bit 4:2:0 grid tiles, a cropped 8-bit 4:2:0 grid tile, and an 8-bit 4:4:4 wavefront tile.
    • Compare complete HEIC presentation output exactly with libheif 1.23.1 and libde265 1.1.1 for single images, multi-tile grids, an auxiliary alpha image, ICC-bearing images, 4:2:0, and 4:4:4 output. Keep the exact HM native-plane tests as the independent codec-reconstruction oracle.
    • Add permanent official HEVC 8/10/12-bit monochrome, 4:2:0, 4:2:2, and 4:4:4 fixtures with exact native-plane references and published per-plane digests. All 10 first pictures from the Sony GENERAL RExt conformance streams match every Y, Cb, and Cr sample with normal SIMD dispatch and with all hardware intrinsics disabled.
    • Extend the permanent HEVC matrix to every exposed profile and each Range Extensions tool not individually isolated by the Sony GENERAL streams, with exact native-plane references from HM or another independent decoder.
    • Implement the shared SIMD-first YUV/CICP paths in both directions for AV1 and HEVC bit depths, chroma formats, ranges, matrix coefficients, transfer functions, color primaries, and chroma positions, with scalar fallback only after the available vector widths are exhausted.
    • Complete the independently generated AV1 and HEVC conformance matrix for every supported color-signaling combination before closing the Phase 5 color exit gate.
    • Apply ICC conversion only after the SIMD YUV/CICP stage, alpha composition, grid assembly, and presentation transforms have produced the presented RGB image; retain ImageSharp's shared ICC converter and optimize reusable bulk kernels rather than creating a HEIF-specific color-management implementation.
    • Verify ICC preservation, conversion, compaction, and metadata skipping for grids, alpha-composited images, every presented sequence frame, and the completed HEVC path.
      • Independently encode AVIF primary, grid, alpha-auxiliary, and two-frame sequence vectors with libavif 1.4.2/libaom 3.14.1; verify exact profile preservation, non-sRGB conversion, canonical-sRGB compaction, non-sRGB compact preservation, metadata skipping, every presented frame, and unchanged composed alpha.
      • Repeat the complete ICC preservation, conversion, compaction, and skipping matrix for HEVC primary, grid, alpha-auxiliary, and sequence paths. Exact ICC-bearing still-image presentation is covered, but it does not close these separate color-management contracts.
    • Benchmark representative 8/10/12-bit end-to-end AVIF and HEIC color pipelines with and without ICC conversion, recording absolute timings and allocations before marking Phase 5 complete.
      • Add permanent frame-wide Av1ColorConversionBenchmarks coverage for sequential 1920x1080 YUV 4:2:0 conversion in both directions at 8, 10, and 12 bits, including managed-allocation reporting.
      • Record the .NET 10 short-run AV1 baseline on 2026-08-25. SIMD decode measured 4.025/2.803/2.742 ms and encode measured 3.750/2.555/2.450 ms at 8/10/12 bits with 488-744 B allocated per frame. Forced-scalar decode measured 31.15/29.47/28.40 ms and encode measured 19.19/15.77/15.14 ms, making the normal SIMD pipeline 7.7-10.5 times faster for decode and 5.1-6.2 times faster for encode on this machine.
      • Add permanent frame-wide HevcColorConversionBenchmarks coverage and record the .NET 10 short-run plane-conversion baseline on 2026-08-26. SIMD decode measured 1.141/2.640/2.668 ms and encode measured 2.089/2.481/2.412 ms at 8/10/12 bits with 504-712 B allocated per frame. Forced-scalar decode measured 13.42/28.35/27.60 ms and encode measured 14.82/14.67/14.45 ms, making the normal SIMD pipeline 10.3-11.8 times faster for decode and 5.9-7.1 times faster for encode on this machine.
      • Re-verify the unified AV1/HEVC driver with all 127 focused color and exact HEIC presentation cases under normal SIMD dispatch and with all hardware intrinsics disabled. Re-run all 12 permanent 1920x1080 plane-conversion benchmarks; BenchmarkDotNet could not hold the configured power plan and the CPU changed performance state during warmup, so do not replace the controlled baselines above with those throttled sustained means.
      • Add representative ICC-enabled AVIF cases and the equivalent HEVC cases after the HEVC reconstruction path is complete.
  • Completed: remove production ToArray() materializations from the HEIF implementation and make every retained copy an explicit ownership decision.
    • Store AV1 palette colors and HEVC short-term reference-picture state inline at their normative fixed bounds.
    • Retain only the AV1 configuration sequence-header payload required for item/sample equivalence instead of every configuration OBU.
    • Allocate HEVC emulation-prevention positions once at the exact validated count without a growing list plus array copy.
    • Use ImageSharp's pooled ChunkedMemoryStream and direct chunk writes for the legacy JPEG item payload.
    • Keep exact owned arrays only where ICC, Exif, XMP, or retained property state must outlive pooled parser buffers.
  • Completed: eliminate per-call scratch allocation across every AV1 forward and inverse 1-D transform, replace shared mutable transform scratch with operation-owned workspace, and complete the libaom-shaped SIMD-first transform paths with scalar fallback.
    • Inventory the existing hot-path scratch use: 15 forward/inverse transformer types contain 30 stackalloc int[...] sites.
    • Refactor every affected forward transform to reuse caller-owned workspace and its output buffer for staged ping-pong storage.
    • Refactor every affected inverse transform to reuse caller-owned workspace and its output buffer for staged ping-pong storage.
    • Remove the shared mutable Av1ForwardTransformer.TemporaryCoefficientsBuffer and give each concurrent transform operation exclusive workspace ownership.
    • Replace the instance-transformer interface with stateless forward and inverse operator structs following the JPEG color-transform static-generic operator pattern.
    • Select transform type, size, bit depth, and ISA once at the 2-D block boundary rather than dispatching through an interface for every row and column.
    • Port the DCT4/8/16/32/64, ADST4/8/16, and identity4/8/16/32 stage networks from the pinned libaom scalar and Highway sources into one static-generic operator architecture.
    • Implement paired add/subtract and whole-butterfly primitives for scalar, Vector128, Vector256, and Vector512 values, including saturated packed arithmetic and shared widening work at each supported SIMD width.
    • Implement the complete libaom two-dimensional load, flip, shift, axis-transform, transpose, rectangle-normalization, promotion, and 64-point coefficient-retention pipeline without per-block allocation.
    • Port the applicable libaom bulk inverse-transform kernels using the same tables, rounding, saturation, and clipping rules as the scalar oracle.
    • Use normal ImageSharp descending-width dispatch and require the actual packed arithmetic ISA when selecting packed Vector512<short> traversal.
    • Document scratch ownership, stage-buffer alternation, fixed-point rounding, lane layout, transposition, and scalar fallback decisions at their implementation points.
    • Remove the separate SIMD files, width-specific forward contracts, and sample-representation suffixes so each transform operator owns one behavior model.
    • Verify every 1-D operator representation and every valid 2-D size/type/bit-depth combination through FeatureTestRunner with normal hardware, AVX-512 disabled, AVX disabled, and all hardware intrinsics disabled.
      • The focused Release run passes all 511 forward and inverse transform cases. The suite covers DCT, ADST, and identity operators, packed overflow-sensitive inputs, padded 2-D input strides, all valid transform configurations, 8/10/12-bit dispatch, inverse reconstruction, and the zero-allocation block contract.
    • Benchmark the production 32x32 DCT path after the complete paired stage port with preferred 256-bit and 512-bit widths.
      • On the measured .NET 10 AVX-512 host, the 8-bit path measured 1.334 microseconds at 256 bits and 1.349 microseconds at 512 bits. The 12-bit path measured 2.977 microseconds at 256 bits and 2.008 microseconds at 512 bits. BenchmarkDotNet reported no managed allocation for any measured path, so production retains the normal runtime-selected width instead of a transform-type or bit-depth patch table.
  • Queued: restore bounded animated HEIC and AVIF image-sequence scope, including the required image-level and per-frame metadata contracts, without introducing unrelated ISO BMFF surfaces.
    • Reconcile the top-level still-image-only scope with the required animated HEIC and AVIF completion boundary before sequence implementation begins.
    • Define the ImageSharp image-level sequence metadata and per-frame metadata contracts, including observable timing, repetition, color, alpha, orientation, and profile behavior.
      • HeifMetadata now carries repetition and root-frame animation behavior through FormatConnectingMetadata. HeifFrameMetadata carries frame duration through FormatConnectingFrameMetadata; frame-local color, alpha, orientation, and profile state remain owned by the existing ImageFrameMetadata contract. The focused Release suite passes all 40 encoder, image metadata, and frame metadata tests.
    • Identify and document the minimum normative ISO BMFF track, sample-description, sample-location, dependency, and timing syntax required by conforming HEIC and AVIF image sequences.
      • The bounded syntax matrix below is derived from ISO/IEC 23008-12:2022 clauses 7 through 10, the codec-specific HEVC Annex B, AVIF 1.2 section 3, AV1-ISOBMFF 1.2 section 2, and the pinned libavif read.c and write.c sequence paths.
    • Parse and write only that bounded image-sequence syntax; do not add audio, arbitrary media, editing, fragmentation, streaming, or general presentation APIs.
      • Share one bounded 32/64-bit box-header reader between the existing item path and the image-sequence path, including short-read handling and parent-boundary validation.
      • Select one enabled pict master track without materializing unrelated tracks, accepting conforming image-sequence tracks that omit track_in_movie, then parse its mvhd/tkhd/mdhd/hdlr, self-contained dref, av01 or hvc1 sample entry, codec configuration, mandatory ccst, and bounded repetition edit.
      • Resolve stsc, stco/co64, stsz/stz2, stts, and stss into one exact value-type descriptor array capped by DecoderOptions.MaxFrames; validate complete run/count syntax through one allocator-owned sequential scratch buffer.
        • Release verification passes the libavif-shaped two-sample parser fixture, the one-frame retention boundary, and a sample offset/length beyond the file. Pinned upstream libavif files independently verify 8-bit and 12-bit sequence identification, alpha-track and Exif/XMP retention, infinite and finite repetition, and safe omission of an unrelated audio track. The parser performs no per-entry allocation and does not buffer moov, mdat, unrelated track payloads, or complete attacker-sized tables.
      • Parse HEVC composition offsets and hidden samples from ctts/cslg, while rejecting the ctts box prohibited for AV1 tracks.
        • Release verification covers signed and unsigned composition-offset syntax, hidden-sample visibility, composition-time calculation, required cslg and edit-list signaling, complete run counts, and the AV1 prohibition without buffering either table.
      • Parse optional direct dependencies from refs sample groups into a compact retained-sample dependency graph.
        • The parser validates complete sbgp runs and sgpd entry lengths, versions, indices, positive identifier uniqueness, decode order, sync-sample independence, and ccst reference limits. It uses pooled value-type sort indexes plus one exact flat reference array, with no per-sample collections or attacker-sized table materialization.
      • Parse track presentation, color, and HDR properties through the same typed property parser used by still-image items.
        • Selected visual sample entries now retain ICC and CICP color profiles, pasp, clap, irot, imir, clli, mdcv, cclv, amve, reve, and ndwt. Fixed-size properties use the parser's existing scratch buffer; only retained ICC bytes use bounded pooled staging before entering ImageSharp's existing profile model.
      • Parse bounded sequence metadata items.
        • Selected image tracks now retain implicitly associated Exif and unencoded application/rdf+xml XMP items from bounded iinf/infe and iloc declarations, including file-relative and idat-relative extents. Item identifiers and extent descriptors use allocator-owned temporary storage; only the exact retained profile payloads survive parsing. DecoderOptions.SkipMetadata bypasses metadata parsing, SegmentIntegrityHandling follows the shared ancillary/image-data policy, and structural errors that prevent safe track parsing remain fatal. The focused Release sequence-parser suite passes all 15 tests.
      • Connect the parsed sequence index to HEIF detection, Identify, frame decode, alpha matching, and frame metadata without changing still-image source selection.
        • Recognize supported avis, hevc, and hevx sequence brands while continuing to reject layered HEVC and JPEG sequence brands.
        • Identify bounded sequence dimensions, frame count, timing, repetition, codec precision, color, HDR, pixel aspect ratio, Exif, and XMP state.
        • Decode all-sync independently decodable AV1 samples into directly adopted ImageSharp frames without cloning complete pixel buffers.
        • Match auxiliary alpha samples by exact decode duration, visibility, and presentation time, and validate premultiplication track identity.
        • Require unity movie and track matrices so image presentation remains on the optimized clap/irot/imir path without a movie compositor.
        • Apply the shared DecoderOptions contract consistently to still items, nested payload codecs, grids, metadata properties, and sequence samples.
          • Strict rejects recoverable ancillary and image-data errors, IgnoreAncillary suppresses only ancillary failures, and IgnoreImageData additionally permits failed image properties or samples to be omitted. SkipMetadata avoids optional property and item-payload validation, while cancellation and the caller configuration flow into nested JPEG and AV1 decoders. Target scaling and ICC conversion remain presentation-level operations after item or grid composition. The focused Release matrix passes all 19 new still-image policy cases, all 3 new sequence-sample cases, and the complete 37-test sequence-parser suite; the Release test-project build completes with zero errors.
          • Apply the same recovery boundary to still-image item relationships, coded payloads, primary-item thumbnail fallback, and optional alpha composition.
            • Unknown item-reference types are skipped within their validated child boundaries. cdsc failures follow ancillary policy and are not parsed when metadata is skipped. dimg, auxl, prem, and thmb failures follow image-data policy. IgnoreImageData can omit an unreadable alpha plane or recover from a failed primary payload through a valid registered thumbnail, but decoding still fails when no color presentation remains. Real AVIF fixtures cover corrupt alpha payloads, corrupt alpha relationships, malformed Exif relationships, and strict, ancillary-only, image-data, and metadata-skipping behavior.
        • Complete reference-dependent AV1 and HEVC sample reconstruction and independent sequence vectors.
      • Write the same bounded movie, track, sample-description, location, dependency, timing, repetition, alpha, and metadata syntax from ImageSharp frames.
    • Decode frame dependencies, durations, repetition, frame-local auxiliary images, and frame-local metadata into the existing ImageSharp multi-frame model.
    • Encode ImageSharp frames, durations, repetition, frame-local auxiliary images, and frame-local metadata as independently decodable HEIC and AVIF image sequences.
    • Add malformed-sequence boundary coverage and independently verify animated inputs and outputs with pinned HEIC and AVIF implementations.
  • Queued: complete the AV1 and HEVC encoding paths and remove the explicit HeifEncoderCore encoding stubs only after each emitted payload passes independent decode.
    • Complete every AV1 encoder task and exit gate in Phase 6; a smallest-valid fixed coding subset is only an intermediate milestone.
    • Complete every HEVC encoder task and exit gate in Phase 7; a smallest-valid fixed coding subset is only an intermediate milestone.
    • Connect both payload encoders to the bounded HEIF writer with the selected bit depth, chroma layout, range, color signaling, alpha, metadata, and animation state.
    • Replace each NotSupportedException branch only when the corresponding payload is accepted by the pinned independent decoder and the ImageSharp decoder.
    • Verify that every public quality, effort, lossless, bit-depth, chroma, alpha, and metadata option changes or constrains the encoded output exactly as documented.
  • Completed: define and document the HEIF encoder option contract using the established ImageSharp encoder pattern.
    • Confirm that IHeifEncoderOptions has only one concrete implementation and remove the unnecessary interface.
    • Document the default, valid range, special values, invalid-value behavior, and format-dependent restrictions of every retained option using observable API behavior only.
    • Pass HeifEncoder directly to HeifEncoderCore, matching the JPEG, PNG, and WebP encoder-core contracts and avoiding interface dispatch.
    • Verify construction-time range validation and legacy-JPEG codec-boundary restrictions with focused tests before the Phase 1 API-review gate is marked complete.
  • Completed: lock public HEIF image metadata to the supported component bit-depth contract.
    • Replace the unrestricted integer bit depth with HeifBitDepth and preserve the 8-bit default.
    • Resolve format-connecting component precision to the nearest supported 8/10/12-bit output without widening the public value domain.
    • Reject HEVC configuration records outside the exposed 8/10/12-bit profile matrix at the external parse boundary.
    • Verify defaults, cloning, format-connecting conversion, pixel-type projection, and current HEIC/HIF/AVIF Identify results.
      • Release build: 0 errors. Focused metadata and Identify tests: 42 passed, 0 failed.

Gain maps, progressive/layered images, sample transforms, and experimental extension brands require explicit conformance and API decisions. They do not create permission to omit any valid color, compression, or bit-depth path from the PR. The container reader must skip unsupported optional extensions safely and reject an unsupported essential property with a useful error.

Container scope

The container implementation is a deliberately narrow HEIF image-format reader and writer, not a general ISO BMFF framework. Implement only the box syntax and relationships required for supported HEIF image items and bounded HEIC/AVIF image sequences.

In scope are the file type, metadata, item location/data, item information, item properties, item references, primary-item selection, idat/mdat payload storage, grids, auxiliary alpha, presentation transforms, color properties, and Exif/XMP paths required by still images. Image sequences additionally include only the brands, tracks, sample descriptions, sample locations, decode dependencies, timing, repetition, and frame-local metadata required to map HEIC and AVIF sequences to ImageSharp frames.

Still files produce one presented primary image. Supported image sequences produce one ordered ImageSharp frame collection with bounded duration, repetition, frame dependency, alpha, color, orientation, and profile state. Supporting items are decoded only when required to construct or describe those presented frames. The implementation does not expose arbitrary HEIF image collections, bursts, pages, audio, or non-image media.

Out of scope are audio, arbitrary video, edit lists, fragments, streaming profiles, general presentation APIs, and reusable movie, track, or sample-table object models. Track, sample, timing, and inter-frame codec state may exist only inside the bounded HEIC/AVIF sequence decoder or encoder that owns it. Unknown optional boxes remain bounded and skippable; unsupported essential image or sequence syntax must fail with a useful image-format error.

Implementation rule: do not introduce a reusable general-purpose ISO BMFF box hierarchy, track model, or media parser. Add box syntax directly to the bounded HEIF model only when a supported still image, image-sequence frame, relationship, property, metadata path, or conformance fixture requires it. Each addition must name the image behavior it enables and have a focused image-format test.

An ISO BMFF construct may be added only when all of the following are true:

  1. A conforming supported still image or HEIC/AVIF image sequence requires it to produce or describe presented ImageSharp frames.
  2. Its owning image item and its effect on the decoded or encoded image are explicit.
  3. It can be parsed or written as bounded state owned by the HEIF image-item or image-sequence path without adding a general media abstraction.
  4. Independent still-image or animated-image fixtures exercise the behavior it enables.

Encountering a box in libavif, ISO BMFF, or a third-party file is not by itself a reason to port it. Constructs not required by the supported still-image or bounded image-sequence presentation must be skipped when optional or rejected when essential.

Codec-configuration rule: parse av1C and hvcC as properties of coded image items and as the bounded decoder configurations referenced by supported image-sequence sample descriptions. Validate profile, level, bit depth, chroma layout, and parameter-set/OBU declarations against the associated item or sequence samples. Do not expose visual sample entries, sample descriptions, layer selection, sample groups, or timing as general public or reusable ISO BMFF models.

AV1 sequence headers and HEVC VPS/SPS/PPS structures remain in scope because they are codec syntax required by image items and image-sequence frames. Retained reference-frame state is permitted only for the lifetime of a bounded HEIC/AVIF sequence decode or encode operation.

Bounded image-sequence syntax

The sequence reader and writer may retain only the following syntax and the resolved image behavior listed beside it:

Syntax Required image behavior
ftyp sequence and structural brands Recognize avis AV1 sequences and the non-layered hevc/hevx HEVC sequence profiles. avio can additionally signal an all-sync AV1 sequence. Layered hevm/hevs, JPEG jpgs, arbitrary video brands, and brands for unimplemented codecs remain unsupported until their image payload and presentation requirements are implemented.
moov/mvhd, trak/tkhd, and mdia/mdhd/hdlr Select an enabled pict master image-sequence track without requiring the optional track_in_movie flag; retain its displayed dimensions, media time scale, media duration, and movie-time-scale track duration. Require unity movie and track matrices because arbitrary movie-canvas composition is outside image-format scope; use clap/irot/imir for supported image presentation. Inspect unrelated tracks only far enough to identify and skip their non-image handlers.
minf/dinf/dref and stbl Accept only self-contained sample data references and own the bounded sample-table state for one selected image sequence plus an optional linked auxiliary-alpha sequence. No reusable data-reference, media-information, or sample-table API is created.
stsd and one supported visual sample entry Require exactly one av01 entry for AVIF or one non-layered hvc1 entry for HEIC. Retain only its dimensions, codec configuration (av1C or hvcC), image presentation/color/HDR properties, and mandatory version-zero ccst coding constraints. Reject an unsupported essential configuration rather than treating it as generic video.
stsc, stco/co64, and stsz/stz2 Resolve each declared image sample directly to a validated file offset and length. Expand run tables once into a compact frame-owned descriptor array bounded by DecoderOptions.MaxFrames; never buffer the movie or complete mdat.
stts Expand decode-order sample durations against the media time scale into exact per-frame rational delays. A sequence with advisory timing still maps every visible sample to an ImageSharp frame.
optional ctts version 0/1 and cslg Retain only composition offsets and the HEIF hidden-sample condition required to determine presented frame order and visibility. No general composition timeline is exposed.
optional stss Mark the one-based random-access samples; absence means every sample is a sync sample. The first retained frame must have a valid decode path from a sync sample.
optional sgpd/sbgp with grouping type refs Resolve direct-reference sample identifiers for bounded dependency validation and random access. Other sample-group types are ignored unless a later image-format requirement explicitly brings them into scope.
optional edts/elst Interpret only normal-rate image-sequence presentation and the HEIF repetition flag. Reject edits that require arbitrary media splicing, rate changes, or a general edit timeline.
optional tref entries auxl, prem, thmb, and cdsc Link only image alpha, premultiplication, image thumbnails, and image metadata. Auxiliary samples are matched to color frames by their decoded time; audio and every unrelated reference type are ignored.
optional track-level meta and image metadata linkage Apply sequence-wide Exif/XMP profiles from the bounded track metadata model. Per-frame metadata is limited to supported metadata items/groups or codec metadata that can be mapped to ImageFrameMetadata; generic timed-metadata tracks are not exposed.

The implementation must validate uniqueness, versions, entry counts, run totals, one-based indices, integer products, cumulative durations, offsets, and lengths before creating retained arrays. Temporary box payloads use the configured MemoryAllocator; persistent sample descriptors are allocated once at the validated retained-frame count. Parsing must skip unrelated boxes by their bounded length and must not copy an entire moov or mdat payload.

Reference hierarchy

Use the references in this order when behavior differs:

  1. The published ISO BMFF, HEIF, HEVC, AV1, AV1-ISOBMFF, and AVIF requirements are normative.
  2. The official AOM AV1 Codec Library is the primary implementation reference for AV1 decode, encode, high-bit-depth behavior, tests, and optimized scalar/SIMD algorithms. Pin one reviewed commit before porting. Use its architecture-specific paths to shape the managed SIMD-first production architecture and its scalar C paths to define the equivalent fallback behavior.
  3. The local D:\GitHub\AOMediaCodec\libavif checkout is the AVIF container, metadata, color-conversion, grid, alpha, and interoperability oracle. At inspection time it identifies itself as 1.4.2-devel.
  4. The official HM reference software at commit 9c1f298659ab0cee9dc13d23d0304221575410b9 is the primary scalar implementation reference for HEVC syntax, reconstruction, and the complete Range Extensions profile matrix. Android libhevc v1.6.0 at commit c83a76b084498d55f252f48b2e3786804cdf24b7 is a secondary production-C comparison for its supported common 8-bit 4:2:0 paths; its decoder rejects other chroma formats and bit depths and therefore is not the feature-coverage authority. The HEVC specification remains normative, and an independent HEIC container implementation is still required as an interoperability oracle. Do not copy from GPL or otherwise incompatible sources.
  5. Existing ImageSharp codecs are the authority for ImageSharp API shape, memory ownership, stream behavior, cancellation, resource limits, pixel conversion, tests, and SIMD dispatch.

The linked ImageSharp discussion establishes the project constraint: the shipped implementation is purely managed and other codec libraries are references, not native runtime dependencies. libaom is the official encoder/decoder implementation reference for AV1, but it does not parse the HEIF container or implement HEVC. libavif dispatches AV1 work to external codec libraries, so it remains an observable AVIF/container oracle rather than the source for every AV1 algorithm.

The AOM source is distributed under the BSD 2-Clause License and the Alliance for Open Media Patent License 1.0. HM is distributed under its BSD license with an explicit patent-rights disclaimer, while the reviewed Android libhevc source is distributed under Apache-2.0. Before porting further code, record the exact upstream file, commit, applicable license/patent notice, and corresponding managed file or method. Audit the existing SVT-AV1-attributed WIP separately rather than relabeling it as libaom-derived. Update THIRD-PARTY-NOTICES.TXT before any referenced implementation code is merged. A pure managed HEVC implementation does not remove HEVC patent or licensing obligations, so those must be resolved before the HEIC work is considered releasable.

Pinned reference and baseline snapshot

The initial post-merge snapshot was established on 2026-08-24:

  • the official libaom reference is tag v3.14.1, commit 03087864cf4bea6abb0d28f95cf7843511413d8f, matching the revision selected by the local libavif ext/aom.cmd dependency script;
  • the local libavif container, color-conversion, and interoperability oracle is commit 092276ce89098ead06db80975173191e5fee1826, described as v1.4.2-66-g092276ce;
  • the official HM reference software is commit 9c1f298659ab0cee9dc13d23d0304221575410b9; its decoder and profile tables cover the HEVC Range Extensions chroma, bit-depth, intra, and still-picture profiles required for the final HEIC feature matrix;
  • the official Android libhevc reference is v1.6.0 at commit c83a76b084498d55f252f48b2e3786804cdf24b7; it remains a secondary common-path comparison because its SPS parser accepts only 8-bit YUV 4:2:0;
  • the official ITU-T H.274 (V4) recommendation from January 2026 is the normative semantics reference for mastering-display and content color-volume fields reused by still-image item properties; its video-SEI persistence and cancellation behavior is outside this container scope;
  • an independent HEIC container interoperability oracle and the complete external HEIC fixture-generation matrix remain to be pinned before HEIC support can pass its final integration gate;
  • dotnet build ImageSharp.sln -c Release --no-restore -m:1 -v minimal succeeds with no errors after the upstream compatibility fixes; and
  • the existing HEIF-focused test run executes 8,198 cases, with 8,184 passing and 14 failing. Thirteen failures are isolated to the WIP AV1 YUV conversion tests, and one is the existing legacy JPEG HIF reference-image mismatch. Golden artifacts have not been changed.

This snapshot pins or classifies the available references and failures; it does not complete Phase 0. The full WIP provenance map, disabled-test inventory, HEIC container interoperability selection, and feature-state matrix remain required.

Provenance map in progress

Managed implementation Normative behavior Reviewed implementation reference Use
HeifColorConversionParameters, HeifColorConverter, IHeifPlanarSampleBuffer, HeifPlanarColorConverter, Av1YuvConverter, and HevcYuvConverter ITU-T H.273 (V4) equations 14-16 and 27-84, including limited/full-range scaling, chromaticity-derived equations 39-47, YCgCo equations 51-57, constant-luminance equations 66-75, IPT-C2, YCgCo-Re/YCgCo-Ro, and the PQ/HLG ICtCp matrices; AV1 section 6.4.2 and HEVC Annex E chroma sample positions Official ITU-T H.273 (V4) (07/2024); libavif src/reformat.c and src/colr.c at 092276ce89098ead06db80975173191e5fee1826; libaom aom/aom_image.h at 03087864cf4bea6abb0d28f95cf7843511413d8f SIMD-first closed operators provide full/limited-range conversion at 8, 10, and 12 bits, with an equivalent scalar fallback and tail. Decode and encode cover every H.273 V4 matrix code point through 17: identity, YCgCo, coefficient-based NCL, both fixed and chromaticity-derived constant/non-constant-luminance systems, SMPTE ST 2085, ICtCp, IPT-C2, YCgCo-Re, and YCgCo-Ro. One closed generic planar driver owns pooled sequential row traversal, chroma reconstruction/downsampling, fixed-point specialization, SIMD dispatch, scalar tails, and packed-pixel conversion for both AV1 and HEVC adapters. Shared paths cover monochrome, YUV 4:2:0, 4:2:2, and 4:4:4; AV1 centered/co-sited placement; and all six HEVC progressive 4:2:0 sample locations. Identity and reversible lifting enforce their full-chroma constraint. Limited-range YCgCo retains the 219-code scale inherited from its R/G/B inputs instead of applying YCbCr's unrelated 224-code chroma range. YCgCo-Re and YCgCo-Ro apply range adjustment at reduced-precision RGB code values before signed integer lifting. Chromaticity derivation uses every defined H.273 primary and matches libavif's BT.709 fallback for unspecified or reserved primaries. ICtCp and IPT-C2 inverses come from the exact H.273 matrices rather than unrelated display conversions. Closed operators provide Vector512, Vector256, and Vector128 implementations with one scalar tail. Eight-bit conversion uses JPEG's RGB plane pack/unpack contracts; high-bit-depth conversion retains native 16-bit packed pixels.
HeifTransferFunctions ITU-T H.273 (V4) Table 3 transfer characteristics 1-18 Official ITU-T H.273 (V4) (07/2024); libavif src/colr.c at 092276ce89098ead06db80975173191e5fee1826 Apply every HEVC/AV1-signallable transfer function required by constant-luminance, ICtCp, and IPT-C2 conversion. Retain the H.273 normalized PQ and HLG definitions; do not import libavif's display-oriented 203-nit scaling or HLG OOTF into codec sample interpretation. Use libavif's midpoint convention only for the non-bijective zero code of the two logarithmic curves.
ObuReader.ReadSequenceHeader, ReadUncompressedFrameHeader, decoder-model parsing, and operating-parameter consumption AV1 sections 5.5.2 through 5.5.4 sequence timing and decoder-model syntax, section 5.9.2 uncompressed frame-header syntax, and section 5.9.31 temporal-point syntax libaom av1/decoder/decodeframe.c functions av1_read_decoder_model_info, av1_read_op_parameters_info, read_temporal_point_info, and read_uncompressed_header, plus common/av1_config.c, at 03087864cf4bea6abb0d28f95cf7843511413d8f Read the normative 32-bit decoding-tick field; consume operating-point buffer delays and the layer-applicable frame removal times needed to keep a non-reduced still-image sequence aligned; and read presentation time only under the normative decoder-model condition. Retain no scheduling or playback behavior from those values and introduce no ISO BMFF timing, track, sample-table, or sequence surface.
Av1FrameBuffer high-bit-depth sample layout and Av1YuvConverter 10/12-bit packed-pixel conversion AV1 section 6.4.1 bit depth and H.273 sample-range scaling libaom aom_scale/yv12config.h, av1/common/idct.c, and av1/common/reconintra.c at 03087864cf4bea6abb0d28f95cf7843511413d8f; libavif src/avif.c and src/reformat.c at 092276ce89098ead06db80975173191e5fee1826 Establish two-byte native sample storage with sample-unit strides for 10/12-bit reconstruction and use ImageSharp's existing Rgb48 pixel-operation paths in both directions so packed-pixel staging does not reduce high-bit-depth samples to eight bits.
Av1PredictionDecoder, Av1IntraPredictor, Av1ChromaFromLumaContext, Av1ChromaFromLumaPredictor, Av1PartitionInfo, and the DC, directional, Paeth, smooth, filter-intra, and chroma-from-luma predictor operators AV1 sections 7.11.2 and 7.11.2.3 intra prediction libaom aom_dsp/intrapred.c, av1/common/reconintra.c, av1/common/av1_common_int.h, av1/common/blockd.h, av1/common/cfl.c, and av1/common/cfl.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Design the shared predictor contract, sample layout, reference preparation, and scratch ownership for widest-to-narrowest SIMD traversal, then provide the equivalent scalar fallback through that contract. Preserve luma/chroma mode-neighbor addressing, directional upsampling, Paeth selection, smooth normalization, filter-intra taps, high-bit-depth clipping, chroma-from-luma storage/subsampling, and chroma-from-luma row strides. The WIP rectangular byte-pipeline smooth digest expectations encode width/height-swapped weights and must be replaced only from an independently generated oracle, not regenerated from this implementation.
Av1TileReader palette mode/color-map parsing, Av1SymbolDecoder palette distributions, Av1BlockModeInfo palette state, and Av1PredictionDecoder palette reconstruction AV1 sections 5.11.46, 5.11.49, and 7.11.2 palette prediction libaom av1/decoder/decodemv.c, av1/decoder/detokenize.c, av1/decoder/decoder.h, av1/common/pred_common.c, av1/common/pred_common.h, and av1/common/entropymode.c at 03087864cf4bea6abb0d28f95cf7843511413d8f Reuse the existing tile range decoder and frame-owned mode map while porting the normative palette cache merge, high-bit-depth color deltas, tile-adaptive mode/size/index distributions, diagonal color-map traversal, edge padding, and direct palette-sample reconstruction. This is AV1 still-image compression syntax and does not add retained video reference state or any ISO BMFF surface.
Av1DeblockingFilter, Av1LoopFilterContext, and Av1LoopFilterDecoder AV1 section 7.14 deblocking loop filter libaom aom_dsp/loopfilter.c and av1/common/av1_loopfilter.c at 03087864cf4bea6abb0d28f95cf7843511413d8f Design the 8- and high-bit-depth edge layout and 4-, 6-, 8-, and 14-tap kernels for SIMD-first traversal, with sharpness thresholds, still-frame intra filter-level derivation, transform-edge selection, and plane traversal shared by the exact scalar fallback. Implement and verify both through the same contract before enabling the stage. This is normative AV1 image reconstruction and adds neither generic ISO BMFF models nor retained video reference state.
Av1CdefDecoder, Av1CdefFilter, and CDEF-unit strength storage AV1 sections 7.15.2 through 7.15.4 constrained directional enhancement filtering libaom av1/common/cdef.c, av1/common/cdef_block.c, av1/common/cdef.h, and av1/common/cdef_block.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Design direction search and constrained primary/secondary filtering around vector-friendly block storage and SIMD-first kernels, with variance adjustment, subsampling direction conversion, skipped-8x8 selection, and frame-edge sentinel behavior shared by the scalar fallback. Use a frame-owned source snapshot so filtering never consumes already modified samples. This is normative AV1 still-image reconstruction and introduces no ISO BMFF, track, timing, or sequence-playback surface.
Av1SuperResolutionDecoder, Av1SuperResolutionKernels, frame-size derivation, and decoded-image dimensions AV1 section 7.16 normative super-resolution upscaling libaom av1/common/resize.c, av1/common/resize.h, av1/common/convolve.c, and aom_dsp/aom_filter.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Port the fixed 64-phase, 8-tap horizontal filter, phase/step derivation, replicated frame edges, chroma width rounding, signed rounding, and 8/10/12-bit clipping. Reuse ImageSharp's existing cross-platform Vector128_.MultiplyAddAdjacent helper for the exact eight-coefficient dot product with a scalar fallback. Generic image resizing is not normative AV1 super-resolution. This adds no track, timing, fragment, animation, or generic ISO BMFF model.
Av1TileReader loop-restoration unit syntax, Av1SymbolDecoder restoration distributions/subexponential codes, and Av1FrameInfo unit storage AV1 section 5.11.57 read_lr and read_lr_unit syntax libaom av1/decoder/decodeframe.c, av1/common/restoration.c, av1/common/restoration.h, av1/common/entropymode.c, aom_dsp/binary_codes_reader.c, and aom_dsp/recenter.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Decode tile-local switchable/Wiener/self-guided selections, finite reference-subexponential coefficients, chroma Wiener windows, self-guided parameter sets, super-resolution-adjusted unit corners, and the AV1 nearest-unit-count rule into frame-owned per-plane grids. This is compressed still-image syntax and adds no movie, track, timing, fragment, audio, or sequence surface.
Av1WienerFilter AV1 sections 7.17.4 and 7.17.5 Wiener restoration filtering and coefficient derivation libaom av1/common/restoration.c, av1/common/restoration.h, av1/common/convolve.c, and av1/common/convolve.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Preserve the implicit center-sample contribution, separable horizontal/vertical rounding, bit-depth-dependent 16-bit intermediate range, and final 8/10/12-bit clipping. Reuse Vector128_.MultiplyAddAdjacent for the contiguous horizontal eight-tap product with an exact scalar fallback. Keep the restoration stage disabled until stripe boundaries and self-guided filtering are both complete.
Av1SelfGuidedFilter AV1 sections 7.17.2 and 7.17.3 self-guided and box-filter processes libaom av1/common/restoration.c and av1/common/restoration.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Design the window layout, caller-owned scratch, local statistics, and projection traversal from libaom's architecture-specific kernels so the production path is SIMD-first. Implement the equivalent scalar fallback through the same contract. Preserve the sixteen normative radius/variance parameter sets, local mean and variance normalization, alternating-row radius-two optimization, decoded projection-coefficient behavior, signed rounding, and 8/10/12-bit clipping. Keep this image-reconstruction stage disabled until restoration stripe boundaries are complete.
Av1LoopRestorationBoundary, Av1LoopRestorationDecoder, and Av1FrameDecoder restoration-stage ordering AV1 section 7.17 loop restoration, including striped boundary semantics libaom av1/common/restoration.c, av1/common/restoration.h, av1/common/resize.c, and av1/decoder/decodeframe.c at 03087864cf4bea6abb0d28f95cf7843511413d8f Preserve two deblocked rows at internal 64-luma stripe boundaries before CDEF, apply the existing normative SIMD-backed super-resolution kernel to saved rows when scaled, use post-CDEF/super-resolution samples at frame edges, extend the final restoration unit up to 150 percent of nominal size, and filter from immutable plane snapshots into separate output planes. This is bounded still-image reconstruction state, not retained reference-frame, track, timing, or playback state.
Av1FilmGrainDecoder and Av1FilmGrainGaussianSequence AV1 section 7.18 film-grain synthesis libaom av1/decoder/grain_synthesis.c, av1/decoder/grain_synthesis.h, and aom_dsp/grain_params.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Preserve the normative 2,048-sample Gaussian sequence, linear-feedback shift register, luma/chroma autoregressive templates, scaling lookup interpolation, 32x32 block selection, boundary overlap, restricted-range clipping, monochrome and 4:2:0/4:2:2/4:4:4 layouts, and 8/10/12-bit arithmetic. Use allocator-owned scratch and runtime-optimized span copies. Apply grain only to the displayed still-image samples after all in-loop filters; reference-frame parameter inheritance remains sequence-playback state and is outside this codec scope.
Av1FrameInfo, Av1TileReader, and Av1BlockDecoder transform/coefficient storage AV1 section 5.11.39 coefficient syntax and section 7.11.2 reconstruction libaom av1/decoder/decodetxb.c and av1/decoder/decoder.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Preserve separate luma and chroma transform coefficients at monotonically advancing per-plane offsets within each superblock so reconstruction consumes the same transform-block order produced by tile parsing.
Av1InverseQuantizer and Av1InverseQuantizationLookup AV1 section 7.12.3 inverse quantization libaom aom_dsp/aom_dsp_common.h, av1/common/quant_common.c, and av1/decoder/decodetxb.c at 03087864cf4bea6abb0d28f95cf7843511413d8f Select the per-segment matrix level, alias 64-pixel transform dimensions to their adjusted matrices, retain a flat level-15 matrix, and apply the five-bit inverse-matrix weight scale. The large managed lookup remains a single process-wide table.
Av1ForwardTransformer, Av1Inverse2dTransformer, Av1Transform2dFlipConfiguration, the forward/inverse 1-D operator structs, and Av1Transform1dMath AV1 forward transform definitions and section 7.11.2 inverse transform and reconstruction libaom av1/encoder/av1_fwd_txfm2d_hwy.h, its AVX-512 and AVX2 instantiations, av1/encoder/av1_fwd_txfm1d.c, av1/common/av1_inv_txfm1d_cfg.h, av1/common/av1_inv_txfm1d.c, av1/common/av1_inv_txfm2d.c, the x86 AVX2/SSE4 implementations, and the corresponding Neon implementations at 03087864cf4bea6abb0d28f95cf7843511413d8f Preserve the normative staged DCT, ADST, and identity arithmetic, direction-specific stage ranges and shifts, transposition, clipping, and high-bit-depth sample addition. Stateless static-generic operators follow ImageSharp's JPEG color-transform pattern. Forward operators share one behavior model across scalar, Vector128, Vector256, and Vector512, selecting packed or expanded lane storage at the 2-D boundary. Inverse production traversal retains the verified scalar, Vector128, and Vector256 tiers until a wider upstream shape and complete-block benefit are both established.
HeifDecoderCore box extension handling and HeifDecoderCore/HeifEncoderCore item-property associations ISO/IEC 14496-12 box extensibility and section 8.11.14 item properties and ipma syntax libavif src/read.c and src/write.c at 092276ce89098ead06db80975173191e5fee1826 Skip unrecognized top-level and metadata child boxes, preserve the position of every property in ipco, reject an unrecognized property only when its item association marks it essential, associate properties by item ID, and read or write the essential bit plus one-based 7-bit or 15-bit property index according to the full-box flags. Independent HEIC, HIF, and AVIF fixtures provide the reader oracle; container-level identification of encoded output guards the writer independently of pixel roundtripping.
HeifCleanAperture, HeifItem presentation state, and HeifDecoderCore transformative-property parsing and application ISO/IEC 14496-12 section 12.1.4 clean aperture; HEIF image rotation and mirror properties; MIAF section 7.3.6.7 presentation order and section 7.3.9 essential transformative properties libavif src/avif.c clean-aperture conversion, src/read.c property parsers and alpha-property validation, and apps/shared/avifutil.c transform application at 092276ce89098ead06db80975173191e5fee1826 Resolve fractional clean-aperture dimensions and center offsets to exact bounded integer pixels, validate the registered rotation/mirror reserved bits, require essential associations, crop after auxiliary-alpha composition, map counter-clockwise HEIF quarter turns to ImageSharp's optimized clockwise rotate modes, then mirror around the signaled axis. Reuse ImageSharp's existing crop, rotation, and flip processors for every pixel type. Retain only the three image-item property values; do not add a generic transform-box or ISO BMFF model.
HeifConstants.IsSupportedFileType, HeifImageFormatDetector, and HeifDecoderCore.CheckFileTypeBox ISO/IEC 14496-12 FileTypeBox syntax and the MP4 Registration Authority HEIF/AVIF still-image and sequence brand registrations libavif src/read.c functions avifParseFileTypeBox, avifFileTypeHasBrand, and avifFileTypeIsCompatible at 092276ce89098ead06db80975173191e5fee1826 Apply one rule to the major and compatible brands, accept implemented still-image brands and the bounded avis, hevc, and hevx image-sequence brands, and distinguish item and sequence presentation before parsing their payloads. Layered HEVC and JPEG sequence brands remain unsupported. The decoder validates the complete ftyp payload; the fixed-size format detector inspects the available prefix.
HeifSequenceParser, HeifTrackMetadataParser, and the bounded HEIF image-sequence writer ISO/IEC 23008-12:2022 clauses 7 through 10 and HEVC Annex B; ISO/IEC 14496-12 sample tables, normal edit lists, and track-level metadata; AVIF 1.2 section 3; AV1-ISOBMFF 1.2 section 2 libavif src/read.c track/sample-table and track-metadata paths and src/write.c sequence path at 092276ce89098ead06db80975173191e5fee1826; independently produced HEIC sequence fixtures remain required Resolve only pict master tracks and linked image alpha/metadata state into validated sample descriptors, exact frame durations, sync/dependency state, repetition, presentation transforms, color/HDR properties, and frame metadata. Track metadata is limited to implicitly associated Exif and unencoded XMP items declared by bounded iinf/infe, iloc, and optional idat syntax. DecoderOptions controls metadata retention and recoverable ancillary/image-data errors through the same rules as other decoders. Consume the required container boxes listed in the bounded syntax matrix without exposing a movie, track, edit, or sample-table API.
HeifDecoderCore.ReadBoxHeader and HeifDecoderCore.ParseBoxHeader ISO/IEC 14496-12 section 4.2.2 basic box syntax libavif src/stream.c functions avifROStreamReadBoxHeaderPartial and avifROStreamReadBoxHeader at 092276ce89098ead06db80975173191e5fee1826 Resolve 32-bit, 64-bit, UUID, and top-level size-zero boxes into content lengths only after validating the complete variable-sized header and the remaining parent boundary. Nested size-zero boxes are invalid; large skips retain 64-bit offsets.
HeifDecoderCore.ParseMetadata ISO/IEC 14496-12 MetaBox and HEIF item declarations, locations, properties, and associations libavif src/read.c functions avifParseMetaBox, avifMetaFindOrCreateItem, avifParseItemLocationBox, and avifParseItemPropertiesBox at 092276ce89098ead06db80975173191e5fee1826 Index unique recognized metadata children by type and payload location, then parse them in dependency order so physical placement does not control item lookup or property association. Duplicate unique children and truncated full-box headers are invalid.
HeifDecoderCore.ApplyAssociatedMetadata HEIF Annex A Exif item data, MIME metadata items, and cdsc item references libavif src/read.c function avifDecoderFindMetadata, src/exif.c function avifGetExifTiffHeaderOffset, and the Exif/XMP item writing paths in src/write.c at 092276ce89098ead06db80975173191e5fee1826 Resolve only metadata items whose cdsc reference identifies the decoded primary image, validate the Exif TIFF-header offset, and attach Exif or application/rdf+xml XMP through ImageSharp's existing profile types. This is a bounded still-image metadata path; it does not introduce a generic ISO BMFF metadata, media, or track model.
HeifPropertyParser, HeifDecoderCore color-property association, HeifItem and HeifSequenceTrack color profiles, and Av1Decoder effective color validation ISO/IEC 14496-12 section 12.1.5 color information; HEIF section 6.5.5.1 color-information properties; AV1-ISOBMFF section 2.3.4 configuration semantics libavif src/read.c functions avifParseColourInformationBox, avifReadColorNclxProperty, and avifReadColorProperties, plus src/write.c function avifEncoderWriteColorProperties, at 092276ce89098ead06db80975173191e5fee1826 Associate at most one ICC and one nclx property with each presented color image item or selected image-sequence sample entry, validate ICC payloads and CICP reserved bits, expose them through ImageSharp's existing profile types, and inherit a grid's CICP description only for tiles that do not declare one. For AV1 images, require every specified sequence-header CICP field and the range flag to match nclx; use the container property only to supply sequence-header fields coded as unspecified before color conversion, then expose the effective CICP values even when no property is present. Retain only the two image color profiles; do not add a reusable color-box or media model.
HeifPropertyParser, HeifPixelAspectRatio, item/track presentation state, and HeifDecoderCore.ApplyItemPixelAspectRatioMetadata ISO/IEC 14496-12 section 12.1.4.3 pixel aspect ratio libavif src/read.c function avifParsePixelAspectRatioBox, src/write.c functions avifEncoderWritePaspProperty and avifEncoderWriteTransformativeProperties, and the presented-image property selection in src/read.c at 092276ce89098ead06db80975173191e5fee1826 Preserve the registered pasp, clap, irot, and imir presentation properties on an associated image item or selected image-sequence sample entry, reject invalid lengths, reserved bits, duplicates, and non-integral clean apertures, and map the displayed pixel ratio and transforms through ImageSharp's existing metadata and processing model. No generic transform or display model is introduced.
Av1CodecConfiguration, HeifItem.Av1CodecConfiguration, Av1HeifItemDecoder, and AV1 grid configuration checks AV1-ISOBMFF sections 2.3.3 and 2.3.4 codec-configuration record syntax and semantics; AVIF sections 2.1, 2.2.1, and 2.2.3 AV1 image-item, item-configuration, and HDR metadata constraints; AV1 sections 5.8.3, 5.8.4, 6.7.3, and 6.7.4 HDR metadata syntax and semantics; ISOBMFF mastering-display and content-light image properties; ITU-T H.274 section 8.9 mastering-display field semantics; MIAF section 7.3.11.4.1 grid input constraints libavif src/read.c functions avifParseCodecConfiguration, avifDecoderItemValidateProperties, avifReadCodecConfigProperty, avifParseContentLightLevelInformation, and avifSkipMasteringDisplayColourVolume at 092276ce89098ead06db80975173191e5fee1826; libaom av1/decoder/obu.c functions read_metadata, read_metadata_hdr_cll, and read_metadata_hdr_mdcv at 03087864cf4bea6abb0d28f95cf7843511413d8f Associate exactly one av1C property with each decoded av01 image item, validate the fixed record and its bit depth/chroma fields against the item's AV1 sequence header and optional pixi channel depths, require matching configurations across grid tiles, and report the encoded image precision and monochrome shape through HeifMetadata. Validate low-overhead OBU framing, require exactly one sequence header in the image item, allow at most one first-position sequence header in configOBUs, and compare a repeated header's extension and payload exactly while ignoring only its legal size-field representation. Decode clli and mdcv as still-image item properties, validate matching HDR CLL and HDR MDCV metadata OBUs from the combined configuration/item sequence, and account for the different primary order and fixed-point precision of the ISOBMFF and AV1 MDCV representations. Expose the effective HDR values without adding sample groups, tracks, or media metadata. Related still-image HDR properties remain required. Consume but do not retain presentation-delay syntax, and introduce no sample entry, sample description, track, timing, or generic decoder-configuration model.
HevcCodecConfiguration, HevcNalUnitArray, and HeifItem.HevcCodecConfiguration ISO/IEC 14496-15 HEVCDecoderConfigurationRecord, HEVC NAL-unit header syntax, and HEVC image-item configuration requirements HM source/Lib/TLibDecoder/NALread.cpp and parameter-header paths at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc v1.6.0 decoder NAL and parameter-header paths at c83a76b084498d55f252f48b2e3786804cdf24b7; independently produced HEIC fixtures for hvcC record layout Associate exactly one hvcC property with each hvc1 image item; validate the fixed profile, level, chroma, bit-depth, reserved, length-size, array, and NAL-header fields within the property boundary; and retain only the image description plus bounded configuration NAL units required to decode that item. Consume but do not retain average-frame-rate or temporal-layer presentation fields. Do not add visual sample entries, sample descriptions, tracks, timing, access-unit timelines, or a generic decoder-configuration model. BSD and Apache-2.0 notice preservation and separate HEVC patent/release clearance remain final integration gates.
HevcNalUnit, HevcNalUnitHeader, HevcRbspDecoder, and HevcBitReader HEVC sections 7.3.1.1 and 7.3.1.2 NAL-unit and header syntax, section 7.3.2.11 RBSP trailing bits, and section 9.1 Exp-Golomb parsing HM source/Lib/TLibDecoder/NALread.cpp and SyntaxElementParser.cpp at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc NAL extraction and bitstream paths at c83a76b084498d55f252f48b2e3786804cdf24b7 Validate the fixed NAL header, remove only legal emulation-prevention bytes, and provide bounded fixed-width, flag, unsigned/signed Exp-Golomb, and RBSP-trailing-bit reads. Configuration arrays immediately use the shared parser. The primitives consume only NAL units delivered by a supported HEIF image item or its hvcC property and introduce no Annex B API, access-unit model, timeline, track, sample table, or generic ISO BMFF surface.
HevcProfileTierLevel, HevcVideoParameterSet, and HevcCodecConfiguration.VideoParameterSets HEVC sections 7.3.2.1 video parameter sets, 7.3.3 profile-tier-level syntax, and E.2.2 hypothetical-reference-decoder syntax; ISO/IEC 14496-15 HEVCDecoderConfigurationRecord matching semantics HM source/Lib/TLibDecoder/TDecCAVLC.cpp functions parseVPS, parsePTL, parseProfileTier, and parseHrdParameters at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoder/ihevcd_parse_headers.c profile-tier-level and VPS paths at c83a76b084498d55f252f48b2e3786804cdf24b7 Retain the base-layer VPS identifier, temporal-sublayer count, nesting flag, and exact general profile/tier/constraint/level fields. Validate profile, tier, compatibility, level, temporal count, and nesting against hvcC, while tolerating the independently observed legacy-muxer practice of zeroing some or all profile-specific constraint bits in the configuration record; SPS validation remains authoritative for chroma and bit depth. Consume but do not retain sublayer ordering, timing, and HRD syntax. Reject multilayer and layer-set state because the supported contract is one independently coded image item; add no layer-selection, access-unit, DPB, scheduling, track, or sample model.
HevcSequenceParameterSet, HevcVideoUsabilityInformation, HevcScalingList, HevcShortTermReferencePictureSet, and HevcParameterSetSyntax HEVC sections 7.3.2.2 sequence parameter sets, 7.3.4 scaling-list data, 7.3.7 short-term reference-picture sets, E.2.1 VUI syntax, and Range Extensions SPS syntax HM source/Lib/TLibDecoder/TDecCAVLC.cpp functions parseSPS, parseScalingList, xDecodeScalingList, parseShortTermRefPicSet, parseVUI, and parseHrdParameters, plus source/Lib/TLibCommon/TComRom.cpp default matrices and diagonal scans, at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoder/ihevcd_parse_headers.c SPS, VUI, scaling-list, and reference-set paths at c83a76b084498d55f252f48b2e3786804cdf24b7 Retain coded/display dimensions, conformance crop, monochrome/4:2:0/4:2:2/4:4:4 and separate-plane layout, 8-through-16-bit precision, coding/transform/PCM geometry, effective scaling matrices, compression-tool flags, bounded reference-set declarations, Range Extensions tools, and still-image VUI aspect/color/range/chroma-location/display-window fields. Consume timing, HRD, ordering, and bitstream-restriction syntax without scheduling state. Reject interlaced fields and multilayer extensions because they do not represent the one progressive independently coded image-item contract; add no DPB pictures, playback state, tracks, or sample tables.
HevcPictureParameterSet and HevcCodecConfiguration.PictureParameterSets HEVC sections 7.3.2.3 picture parameter sets, 7.3.4 scaling-list data, and Range Extensions PPS syntax HM source/Lib/TLibDecoder/TDecCAVLC.cpp function parsePPS, source/Lib/TLibDecoder/TDecConformance.cpp PPS range checks, and source/Lib/TLibCommon/TComPicSym.cpp tile derivation at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoder/ihevcd_parse_headers.c PPS and tile paths at c83a76b084498d55f252f48b2e3786804cdf24b7 Link each PPS to a configuration-local SPS and retain the CABAC, quantization, transform-skip, scaling-list, tile/wavefront, deblocking, SAO, slice-header, and Range Extensions controls required to reconstruct the independently coded picture. Derive the complete CTB tile grid and validate every SPS-dependent depth, size, precision, and offset bound during parsing. Reject multilayer extensions and add no access-unit, reference-picture lifetime, playback, track, timing, or sample-table model.
HevcImageItemBitstream, HevcSliceSegmentHeader, and HevcSliceType HEVC sections 7.3.1.2 NAL-unit syntax, 7.3.6 slice-segment headers, and 7.4.7 slice-segment-header semantics; HEVC image-item length delimiting HM source/Lib/TLibDecoder/TDecCAVLC.cpp function parseSliceHeader, source/Lib/TLibDecoder/NALread.cpp, and slice byte-alignment and entry-point handling at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoder/ihevcd_nal.c, ihevcd_parse_slice_header.c, and ihevcd_parse_headers.c at c83a76b084498d55f252f48b2e3786804cdf24b7 Split only the selected hvc1 image item's one-through-four-byte length-delimited NAL units, reject in-band parameter-set replacement and end-of-sequence state, and require exactly one base-layer IDR picture. Retain the intra slice, separate-plane, SAO, quantization, deblocking, cross-slice filtering, tile/wavefront entry-point, extension, byte-alignment, and entropy-payload fields required by reconstruction. Keep emulation-prevention positions only to translate slice substream byte boundaries; add no Annex B scanner, access-unit collection, non-IDR reference lifetime, playback, timing, track, or sample model.
HevcHeifItemDecoder, HeifCompressionFactory, and HevcYuvConverter coded-window output HEVC sections 6.1 through 6.3 decoded picture and conformance-window semantics; ISO/IEC 14496-12 section 12.1.5 color information and ISO/IEC 23008-12 HEVC image-item binding HM source/Lib/TLibDecoder/TDecTop.cpp decoded-picture output and source/Lib/TLibCommon/TComPicYuv.cpp conformance-window handling at 9c1f298659ab0cee9dc13d23d0304221575410b9; ImageSharp's shared HEIF H.273 SIMD operators, pooled row conversion, and item-decoder lifecycle Register only hvc1 still-image items, bind the selected slice PPS/SPS to one picture decoder, convert the conformance-window region directly from native component planes without an intermediate packed image, and preserve the effective container-or-VUI CICP description used by conversion. Reuse the shared descending-width SIMD color operators and pixel packers with one pooled row workspace. Add no Annex B entry point, decoded-picture queue, inter-picture state, track, timing, or general-purpose HEVC API.
HevcCabacContext and HevcCabacDecoder HEVC sections 9.3.3.2 through 9.3.3.4 context initialization, arithmetic decoding, bypass decoding, and termination HM source/Lib/TLibCommon/ContextModel.cpp, source/Lib/TLibCommon/TComCABACTables.cpp, and source/Lib/TLibDecoder/TDecBinCoderCABAC.cpp at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc common/ihevc_cabac_tables.c, common/ihevc_cabac_tables.h, decoder/ihevcd_cabac.c, and decoder/ihevcd_cabac.h at c83a76b084498d55f252f48b2e3786804cdf24b7 Implement the bit-exact scalar probability-state transitions, range normalization, bounded byte refills, bypass modes, termination, and alignment validation for one slice tile or wavefront substream. CABAC decisions are serial and adaptive, so this primitive is the scalar oracle rather than a SIMD target; later prediction, transform, filtering, and color-conversion stages must reuse ImageSharp's compatible SIMD kernels. The decoder owns no container boxes, access units, reference pictures, tracks, timing, playback, or general-purpose bitstream surface.
HevcCabacContexts HEVC section 9.3.2.2 initialization of context variables for an intra-coded slice and Range Extensions context additions HM source/Lib/TLibCommon/ContextTables.h and source/Lib/TLibDecoder/TDecSbac.cpp function resetEntropy at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc common/ihevc_cabac_tables.c and decoder/ihevcd_cabac.c at c83a76b084498d55f252f48b2e3786804cdf24b7 Initialize only the context groups reachable from the independently coded intra-picture syntax, including transform skip, chroma quantization adjustment, and cross-component prediction. Keep one contiguous mutable context set per entropy substream so tile and wavefront synchronization can copy the normative adaptive state without retaining reference-picture or playback state. Omit skip, merge, motion, reference-index, and other inter-picture-only groups.
HevcCabacSyntaxReader HEVC sections 9.3.3 through 9.3.4 CABAC binarization and context selection for intra coding units, transform trees, residual coefficients, sample-adaptive offset, and Range Extensions tools HM source/Lib/TLibDecoder/TDecSbac.cpp, source/Lib/TLibCommon/TComChromaFormat.h, and source/Lib/TLibCommon/TComTrQuant.cpp at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoder/ihevcd_cabac.c, decoder/ihevcd_parse_slice.c, and decoder/ihevcd_parse_residual.c at c83a76b084498d55f252f48b2e3786804cdf24b7 Decode only syntax values reachable while reconstructing the selected independently coded still picture: intra partition and mode selectors, transform splitting and coded-block flags, quantization adjustments, transform skip, coefficient significance and levels, cross-component prediction, SAO, bypass alignment, and termination. Keep neighbor-dependent context selection in the owning coding-tree/reconstruction layer and expose no inter prediction, reference-picture, access-unit, track, timing, or playback syntax.
HevcCoefficientScanOrder, HevcCoefficientCodingParameters, and HevcCoefficientDecoder HEVC sections 9.3.4.2.4 through 9.3.4.2.8 last-position, significance-map, level, sign-data-hiding, and Rice-adaptation processes; Range Extensions transform-skip context and bypass-alignment rules HM source/Lib/TLibCommon/TComRom.cpp grouped scan generation, source/Lib/TLibCommon/TComChromaFormat.cpp transform-unit entropy parameters, source/Lib/TLibCommon/TComTrQuant.cpp significance-context derivation, and source/Lib/TLibDecoder/TDecSbac.cpp function parseCoeffNxN at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoder/ihevcd_parse_residual.c at c83a76b084498d55f252f48b2e3786804cdf24b7; ImageSharp MemoryAllocator and fixed inline buffers Generate diagonal, horizontal, and vertical grouped scans into entropy-substream-owned scratch; derive directional scan selection, last-significant positions, right/lower group contexts, significant coefficient flags, greater-than-one/two levels, escape remainders, sign hiding, and the four persistent Rice states. Allocate one pooled scratch block per entropy substream and perform no allocation or dynamic stack growth per transform block. This serial adaptive stage intentionally remains scalar and feeds the existing SIMD-first inverse quantization, inverse transform, transform-skip, and residual-reconstruction kernels.
HevcPlane and HevcPictureBuffer HEVC sections 6.2 and 6.3 source and decoded picture sample-array dimensions for monochrome, 4:2:0, 4:2:2, 4:4:4, and separate-color-plane coding HM source/Lib/TLibCommon/TComPicYuv.cpp at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoded-picture buffer layout paths at c83a76b084498d55f252f48b2e3786804cdf24b7; ImageSharp MemoryAllocator and Buffer2D<T> Allocate one allocator-owned native ushort plane set for the selected still picture so the same reconstruction path preserves every supported 8-through-16-bit sample without byte-backed reinterpretation. Derive chroma dimensions by ceiling division for each sampling layout and treat separate color planes as full-resolution independently coded arrays. The buffer contains one picture only and introduces no decoded-picture buffer, reference lifetime, frame queue, or playback state.
HevcCodingTreeState HEVC sections 6.4 coding-tree block and coding-block availability plus section 9.3 split-flag context derivation HM source/Lib/TLibCommon/TComDataCU.cpp function getCtxSplitFlag and source/Lib/TLibDecoder/TDecCu.cpp coding-tree traversal at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoder/ihevcd_parse_slice.c coding-tree paths at c83a76b084498d55f252f48b2e3786804cdf24b7; ImageSharp MemoryAllocator and Buffer2D<T> Store only leaf depth, effective luma QP, transquant-bypass, and PCM state at minimum-coding-block resolution for the selected picture. Derive split contexts from caller-approved left and above availability so slice and tile boundaries remain owned by traversal rather than hidden in a general block graph. Clip edge writes to the coded still-picture state and add no prediction units, motion fields, references, decoded-picture queue, or sequence lifetime.
HevcIntraPredictionState HEVC sections 8.4.2 and 9.3 intra luma/chroma prediction-mode derivation and binarization HM source/Lib/TLibCommon/TComDataCU.cpp functions getIntraDirPredictor and getAllowedChromaDir, plus source/Lib/TLibDecoder/TDecSbac.cpp functions parseIntraDirLumaAng and parseIntraDirChroma, at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc intra-mode parsing and neighbor derivation paths at c83a76b084498d55f252f48b2e3786804cdf24b7; ImageSharp MemoryAllocator and Buffer2D<T> Preserve the normative two-pass luma flag/suffix order, spatial most-probable-mode derivation, omitted-mode reinsertion, four-way minimum-CU partition order, explicit chroma candidate substitution, and derived-chroma mode at 4x4 luma resolution. Accept slice/tile availability from traversal, fill only the selected still-picture map, and add no inter prediction unit, motion, reference, or sequence state.
HevcIntraPredictor HEVC section 8.4.4.2 intra sample prediction and section 8.4.4.2.3 reference-sample substitution and filtering HM source/Lib/TLibCommon/TComPrediction.cpp functions predIntraGetPredValDC, xPredIntraAng, xPredIntraPlanar, and xDCPredFiltering, plus source/Lib/TLibCommon/TComPattern.cpp function fillReferenceSamples and its reference-filter selection at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc intra-prediction kernels at c83a76b084498d55f252f48b2e3786804cdf24b7 Predict from caller-prepared references with planar, DC, and all 33 angular modes; negative-angle extension; three-tap and strong-bilinear smoothing; and the normative luma boundary filters. Traverse Vector512, Vector256, and Vector128 widths before one scalar tail, use caller-owned reusable scratch for horizontal transposition, and retain no generic video, inter-picture, or reference-picture state.
HeifContentColorVolume, HeifItem.ContentColorVolume, and HeifDecoderCore content color-volume parsing and presentation HEIF content color-volume item property; AVIF 1.2 content color-volume requirements; ITU-T H.274 (V4) content colour volume syntax and semantics libavif src/read.c function avifSkipContentColourVolume at 092276ce89098ead06db80975173191e5fee1826; official ITU-T H.274 (V4), January 2026 Decode only the bounded per-image cclv property: require zero cancellation, persistence, and reserved bits; preserve optional signed G/B/R primary coordinates and normalized minimum, maximum, and average luminance values; and validate their registered ranges and ordering. Expose the effective grid-or-tile still-image value through HeifMetadata. Do not add SEI persistence, retained video state, tracks, samples, timing, or a generic ISO BMFF color-volume box model.
HeifAmbientViewingEnvironment, HeifReferenceViewingEnvironment, HeifNominalDiffuseWhite, and their per-item presentation metadata ISOBMFF ambient viewing environment; ITU-T H.274 (V4) section 8.13; HEIF Amendment 1 sections 6.5.44 and 6.5.45; AVIF 1.2 image-item box requirements libavif src/read.c functions avifSkipAmbientViewingEnvironment, avifSkipReferenceViewingEnvironment, and avifSkipNominalDiffuseWhite at 092276ce89098ead06db80975173191e5fee1826; official ITU-T H.274 (V4), January 2026 Decode the fixed-size amve, version-zero reve, and version-zero ndwt properties only when associated with the presented still-image item. Preserve physical illuminance and luminance units, distinct surround/periphery chromaticities, and the coded zero that requests the standard nominal diffuse-white default. Validate registered coordinate ranges and duplicate associations. Retain no video-SEI persistence, visual sample entry, display pipeline, track, timing, or generic viewing-environment box model.
GridHeifItemDecoder and HeifDecoderCore grid/thumbnail selection ISO/IEC 23008-12 section 6.6.2.3 image-grid syntax and MIAF grid-cell constraints libavif src/read.c functions avifParseImageGridBox, avifDecoderDataAllocateImagePlanes, and avifDecoderDataCopyTileToImage at 092276ce89098ead06db80975173191e5fee1826 Parse version-zero 16-bit and 32-bit grid descriptors, preserve row-major dimg order, require the declared tile count and one coding format, validate canvas coverage and edge overlap, and crop only the rightmost column and bottom row while copying through ImageSharp pixel buffers. A primary grid whose tile codec is unavailable may use only a decodable thumbnail that explicitly references that grid.
HeifDecoderCore alpha auxiliary selection/composition and GridHeifItemDecoder auxiliary tile ordering ISO/IEC 23008-12 alpha auxiliary image semantics, auxC, auxl, prem, and per-grid-tile alpha relationships libavif src/read.c functions avifParseAuxiliaryTypeProperty, avifDecoderItemIsAlphaAux, avifMetaFindAlphaItem, and avifDecoderCheckAlphaProperties, plus src/scale.c box-filter scaling at 092276ce89098ead06db80975173191e5fee1826 Recognize both registered alpha URNs, decode a direct alpha image/grid or the complete row-major set of per-color-tile alpha auxiliaries, normalize through L16, box-resample differing auxiliary dimensions, compose through Rgba64 and PixelOperations<TPixel>, and unassociate prem color samples with transparent-black handling. This remains an image-item relationship only; no track or generic media-reference model is introduced.

This table is intentionally incomplete. Add a row before each additional AV1 or HEVC algorithm is ported or materially reshaped.

Current implementation assessment

This assessment was reconciled with the source tree on 2026-08-26. Unless a result is stated explicitly, each item is a source-inspection finding rather than a verified interoperability claim.

Public integration

  • HeifFormat combines the HEIF, HEIC, HIF, and AVIF identities and extensions, but the implementation does not yet decode all payloads that contract implies.
  • HeifDecoder now defaults to Rgba32, preserving decoded auxiliary alpha for non-generic loads.
  • HeifMetadata now reports alpha presence and the corresponding 24/32-bit RGB pixel shape, but complete decoded HEVC/AV1 bit depth, monochrome/chroma layout, color signaling, and profiles remain absent.
  • HeifEncoder defines quality, alpha quality, effort, lossless, chroma-subsampling, and bit-depth contracts directly, without a single-implementation options interface. The legacy JPEG path applies its supported quality, bit-depth, and chroma options and rejects unsupported combinations; AV1 and HEVC must implement the same public contracts before the Phase 1 API-review gate can pass.
  • HEIF/HEIC/AVIF is absent from the format source-generation list in _Formats.ttinclude, so the standard ImageSharp save extensions are not generated.
  • Configuration registration exists, but it currently registers capabilities broader than the implementation provides.

HEIF/ISO BMFF container

  • The bounded reader now handles basic, extended-size, UUID, and permitted top-level to-end boxes, skips unknown optional top-level and metadata children, and rejects child boxes that escape their parent.
  • File type handling evaluates supported still-image major and compatible brands and recognizes the bounded avis, hevc, and hevx image-sequence brands. The sequence parser retains only the selected picture and auxiliary-alpha tracks, timing, sample locations, dependencies, presentation properties, and image metadata required by animated HEIF; unrelated media tracks and general ISO BMFF playback state are not materialized.
  • Item IDs are resolved as keys rather than list indices; metadata children are indexed and parsed in dependency order rather than physical order.
  • Item property associations now preserve physical ipco indices, apply one-based 7-bit or 15-bit indices and essential flags, associate by item ID, and reject arbitrary unknown essential properties.
  • Item locations now support bounded file-relative and idat-relative storage, multiple ordered extents, versioned item IDs, 0/4/8-byte registered field sizes, and 64-bit offsets. Referenced-item construction method two and external data references remain explicitly unsupported.
  • Grid derived-image decoding now parses both registered descriptor widths, resolves the ordered dimg cells, validates tile count, coding format, dimensions, canvas coverage, and edge overlap, then composes the output through ImageSharp row buffers. Unsupported grid tile codecs can select only a decodable thumbnail of the same primary grid. HEVC grids have exact complete-image libheif references and exact HM tile-plane references; independent AV1 and JPEG grid fixtures are still required.
  • Alpha auxiliary decoding recognizes auxC, auxl, and prem, supports direct auxiliary images, auxiliary grids, and per-color-grid-tile alpha, and reports alpha presence. The common-size path normalizes the native luma plane and composes each row directly into the destination; the scaled path uses allocator-owned row staging and the bounded ImageSharp resize window instead of materializing a complete Image<L16>. HEVC auxiliary-alpha presentation matches the independent libheif reference exactly, and independently encoded AVIF alpha fixtures cover composed alpha and ICC interaction.
  • Clean aperture, image rotation, and image mirror properties now validate their registered payloads, exact integer crop geometry, and essential associations; affect Identify dimensions; and reuse ImageSharp's optimized crop/rotate/flip processors after auxiliary alpha composition in the MIAF-defined order. Independent transform vectors must still verify every crop/rotation/mirror/alpha combination.
  • Decode now resolves cdsc-associated Exif and application/rdf+xml XMP items for the primary still image, validates the declared Exif TIFF-header offset, and attaches the payloads through ImageSharp's existing profile types before presentation transforms. Independent AVIF, HEIC, and HIF metadata fixtures and Identify-time profile reporting remain required.
  • ICC and nclx CICP color properties are now associated with the presented color item instead of global parser state, validated, and exposed on Decode and Identify through the existing ImageSharp profiles. AV1 decode requires specified sequence-header fields and the range flag to match nclx, uses the property only for fields the bitstream leaves unspecified, and exposes the effective sequence-header CICP description when no property is present. Independent libavif fixtures verify exact ICC byte preservation, observable non-sRGB conversion, canonical-sRGB compaction, non-sRGB compact preservation, and metadata skipping for primary, grid, alpha-composited, and every presented sequence frame; the alpha vector also verifies that ICC conversion does not alter composed alpha. Identify-time bitstream fallback, independent CICP vectors, and HEVC integration remain required.
  • Pixel aspect ratio now preserves the complete unsigned spacing pair, affects Decode and Identify through ImageSharp's existing unitless resolution metadata, and follows quarter-turn presentation rotation. Independent grid, rotation, and maximum-spacing fixtures remain required.
  • AV1 codec configuration is now retained per av01 image item rather than in decoder-global state. Decode requires the property, validates its fixed record against the item's sequence header and any associated pixi channel depths, and requires matching configurations across grid tiles. Identify now reports the configuration's 8/10/12-bit precision and monochrome shape. The optional configOBUs sequence is bounded and validated, including its mandatory size fields, first-position/at-most-one sequence-header rule, the image item's exactly-one sequence-header rule, and exact comparison of a repeated configuration header with the item header. Content light-level and mastering-display color-volume information are decoded from the bounded clli and mdcv image properties and matching AV1 HDR metadata OBUs, with the representations' distinct fixed-point precision, grid/property precedence, and SkipMetadata behavior preserved. The still-image cclv, amve, reve, and ndwt properties preserve content-volume, ambient-viewing, mastering-environment, and diffuse-white descriptions without importing their video-SEI or sample-entry state models. Independent fixtures and encoder preservation remain required. HEVC hvcC is associated with each hvc1 image item, structurally validated through its VPS/SPS/PPS arrays, and used to report encoded precision and monochrome shape. The bounded decoder reconstructs exact native planes for the verified 8-bit 4:2:0 and 4:4:4 item payloads; the remaining profile, bit-depth, chroma-layout, and Range Extensions matrix is not yet established.
  • Several image-item properties and relationships remain missing or parsed without fully affecting output.
  • Identify and decode now use the same bounded metadata parser and both validate the complete leading file type box. The parsed state is still mutable and Identify does not yet report the complete bit depth, color, profile, or transform model.

HEVC decoder and encoder

  • Heif4CharCode recognizes hvc1 image items, HevcCodecConfiguration validates and associates the bounded hvcC property, Identify reports its HEVC precision and monochrome shape, and HeifCompressionFactory registers the bounded HEVC still-image decoder.
  • The HEVC path validates NAL headers and RBSP escaping, links hvcC VPS/SPS/PPS structures through Range Extensions, parses one length-delimited base-layer IDR picture, and reconstructs its complete intra-coded coding tree. It includes CABAC, transform traversal, coefficient decoding, intra prediction, inverse quantization and transform, PCM, deblocking, sample-adaptive offset, and allocator-owned native ushort planes for monochrome, 4:2:0, 4:2:2, 4:4:4, separate-color-plane, and 8-through-16-bit samples. The item adapter converts the SPS conformance window directly through the shared pooled SIMD-first H.273 pipeline and preserves the effective container-or-VUI CICP description. Four real item payloads match HM 18 native planes exactly, and the first pictures from 10 official Sony GENERAL Range Extensions streams match every 8/10/12-bit monochrome, 4:2:0, 4:2:2, and 4:4:4 sample with normal and forced-scalar dispatch. Complete images covering grids, auxiliary alpha, ICC, 4:2:0, and 4:4:4 presentation also match independent libheif output exactly. Profiles and Range Extensions tools not isolated by that matrix, plus the complete color matrix, remain open.
  • There is no HEVC encoder. HeifEncoderCore rejects HeifCompressionMethod.Hevc; the separately selected legacy JPEG path emits a generic HEIF/HIF-style JPEG image item and does not provide HEIC output.
  • Permanent HEVC conformance tests compare extracted item planes exactly with HM output and complete presented images exactly with pinned libheif/libde265 reference PNGs. The fixtures do not yet cover the complete exposed profile matrix.

AV1 decoder

  • The single-still Av1Decoder path now parses tile state before allocating and reconstructing one independently decodable frame, and it disposes the reconstruction planes after pixel conversion. It deliberately does not retain animation/video reference frames or implement show_existing_frame playback state.
  • Transform coefficient entropy derivation and updates now address the above contexts relative to the tile column and the left contexts relative to the current superblock row, preserve luma coordinates independently of chroma subsampling, and test every packed context entry for the libaom any-nonzero rule. Extended vertical partition updates advance the mode-information column rather than the row. The existing multi-superblock 4:4:4 AVIF fixture now completes tile parsing; independent coefficient-context vectors across tile boundaries, chroma layouts, bit depths, and edge-clipped transforms remain required.
  • The reconstruction pipeline now records plane-relative transform geometry, preserves tile-local delta-Q and delta-LF predictors, derives segmentation and reference-adjusted filter levels, and runs the exact AV1 4-, 6-, 8-, and 14-tap deblocking kernels in normative vertical-then-horizontal order. Deblocking uses the same closed edge-operator architecture as the HEVC filter, with operators specialized by sample storage and orientation, Vector128<int> lanes representing the four rows or columns along an edge, and an allocation-free scalar fallback for disabled intrinsics. Exact native-plane comparison with pinned scalar libaom output now verifies active deblocking and complete reconstruction for real 8-bit 4:2:0, 10-bit 4:4:4, and 12-bit 4:4:4 content; genuine AVIF containers separately verify presentation and public bit-depth metadata. The pipeline then applies CDEF through one semantic filter architecture: paired AVX2 and single-block Vector128 direction analysis, closed primary/secondary strength operators, packed 4x4/4x8/8x4/8x8 constrained filtering, byte/16-bit output operators, and an exact allocation-free scalar fallback. The decoder still uses heap-allocated immutable plane snapshots and direction/variance maps; moving those to allocator-owned memory and using paired direction traversal is the next CDEF step. Active super-resolution derives the Appendix A bounded coded width and applies the exact 64-phase, 8-tap horizontal filter with aligned reconstruction-edge input, 8/10/12-bit clipping, and the existing cross-platform Vector128_.MultiplyAddAdjacent helper. Loop restoration follows super-resolution, preserves the required pre-CDEF deblocked context at internal stripes, and applies decoded Wiener or self-guided units from immutable plane snapshots. The visible still-image path then applies the complete self-contained film-grain parameter set after all in-loop filters. Independent 8-, 10-, and 12-bit vectors exercising CDEF, super-resolution, restoration, and grain remain required before those stages have external pixel-level verification.
  • Palette mode now reads the normative luma/chroma mode and size CDFs, neighbor color caches, high-bit-depth color syntax, diagonal color-index maps, and clipped-edge padding. Reconstruction retains libaom's exact indexed-palette contract while using repeated eight-entry tables and native lane-local shuffles across runtime-selected Vector512, Vector256, and Vector128 paths before an exact scalar fallback. FeatureTestRunner verifies palette sizes 2-8, transform widths 4-64, padded rows, and 8/10/12-bit samples. This path is not production-complete until its syntax and reconstruction are verified with an independently encoded palette AVIF fixture.
  • Non-reduced still-image sequence parsing now consumes decoder-model operating parameters, temporal presentation fields, and OBU-layer-applicable buffer-removal fields only to preserve AV1 bit alignment. The scheduling values are not retained, and no movie, track, timing, playback, or generic ISO BMFF surface has been introduced. Existing focused sequence-header coverage exercises only reduced-still syntax, so an independent non-reduced still AVIF vector remains required.
  • Loop-restoration unit parsing records tile-local switchable/Wiener/self-guided filter selections and coefficients in frame-owned plane grids, including super-resolution-adjusted unit corners and the corrected conditional 64x64-superblock unit-size bit. The active restoration stage implements the normative unit geometry, striped deblocked boundaries, Wiener filtering, self-guided projection, and 8/10/12-bit clipping, while reusing the existing SIMD-backed super-resolution and adjacent multiply/add primitives. Independently encoded fixtures covering every parameter set, plane layout, bit depth, and frame-edge geometry are still required. Other normative independently decodable still-image syntax paths still contain NotImplementedException or equivalent unsupported branches. Tile-local palette CDF adaptation is present; the remaining still-image frame-context behavior requires a separate source audit without introducing sequence playback state.
  • The frame buffer now establishes two-byte native sample storage, logical plane rows, and sample-unit block strides for 10/12-bit frames. The active intra-prediction, inverse-transform, and block-reconstruction path selects native 16-bit samples for 10/12-bit frames and has focused pipeline wiring coverage. Chroma-from-luma storage, 4:4:4/4:2:2/4:2:0 subsampling, rounded mean subtraction, U/V sharing, and 8/10/12-bit prediction now traverse AVX2 and cross-platform Vector128 paths before an exact scalar fallback. FeatureTestRunner verifies every tier against independent fixed-point definitions across every supported block width; independently encoded high-bit-depth and chroma-from-luma AVIF conformance files are still required.
  • Av1YuvConverter now consumes the signaled full or limited range, every non-reserved AV1 H.273 matrix coefficient, transfer characteristics where the matrix definition requires them, subsampling, and chroma sample position for 8, 10, and 12-bit output. Its high-bit-depth decode and encode paths use allocator-backed Rgb48 rows and the existing PixelOperations<TPixel> conversions, avoiding the former eight-bit intermediate. Encoder conversion covers monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with libavif-compatible box averaging. Identity, full/limited-range YCgCo, the fixed non-constant-luminance matrices, both fixed and chromaticity-derived constant/non-constant-luminance systems, SMPTE ST 2085, and PQ/HLG ICtCp are active in both directions. Independent vectors for every matrix, transfer, range, bit depth, sampling layout, and chroma position remain required before the complete color matrix is externally verified.
  • Forward and inverse transforms use operation-owned workspace, explicit sequential fixed storage, and stateless static-generic operator structs for every valid DCT, ADST, and identity size. Named configuration factories keep the encoder's three shifts and variable cosine precision separate from the decoder's two shifts, fixed 12-bit cosine precision, and 8/10/12-bit clamp ranges. Forward traversal shares one libaom-shaped stage network across scalar, Vector128, Vector256, and Vector512; inverse production traversal uses the verified scalar, Vector128, and Vector256 tiers. Lossless segments now bypass the DCT pipeline and apply the reversible four-by-four inverse Walsh-Hadamard transform through the same byte/high-bit-depth output operators, with a Vector128 production path and caller-workspace scalar fallback. The focused Release matrix passes across the FeatureTestRunner hardware configurations, and the production transform benchmarks report zero allocation.
  • Core intra prediction, chroma-from-luma, and palette reconstruction now use SIMD-first operator or packed-kernel architectures with exact scalar fallbacks. Nonlinear self-guided restoration remains scalar and therefore does not meet the completion architecture. It must be redesigned around vector-friendly storage, shared operator contracts, and SIMD-first traversal with benchmark-gated lane selection, with its current results retained only as fallback/parity behavior. Transform traversal is already SIMD-first, while normative super-resolution and Wiener horizontal products reuse ImageSharp's cross-platform adjacent multiply/add SIMD helper with exact scalar fallbacks.

AV1 encoder

  • HeifEncoderCore.Encode() is synchronous, matching the ImageSharp encoder contract.
  • The production switch writes only an explicitly selected legacy JPEG image item. It throws for HeifCompressionMethod.Av1 and HeifCompressionMethod.Hevc, so it produces neither AVIF nor HEIC.
  • Av1FrameEncoder.Encode() is an outline of an SVT-style pipeline rather than an implementation.
  • Required mode-decision, block-geometry, forward-transform, token-writing, neighbor-context, palette, intra-block-copy, transform-size, quantization/rate-control, and OBU-writing paths are absent or throw.
  • Encoder tests currently cover the public option contract and legacy JPEG boundaries. There is no AV1 or HEVC payload output to cross-decode.

Legacy JPEG image items

  • A JPEG item decoder exists, and the Fujifilm .hif fixture is identified and decoded as a legacy JPEG image item.
  • The encoder uses the existing ImageSharp JPEG codec only when HeifCompressionMethod.LegacyJpeg is explicitly selected and emits a jpeg primary item.
  • The legacy JPEG path still needs independent container interoperability coverage, but its quality, bit-depth, lossless, and chroma restrictions are explicit at the public option and codec boundaries.

Tests

  • HEVC coverage includes exact native-plane comparison with HM output, exact complete-image comparison with pinned libheif/libde265 references, and the 10 official Sony GENERAL Range Extensions first-picture fixtures across 8/10/12-bit monochrome, 4:2:0, 4:2:2, and 4:4:4 reconstruction. The remaining exposed profiles and individual Range Extensions tools still need exact independent vectors.
  • AV1 has focused bitstream, prediction, entropy, reconstruction, filter, film-grain, color, and transform coverage, plus real libavif inputs. Valid still-image syntax paths still contain explicit unsupported branches, so the independent AV1 decode matrix is not complete.
  • The AV1 transform matrix verifies scalar, Vector128, Vector256, and Vector512 forward representations plus the production inverse tiers across every valid size/type combination and supported bit depth. All 511 focused forward and inverse cases pass in Release; FeatureTestRunner isolates hardware tiers, every two-dimensional configuration exercises production dispatch, and the complete-block benchmark records zero managed allocation.
  • Independent libavif fixtures cover primary, grid, auxiliary-alpha, ICC, metadata-skipping, and all-sync image-sequence presentation. Reference-dependent AV1 and HEVC sequence reconstruction and independent HEVC ICC sequence coverage remain open.
  • Focused decoder-option tests cover strict, ancillary-only, image-data, and metadata-skipping behavior for still items and sequence samples. Complete adversarial dimension, allocation, malformed-container, and resource-limit coverage remains open.
  • No cross-codec encode test exists for AV1 or HEVC because both production encoding branches still throw before producing a payload.

Architectural direction

Public format identities, shared internal HEIF container

Expose HEIC and AVIF as distinct public format identities backed by one internal HEIF/ISO BMFF container implementation. Provide format-appropriate decoders, encoders, metadata, options, MIME types, extensions, and generated save methods. Generic .heif and .hif input must be dispatched by brands and primary item type to HEVC, AV1, or legacy JPEG rather than by extension. Avoid a broad internal namespace rename until the container model is stable.

Generate the standard sync and async SaveAsHeic and SaveAsAvif overloads through the existing format templates. If a generic SaveAsHeif API is retained, its encoder options must require an explicit supported payload codec rather than infer one from an extension. Public XML documentation must describe observable behavior and option effects only.

Parsed container model

Parse the meta box into an immutable logical model before decoding payloads:

  • dictionaries keyed by item ID, not collection position;
  • a one-based property table with explicit association records and essential flags;
  • resolved item references (dimg, auxl, thmb, and metadata relationships);
  • item locations represented as validated extents and construction methods;
  • typed properties for ispe, pixi, hvcC, av1C, colr, auxC, pasp, clap, irot, and imir;
  • bounded views over item payloads, independent of source box order; and
  • explicit primary, alpha, grid-tile, thumbnail, Exif, and XMP roles.

Use existing ImageSharp buffered stream and allocation abstractions. Do not copy the whole file into an unbounded byte array. Every length, offset, multiplication, allocation, tile count, and image dimension coming from the file is an external boundary and must be checked against the enclosing box and ImageSharp limits. Internal decode stages should rely on the validated model rather than repeat defensive checks.

Codec state and sample storage

Keep HEVC and AV1 bitstream state in separate codec implementations. Within each codec, separate parameter/sequence state, frame or picture headers, tile/slice entropy state, and reconstructed images. Still-image operations retain no reference pictures; bounded image-sequence operations retain only the reference frames required by their declared dependencies. Give each allocation one owner and a deterministic disposal point.

Represent 8-bit samples with bytes and high-bit-depth samples with unsigned 16-bit storage. Plane dimensions and strides must reflect monochrome and chroma subsampling instead of pretending every plane is full-resolution 4:4:4. Design storage and reconstruction operators for SIMD-first traversal, with the scalar fallback sharing the same layout and contract for behavioral parity.

Decode directly into ImageSharp pixels

Perform chroma upsampling, range expansion, matrix conversion, alpha composition, and pixel packing into allocator-backed row buffers or the destination frame. Use PixelOperations<TPixel> and the existing packed pixel conversion paths instead of creating an intermediate Image<Rgb24>.

Do not reuse JPEG or WebP color constants merely because those codecs already contain vectorized YUV conversion. Reuse their vector dispatch, lane handling, row processing, and pixel packing patterns only when the signaled HEVC/AV1 range and matrix semantics are preserved.

Existing ImageSharp code to reuse

Need Reuse target Constraint
Stream parsing BufferedReadStream, existing endian readers, bounded decoder-core patterns HEIF box extents remain the source of truth.
Memory ownership MemoryAllocator, IMemoryOwner<T>, Buffer2D<T>, allocator-backed row buffers No unbounded file-sized arrays or per-block allocations.
Pixel output PixelOperations<TPixel>, Rgba32, Rgba64, existing packed conversion methods Preserve alpha and high-bit-depth precision.
SIMD utilities SimdUtils, Vector128_, Vector256_, Vector512_, Numerics helpers Add codec-specific math only where semantics differ.
SIMD structure JPEG color-converter factories and WebP YUV row converters Reuse dispatch/tail patterns, not incompatible coefficients.
Decode lifecycle JPEG/WebP decoder cores, ImageSharp cancellation and dimension-limit handling Identify must not reconstruct pixels.
Public format API PNG, JPEG, and WebP format/decoder/encoder/options/metadata patterns Keep HEIC, AVIF, and generic HEIF dispatch behavior explicit.
Generated save APIs _Formats.ttinclude and existing format templates Generate SaveAsHeic and SaveAsAvif; require an explicit codec for generic HEIF output.
Metadata Existing ICC, Exif, XMP, and CICP-related metadata structures where available Preserve payloads and apply only specified transforms.
Test infrastructure ReferenceCodec, image comparers, feature-test helpers, codec test base classes Include independent artifacts and cross-codec tests.
Performance tests Existing ImageSharp benchmark project and hardware-intrinsics test controls Benchmark real decode stages and representative images.

Implementation phases

Every phase exit gate is an internal prerequisite for the next phase. Only the Phase 9 exit gate together with a fully passing verification matrix marks the PR complete.

Phase 0: establish a reproducible baseline

Tasks:

  • Build the merged solution in Release and record compile errors and warnings attributable to the WIP.
  • Run only the existing HEIF/HEVC/AV1 tests first, then record disabled tests and unexecuted asset coverage.
  • Record the pinned official libaom, local libavif, HM, and Android libhevc revisions, then pin the independent HEIC container interoperability oracle and external fixture toolchain.
  • Create a provenance map from each WIP codec file to its specification section and exact upstream source. Preserve the current SVT-AV1 origins where applicable and identify which missing paths will use libaom.
  • Convert the completion boundary above into a feature matrix with unsupported, parses, decodes, encodes, and verified independently states.
  • Audit every source file under src/ImageSharp/Formats/Heif. Document every type and shared contract, and add technical comments wherever the code depends on non-obvious specification syntax, fixed-point arithmetic, transform staging, entropy state, buffer layout, or SIMD behavior. Keep public XML documentation limited to observable API behavior.

Exit gate:

  • The post-merge branch has a recorded Release baseline, every existing failure is classified, all upstream code origins are known, and the complete HEIF source tree passes the documentation audit before additional porting begins.

Phase 1: correct the format contract

Tasks:

  • Define distinct HEIC and AVIF public format types over the shared internal HEIF container and register the correct brands, MIME types, and extensions.
  • Add generated SaveAsHeic and SaveAsAvif APIs and format metadata integration through the same mechanisms as established codecs. Define generic SaveAsHeif only if its options require an explicit supported payload codec.
  • Define decoder options using existing DecoderOptions behavior, including target pixel type, metadata handling, cancellation, and image-size limits.
    • Generic decode selects the caller's pixel type and the non-generic entry point defaults to Rgba32. TargetSize and Sampler are applied once after HEIF presentation composition. MaxFrames bounds retained sequence samples. SkipMetadata, SegmentIntegrityHandling, ColorProfileHandling, Configuration, and cancellation flow through the container, item, sequence, and nested-codec boundaries.
    • Complete adversarial dimension and allocation-limit coverage for still items, grids, auxiliary images, and sequence tracks before closing this contract item. Large payload and image buffers already use the configured allocator, but the complete cross-product has not been verified.
  • Define codec-specific encoder options with observable semantics for quality, speed/effort, lossless mode, chroma subsampling, bit depth, alpha quality, and metadata handling. Avoid exposing internal HEVC or AV1 tuning knobs without a stable user-facing meaning.
  • Make Rgba32 the default 8-bit decode output so alpha is not silently lost.
  • Remove unsupported JPEG 2000, JPEG-XR, JPEG-XS, and AVC capability claims unless those payload codecs are added to the completion matrix. Retain legacy JPEG as an explicit supported HEIF image-item codec.

Exit gate:

  • API review confirms that names and documented behavior match existing ImageSharp patterns and promise only the completed HEVC, AV1, and legacy JPEG HEIF payload paths. Capabilities must not be registered before their implementation reaches the final PR gate.

Phase 2: rebuild the bounded HEIF still-image reader around validated items

Tasks:

  • Implement a bounded box reader supporting 32-bit, 64-bit, and to-end box sizes where allowed, with overflow-safe arithmetic and correct parent bounds.
  • Accept the applicable HEIF, HEIC, and AVIF still-image brands through the major or compatible brand rules. Recognize and reject sequence/movie brands without implementing their track surface.
  • Skip unknown non-essential boxes and properties. Reject unknown essential properties attached to a decoded item.
  • Parse meta children independently of physical order and build the item/property/reference model described above.
  • Correct item ID lookup, one-based property indices, association flag masks, full-box versions, and large IDs/offsets.
  • Resolve idat and mdat item data, multiple extents, construction methods, and 64-bit offsets through bounded item streams.
  • Parse and associate HEVC and AV1 configuration, dimensions, plane information, ICC/CICP color, alpha auxiliary type, grids, transforms, Exif, XMP, and thumbnails.
  • Make Identify return dimensions, bit depth, color type, alpha presence, profiles, and metadata from the parsed model without decoding AV1 tiles.
  • Add malformed-container tests for every external length, offset, count, ID, association, extent, and relationship boundary.

Exit gate:

  • The parser resolves each current HEIC, HIF, and AVIF asset into a stable logical model, malformed inputs fail without escaping bounds or allocating attacker-controlled sizes, and Identify has reference-verified metadata and payload classification.

Phase 3: complete SIMD-first AV1 still-image reconstruction

Implement and verify in dependency order:

  • OBU framing, sequence headers, frame headers, tile groups, byte alignment, and trailing bits.
  • One coherent decoder lifecycle that retains parsed frame and tile state and disposes all buffers deterministically.
  • Tile partitioning, mode information, segmentation, delta quantization, transform-size selection, coefficient token decode, inverse quantization, and inverse transforms.
  • Intra prediction, including every directional, smooth, Paeth, CFL, filter-intra, and palette case permitted by AV1.
    • Implement SIMD-first chroma-from-luma storage, 4:4:4/4:2:2/4:2:0 subsampling, rounded mean subtraction, and 8/10/12-bit prediction with exact scalar fallback and FeatureTestRunner parity.
    • Implement allocation-free SIMD-first palette reconstruction for palette sizes 2-8, transform widths 4-64, and 8/10/12-bit samples with exact scalar fallback and FeatureTestRunner parity.
    • Verify palette mode syntax and presented reconstruction with an independently encoded palette AVIF fixture.
  • Lossless and high-bit-depth reconstruction with correct clipping and intermediate precision.
    • Route lossless 4x4 blocks through allocation-free reversible inverse Walsh-Hadamard reconstruction for 8/10/12-bit samples, including the DC-only specialization, Vector128 production traversal, scalar fallback, exact clipping, and FeatureTestRunner parity.
    • Verify lossless syntax, inverse quantization, prediction, and presented reconstruction with independently encoded 8/10/12-bit AVIF fixtures.
  • Deblocking loop filter.
    • Implement allocation-free SIMD-first 4-, 6-, 8-, and 14-tap filtering for vertical and horizontal edges in 8/10/12-bit storage through closed edge operators, with exact scalar fallback and FeatureTestRunner parity against an independent definition.
    • Verify deblocking syntax, filter-level derivation, and boundary traversal with independently encoded 8/10/12-bit AV1 samples and exact scalar-libaom planes; verify presented reconstruction and public precision with genuine AVIF containers at every supported bit depth.
  • CDEF.
    • Replace the scalar helper bucket with an allocation-free SIMD-first Av1CdefFilter: vector direction analysis, 4x4/4x8/8x4/8x8 constrained filtering, byte/16-bit output operators, and one exact scalar fallback verified through FeatureTestRunner.
    • Move the immutable deblocked-plane snapshot and frame-wide direction/variance maps to allocator-owned decoder memory, preserving luma-first direction derivation, skipped-block selection, asymmetric chroma direction conversion, and frame-edge sentinels.
    • Verify CDEF syntax, strength derivation, unit/block traversal, subsampling, frame edges, and presented reconstruction with independently encoded active-CDEF AV1/AVIF fixtures across 8/10/12-bit samples.
  • Super-resolution scaling.
  • Loop restoration.
  • Frame padding and film-grain synthesis when signaled.

For each SIMD-suitable item, design the data layout, operator contract, scratch ownership, traversal, and descending-width dispatch around the production SIMD path first. Implement the behaviorally equivalent scalar fallback through the same contract, then verify every supported vector width and the scalar path against normative or independent results before enabling the stage. Serial adaptive work such as entropy decoding remains scalar where data dependencies prevent meaningful vectorization. Remove constant feature-disable flags and unsupported branches only when their replacement is verified. Unsupported syntax must produce a codec-specific invalid-image error; it must never silently skip a normative reconstruction stage.

Exit gate:

  • Independently encoded, opaque, single-item AVIF files reconstruct correctly across all AVIF profiles, bit depths, subsampling modes, and normative still-image compression tools. Pixel comparisons are made after applying the same signaled color conversion in the reference path.

Phase 4: complete HEVC still-image reconstruction

Implement and verify in dependency order:

  • HEIF item-local length-delimited NAL units, hvcC, VPS, SPS, PPS, and the parameter-set and slice-header syntax needed to decode the one independently coded picture carried by a supported still-image item. Do not add an Annex B byte-stream API or an access-unit/timeline abstraction.
  • One bounded decoder lifecycle that owns only the parameter sets, current-picture state, slice/tile entropy state, and reconstructed planes required by that image item.
  • CABAC arithmetic decoding and every required context transition.
  • Coding-tree, coding-unit, prediction-unit, and transform-unit traversal across all permitted sizes and partition modes.
  • Intra prediction for every luma and chroma mode, including strong intra smoothing and constrained prediction rules.
    • Implement and verify the allocation-free SIMD-first predictor primitive for all 35 modes, 4/8/16/32 blocks, and 8/10/12-bit samples.
    • Implement reconstructed-plane reference collection, substitution, asymmetric chroma-unit handling, and reference-filter selection from caller-owned availability and scratch spans.
    • Build each availability flag from reconstructed-plane and coding state, and connect the predictor to transform-unit traversal for the independently verified 8-bit 4:2:0 and 4:4:4 item paths.
  • Scaling lists, inverse quantization, transform skip, every required inverse transform, range-extension precision, and lossless reconstruction.
    • Implement allocation-free SIMD-first inverse DCT for every 4/8/16/32 width and height combination, the four-by-four intra-luma inverse DST, normative intermediate and residual clipping, and saturated prediction addition for 8/10/12-bit samples. Verify normal and forced-scalar execution against fixed results and a dense scalar oracle.
    • Implement the SIMD-first inverse-quantization primitive for flat and scaling-list paths across every transform size, component matrix, prediction mode, transform-skip scaling rule, signed normalization direction, and range-extension precision path.
    • Derive effective luma and chroma quantization parameters, including bit-depth offsets and chroma-format mapping, then connect scaling-list selection and inverse quantization to transform-unit traversal.
      • Implement the immutable effective-QP value used by reconstruction, including independent luma/chroma bit-depth offsets, the normative 4:2:0 mapping plateaus, the 4:2:2/4:4:4 saturation rule, and combined picture/slice/coding-unit chroma offsets.
      • Select each transform unit's coding-unit luma QP and chroma-adjustment-list entry, then pass the derived component QP into inverse quantization.
    • Implement transform skip, coefficient rotation, implicit and explicit residual DPCM, transquant bypass, and lossless reconstruction.
      • Implement allocation-free SIMD-first transform-skip normalization, complete-block coefficient rotation, transquant-bypass copying, implicit intra-direction selection, and horizontal/vertical inverse residual DPCM with a scalar fallback and signed residual clipping.
      • Decode explicit inter residual-DPCM modes and connect bypass, transform skip, residual DPCM, prediction addition, and lossless reconstruction through transform-unit traversal.
    • Connect coefficient decoding, inverse quantization, transform selection, reusable scratch, and add/clip to transform-unit traversal.
      • Implement grouped diagonal/horizontal/vertical scans, last-significant and significance context derivation, coefficient levels and signs, sign-data hiding, Range Extensions bypass alignment, four persistent Rice states, and one pooled scratch owner with no per-transform allocation or stackalloc.
      • Invoke coefficient decoding and the existing reconstruction kernels from transform-unit traversal, then verify complete transform blocks from independently encoded HEIC fixtures.
  • Deblocking and sample-adaptive offset for every signaled luma/chroma and bit-depth path.
  • Tiles, wavefront entry points, dependent slices, and all other parallelization syntax permitted by the exposed still-image profiles.
  • Supplemental enhancement information that changes image presentation or metadata exposed by ImageSharp.
    • Establish exact HM 18 native-plane conformance for four real 8-bit item payloads covering 4:2:0, 4:4:4, cropping, multiple grid positions, and wavefront entry points.
    • Establish exact independent presentation verification for single images, grids, auxiliary alpha, ICC, 4:2:0, and 4:4:4 output without weakening the exact native-plane codec oracle.
    • Establish exact native-plane conformance for 8/10/12-bit monochrome, 4:2:0, 4:2:2, and 4:4:4 across every exposed HEVC profile and required Range Extensions tool.

Each subsystem keeps a specification-shaped scalar oracle derived from the HEVC specification and the pinned implementation reference, while production hot paths are designed SIMD-first with scalar fallback. No valid syntax in the exposed HEIC profiles may terminate in an unsupported branch or silently omit a normative reconstruction stage.

Exit gate:

  • Independently encoded HEIC files reconstruct correctly across every exposed HEVC profile, chroma format, bit depth, range-extension tool, and normative still-image compression path.

Phase 5: shared color conversion, alpha, grids, and presentation transforms

Tasks:

  • Implement monochrome, 4:2:0, 4:2:2, and 4:4:4 plane access with every valid signaled chroma sample position.
    • Implement SIMD AV1 4:2:0 and 4:2:2 chroma reconstruction and downsampling, including centered/colocated reconstruction and odd image edges.
    • Implement pooled HEVC plane conversion for monochrome, 4:2:0, 4:2:2, and 4:4:4, including exact encode-coordinate and decode-interpolation coverage for all six progressive 4:2:0 sample locations.
    • Connect reconstructed HEVC pictures and complete the independently verified AV1/HEVC chroma-position matrix before closing this item.
  • Implement full- and limited-range expansion for 8, 10, and 12-bit samples across every supported plane layout.
    • Implement AV1 range expansion and quantization with SIMD-first byte/ushort loaders and storers and pooled planar rows.
    • Implement the equivalent HEVC plane paths with independent luma/chroma precision, native 16-bit high-depth staging, and exact limited-range endpoint coverage.
    • Independently verify the complete HEVC range/precision/plane-layout cross-product after reconstruction is connected.
  • Implement every non-reserved HEVC/AV1 color-primary, transfer-characteristic, and matrix-coefficient signaling path, including identity conversion, with correct fixed-point rounding and clipping.
    • Implement one shared H.273 matrix and transfer-function operator set in both directions for AV1 and HEVC, including identity, derived coefficients, constant luminance, SMPTE ST 2085, PQ/HLG ICtCp, IPT-C2, YCgCo-Re, and YCgCo-Ro.
    • Add independent conformance vectors for signaling combinations that libavif deliberately rejects, then verify the operator set through completed HEVC reconstruction.
  • Make SIMD the default decode path for YUV range expansion, chroma reconstruction, H.273 matrix and transfer conversion, clamping, and 8/10/12-bit RGB packing. Use the established static operator pattern with Vector512, Vector256, and Vector128 dispatch followed by one scalar tail that remains the behavioral oracle.
    • AV1 uses the JPEG-style static operator contract and descending SIMD-width traversal for decode and encode, with scalar execution limited to the remaining lanes or hardware without vector support.
    • HEVC plane conversion uses the same shared static operators, descending SIMD-width traversal, JPEG byte-plane pack/unpack, pooled high-depth staging, and scalar fallback; full-HD SIMD/scalar timings are recorded above.
    • Connect the HEVC reconstruction path and record complete HEIC item and sequence measurements before closing this item.
  • Apply embedded ICC profiles after codec YUV/CICP conversion and image composition through ImageSharp's existing color-profile converter. Cover preserve, convert, compact, and ignore behavior for primary items, grids, alpha-composited images, and every presented sequence frame; never substitute CICP or an assumed sRGB profile for an embedded ICC profile.
    • Primary AVIF verification passes four independently scoped cases: exact profile preservation, non-sRGB conversion with changed pixels and paired-source comparison, canonical-sRGB compaction without pixel changes, and metadata skipping. Grid, auxiliary-alpha, sequence, and HEVC cases remain open.
  • Decode alpha auxiliary items as monochrome planes, validate dimensions and bit depth, and compose them without losing precision. Define premultiplication behavior from AVIF signaling and ImageSharp's pixel contract.
  • Validate grid tile type, dimensions, properties, order, and canvas coverage. Compose directly into the destination frame, cropping only the permitted right and bottom tile overlap.
  • Apply clean aperture, rotation, mirroring, and pixel aspect ratio according to the container property order and ImageSharp metadata/processing conventions.
  • Preserve ICC, CICP, Exif, and XMP metadata using existing ImageSharp profile types. ICC preservation and ICC color conversion are separate contracts and must both be independently verified.

Exit gate:

  • The complete cross-product of valid HEVC/AV1 bit depths, subsampling modes, ranges, color signaling, alpha, grids, and transforms is covered by focused vectors and representative HEIC/AVIF integration files and matches independently decoded references within a documented conversion tolerance. Tests must prove ICC conversion changes pixel values according to an independent profile transform instead of checking profile retention alone. Integer identity/lossless cases must match exactly.

Phase 6: implement a real AV1 still-image encoder

Delete the JPEG payload path from the production encoder. Keep the synchronous ImageSharp encoder contract synchronous unless the established base contract provides an async implementation; never use async void.

Implement in vertical slices that always produce a decodable AV1 bitstream:

  • Convert RGB/RGBA to AV1 planes for every 8, 10, and 12-bit output, range, matrix, and monochrome/4:2:0/4:2:2/4:4:4 combination permitted by the selected AV1 profile.
    • Implement the pooled sequential RGB-to-planar pipeline, 8/10/12-bit quantization, every implemented H.273 operator, and SIMD 4:2:0/4:2:2 downsampling with odd-edge handling.
    • Complete the exhaustive format/range/matrix/transfer/primary cross-product and validate encoded planes against an independent implementation before closing this item.
  • Write sequence, frame, tile-group, and metadata OBUs for a reduced still picture.
  • Implement a temporary smallest-valid intra-only vertical slice using existing partition, prediction, transform, quantization, coefficient, and entropy structures.
  • Complete block geometry, neighbor/context updates, transform selection and forward transforms, quantization, coefficient tokenization, and range coding.
  • Add mode decision and rate/distortion selection in increasing effort levels. Reuse computed prediction and transform results instead of duplicating analysis across stages.
  • Add lossless mode and validate the exact lossless constraints rather than treating quality 100 as lossless.
  • Add in-loop filter decisions and signaling. A legal choice to disable an encoder feature is distinct from a decoder skipping a signaled feature.
  • Add alpha as an auxiliary AV1 item with independently controllable quality where the public option warrants it.
  • Write the AVIF item graph, extents, av1C, pixel information, color properties, metadata, and alpha relationships with correct brands.
  • Write embedded ICC and CICP color properties independently, preserve the source profile selected by the encoder options, and verify both metadata round-trip and independently converted output pixels.
  • Make output deterministic for identical pixels, metadata, options, and configuration.
  • Complete the partition, prediction, transform, quantization, entropy, filter, and rate/distortion choices needed for quality and effort settings to provide a genuine compression tradeoff rather than selecting from a fixed coding subset.
  • Exercise every encoder bit-depth, plane-layout, range, color, alpha, lossless/lossy, quality, and effort combination through independent decode.

Every vertical slice must be decoded by libavif before additional compression features are added. Self-round-trip tests are supplementary because matching encoder and decoder bugs can otherwise hide invalid bitstreams.

Exit gate:

  • libavif and another independent AV1 decoder accept ImageSharp output across the complete encoder matrix, and ImageSharp reconstructs the same files. Lossless output is pixel-exact; lossy output demonstrates effective quality/effort tradeoffs and meets recorded quality and size expectations without malformed or non-AV1 payloads.

Phase 7: implement a real HEVC still-image encoder

Implement in vertical slices that always produce a HEVC bitstream accepted by an independent HEIC decoder:

  • Convert ImageSharp pixels into every exposed HEVC bit depth, range, and monochrome/4:2:0/4:2:2/4:4:4 combination.
  • Write VPS, SPS, PPS, slice headers, parameter arrays in hvcC, and a smallest-valid intra picture as a temporary vertical slice.
  • Complete coding-tree partitioning, intra prediction selection, forward transforms, scaling/quantization, coefficient scanning, and CABAC encoding.
  • Add real rate/distortion mode decision, quality and effort controls, lossless mode, deblocking decisions, and sample-adaptive-offset decisions.
  • Add alpha auxiliary images, grids, metadata, color properties, image relationships, and correct HEIC brands.
  • Write embedded ICC and CICP color properties independently, preserve the source profile selected by the encoder options, and verify both metadata round-trip and independently converted output pixels.
  • Exercise every encoder profile, bit depth, chroma format, range, color, alpha, lossless/lossy, quality, and effort combination through independent decode.

Exit gate:

  • Independent HEIC decoders accept ImageSharp output across the complete encoder matrix. Lossless output is pixel-exact; lossy output demonstrates effective quality/effort tradeoffs and meets recorded quality and size expectations.

Phase 8: SIMD and allocation optimization

Production hot paths are designed and implemented SIMD-first. The same architecture must provide a behaviorally equivalent scalar fallback for unsupported hardware, remainder work, and parity verification; scalar code is not a temporary production architecture that SIMD is added to later. Every supported hardware path and the scalar fallback are verified against independent specification-shaped results before the subsystem is complete.

Tasks:

  • Remove known avoidable allocations first: per-transform arrays, the intermediate RGB image, repeated block scratch arrays, and file-sized buffering.
    • Complete the active AV1 forward and inverse transform-family checklist above.
  • Benchmark codec-specific costs for CABAC/range decode, inverse transforms, still-image prediction, deblocking, SAO, CDEF, restoration, chroma upsampling, color conversion, alpha packing, and grid copies.
    • Add a permanent frame-wide HEVC intra-prediction benchmark. On .NET 10, SIMD planar, vertical-angular, and horizontal-angular prediction measured 155.2, 150.7, and 255.0 microseconds per padded 1920x1088 frame, compared with forced-scalar timings of 1.890, 1.081, and 0.979 milliseconds: 12.2, 7.2, and 3.8 times faster with zero managed allocations.
    • Add a permanent frame-wide HEVC reference-preparation benchmark. On .NET 10, complete and partially substituted borders measured 465.3 and 566.7 microseconds per 2,040-block padded 1920x1088 frame, compared with forced-scalar timings of 477.5 and 608.0 microseconds, with zero managed allocations.
    • Add a permanent frame-wide HEVC inverse-transform benchmark. On .NET 10, dense-coefficient 32x32 twelve-bit inverse DCT, transposition, and add/clip measured 3.643 milliseconds per padded 1920x1088 frame, compared with 45.23 milliseconds with hardware intrinsics disabled: 12.4 times faster with zero managed allocations.
    • Add a permanent frame-wide HEVC inverse-quantization benchmark. On .NET 10, dense 32x32 twelve-bit flat and scaling-list paths measured 69.98 and 238.4 microseconds per padded 1920x1088 frame, compared with forced-scalar timings of 1.469 and 1.609 milliseconds: 21.0 and 6.7 times faster with zero managed allocations. Pre-expanding the scaling matrices once reduced the SIMD scaling-list path from 4.974 milliseconds to 238.4 microseconds.
    • Add a permanent frame-wide HEVC residual-reconstruction benchmark. On .NET 10, dense 32x32 twelve-bit transform skip, horizontal RDPCM, and vertical RDPCM measured 79.64, 275.61, and 110.81 microseconds per padded 1920x1088 frame, compared with forced-scalar timings of 571.5 microseconds, 1.512 milliseconds, and 1.257 milliseconds: 7.2, 5.5, and 11.3 times faster with zero managed allocations.
    • Add a permanent frame-wide HEVC grouped coefficient-scan benchmark. On .NET 10, generating the complete diagonal 32x32 scan for all 2,040 transform blocks in a padded 1920x1088 frame measured 1.432 milliseconds with zero managed allocations. CABAC decisions remain serial and require an independently encoded complete-slice workload after transform-unit traversal is connected.
    • Add a permanent frame-wide AV1 chroma-from-luma prediction benchmark. The lane-selection run rejected the explicit Vector512 experiment because AVX2 was 2.6% faster at 8-bit and 4.8% faster at 12-bit on the test system. With that path removed, the selected hardware and forced-AVX2 paths are equivalent; the final .NET 10 run measured 88.76 and 74.89 microseconds per padded 1920x1088 frame at 8 and 12 bits, compared with forced-scalar timings of 1.594 and 1.604 milliseconds: 18.0 and 21.4 times faster with zero managed allocations.
    • Add a permanent frame-wide AV1 palette-reconstruction benchmark. On .NET 10, runtime-selected hardware measured 20.40 and 62.99 microseconds per padded 1920x1088 frame at 8 and 12 bits, compared with forced-scalar timings of 488.96 and 509.58 microseconds: 24.0 and 8.1 times faster with zero managed allocations. Forced Vector512 measured 20.81 and 62.70 microseconds, so production continues to honor the runtime's preferred vector width instead of overriding this machine's Vector256 selection.
    • Add a permanent frame-wide AV1 lossless inverse-transform benchmark. Dense four-by-four reconstruction across a padded 1920x1088 frame measured 595.6 microseconds at 8 bits and 512.4 microseconds at 12 bits, compared with forced-scalar timings of 2.074 and 2.270 milliseconds: 3.5 and 4.4 times faster with zero managed allocations.
    • Add a permanent frame-wide AV1 deblocking benchmark. Complete vertical and horizontal fourteen-tap passes across a padded 1920x1088 frame measured 4.040 milliseconds at 8 bits and 4.184 milliseconds at 12 bits, compared with forced-scalar timings of 9.416 and 9.410 milliseconds: 2.33 and 2.25 times faster with zero managed allocations.
  • Implement vector paths only for confirmed hot loops, using existing Vector128, Vector256, and Vector512 helper and dispatch patterns where supported.
  • Prioritize shared color conversion and pixel packing, chroma upsampling, inverse-transform add-and-clip, intra predictors, HEVC deblock/SAO, AV1 loop filter/CDEF/restoration, and contiguous grid copies.
  • Benchmark the complete decode color pipeline on representative 8/10/12-bit AVIF and HEIC images with and without embedded ICC profiles. Report absolute end-to-end timings and allocations in addition to the isolated YUV/CICP and ICC stage costs.
  • Design each hot-path operator and data layout from the applicable upstream SIMD algorithm, mapping lane width, signedness, intermediate precision, rounding, saturation, edge extension, and high-bit-depth behavior before implementation.
  • Implement the scalar fallback through the same operator contract, tables, constants, scratch ownership, and traversal as the SIMD paths so it remains a specification-shaped oracle without becoming a separate architecture.
  • Test scalar and each available hardware path with intrinsics explicitly enabled and disabled, including widths shorter than a vector, exact-vector widths, non-multiples, edges, maximum sample values, and high-bit-depth overflow cases.
  • Remove dead or commented SIMD experiments once a verified production path replaces them.

Exit gate:

  • Benchmarks show a material improvement on representative AVIF files, allocation measurements meet an agreed budget, and every vector path is behaviorally identical to the scalar path for integer reconstruction or within the documented color-conversion tolerance.

Phase 9: hardening, documentation, and release readiness

Tasks:

  • Fuzz the box parser, AV1 OBU parser, HEVC NAL/parser, entropy decoders, and dimension/allocation boundaries using the same safety expectations as established ImageSharp codecs.
  • Test seekable and non-seekable streams, short reads, cancellation, truncated data, unknown optional boxes, unknown essential properties, oversized dimensions, malicious counts, and offset arithmetic overflow.
  • Run the focused HEIF/HEIC/AVIF suite after every final codec edit, then the full ImageSharp suite in Release.
  • Build all supported target frameworks and run packaging/API compatibility checks used by the repository.
  • Update public documentation, format tables, MIME/extension lists, samples, and THIRD-PARTY-NOTICES.TXT.
  • Remove placeholder images, legacy JPEG-in-HEIF production paths, stale TODO-only code, disabled tests that now have coverage, and unsupported capability claims.

Exit gate:

  • The full Release build and test matrix passes, every valid HEVC/AV1 color/compression/bit-depth entry is implemented, independent HEIC and AVIF interoperability is recorded for the complete feature matrix, the public API has been reviewed, provenance and patent/license obligations are complete, and no advertised feature depends on a placeholder, narrow temporary subset, silent fallback, disabled normative stage, or unsupported valid syntax branch.

Verification matrix

Every valid combination in the HEVC and AV1 profiles exposed by the final public contract needs focused coverage and representative integration coverage. Each axis needs independently produced HEIC/AVIF inputs and ImageSharp-produced outputs; pairwise and targeted cross-product cases must cover interactions where exhaustive media fixtures would be redundant.

Status Area Required coverage
- [ ] Container/payload HEIC with HEVC, AVIF with AV1, and generic HEIF/HIF with supported HEVC, AV1, or legacy JPEG items; detection by brands/items.
- [ ] Bit depth Every bit depth permitted by the exposed HEVC and AV1 profiles, including 8, 10, and 12-bit decode and encode.
- [ ] Planes monochrome, 4:2:0, 4:2:2, and 4:4:4 decode and encode in every valid HEVC/AV1 profile and depth combination.
- [ ] Range full and limited decode and encode.
- [ ] Color every valid non-reserved color-primary, transfer-characteristic, matrix-coefficient, and chroma-position signaling path; identity RGB; ICC; CICP defaults and overrides.
- [ ] Alpha opaque, binary, gradient, different alpha quality, high bit depth, malformed relationship.
- [ ] Structure single item, multiple extents, idat, mdat, grids with cropped edge tiles, metadata items.
- [ ] Transform pasp, clap, irot, imir, and valid combinations.
- [ ] AV1 decode tools every normative transform type/size, still-image predictor, partition, palette, segmentation, quantization, entropy/context, lossless, deblock, CDEF, super-resolution, restoration, and film-grain path valid in independently decodable AVIF still-image items.
- [ ] AV1 encode compression real mode decision and rate/distortion selection across partitions, predictions, transforms, quantization, entropy coding, filters, lossless/lossy quality, and effort settings; no permanent fixed coding subset.
- [ ] HEVC decode tools every normative NAL/parameter/slice, CABAC, coding-tree, intra prediction, transform, quantization, range-extension, lossless, tile/wavefront, deblock, and SAO path valid in independently decodable still-image items for the exposed HEIC profiles.
- [ ] HEVC encode compression real coding-tree, prediction, transform, quantization, CABAC, filter, and rate/distortion decisions across lossless/lossy quality and effort settings; no permanent fixed coding subset.
- [ ] Streams file, memory, non-seekable, short-read wrapper, cancellation.
- [ ] Failure truncation at every box/OBU layer, invalid sizes/offsets/counts, unknown essential properties, unsupported profile.
- [ ] Interop libavif and libaom for AVIF/AV1; a pinned independent HEIC container and HEVC codec implementation for HEIC/HEVC.
- [ ] SIMD scalar, 128, 256, and 512-bit paths where supported; tails and edge blocks.

Reference outputs must be versioned artifacts or generated by a pinned reference command whose exact tool version and arguments are recorded. Do not use ImageSharp's own decoder to establish the expected pixels for its encoder, and do not replace final-image assertions with internal buffer or non-zero checks.

No valid HEVC or AV1 color, compression, or bit-depth row may remain unsupported, partially implemented, disabled, or deferred when the PR is marked complete.

Working rules for implementation

  • Keep changes vertical and reviewable. A slice should add one behavior, its focused tests, independent evidence, and any required notice update.
  • Design SIMD-suitable codec work SIMD-first. Establish vector-friendly storage, operator boundaries, scratch ownership, traversal, every applicable lane width, and benchmark-gated dispatch before implementing the equivalent scalar fallback; never build a scalar production architecture and bolt SIMD onto it later.
  • Inspect every owning method and upstream invariant before adding guards. Validate external file data at the parser/model boundary and rely on those established invariants internally.
  • Do not extract one-use helpers merely to label code. Extract shared primitives only when they have genuine reuse or remove substantial complexity.
  • Put comments at the points where HEVC/AV1 rounding, edge extension, context propagation, or SIMD lane behavior is not evident from the code. Comments should explain why the algorithm has that shape and identify the normative rule and pinned implementation reference.
  • Use observable behavior only in public API documentation.
  • Do not use reflection, built-assembly probing, native runtime fallbacks, fabricated images, or self-round-trip-only evidence.
  • Build and test in Release configuration.

The dashboard and immediate execution queue define the remaining critical path. In phase terms, work proceeds as follows:

  • Finish the Phase 8 checkpoint for the implemented AV1 Vector512 transforms, including Release, feature-isolation, and benchmark evidence.
  • Current: close Phase 3 by removing every unsupported valid AV1 still-image syntax path and proving the complete AVIF decode matrix with independent inputs and scalar/SIMD parity.
  • Close Phase 4 by completing the remaining HEVC profile and Range Extensions matrix with exact native-plane and presented-image evidence.
  • Close Phase 5 and the decode portion of the bounded sequence ledger: color, ICC, alpha, grids, presentation transforms, reference-dependent samples, and complete animated AVIF/HEIC decode.
  • Close the still-image portions of Phases 0, 1, and 2 that remain as release gates: documentation, provenance, public format boundaries, API review, parser hardening, and malformed-input coverage.
  • Complete Phase 6 with a real AV1 encoder and independent AVIF cross-decode across every public option and metadata combination.
  • Complete Phase 7 with a real HEVC encoder and independent HEIC cross-decode across every public option and metadata combination.
  • Complete animated AVIF and HEIC writing only after both still payload encoders pass their independent exit gates.
  • Close Phase 8 end-to-end performance and allocation budgets for every completed subsystem, then close Phase 9 and the full verification matrix.

The phase numbers describe technical workstreams; they are not permission to work out of order. SIMD, allocation control, documentation, and independent evidence are completion requirements inside each active workstream. The PR remains incomplete until every dashboard stage and verification-matrix row is complete.