📷 A modern, cross-platform, 2D Graphics library for .NET
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

122 KiB

HEIF family implementation plan

Goal

Complete a production-quality, fully managed HEIF family image-format implementation for ImageSharp. The implementation must support still and animated HEIC, HIF/HEIF, and AVIF image files, interoperate with independent encoders and decoders, follow the existing ImageSharp architecture and code style, reuse existing ImageSharp infrastructure wherever its semantics match, and use SIMD for measured hot paths without maintaining a separate behavior model.

HEIF is the shared ISO BMFF-derived container. HEIC carries HEVC image items, AVIF carries AV1 image items, and .hif/.heif are container extensions whose payload codec must be determined from brands and item types rather than the filename. The completed implementation will support HEVC, AV1, and legacy JPEG image items. Other registered HEIF payload codecs must not be advertised unless they are implemented and independently verified.

Completion boundary

This plan has one PR completion gate. The phases below are dependency order and internal verification points only; none is a separately releasable or merge-complete subset. The PR is not complete until the complete matrix is implemented and independently verified.

Full completion includes:

  • XML documentation covers every type and contract in the HEIF implementation, including the AV1 and HEVC codec internals, and inline comments explain non-obvious container layouts, bitstream rules, numerical algorithms, SIMD choices, and interoperability constraints.
  • HEIF, HEIC, and AVIF brands and item types are detected without relying on file extensions.
  • HEVC/HEIC and AV1/AVIF decode and encode are complete for every image behavior included in the final scope.
  • Legacy JPEG image-item decode and encode are standards compliant for HEIF/HIF files.
  • Every bit depth and chroma format permitted by the exposed HEVC and AV1 profiles is supported, including 8, 10, and 12-bit and monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 paths.
  • Full- and limited-range conversion supports every valid signaled color-primary, transfer-characteristic, matrix-coefficient, and chroma-sample-position combination, including identity RGB signaling.
  • Every normative AV1 compression tool permitted by the final image scope decodes without an unsupported branch or silently skipped reconstruction.
  • The AV1 encoder implements real lossy and lossless compression with working quality and effort controls, complete mode decision, prediction, transform, quantization, entropy coding, and legal in-loop filter decisions.
  • Every normative HEVC compression tool permitted by the final image scope decodes across the exposed profiles, including the range-extension tools required for high bit depth and 4:2:2/4:4:4.
  • The HEVC encoder implements real lossy and lossless compression with working quality and effort controls, complete coding-tree, prediction, transform, quantization, CABAC, deblocking, and sample-adaptive-offset decisions.
  • Primary images, alpha auxiliary images, image grids, ICC and CICP color information, Exif, XMP, pixel aspect ratio, clean aperture, rotation, and mirroring are complete.
  • AVIF interoperability is independently verified with libavif and libaom, and HEIC interoperability is independently verified with a separately selected HEVC/HEIF implementation.

Checkboxes may be marked complete only when the implementation and the verification required by the corresponding phase exit gate are both recorded. Source presence, compilation, self-roundtrip, or a temporary vertical slice is not completion evidence.

Active and queued work

  • Active: complete the SIMD-first shared HEIF color pipeline, including ICC color management after codec color conversion and image composition.
    • Correct AV1 clipped-edge partition entropy handling and verify every block-mode cell from a real libavif AVIF against libaom 3.14.1.
    • Preserve the exact ICC payload from an independently encoded AVIF primary item.
    • Prove that a genuine non-sRGB AVIF profile changes decoded pixels and matches the independently converted source image within the documented AV1 tolerance.
    • Verify canonical-sRGB compaction and metadata skipping independently from ICC preservation and color conversion.
    • Replace the AV1 RGB/YUV arithmetic layer with JPEG's closed-generic static operator pattern in both directions: frame-scoped operator selection, in-place planar decode, RGB inputs with out component planes for encode, Vector512/Vector256/Vector128 traversal, and one scalar tail.
    • Keep codec row scheduling sequential and reuse frame-scoped allocator-backed buffers. Reuse JPEG's optimized UnpackIntoRgbPlanes contract for 8-bit encoding and pooled Rgb48 staging for 10/12-bit encoding so high-precision input is not truncated through an eight-bit or Vector4 intermediate.
    • Implement SIMD range expansion, chroma reconstruction, matrix/transfer conversion, clamping, and RGB packing for coefficient YCbCr, identity GBR, YCgCo, constant luminance, SMPTE ST 2085, and PQ/HLG ICtCp, with focused SIMD-lane/scalar-tail parity coverage.
    • Implement SIMD 4:2:0 and 4:2:2 encoder downsampling with odd-width and odd-height tails, and verify the stored 8/12-bit chroma samples against independently encoded full-resolution planes.
    • Complete the SIMD YUV/CICP paths for every supported AV1 bit depth, chroma format, range, matrix, transfer function, color primary, and chroma position, with scalar fallback only when hardware vectorization is unavailable or the operation is inherently non-vectorizable.
    • Apply ICC conversion only after the SIMD YUV/CICP stage, alpha composition, grid assembly, and presentation transforms have produced the presented RGB image; retain ImageSharp's shared ICC converter and optimize reusable bulk kernels rather than creating a HEIF-specific color-management implementation.
    • Verify ICC preservation, conversion, compaction, and metadata skipping for grids, alpha-composited images, every presented sequence frame, and the completed HEVC path.
      • Independently encode AVIF primary, grid, alpha-auxiliary, and two-frame sequence vectors with libavif 1.4.2/libaom 3.14.1; verify exact profile preservation, non-sRGB conversion, canonical-sRGB compaction, non-sRGB compact preservation, metadata skipping, every presented frame, and unchanged composed alpha.
      • Repeat the same matrix for HEVC primary, grid, alpha-auxiliary, and sequence paths after HEVC reconstruction is complete.
    • Benchmark representative 8/10/12-bit end-to-end AVIF and HEIC color pipelines with and without ICC conversion, recording absolute timings and allocations before marking Phase 5 complete.
      • Add permanent frame-wide Av1ColorConversionBenchmarks coverage for sequential 1920x1080 YUV 4:2:0 conversion in both directions at 8, 10, and 12 bits, including managed-allocation reporting.
      • Record the .NET 10 short-run AV1 baseline on 2026-08-25. SIMD decode measured 4.025/2.803/2.742 ms and encode measured 3.750/2.555/2.450 ms at 8/10/12 bits with 488-744 B allocated per frame. Forced-scalar decode measured 31.15/29.47/28.40 ms and encode measured 19.19/15.77/15.14 ms, making the normal SIMD pipeline 7.7-10.5 times faster for decode and 5.1-6.2 times faster for encode on this machine.
      • Add representative ICC-enabled AVIF cases and the equivalent HEVC cases after the HEVC reconstruction path is complete.
  • Completed: remove production ToArray() materializations from the HEIF implementation and make every retained copy an explicit ownership decision.
    • Store AV1 palette colors and HEVC short-term reference-picture state inline at their normative fixed bounds.
    • Retain only the AV1 configuration sequence-header payload required for item/sample equivalence instead of every configuration OBU.
    • Allocate HEVC emulation-prevention positions once at the exact validated count without a growing list plus array copy.
    • Use ImageSharp's pooled ChunkedMemoryStream and direct chunk writes for the legacy JPEG item payload.
    • Keep exact owned arrays only where ICC, Exif, XMP, or retained property state must outlive pooled parser buffers.
  • Completed: eliminate per-call scratch allocation across every AV1 forward and inverse 1-D transform, replace shared mutable transform scratch with operation-owned workspace, and implement libaom-shaped bulk transform paths with scalar parity.
    • Inventory the existing hot-path scratch use: 15 forward/inverse transformer types contain 30 stackalloc int[...] sites.
    • Refactor every affected forward transform to reuse caller-owned workspace and its output buffer for staged ping-pong storage.
    • Refactor every affected inverse transform to reuse caller-owned workspace and its output buffer for staged ping-pong storage.
    • Remove the shared mutable Av1ForwardTransformer.TemporaryCoefficientsBuffer and give each concurrent transform operation exclusive workspace ownership.
    • Replace the instance-transformer interface with stateless forward and inverse operator structs following the JPEG color-transform static-generic operator pattern.
    • Select transform type, size, bit depth, and ISA once at the 2-D block boundary rather than dispatching through an interface for every row and column.
    • Port the applicable libaom bulk forward-transform kernels using the existing ImageSharp Vector128 and Vector256 infrastructure while retaining the scalar oracle; add a managed Vector512 path only if complete-block measurements justify extending beyond the pinned upstream SIMD implementations.
    • Port the applicable libaom bulk inverse-transform kernels using the same tables, rounding, saturation, and clipping rules as the scalar oracle.
    • Document scratch ownership, stage-buffer alternation, fixed-point rounding, lane layout, transposition, and scalar fallback decisions at their implementation points.
    • Add complete-block BenchmarkDotNet coverage for scalar, Vector128, Vector256, and runtime-dispatched 8x8 and 32x32 forward/inverse DCT paths, including managed-allocation reporting.
    • Verify every scalar and hardware path across all transform types, sizes, bit depths, edge blocks, and bounded overflow cases, then record representative complete-block timings and allocations.
      • Verification covers all 159 normative size/type combinations at 8, 10, and 12 bits with padded input, prediction, and destination strides. Each of the 477 configurations compares scalar and Vector128 output, and every configuration that contains a complete eight-lane tile also compares Vector256 output. The 989-test focused suite passes with normal AVX2 dispatch, with AVX2 disabled, and with all hardware intrinsics disabled.
  • Queued: restore bounded animated HEIC and AVIF image-sequence scope, including the required image-level and per-frame metadata contracts, without introducing unrelated ISO BMFF surfaces.
    • Reconcile the top-level still-image-only scope with the required animated HEIC and AVIF completion boundary before sequence implementation begins.
    • Define the ImageSharp image-level sequence metadata and per-frame metadata contracts, including observable timing, repetition, color, alpha, orientation, and profile behavior.
      • HeifMetadata now carries repetition and root-frame animation behavior through FormatConnectingMetadata. HeifFrameMetadata carries frame duration through FormatConnectingFrameMetadata; frame-local color, alpha, orientation, and profile state remain owned by the existing ImageFrameMetadata contract. The focused Release suite passes all 40 encoder, image metadata, and frame metadata tests.
    • Identify and document the minimum normative ISO BMFF track, sample-description, sample-location, dependency, and timing syntax required by conforming HEIC and AVIF image sequences.
      • The bounded syntax matrix below is derived from ISO/IEC 23008-12:2022 clauses 7 through 10, the codec-specific HEVC Annex B, AVIF 1.2 section 3, AV1-ISOBMFF 1.2 section 2, and the pinned libavif read.c and write.c sequence paths.
    • Parse and write only that bounded image-sequence syntax; do not add audio, arbitrary media, editing, fragmentation, streaming, or general presentation APIs.
      • Share one bounded 32/64-bit box-header reader between the existing item path and the image-sequence path, including short-read handling and parent-boundary validation.
      • Select one enabled pict master track without materializing unrelated tracks, accepting conforming image-sequence tracks that omit track_in_movie, then parse its mvhd/tkhd/mdhd/hdlr, self-contained dref, av01 or hvc1 sample entry, codec configuration, mandatory ccst, and bounded repetition edit.
      • Resolve stsc, stco/co64, stsz/stz2, stts, and stss into one exact value-type descriptor array capped by DecoderOptions.MaxFrames; validate complete run/count syntax through one allocator-owned sequential scratch buffer.
        • Release verification passes the libavif-shaped two-sample parser fixture, the one-frame retention boundary, and a sample offset/length beyond the file. Pinned upstream libavif files independently verify 8-bit and 12-bit sequence identification, alpha-track and Exif/XMP retention, infinite and finite repetition, and safe omission of an unrelated audio track. The parser performs no per-entry allocation and does not buffer moov, mdat, unrelated track payloads, or complete attacker-sized tables.
      • Parse HEVC composition offsets and hidden samples from ctts/cslg, while rejecting the ctts box prohibited for AV1 tracks.
        • Release verification covers signed and unsigned composition-offset syntax, hidden-sample visibility, composition-time calculation, required cslg and edit-list signaling, complete run counts, and the AV1 prohibition without buffering either table.
      • Parse optional direct dependencies from refs sample groups into a compact retained-sample dependency graph.
        • The parser validates complete sbgp runs and sgpd entry lengths, versions, indices, positive identifier uniqueness, decode order, sync-sample independence, and ccst reference limits. It uses pooled value-type sort indexes plus one exact flat reference array, with no per-sample collections or attacker-sized table materialization.
      • Parse track presentation, color, and HDR properties through the same typed property parser used by still-image items.
        • Selected visual sample entries now retain ICC and CICP color profiles, pasp, clap, irot, imir, clli, mdcv, cclv, amve, reve, and ndwt. Fixed-size properties use the parser's existing scratch buffer; only retained ICC bytes use bounded pooled staging before entering ImageSharp's existing profile model.
      • Parse bounded sequence metadata items.
        • Selected image tracks now retain implicitly associated Exif and unencoded application/rdf+xml XMP items from bounded iinf/infe and iloc declarations, including file-relative and idat-relative extents. Item identifiers and extent descriptors use allocator-owned temporary storage; only the exact retained profile payloads survive parsing. DecoderOptions.SkipMetadata bypasses metadata parsing, SegmentIntegrityHandling follows the shared ancillary/image-data policy, and structural errors that prevent safe track parsing remain fatal. The focused Release sequence-parser suite passes all 15 tests.
      • Connect the parsed sequence index to HEIF detection, Identify, frame decode, alpha matching, and frame metadata without changing still-image source selection.
        • Recognize supported avis, hevc, and hevx sequence brands while continuing to reject layered HEVC and JPEG sequence brands.
        • Identify bounded sequence dimensions, frame count, timing, repetition, codec precision, color, HDR, pixel aspect ratio, Exif, and XMP state.
        • Decode all-sync independently decodable AV1 samples into directly adopted ImageSharp frames without cloning complete pixel buffers.
        • Match auxiliary alpha samples by exact decode duration, visibility, and presentation time, and validate premultiplication track identity.
        • Require unity movie and track matrices so image presentation remains on the optimized clap/irot/imir path without a movie compositor.
        • Apply the shared DecoderOptions contract consistently to still items, nested payload codecs, grids, metadata properties, and sequence samples.
          • Strict rejects recoverable ancillary and image-data errors, IgnoreAncillary suppresses only ancillary failures, and IgnoreImageData additionally permits failed image properties or samples to be omitted. SkipMetadata avoids optional property and item-payload validation, while cancellation and the caller configuration flow into nested JPEG and AV1 decoders. Target scaling and ICC conversion remain presentation-level operations after item or grid composition. The focused Release matrix passes all 19 new still-image policy cases, all 3 new sequence-sample cases, and the complete 37-test sequence-parser suite; the Release test-project build completes with zero errors.
          • Apply the same recovery boundary to still-image item relationships, coded payloads, primary-item thumbnail fallback, and optional alpha composition.
            • Unknown item-reference types are skipped within their validated child boundaries. cdsc failures follow ancillary policy and are not parsed when metadata is skipped. dimg, auxl, prem, and thmb failures follow image-data policy. IgnoreImageData can omit an unreadable alpha plane or recover from a failed primary payload through a valid registered thumbnail, but decoding still fails when no color presentation remains. Real AVIF fixtures cover corrupt alpha payloads, corrupt alpha relationships, malformed Exif relationships, and strict, ancillary-only, image-data, and metadata-skipping behavior.
        • Complete reference-dependent AV1 and HEVC sample reconstruction and independent sequence vectors.
      • Write the same bounded movie, track, sample-description, location, dependency, timing, repetition, alpha, and metadata syntax from ImageSharp frames.
    • Decode frame dependencies, durations, repetition, frame-local auxiliary images, and frame-local metadata into the existing ImageSharp multi-frame model.
    • Encode ImageSharp frames, durations, repetition, frame-local auxiliary images, and frame-local metadata as independently decodable HEIC and AVIF image sequences.
    • Add malformed-sequence boundary coverage and independently verify animated inputs and outputs with pinned HEIC and AVIF implementations.
  • Queued: complete the AV1 and HEVC encoding paths and remove the explicit HeifEncoderCore encoding stubs only after each emitted payload passes independent decode.
    • Complete every AV1 encoder task and exit gate in Phase 6; a smallest-valid fixed coding subset is only an intermediate milestone.
    • Complete every HEVC encoder task and exit gate in Phase 7; a smallest-valid fixed coding subset is only an intermediate milestone.
    • Connect both payload encoders to the bounded HEIF writer with the selected bit depth, chroma layout, range, color signaling, alpha, metadata, and animation state.
    • Replace each NotSupportedException branch only when the corresponding payload is accepted by the pinned independent decoder and the ImageSharp decoder.
    • Verify that every public quality, effort, lossless, bit-depth, chroma, alpha, and metadata option changes or constrains the encoded output exactly as documented.
  • Completed: define and document the HEIF encoder option contract using the established ImageSharp encoder pattern.
    • Confirm that IHeifEncoderOptions has only one concrete implementation and remove the unnecessary interface.
    • Document the default, valid range, special values, invalid-value behavior, and format-dependent restrictions of every retained option using observable API behavior only.
    • Pass HeifEncoder directly to HeifEncoderCore, matching the JPEG, PNG, and WebP encoder-core contracts and avoiding interface dispatch.
    • Verify construction-time range validation and legacy-JPEG codec-boundary restrictions with focused tests before the Phase 1 API-review gate is marked complete.
  • Completed: lock public HEIF image metadata to the supported component bit-depth contract.
    • Replace the unrestricted integer bit depth with HeifBitDepth and preserve the 8-bit default.
    • Resolve format-connecting component precision to the nearest supported 8/10/12-bit output without widening the public value domain.
    • Reject HEVC configuration records outside the exposed 8/10/12-bit profile matrix at the external parse boundary.
    • Verify defaults, cloning, format-connecting conversion, pixel-type projection, and current HEIC/HIF/AVIF Identify results.
      • Release build: 0 errors. Focused metadata and Identify tests: 42 passed, 0 failed.

Gain maps, progressive/layered images, sample transforms, and experimental extension brands require explicit conformance and API decisions. They do not create permission to omit any valid color, compression, or bit-depth path from the PR. The container reader must skip unsupported optional extensions safely and reject an unsupported essential property with a useful error.

Container scope

The container implementation is a deliberately narrow HEIF image-format reader and writer, not a general ISO BMFF framework. Implement only the box syntax and relationships required for supported HEIF image items and bounded HEIC/AVIF image sequences.

In scope are the file type, metadata, item location/data, item information, item properties, item references, primary-item selection, idat/mdat payload storage, grids, auxiliary alpha, presentation transforms, color properties, and Exif/XMP paths required by still images. Image sequences additionally include only the brands, tracks, sample descriptions, sample locations, decode dependencies, timing, repetition, and frame-local metadata required to map HEIC and AVIF sequences to ImageSharp frames.

Still files produce one presented primary image. Supported image sequences produce one ordered ImageSharp frame collection with bounded duration, repetition, frame dependency, alpha, color, orientation, and profile state. Supporting items are decoded only when required to construct or describe those presented frames. The implementation does not expose arbitrary HEIF image collections, bursts, pages, audio, or non-image media.

Out of scope are audio, arbitrary video, edit lists, fragments, streaming profiles, general presentation APIs, and reusable movie, track, or sample-table object models. Track, sample, timing, and inter-frame codec state may exist only inside the bounded HEIC/AVIF sequence decoder or encoder that owns it. Unknown optional boxes remain bounded and skippable; unsupported essential image or sequence syntax must fail with a useful image-format error.

Implementation rule: do not introduce a reusable general-purpose ISO BMFF box hierarchy, track model, or media parser. Add box syntax directly to the bounded HEIF model only when a supported still image, image-sequence frame, relationship, property, metadata path, or conformance fixture requires it. Each addition must name the image behavior it enables and have a focused image-format test.

An ISO BMFF construct may be added only when all of the following are true:

  1. A conforming supported still image or HEIC/AVIF image sequence requires it to produce or describe presented ImageSharp frames.
  2. Its owning image item and its effect on the decoded or encoded image are explicit.
  3. It can be parsed or written as bounded state owned by the HEIF image-item or image-sequence path without adding a general media abstraction.
  4. Independent still-image or animated-image fixtures exercise the behavior it enables.

Encountering a box in libavif, ISO BMFF, or a third-party file is not by itself a reason to port it. Constructs not required by the supported still-image or bounded image-sequence presentation must be skipped when optional or rejected when essential.

Codec-configuration rule: parse av1C and hvcC as properties of coded image items and as the bounded decoder configurations referenced by supported image-sequence sample descriptions. Validate profile, level, bit depth, chroma layout, and parameter-set/OBU declarations against the associated item or sequence samples. Do not expose visual sample entries, sample descriptions, layer selection, sample groups, or timing as general public or reusable ISO BMFF models.

AV1 sequence headers and HEVC VPS/SPS/PPS structures remain in scope because they are codec syntax required by image items and image-sequence frames. Retained reference-frame state is permitted only for the lifetime of a bounded HEIC/AVIF sequence decode or encode operation.

Bounded image-sequence syntax

The sequence reader and writer may retain only the following syntax and the resolved image behavior listed beside it:

Syntax Required image behavior
ftyp sequence and structural brands Recognize avis AV1 sequences and the non-layered hevc/hevx HEVC sequence profiles. avio can additionally signal an all-sync AV1 sequence. Layered hevm/hevs, JPEG jpgs, arbitrary video brands, and brands for unimplemented codecs remain unsupported until their image payload and presentation requirements are implemented.
moov/mvhd, trak/tkhd, and mdia/mdhd/hdlr Select an enabled pict master image-sequence track without requiring the optional track_in_movie flag; retain its displayed dimensions, media time scale, media duration, and movie-time-scale track duration. Require unity movie and track matrices because arbitrary movie-canvas composition is outside image-format scope; use clap/irot/imir for supported image presentation. Inspect unrelated tracks only far enough to identify and skip their non-image handlers.
minf/dinf/dref and stbl Accept only self-contained sample data references and own the bounded sample-table state for one selected image sequence plus an optional linked auxiliary-alpha sequence. No reusable data-reference, media-information, or sample-table API is created.
stsd and one supported visual sample entry Require exactly one av01 entry for AVIF or one non-layered hvc1 entry for HEIC. Retain only its dimensions, codec configuration (av1C or hvcC), image presentation/color/HDR properties, and mandatory version-zero ccst coding constraints. Reject an unsupported essential configuration rather than treating it as generic video.
stsc, stco/co64, and stsz/stz2 Resolve each declared image sample directly to a validated file offset and length. Expand run tables once into a compact frame-owned descriptor array bounded by DecoderOptions.MaxFrames; never buffer the movie or complete mdat.
stts Expand decode-order sample durations against the media time scale into exact per-frame rational delays. A sequence with advisory timing still maps every visible sample to an ImageSharp frame.
optional ctts version 0/1 and cslg Retain only composition offsets and the HEIF hidden-sample condition required to determine presented frame order and visibility. No general composition timeline is exposed.
optional stss Mark the one-based random-access samples; absence means every sample is a sync sample. The first retained frame must have a valid decode path from a sync sample.
optional sgpd/sbgp with grouping type refs Resolve direct-reference sample identifiers for bounded dependency validation and random access. Other sample-group types are ignored unless a later image-format requirement explicitly brings them into scope.
optional edts/elst Interpret only normal-rate image-sequence presentation and the HEIF repetition flag. Reject edits that require arbitrary media splicing, rate changes, or a general edit timeline.
optional tref entries auxl, prem, thmb, and cdsc Link only image alpha, premultiplication, image thumbnails, and image metadata. Auxiliary samples are matched to color frames by their decoded time; audio and every unrelated reference type are ignored.
optional track-level meta and image metadata linkage Apply sequence-wide Exif/XMP profiles from the bounded track metadata model. Per-frame metadata is limited to supported metadata items/groups or codec metadata that can be mapped to ImageFrameMetadata; generic timed-metadata tracks are not exposed.

The implementation must validate uniqueness, versions, entry counts, run totals, one-based indices, integer products, cumulative durations, offsets, and lengths before creating retained arrays. Temporary box payloads use the configured MemoryAllocator; persistent sample descriptors are allocated once at the validated retained-frame count. Parsing must skip unrelated boxes by their bounded length and must not copy an entire moov or mdat payload.

Reference hierarchy

Use the references in this order when behavior differs:

  1. The published ISO BMFF, HEIF, HEVC, AV1, AV1-ISOBMFF, and AVIF requirements are normative.
  2. The official AOM AV1 Codec Library is the primary implementation reference for AV1 decode, encode, high-bit-depth behavior, tests, and optimized scalar/SIMD algorithms. Pin one reviewed commit before porting. Use its scalar C paths as behavioral references and its architecture-specific paths as SIMD references to be expressed with ImageSharp's existing managed intrinsics.
  3. The local D:\GitHub\AOMediaCodec\libavif checkout is the AVIF container, metadata, color-conversion, grid, alpha, and interoperability oracle. At inspection time it identifies itself as 1.4.2-devel.
  4. The official HM reference software at commit 9c1f298659ab0cee9dc13d23d0304221575410b9 is the primary scalar implementation reference for HEVC syntax, reconstruction, and the complete Range Extensions profile matrix. Android libhevc v1.6.0 at commit c83a76b084498d55f252f48b2e3786804cdf24b7 is a secondary production-C comparison for its supported common 8-bit 4:2:0 paths; its decoder rejects other chroma formats and bit depths and therefore is not the feature-coverage authority. The HEVC specification remains normative, and an independent HEIC container implementation is still required as an interoperability oracle. Do not copy from GPL or otherwise incompatible sources.
  5. Existing ImageSharp codecs are the authority for ImageSharp API shape, memory ownership, stream behavior, cancellation, resource limits, pixel conversion, tests, and SIMD dispatch.

The linked ImageSharp discussion establishes the project constraint: the shipped implementation is purely managed and other codec libraries are references, not native runtime dependencies. libaom is the official encoder/decoder implementation reference for AV1, but it does not parse the HEIF container or implement HEVC. libavif dispatches AV1 work to external codec libraries, so it remains an observable AVIF/container oracle rather than the source for every AV1 algorithm.

The AOM source is distributed under the BSD 2-Clause License and the Alliance for Open Media Patent License 1.0. HM is distributed under its BSD license with an explicit patent-rights disclaimer, while the reviewed Android libhevc source is distributed under Apache-2.0. Before porting further code, record the exact upstream file, commit, applicable license/patent notice, and corresponding managed file or method. Audit the existing SVT-AV1-attributed WIP separately rather than relabeling it as libaom-derived. Update THIRD-PARTY-NOTICES.TXT before any referenced implementation code is merged. A pure managed HEVC implementation does not remove HEVC patent or licensing obligations, so those must be resolved before the HEIC work is considered releasable.

Pinned reference and baseline snapshot

The initial post-merge snapshot was established on 2026-08-24:

  • the official libaom reference is tag v3.14.1, commit 03087864cf4bea6abb0d28f95cf7843511413d8f, matching the revision selected by the local libavif ext/aom.cmd dependency script;
  • the local libavif container, color-conversion, and interoperability oracle is commit 092276ce89098ead06db80975173191e5fee1826, described as v1.4.2-66-g092276ce;
  • the official HM reference software is commit 9c1f298659ab0cee9dc13d23d0304221575410b9; its decoder and profile tables cover the HEVC Range Extensions chroma, bit-depth, intra, and still-picture profiles required for the final HEIC feature matrix;
  • the official Android libhevc reference is v1.6.0 at commit c83a76b084498d55f252f48b2e3786804cdf24b7; it remains a secondary common-path comparison because its SPS parser accepts only 8-bit YUV 4:2:0;
  • the official ITU-T H.274 (V4) recommendation from January 2026 is the normative semantics reference for mastering-display and content color-volume fields reused by still-image item properties; its video-SEI persistence and cancellation behavior is outside this container scope;
  • an independent HEIC container interoperability oracle and the complete external HEIC fixture-generation matrix remain to be pinned before HEIC support can pass its final integration gate;
  • dotnet build ImageSharp.sln -c Release --no-restore -m:1 -v minimal succeeds with no errors after the upstream compatibility fixes; and
  • the existing HEIF-focused test run executes 8,198 cases, with 8,184 passing and 14 failing. Thirteen failures are isolated to the WIP AV1 YUV conversion tests, and one is the existing legacy JPEG HIF reference-image mismatch. Golden artifacts have not been changed.

This snapshot pins or classifies the available references and failures; it does not complete Phase 0. The full WIP provenance map, disabled-test inventory, HEIC container interoperability selection, and feature-state matrix remain required.

Provenance map in progress

Managed implementation Normative behavior Reviewed implementation reference Use
Av1YuvConverter.ConvertToRgb, ConvertFromRgb, scalar row conversion, chroma reconstruction, and chroma downsampling ITU-T H.273 (V4) equations 14-16 and 27-84, including limited/full-range scaling, chromaticity-derived equations 39-47, YCgCo equations 51-57, constant-luminance equations 66-75, and the PQ/HLG ICtCp matrices; AV1 section 6.4.2 chroma sample positions Official ITU-T H.273 (V4) (07/2024); libavif src/reformat.c and src/colr.c at 092276ce89098ead06db80975173191e5fee1826; libaom aom/aom_image.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Scalar behavioral oracle for full/limited-range conversion at 8, 10, and 12 bits. Decode and encode cover identity, YCgCo, coefficient-based NCL, both fixed and chromaticity-derived constant/non-constant-luminance systems, SMPTE ST 2085, and ICtCp across monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with AV1 chroma sample positioning. Limited-range YCgCo retains the 219-code scale inherited from its R/G/B inputs instead of applying YCbCr's unrelated 224-code chroma range. Chromaticity derivation uses every defined H.273 primary and matches libavif's BT.709 fallback for unspecified or reserved primaries. The ICtCp inverse is derived from the exact H.273 integer matrices rather than an unrelated display conversion. Later SIMD paths must preserve this scalar behavior.
Av1TransferFunctions ITU-T H.273 (V4) Table 3 transfer characteristics 1-18 Official ITU-T H.273 (V4) (07/2024); libavif src/colr.c at 092276ce89098ead06db80975173191e5fee1826 Apply every AV1-signallable transfer function required by constant-luminance and ICtCp color conversion. Retain the H.273 normalized PQ and HLG definitions; do not import libavif's display-oriented 203-nit scaling or HLG OOTF into codec sample interpretation. Use libavif's midpoint convention only for the non-bijective zero code of the two logarithmic curves.
ObuReader.ReadSequenceHeader, ReadUncompressedFrameHeader, decoder-model parsing, and operating-parameter consumption AV1 sections 5.5.2 through 5.5.4 sequence timing and decoder-model syntax, section 5.9.2 uncompressed frame-header syntax, and section 5.9.31 temporal-point syntax libaom av1/decoder/decodeframe.c functions av1_read_decoder_model_info, av1_read_op_parameters_info, read_temporal_point_info, and read_uncompressed_header, plus common/av1_config.c, at 03087864cf4bea6abb0d28f95cf7843511413d8f Read the normative 32-bit decoding-tick field; consume operating-point buffer delays and the layer-applicable frame removal times needed to keep a non-reduced still-image sequence aligned; and read presentation time only under the normative decoder-model condition. Retain no scheduling or playback behavior from those values and introduce no ISO BMFF timing, track, sample-table, or sequence surface.
Av1FrameBuffer high-bit-depth sample layout and Av1YuvConverter 10/12-bit packed-pixel conversion AV1 section 6.4.1 bit depth and H.273 sample-range scaling libaom aom_scale/yv12config.h, av1/common/idct.c, and av1/common/reconintra.c at 03087864cf4bea6abb0d28f95cf7843511413d8f; libavif src/avif.c and src/reformat.c at 092276ce89098ead06db80975173191e5fee1826 Establish two-byte native sample storage with sample-unit strides for 10/12-bit reconstruction and use ImageSharp's existing Rgb48 pixel-operation paths in both directions so packed-pixel staging does not reduce high-bit-depth samples to eight bits.
Av1PredictionDecoder, Av1HighBitDepthPredictor, Av1ChromaFromLumaContext, Av1PartitionInfo, and the scalar DC, directional, Paeth, smooth, filter-intra, and chroma-from-luma predictors AV1 sections 7.11.2 and 7.11.2.3 intra prediction libaom aom_dsp/intrapred.c, av1/common/reconintra.c, av1/common/av1_common_int.h, av1/common/blockd.h, av1/common/cfl.c, and av1/common/cfl.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Behavioral oracle for luma/chroma mode-neighbor addressing, directional upsampling, Paeth selection, smooth normalization, filter-intra taps, high-bit-depth clipping, chroma-from-luma storage/subsampling, and chroma-from-luma row strides. Existing managed scalar tables and predictors remain the implementation base. The WIP rectangular byte-pipeline smooth digest expectations encode width/height-swapped weights and must be replaced only from an independently generated oracle, not regenerated from this implementation.
Av1TileReader palette mode/color-map parsing, Av1SymbolDecoder palette distributions, Av1BlockModeInfo palette state, and Av1PredictionDecoder palette reconstruction AV1 sections 5.11.46, 5.11.49, and 7.11.2 palette prediction libaom av1/decoder/decodemv.c, av1/decoder/detokenize.c, av1/decoder/decoder.h, av1/common/pred_common.c, av1/common/pred_common.h, and av1/common/entropymode.c at 03087864cf4bea6abb0d28f95cf7843511413d8f Reuse the existing tile range decoder and frame-owned mode map while porting the normative palette cache merge, high-bit-depth color deltas, tile-adaptive mode/size/index distributions, diagonal color-map traversal, edge padding, and direct palette-sample reconstruction. This is AV1 still-image compression syntax and does not add retained video reference state or any ISO BMFF surface.
Av1LoopFilterKernels, Av1LoopFilterContext, and Av1LoopFilterDecoder AV1 section 7.14 deblocking loop filter libaom aom_dsp/loopfilter.c and av1/common/av1_loopfilter.c at 03087864cf4bea6abb0d28f95cf7843511413d8f Port the scalar 4-, 6-, 8-, and 14-tap low/high-bit-depth filters, sharpness thresholds, still-frame intra filter-level derivation, transform-edge selection, and plane traversal before enabling the stage. Later SIMD must preserve the scalar result. This is normative AV1 image reconstruction and adds neither generic ISO BMFF models nor retained video reference state.
Av1CdefDecoder, Av1CdefKernels, and CDEF-unit strength storage AV1 sections 7.15.2 through 7.15.4 constrained directional enhancement filtering libaom av1/common/cdef.c, av1/common/cdef_block.c, av1/common/cdef.h, and av1/common/cdef_block.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Port the scalar direction search, variance adjustment, constrained primary/secondary taps, subsampling direction conversion, skipped-8x8 selection, and frame-edge sentinel behavior. Use a frame-owned source snapshot so filtering never consumes already modified samples. This is normative AV1 still-image reconstruction and introduces no ISO BMFF, track, timing, or sequence-playback surface.
Av1SuperResolutionDecoder, Av1SuperResolutionKernels, frame-size derivation, and decoded-image dimensions AV1 section 7.16 normative super-resolution upscaling libaom av1/common/resize.c, av1/common/resize.h, av1/common/convolve.c, and aom_dsp/aom_filter.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Port the fixed 64-phase, 8-tap horizontal filter, phase/step derivation, replicated frame edges, chroma width rounding, signed rounding, and 8/10/12-bit clipping. Reuse ImageSharp's existing cross-platform Vector128_.MultiplyAddAdjacent helper for the exact eight-coefficient dot product with a scalar fallback. Generic image resizing is not normative AV1 super-resolution. This adds no track, timing, fragment, animation, or generic ISO BMFF model.
Av1TileReader loop-restoration unit syntax, Av1SymbolDecoder restoration distributions/subexponential codes, and Av1FrameInfo unit storage AV1 section 5.11.57 read_lr and read_lr_unit syntax libaom av1/decoder/decodeframe.c, av1/common/restoration.c, av1/common/restoration.h, av1/common/entropymode.c, aom_dsp/binary_codes_reader.c, and aom_dsp/recenter.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Decode tile-local switchable/Wiener/self-guided selections, finite reference-subexponential coefficients, chroma Wiener windows, self-guided parameter sets, super-resolution-adjusted unit corners, and the AV1 nearest-unit-count rule into frame-owned per-plane grids. This is compressed still-image syntax and adds no movie, track, timing, fragment, audio, or sequence surface.
Av1WienerFilter AV1 sections 7.17.4 and 7.17.5 Wiener restoration filtering and coefficient derivation libaom av1/common/restoration.c, av1/common/restoration.h, av1/common/convolve.c, and av1/common/convolve.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Preserve the implicit center-sample contribution, separable horizontal/vertical rounding, bit-depth-dependent 16-bit intermediate range, and final 8/10/12-bit clipping. Reuse Vector128_.MultiplyAddAdjacent for the contiguous horizontal eight-tap product with an exact scalar fallback. Keep this scalar/SIMD oracle disabled until restoration stripe boundaries and self-guided filtering are both complete.
Av1SelfGuidedFilter AV1 sections 7.17.2 and 7.17.3 self-guided and box-filter processes libaom av1/common/restoration.c and av1/common/restoration.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Preserve the sixteen normative radius/variance parameter sets, local mean and variance normalization, alternating-row radius-two optimization, decoded projection-coefficient behavior, signed rounding, and 8/10/12-bit clipping. Use caller-owned scratch storage and a scalar oracle before considering a managed SIMD translation of libaom's architecture-specific nonlinear filter kernels. Keep this image-reconstruction stage disabled until restoration stripe boundaries are complete.
Av1LoopRestorationBoundary, Av1LoopRestorationDecoder, and Av1FrameDecoder restoration-stage ordering AV1 section 7.17 loop restoration, including striped boundary semantics libaom av1/common/restoration.c, av1/common/restoration.h, av1/common/resize.c, and av1/decoder/decodeframe.c at 03087864cf4bea6abb0d28f95cf7843511413d8f Preserve two deblocked rows at internal 64-luma stripe boundaries before CDEF, apply the existing normative SIMD-backed super-resolution kernel to saved rows when scaled, use post-CDEF/super-resolution samples at frame edges, extend the final restoration unit up to 150 percent of nominal size, and filter from immutable plane snapshots into separate output planes. This is bounded still-image reconstruction state, not retained reference-frame, track, timing, or playback state.
Av1FilmGrainDecoder and Av1FilmGrainGaussianSequence AV1 section 7.18 film-grain synthesis libaom av1/decoder/grain_synthesis.c, av1/decoder/grain_synthesis.h, and aom_dsp/grain_params.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Preserve the normative 2,048-sample Gaussian sequence, linear-feedback shift register, luma/chroma autoregressive templates, scaling lookup interpolation, 32x32 block selection, boundary overlap, restricted-range clipping, monochrome and 4:2:0/4:2:2/4:4:4 layouts, and 8/10/12-bit arithmetic. Use allocator-owned scratch and runtime-optimized span copies. Apply grain only to the displayed still-image samples after all in-loop filters; reference-frame parameter inheritance remains sequence-playback state and is outside this codec scope.
Av1FrameInfo, Av1TileReader, and Av1BlockDecoder transform/coefficient storage AV1 section 5.11.39 coefficient syntax and section 7.11.2 reconstruction libaom av1/decoder/decodetxb.c and av1/decoder/decoder.h at 03087864cf4bea6abb0d28f95cf7843511413d8f Preserve separate luma and chroma transform coefficients at monotonically advancing per-plane offsets within each superblock so reconstruction consumes the same transform-block order produced by tile parsing.
Av1InverseQuantizer and Av1InverseQuantizationLookup AV1 section 7.12.3 inverse quantization libaom aom_dsp/aom_dsp_common.h, av1/common/quant_common.c, and av1/decoder/decodetxb.c at 03087864cf4bea6abb0d28f95cf7843511413d8f Select the per-segment matrix level, alias 64-pixel transform dimensions to their adjusted matrices, retain a flat level-15 matrix, and apply the five-bit inverse-matrix weight scale. The large managed lookup remains a single process-wide table.
Av1ForwardTransformer, Av1Inverse2dTransformer, Av1Transform2dFlipConfiguration, the forward/inverse 1-D operator structs, and Av1Transform1dMath AV1 forward transform definitions and section 7.11.2 inverse transform and reconstruction libaom av1/encoder/av1_fwd_txfm1d.c, av1/encoder/av1_fwd_txfm2d.c, av1/common/av1_inv_txfm1d_cfg.h, av1/common/av1_inv_txfm1d.c, av1/common/av1_inv_txfm2d.c, the x86 AVX2/SSE4 implementations, and the corresponding Neon implementations at 03087864cf4bea6abb0d28f95cf7843511413d8f Preserve the normative staged DCT, ADST, and identity arithmetic, direction-specific stage ranges and shifts, transposition, clipping, and high-bit-depth sample addition. Stateless static-generic operators follow ImageSharp's JPEG color-transform pattern. Vector128 and Vector256 kernels retain wrapping 32-bit lane arithmetic before the bounded rounding shift, matching the optimized upstream implementations, with the scalar operators as the behavioral oracle. The pinned upstream transform paths do not provide an AVX-512 implementation.
HeifDecoderCore box extension handling and HeifDecoderCore/HeifEncoderCore item-property associations ISO/IEC 14496-12 box extensibility and section 8.11.14 item properties and ipma syntax libavif src/read.c and src/write.c at 092276ce89098ead06db80975173191e5fee1826 Skip unrecognized top-level and metadata child boxes, preserve the position of every property in ipco, reject an unrecognized property only when its item association marks it essential, associate properties by item ID, and read or write the essential bit plus one-based 7-bit or 15-bit property index according to the full-box flags. Independent HEIC, HIF, and AVIF fixtures provide the reader oracle; container-level identification of encoded output guards the writer independently of pixel roundtripping.
HeifCleanAperture, HeifItem presentation state, and HeifDecoderCore transformative-property parsing and application ISO/IEC 14496-12 section 12.1.4 clean aperture; HEIF image rotation and mirror properties; MIAF section 7.3.6.7 presentation order and section 7.3.9 essential transformative properties libavif src/avif.c clean-aperture conversion, src/read.c property parsers and alpha-property validation, and apps/shared/avifutil.c transform application at 092276ce89098ead06db80975173191e5fee1826 Resolve fractional clean-aperture dimensions and center offsets to exact bounded integer pixels, validate the registered rotation/mirror reserved bits, require essential associations, crop after auxiliary-alpha composition, map counter-clockwise HEIF quarter turns to ImageSharp's optimized clockwise rotate modes, then mirror around the signaled axis. Reuse ImageSharp's existing crop, rotation, and flip processors for every pixel type. Retain only the three image-item property values; do not add a generic transform-box or ISO BMFF model.
HeifConstants.IsSupportedFileType, HeifImageFormatDetector, and HeifDecoderCore.CheckFileTypeBox ISO/IEC 14496-12 FileTypeBox syntax and the MP4 Registration Authority HEIF/AVIF still-image and sequence brand registrations libavif src/read.c functions avifParseFileTypeBox, avifFileTypeHasBrand, and avifFileTypeIsCompatible at 092276ce89098ead06db80975173191e5fee1826 Apply one rule to the major and compatible brands, accept implemented still-image brands and the bounded avis, hevc, and hevx image-sequence brands, and distinguish item and sequence presentation before parsing their payloads. Layered HEVC and JPEG sequence brands remain unsupported. The decoder validates the complete ftyp payload; the fixed-size format detector inspects the available prefix.
HeifSequenceParser, HeifTrackMetadataParser, and the bounded HEIF image-sequence writer ISO/IEC 23008-12:2022 clauses 7 through 10 and HEVC Annex B; ISO/IEC 14496-12 sample tables, normal edit lists, and track-level metadata; AVIF 1.2 section 3; AV1-ISOBMFF 1.2 section 2 libavif src/read.c track/sample-table and track-metadata paths and src/write.c sequence path at 092276ce89098ead06db80975173191e5fee1826; independently produced HEIC sequence fixtures remain required Resolve only pict master tracks and linked image alpha/metadata state into validated sample descriptors, exact frame durations, sync/dependency state, repetition, presentation transforms, color/HDR properties, and frame metadata. Track metadata is limited to implicitly associated Exif and unencoded XMP items declared by bounded iinf/infe, iloc, and optional idat syntax. DecoderOptions controls metadata retention and recoverable ancillary/image-data errors through the same rules as other decoders. Consume the required container boxes listed in the bounded syntax matrix without exposing a movie, track, edit, or sample-table API.
HeifDecoderCore.ReadBoxHeader and HeifDecoderCore.ParseBoxHeader ISO/IEC 14496-12 section 4.2.2 basic box syntax libavif src/stream.c functions avifROStreamReadBoxHeaderPartial and avifROStreamReadBoxHeader at 092276ce89098ead06db80975173191e5fee1826 Resolve 32-bit, 64-bit, UUID, and top-level size-zero boxes into content lengths only after validating the complete variable-sized header and the remaining parent boundary. Nested size-zero boxes are invalid; large skips retain 64-bit offsets.
HeifDecoderCore.ParseMetadata ISO/IEC 14496-12 MetaBox and HEIF item declarations, locations, properties, and associations libavif src/read.c functions avifParseMetaBox, avifMetaFindOrCreateItem, avifParseItemLocationBox, and avifParseItemPropertiesBox at 092276ce89098ead06db80975173191e5fee1826 Index unique recognized metadata children by type and payload location, then parse them in dependency order so physical placement does not control item lookup or property association. Duplicate unique children and truncated full-box headers are invalid.
HeifDecoderCore.ApplyAssociatedMetadata HEIF Annex A Exif item data, MIME metadata items, and cdsc item references libavif src/read.c function avifDecoderFindMetadata, src/exif.c function avifGetExifTiffHeaderOffset, and the Exif/XMP item writing paths in src/write.c at 092276ce89098ead06db80975173191e5fee1826 Resolve only metadata items whose cdsc reference identifies the decoded primary image, validate the Exif TIFF-header offset, and attach Exif or application/rdf+xml XMP through ImageSharp's existing profile types. This is a bounded still-image metadata path; it does not introduce a generic ISO BMFF metadata, media, or track model.
HeifPropertyParser, HeifDecoderCore color-property association, HeifItem and HeifSequenceTrack color profiles, and Av1Decoder effective color validation ISO/IEC 14496-12 section 12.1.5 color information; HEIF section 6.5.5.1 color-information properties; AV1-ISOBMFF section 2.3.4 configuration semantics libavif src/read.c functions avifParseColourInformationBox, avifReadColorNclxProperty, and avifReadColorProperties, plus src/write.c function avifEncoderWriteColorProperties, at 092276ce89098ead06db80975173191e5fee1826 Associate at most one ICC and one nclx property with each presented color image item or selected image-sequence sample entry, validate ICC payloads and CICP reserved bits, expose them through ImageSharp's existing profile types, and inherit a grid's CICP description only for tiles that do not declare one. For AV1 images, require every specified sequence-header CICP field and the range flag to match nclx; use the container property only to supply sequence-header fields coded as unspecified before color conversion, then expose the effective CICP values even when no property is present. Retain only the two image color profiles; do not add a reusable color-box or media model.
HeifPropertyParser, HeifPixelAspectRatio, item/track presentation state, and HeifDecoderCore.ApplyItemPixelAspectRatioMetadata ISO/IEC 14496-12 section 12.1.4.3 pixel aspect ratio libavif src/read.c function avifParsePixelAspectRatioBox, src/write.c functions avifEncoderWritePaspProperty and avifEncoderWriteTransformativeProperties, and the presented-image property selection in src/read.c at 092276ce89098ead06db80975173191e5fee1826 Preserve the registered pasp, clap, irot, and imir presentation properties on an associated image item or selected image-sequence sample entry, reject invalid lengths, reserved bits, duplicates, and non-integral clean apertures, and map the displayed pixel ratio and transforms through ImageSharp's existing metadata and processing model. No generic transform or display model is introduced.
Av1CodecConfiguration, HeifItem.Av1CodecConfiguration, Av1HeifItemDecoder, and AV1 grid configuration checks AV1-ISOBMFF sections 2.3.3 and 2.3.4 codec-configuration record syntax and semantics; AVIF sections 2.1, 2.2.1, and 2.2.3 AV1 image-item, item-configuration, and HDR metadata constraints; AV1 sections 5.8.3, 5.8.4, 6.7.3, and 6.7.4 HDR metadata syntax and semantics; ISOBMFF mastering-display and content-light image properties; ITU-T H.274 section 8.9 mastering-display field semantics; MIAF section 7.3.11.4.1 grid input constraints libavif src/read.c functions avifParseCodecConfiguration, avifDecoderItemValidateProperties, avifReadCodecConfigProperty, avifParseContentLightLevelInformation, and avifSkipMasteringDisplayColourVolume at 092276ce89098ead06db80975173191e5fee1826; libaom av1/decoder/obu.c functions read_metadata, read_metadata_hdr_cll, and read_metadata_hdr_mdcv at 03087864cf4bea6abb0d28f95cf7843511413d8f Associate exactly one av1C property with each decoded av01 image item, validate the fixed record and its bit depth/chroma fields against the item's AV1 sequence header and optional pixi channel depths, require matching configurations across grid tiles, and report the encoded image precision and monochrome shape through HeifMetadata. Validate low-overhead OBU framing, require exactly one sequence header in the image item, allow at most one first-position sequence header in configOBUs, and compare a repeated header's extension and payload exactly while ignoring only its legal size-field representation. Decode clli and mdcv as still-image item properties, validate matching HDR CLL and HDR MDCV metadata OBUs from the combined configuration/item sequence, and account for the different primary order and fixed-point precision of the ISOBMFF and AV1 MDCV representations. Expose the effective HDR values without adding sample groups, tracks, or media metadata. Related still-image HDR properties remain required. Consume but do not retain presentation-delay syntax, and introduce no sample entry, sample description, track, timing, or generic decoder-configuration model.
HevcCodecConfiguration, HevcNalUnitArray, and HeifItem.HevcCodecConfiguration ISO/IEC 14496-15 HEVCDecoderConfigurationRecord, HEVC NAL-unit header syntax, and HEVC image-item configuration requirements HM source/Lib/TLibDecoder/NALread.cpp and parameter-header paths at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc v1.6.0 decoder NAL and parameter-header paths at c83a76b084498d55f252f48b2e3786804cdf24b7; independently produced HEIC fixtures for hvcC record layout Associate exactly one hvcC property with each hvc1 image item; validate the fixed profile, level, chroma, bit-depth, reserved, length-size, array, and NAL-header fields within the property boundary; and retain only the image description plus bounded configuration NAL units required to decode that item. Consume but do not retain average-frame-rate or temporal-layer presentation fields. Do not add visual sample entries, sample descriptions, tracks, timing, access-unit timelines, or a generic decoder-configuration model. BSD and Apache-2.0 notice preservation and separate HEVC patent/release clearance remain final integration gates.
HevcNalUnit, HevcNalUnitHeader, HevcRbspDecoder, and HevcBitReader HEVC sections 7.3.1.1 and 7.3.1.2 NAL-unit and header syntax, section 7.3.2.11 RBSP trailing bits, and section 9.1 Exp-Golomb parsing HM source/Lib/TLibDecoder/NALread.cpp and SyntaxElementParser.cpp at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc NAL extraction and bitstream paths at c83a76b084498d55f252f48b2e3786804cdf24b7 Validate the fixed NAL header, remove only legal emulation-prevention bytes, and provide bounded fixed-width, flag, unsigned/signed Exp-Golomb, and RBSP-trailing-bit reads. Configuration arrays immediately use the shared parser. The primitives consume only NAL units delivered by a supported HEIF image item or its hvcC property and introduce no Annex B API, access-unit model, timeline, track, sample table, or generic ISO BMFF surface.
HevcProfileTierLevel, HevcVideoParameterSet, and HevcCodecConfiguration.VideoParameterSets HEVC sections 7.3.2.1 video parameter sets, 7.3.3 profile-tier-level syntax, and E.2.2 hypothetical-reference-decoder syntax; ISO/IEC 14496-15 HEVCDecoderConfigurationRecord matching semantics HM source/Lib/TLibDecoder/TDecCAVLC.cpp functions parseVPS, parsePTL, parseProfileTier, and parseHrdParameters at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoder/ihevcd_parse_headers.c profile-tier-level and VPS paths at c83a76b084498d55f252f48b2e3786804cdf24b7 Retain the base-layer VPS identifier, temporal-sublayer count, nesting flag, and exact general profile/tier/constraint/level fields. Validate profile, tier, compatibility, level, temporal count, and nesting against hvcC, while tolerating the independently observed legacy-muxer practice of zeroing some or all profile-specific constraint bits in the configuration record; SPS validation remains authoritative for chroma and bit depth. Consume but do not retain sublayer ordering, timing, and HRD syntax. Reject multilayer and layer-set state because the supported contract is one independently coded image item; add no layer-selection, access-unit, DPB, scheduling, track, or sample model.
HevcSequenceParameterSet, HevcVideoUsabilityInformation, HevcScalingList, HevcShortTermReferencePictureSet, and HevcParameterSetSyntax HEVC sections 7.3.2.2 sequence parameter sets, 7.3.4 scaling-list data, 7.3.7 short-term reference-picture sets, E.2.1 VUI syntax, and Range Extensions SPS syntax HM source/Lib/TLibDecoder/TDecCAVLC.cpp functions parseSPS, parseScalingList, xDecodeScalingList, parseShortTermRefPicSet, parseVUI, and parseHrdParameters, plus source/Lib/TLibCommon/TComRom.cpp default matrices and diagonal scans, at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoder/ihevcd_parse_headers.c SPS, VUI, scaling-list, and reference-set paths at c83a76b084498d55f252f48b2e3786804cdf24b7 Retain coded/display dimensions, conformance crop, monochrome/4:2:0/4:2:2/4:4:4 and separate-plane layout, 8-through-16-bit precision, coding/transform/PCM geometry, effective scaling matrices, compression-tool flags, bounded reference-set declarations, Range Extensions tools, and still-image VUI aspect/color/range/chroma-location/display-window fields. Consume timing, HRD, ordering, and bitstream-restriction syntax without scheduling state. Reject interlaced fields and multilayer extensions because they do not represent the one progressive independently coded image-item contract; add no DPB pictures, playback state, tracks, or sample tables.
HevcPictureParameterSet and HevcCodecConfiguration.PictureParameterSets HEVC sections 7.3.2.3 picture parameter sets, 7.3.4 scaling-list data, and Range Extensions PPS syntax HM source/Lib/TLibDecoder/TDecCAVLC.cpp function parsePPS, source/Lib/TLibDecoder/TDecConformance.cpp PPS range checks, and source/Lib/TLibCommon/TComPicSym.cpp tile derivation at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoder/ihevcd_parse_headers.c PPS and tile paths at c83a76b084498d55f252f48b2e3786804cdf24b7 Link each PPS to a configuration-local SPS and retain the CABAC, quantization, transform-skip, scaling-list, tile/wavefront, deblocking, SAO, slice-header, and Range Extensions controls required to reconstruct the independently coded picture. Derive the complete CTB tile grid and validate every SPS-dependent depth, size, precision, and offset bound during parsing. Reject multilayer extensions and add no access-unit, reference-picture lifetime, playback, track, timing, or sample-table model.
HevcImageItemBitstream, HevcSliceSegmentHeader, and HevcSliceType HEVC sections 7.3.1.2 NAL-unit syntax, 7.3.6 slice-segment headers, and 7.4.7 slice-segment-header semantics; HEVC image-item length delimiting HM source/Lib/TLibDecoder/TDecCAVLC.cpp function parseSliceHeader, source/Lib/TLibDecoder/NALread.cpp, and slice byte-alignment and entry-point handling at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoder/ihevcd_nal.c, ihevcd_parse_slice_header.c, and ihevcd_parse_headers.c at c83a76b084498d55f252f48b2e3786804cdf24b7 Split only the selected hvc1 image item's one-through-four-byte length-delimited NAL units, reject in-band parameter-set replacement and end-of-sequence state, and require exactly one base-layer IDR picture. Retain the intra slice, separate-plane, SAO, quantization, deblocking, cross-slice filtering, tile/wavefront entry-point, extension, byte-alignment, and entropy-payload fields required by reconstruction. Keep emulation-prevention positions only to translate slice substream byte boundaries; add no Annex B scanner, access-unit collection, non-IDR reference lifetime, playback, timing, track, or sample model.
HevcCabacContext and HevcCabacDecoder HEVC sections 9.3.3.2 through 9.3.3.4 context initialization, arithmetic decoding, bypass decoding, and termination HM source/Lib/TLibCommon/ContextModel.cpp, source/Lib/TLibCommon/TComCABACTables.cpp, and source/Lib/TLibDecoder/TDecBinCoderCABAC.cpp at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc common/ihevc_cabac_tables.c, common/ihevc_cabac_tables.h, decoder/ihevcd_cabac.c, and decoder/ihevcd_cabac.h at c83a76b084498d55f252f48b2e3786804cdf24b7 Implement the bit-exact scalar probability-state transitions, range normalization, bounded byte refills, bypass modes, termination, and alignment validation for one slice tile or wavefront substream. CABAC decisions are serial and adaptive, so this primitive is the scalar oracle rather than a SIMD target; later prediction, transform, filtering, and color-conversion stages must reuse ImageSharp's compatible SIMD kernels. The decoder owns no container boxes, access units, reference pictures, tracks, timing, playback, or general-purpose bitstream surface.
HevcCabacContexts HEVC section 9.3.2.2 initialization of context variables for an intra-coded slice and Range Extensions context additions HM source/Lib/TLibCommon/ContextTables.h and source/Lib/TLibDecoder/TDecSbac.cpp function resetEntropy at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc common/ihevc_cabac_tables.c and decoder/ihevcd_cabac.c at c83a76b084498d55f252f48b2e3786804cdf24b7 Initialize only the context groups reachable from the independently coded intra-picture syntax, including transform skip, chroma quantization adjustment, and cross-component prediction. Keep one contiguous mutable context set per entropy substream so tile and wavefront synchronization can copy the normative adaptive state without retaining reference-picture or playback state. Omit skip, merge, motion, reference-index, and other inter-picture-only groups.
HevcCabacSyntaxReader HEVC sections 9.3.3 through 9.3.4 CABAC binarization and context selection for intra coding units, transform trees, residual coefficients, sample-adaptive offset, and Range Extensions tools HM source/Lib/TLibDecoder/TDecSbac.cpp, source/Lib/TLibCommon/TComChromaFormat.h, and source/Lib/TLibCommon/TComTrQuant.cpp at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoder/ihevcd_cabac.c, decoder/ihevcd_parse_slice.c, and decoder/ihevcd_parse_residual.c at c83a76b084498d55f252f48b2e3786804cdf24b7 Decode only syntax values reachable while reconstructing the selected independently coded still picture: intra partition and mode selectors, transform splitting and coded-block flags, quantization adjustments, transform skip, coefficient significance and levels, cross-component prediction, SAO, bypass alignment, and termination. Keep neighbor-dependent context selection in the owning coding-tree/reconstruction layer and expose no inter prediction, reference-picture, access-unit, track, timing, or playback syntax.
HevcPlane and HevcPictureBuffer HEVC sections 6.2 and 6.3 source and decoded picture sample-array dimensions for monochrome, 4:2:0, 4:2:2, 4:4:4, and separate-color-plane coding HM source/Lib/TLibCommon/TComPicYuv.cpp at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoded-picture buffer layout paths at c83a76b084498d55f252f48b2e3786804cdf24b7; ImageSharp MemoryAllocator and Buffer2D<T> Allocate one allocator-owned native ushort plane set for the selected still picture so the same reconstruction path preserves every supported 8-through-16-bit sample without byte-backed reinterpretation. Derive chroma dimensions by ceiling division for each sampling layout and treat separate color planes as full-resolution independently coded arrays. The buffer contains one picture only and introduces no decoded-picture buffer, reference lifetime, frame queue, or playback state.
HevcCodingTreeState HEVC sections 6.4 coding-tree block and coding-block availability plus section 9.3 split-flag context derivation HM source/Lib/TLibCommon/TComDataCU.cpp function getCtxSplitFlag and source/Lib/TLibDecoder/TDecCu.cpp coding-tree traversal at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc decoder/ihevcd_parse_slice.c coding-tree paths at c83a76b084498d55f252f48b2e3786804cdf24b7; ImageSharp MemoryAllocator and Buffer2D<T> Store only leaf depth, effective luma QP, transquant-bypass, and PCM state at minimum-coding-block resolution for the selected picture. Derive split contexts from caller-approved left and above availability so slice and tile boundaries remain owned by traversal rather than hidden in a general block graph. Clip edge writes to the coded still-picture state and add no prediction units, motion fields, references, decoded-picture queue, or sequence lifetime.
HevcIntraPredictionState HEVC sections 8.4.2 and 9.3 intra luma/chroma prediction-mode derivation and binarization HM source/Lib/TLibCommon/TComDataCU.cpp functions getIntraDirPredictor and getAllowedChromaDir, plus source/Lib/TLibDecoder/TDecSbac.cpp functions parseIntraDirLumaAng and parseIntraDirChroma, at 9c1f298659ab0cee9dc13d23d0304221575410b9; Android libhevc intra-mode parsing and neighbor derivation paths at c83a76b084498d55f252f48b2e3786804cdf24b7; ImageSharp MemoryAllocator and Buffer2D<T> Preserve the normative two-pass luma flag/suffix order, spatial most-probable-mode derivation, omitted-mode reinsertion, four-way minimum-CU partition order, explicit chroma candidate substitution, and derived-chroma mode at 4x4 luma resolution. Accept slice/tile availability from traversal, fill only the selected still-picture map, and add no inter prediction unit, motion, reference, or sequence state.
HeifContentColorVolume, HeifItem.ContentColorVolume, and HeifDecoderCore content color-volume parsing and presentation HEIF content color-volume item property; AVIF 1.2 content color-volume requirements; ITU-T H.274 (V4) content colour volume syntax and semantics libavif src/read.c function avifSkipContentColourVolume at 092276ce89098ead06db80975173191e5fee1826; official ITU-T H.274 (V4), January 2026 Decode only the bounded per-image cclv property: require zero cancellation, persistence, and reserved bits; preserve optional signed G/B/R primary coordinates and normalized minimum, maximum, and average luminance values; and validate their registered ranges and ordering. Expose the effective grid-or-tile still-image value through HeifMetadata. Do not add SEI persistence, retained video state, tracks, samples, timing, or a generic ISO BMFF color-volume box model.
HeifAmbientViewingEnvironment, HeifReferenceViewingEnvironment, HeifNominalDiffuseWhite, and their per-item presentation metadata ISOBMFF ambient viewing environment; ITU-T H.274 (V4) section 8.13; HEIF Amendment 1 sections 6.5.44 and 6.5.45; AVIF 1.2 image-item box requirements libavif src/read.c functions avifSkipAmbientViewingEnvironment, avifSkipReferenceViewingEnvironment, and avifSkipNominalDiffuseWhite at 092276ce89098ead06db80975173191e5fee1826; official ITU-T H.274 (V4), January 2026 Decode the fixed-size amve, version-zero reve, and version-zero ndwt properties only when associated with the presented still-image item. Preserve physical illuminance and luminance units, distinct surround/periphery chromaticities, and the coded zero that requests the standard nominal diffuse-white default. Validate registered coordinate ranges and duplicate associations. Retain no video-SEI persistence, visual sample entry, display pipeline, track, timing, or generic viewing-environment box model.
GridHeifItemDecoder and HeifDecoderCore grid/thumbnail selection ISO/IEC 23008-12 section 6.6.2.3 image-grid syntax and MIAF grid-cell constraints libavif src/read.c functions avifParseImageGridBox, avifDecoderDataAllocateImagePlanes, and avifDecoderDataCopyTileToImage at 092276ce89098ead06db80975173191e5fee1826 Parse version-zero 16-bit and 32-bit grid descriptors, preserve row-major dimg order, require the declared tile count and one coding format, validate canvas coverage and edge overlap, and crop only the rightmost column and bottom row while copying through ImageSharp pixel buffers. A primary grid whose tile codec is unavailable may use only a decodable thumbnail that explicitly references that grid.
HeifDecoderCore alpha auxiliary selection/composition and GridHeifItemDecoder auxiliary tile ordering ISO/IEC 23008-12 alpha auxiliary image semantics, auxC, auxl, prem, and per-grid-tile alpha relationships libavif src/read.c functions avifParseAuxiliaryTypeProperty, avifDecoderItemIsAlphaAux, avifMetaFindAlphaItem, and avifDecoderCheckAlphaProperties, plus src/scale.c box-filter scaling at 092276ce89098ead06db80975173191e5fee1826 Recognize both registered alpha URNs, decode a direct alpha image/grid or the complete row-major set of per-color-tile alpha auxiliaries, normalize through L16, box-resample differing auxiliary dimensions, compose through Rgba64 and PixelOperations<TPixel>, and unassociate prem color samples with transparent-black handling. This remains an image-item relationship only; no track or generic media-reference model is introduced.

This table is intentionally incomplete. Add a row before each additional AV1 or HEVC algorithm is ported or materially reshaped.

Current implementation assessment

This assessment is based on the current source after the upstream ImageSharp merge and the baseline recorded above. Unless a result is stated explicitly, each item is a source-inspection finding rather than a verified interoperability claim.

Public integration

  • HeifFormat combines the HEIF, HEIC, HIF, and AVIF identities and extensions, but the implementation does not yet decode all payloads that contract implies.
  • HeifDecoder now defaults to Rgba32, preserving decoded auxiliary alpha for non-generic loads.
  • HeifMetadata now reports alpha presence and the corresponding 24/32-bit RGB pixel shape, but complete decoded HEVC/AV1 bit depth, monochrome/chroma layout, color signaling, and profiles remain absent.
  • HeifEncoder defines quality, alpha quality, effort, lossless, chroma-subsampling, and bit-depth contracts directly, without a single-implementation options interface. The legacy JPEG path applies its supported quality, bit-depth, and chroma options and rejects unsupported combinations; AV1 and HEVC must implement the same public contracts before the Phase 1 API-review gate can pass.
  • HEIF/HEIC/AVIF is absent from the format source-generation list in _Formats.ttinclude, so the standard ImageSharp save extensions are not generated.
  • Configuration registration exists, but it currently registers capabilities broader than the implementation provides.

HEIF/ISO BMFF container

  • The bounded reader now handles basic, extended-size, UUID, and permitted top-level to-end boxes, skips unknown optional top-level and metadata children, and rejects child boxes that escape their parent.
  • File type handling now evaluates supported still-image major and compatible brands while rejecting registered sequence major brands without adding track or timing support.
  • Item IDs are resolved as keys rather than list indices; metadata children are indexed and parsed in dependency order rather than physical order.
  • Item property associations now preserve physical ipco indices, apply one-based 7-bit or 15-bit indices and essential flags, associate by item ID, and reject arbitrary unknown essential properties.
  • Item locations now support bounded file-relative and idat-relative storage, multiple ordered extents, versioned item IDs, 0/4/8-byte registered field sizes, and 64-bit offsets. Referenced-item construction method two and external data references remain explicitly unsupported.
  • Grid derived-image decoding now parses both registered descriptor widths, resolves the ordered dimg cells, validates tile count, coding format, dimensions, canvas coverage, and edge overlap, then composes the output through ImageSharp row buffers. Unsupported grid tile codecs can select only a decodable thumbnail of the same primary grid. Independent AV1 and JPEG grid fixtures are still required, and HEVC grids remain blocked on the HEVC decoder.
  • Alpha auxiliary decoding now recognizes auxC, auxl, and prem, supports a direct auxiliary image/grid and the per-color-grid-tile form, preserves normalized alpha through L16/Rgba64, box-resamples differing plane sizes, and reports alpha presence. HEVC alpha remains blocked on the HEVC image-item decoder, while independent AVIF alpha fixtures are still required against the incomplete AV1 reconstruction pipeline.
  • Clean aperture, image rotation, and image mirror properties now validate their registered payloads, exact integer crop geometry, and essential associations; affect Identify dimensions; and reuse ImageSharp's optimized crop/rotate/flip processors after auxiliary alpha composition in the MIAF-defined order. Independent transform vectors must still verify every crop/rotation/mirror/alpha combination.
  • Decode now resolves cdsc-associated Exif and application/rdf+xml XMP items for the primary still image, validates the declared Exif TIFF-header offset, and attaches the payloads through ImageSharp's existing profile types before presentation transforms. Independent AVIF, HEIC, and HIF metadata fixtures and Identify-time profile reporting remain required.
  • ICC and nclx CICP color properties are now associated with the presented color item instead of global parser state, validated, and exposed on Decode and Identify through the existing ImageSharp profiles. AV1 decode requires specified sequence-header fields and the range flag to match nclx, uses the property only for fields the bitstream leaves unspecified, and exposes the effective sequence-header CICP description when no property is present. Independent libavif fixtures verify exact ICC byte preservation, observable non-sRGB conversion, canonical-sRGB compaction, non-sRGB compact preservation, and metadata skipping for primary, grid, alpha-composited, and every presented sequence frame; the alpha vector also verifies that ICC conversion does not alter composed alpha. Identify-time bitstream fallback, independent CICP vectors, and HEVC integration remain required.
  • Pixel aspect ratio now preserves the complete unsigned spacing pair, affects Decode and Identify through ImageSharp's existing unitless resolution metadata, and follows quarter-turn presentation rotation. Independent grid, rotation, and maximum-spacing fixtures remain required.
  • AV1 codec configuration is now retained per av01 image item rather than in decoder-global state. Decode requires the property, validates its fixed record against the item's sequence header and any associated pixi channel depths, and requires matching configurations across grid tiles. Identify now reports the configuration's 8/10/12-bit precision and monochrome shape. The optional configOBUs sequence is bounded and validated, including its mandatory size fields, first-position/at-most-one sequence-header rule, the image item's exactly-one sequence-header rule, and exact comparison of a repeated configuration header with the item header. Content light-level and mastering-display color-volume information are decoded from the bounded clli and mdcv image properties and matching AV1 HDR metadata OBUs, with the representations' distinct fixed-point precision, grid/property precedence, and SkipMetadata behavior preserved. The still-image cclv, amve, reve, and ndwt properties preserve content-volume, ambient-viewing, mastering-environment, and diffuse-white descriptions without importing their video-SEI or sample-entry state models. Independent fixtures and encoder preservation remain required. HEVC hvcC is now associated with each hvc1 image item, structurally validated through its VPS/SPS/PPS arrays, and used to report encoded precision and monochrome shape; item-payload validation is implemented for the bounded decoder path, while pixel reconstruction remains incomplete.
  • Several image-item properties and relationships remain missing or parsed without fully affecting output.
  • Identify and decode now use the same bounded metadata parser and both validate the complete leading file type box. The parsed state is still mutable and Identify does not yet report the complete bit depth, color, profile, or transform model.

HEVC decoder and encoder

  • Heif4CharCode recognizes hvc1 image items, HevcCodecConfiguration validates and associates the bounded hvcC property, and Identify reports its HEVC precision and monochrome shape, but HeifCompressionFactory has no HEVC item decoder.
  • The HEVC path now validates NAL headers and RBSP escaping, parses and links hvcC VPS/SPS/PPS structures through Range Extensions, and parses the selected image item's length-delimited base-layer IDR slice headers and entropy payload boundaries. Its scalar CABAC primitive implements the exact adaptive state transitions, arithmetic range normalization, bypass modes, bounded refills, termination, and stop-bit validation from the pinned HM reference. The one-allocation context owner initializes every intra-picture and Range Extensions context that still-image reconstruction can select while omitting inter-picture-only groups. The codec-local syntax reader decodes the intra coding-unit, transform, coefficient, SAO, and Range Extensions binarizations. Allocator-owned native ushort picture planes cover monochrome, 4:2:0, 4:2:2, 4:4:4, separate-color-plane, and 8-through-16-bit reconstruction without a decoded-picture queue. Coding-tree traversal and neighbor-derived context selection are not yet wired. The path is deliberately not registered as an item decoder until coding-tree reconstruction, still-image intra prediction, inverse quantization/transform, deblocking, sample-adaptive offset, color conversion, and pixel output are implemented.
  • There is no HEVC encoder. The current HEIC-branded encoder writes a legacy JPEG payload and therefore cannot provide HEIC output.
  • Existing HEVC tests prove container identification only; they do not decode or compare HEIC pixels.

AV1 decoder

  • The single-still Av1Decoder path now parses tile state before allocating and reconstructing one independently decodable frame, and it disposes the reconstruction planes after pixel conversion. It deliberately does not retain animation/video reference frames or implement show_existing_frame playback state.
  • Transform coefficient entropy derivation and updates now address the above contexts relative to the tile column and the left contexts relative to the current superblock row, preserve luma coordinates independently of chroma subsampling, and test every packed context entry for the libaom any-nonzero rule. Extended vertical partition updates advance the mode-information column rather than the row. The existing multi-superblock 4:4:4 AVIF fixture now completes tile parsing; independent coefficient-context vectors across tile boundaries, chroma layouts, bit depths, and edge-clipped transforms remain required.
  • The reconstruction pipeline now records plane-relative transform geometry, preserves tile-local delta-Q and delta-LF predictors, derives segmentation and reference-adjusted filter levels, and runs the exact scalar low/high-bit-depth AV1 4-, 6-, 8-, and 14-tap deblocking kernels in normative vertical-then-horizontal order. It then applies scalar CDEF direction search, luma variance adjustment, primary and secondary constrained taps, chroma direction conversion, high-bit-depth scaling, skipped-block selection, and frame-edge sentinel handling from immutable per-plane snapshots. Active super-resolution derives the Appendix A bounded coded width and applies the exact 64-phase, 8-tap horizontal filter with aligned reconstruction-edge input, 8/10/12-bit clipping, and the existing cross-platform Vector128_.MultiplyAddAdjacent helper. Loop restoration follows super-resolution, preserves the required pre-CDEF deblocked context at internal stripes, and applies decoded Wiener or self-guided units from immutable plane snapshots. The visible still-image path then applies the complete self-contained film-grain parameter set after all in-loop filters. Independent 8-, 10-, and 12-bit AVIF vectors exercising every active filter and grain stage remain required before these paths have external pixel-level verification.
  • Palette mode now reads the normative luma/chroma mode and size CDFs, neighbor color caches, high-bit-depth color syntax, diagonal color-index maps, clipped-edge padding, and direct sample prediction through the existing reconstruction pipeline. The scalar implementation matches the pinned libaom source, but an independently encoded palette AVIF fixture is still required before this path is independently verified.
  • Non-reduced still-image sequence parsing now consumes decoder-model operating parameters, temporal presentation fields, and OBU-layer-applicable buffer-removal fields only to preserve AV1 bit alignment. The scheduling values are not retained, and no movie, track, timing, playback, or generic ISO BMFF surface has been introduced. Existing focused sequence-header coverage exercises only reduced-still syntax, so an independent non-reduced still AVIF vector remains required.
  • Loop-restoration unit parsing records tile-local switchable/Wiener/self-guided filter selections and coefficients in frame-owned plane grids, including super-resolution-adjusted unit corners and the corrected conditional 64x64-superblock unit-size bit. The active restoration stage implements the normative unit geometry, striped deblocked boundaries, Wiener filtering, self-guided projection, and 8/10/12-bit clipping, while reusing the existing SIMD-backed super-resolution and adjacent multiply/add primitives. Independently encoded fixtures covering every parameter set, plane layout, bit depth, and frame-edge geometry are still required. Other normative independently decodable still-image syntax paths still contain NotImplementedException or equivalent unsupported branches. Tile-local palette CDF adaptation is present; the remaining still-image frame-context behavior requires a separate source audit without introducing sequence playback state.
  • The frame buffer now establishes two-byte native sample storage, logical plane rows, and sample-unit block strides for 10/12-bit frames. The active intra-prediction, inverse-transform, and block-reconstruction path selects native 16-bit samples for 10/12-bit frames and has focused pipeline wiring coverage. Chroma-from-luma storage, subsampling, parameter derivation, U/V sharing, and 8/10/12-bit prediction are active; independently encoded high-bit-depth and chroma-from-luma AVIF conformance files are still required.
  • Av1YuvConverter now consumes the signaled full or limited range, every non-reserved AV1 H.273 matrix coefficient, transfer characteristics where the matrix definition requires them, subsampling, and chroma sample position for 8, 10, and 12-bit output. Its high-bit-depth decode and encode paths use allocator-backed Rgb48 rows and the existing PixelOperations<TPixel> conversions, avoiding the former eight-bit intermediate. Encoder conversion covers monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with libavif-compatible box averaging. Identity, full/limited-range YCgCo, the fixed non-constant-luminance matrices, both fixed and chromaticity-derived constant/non-constant-luminance systems, SMPTE ST 2085, and PQ/HLG ICtCp are active in both directions. Independent vectors for every matrix, transfer, range, bit depth, sampling layout, and chroma position remain required before the complete color matrix is externally verified.
  • Forward and inverse transforms now use operation-owned allocator workspace and stateless static-generic operator structs for every valid DCT, ADST, and identity size. Named configuration factories keep the encoder's three shifts and variable cosine precision separate from the decoder's two shifts, fixed 12-bit cosine precision, and 8/10/12-bit clamp ranges. The two-dimensional traversal selects Vector256, then Vector128, with scalar only when hardware vectorization is unavailable. Focused tests prove exact scalar/SIMD parity for all 159 normative size/type combinations at every supported bit depth, padded edge strides, bounded residual extrema, the complete inverse shift/range tables, and zero per-block managed allocations. The 989-test suite passes with normal AVX2 dispatch, AVX2 disabled, and all hardware intrinsics disabled. On the measured .NET 10 AVX2 host, runtime-dispatched 8x8 DCT blocks take 102.42 ns forward and 112.14 ns inverse versus 441.19 ns and 602.52 ns scalar; 32x32 blocks take 1.439 microseconds forward and 8.825 microseconds inverse versus 9.112 microseconds and 23.455 microseconds scalar. BenchmarkDotNet reports no managed allocation for any measured path. A managed Vector512 path remains deliberately absent: pinned libaom has an optional Highway AVX-512 low-bit-depth forward path, but no matching complete inverse and high-bit-depth transform family, and the complete-block results do not justify maintaining a separate partial algorithm.
  • The production prediction and nonlinear self-guided paths remain predominantly scalar. Transform traversal is SIMD-first on supported hardware, while normative super-resolution and Wiener horizontal products reuse ImageSharp's cross-platform adjacent multiply/add SIMD helper with exact scalar fallbacks; further SIMD work must preserve these scalar reconstruction oracles.

AV1 encoder

  • HeifEncoderCore.Encode() now remains synchronous and waits for its temporary JPEG item encoding, so work and exceptions cannot outlive the ImageSharp encoder contract.
  • The current container encoder compresses pixels with the ImageSharp JPEG encoder and writes that payload into a HEIC-branded HEIF file. It does not produce AVIF.
  • Av1FrameEncoder.Encode() is an outline of an SVT-style pipeline rather than an implementation.
  • Required mode-decision, block-geometry, forward-transform, token-writing, neighbor-context, palette, intra-block-copy, transform-size, quantization/rate-control, and OBU-writing paths are absent or throw.
  • The current encoder test is a self-round-trip through the JPEG-in-HEIF path. It does not prove that the output is AVIF or that an independent decoder can read it.

Legacy JPEG image items

  • A JPEG item decoder exists, and the Fujifilm .hif fixture is identified and decoded as a legacy JPEG image item.
  • The encoder currently uses JPEG as an accidental fallback for all output rather than as an explicitly selected, correctly branded HEIF image-item codec.
  • The legacy JPEG path needs container interoperability tests and explicit public option semantics, but the existing ImageSharp JPEG codec should remain the payload implementation.

Tests

  • The repository contains HEIC, HIF, and AVIF assets, but only the legacy JPEG HIF path reaches a full reference-image comparison. Pinned upstream libavif animations now verify real 8-bit, 12-bit, alpha, Exif/XMP, repetition, and unrelated-audio sequence parsing without claiming inter-frame pixel reconstruction.
  • HEVC fixtures are identified but not decoded, and there are no HEVC algorithm tests.
  • The strongest AV1 integration test now drives the single-still decoder through tile parsing, reconstruction, and pixel conversion, but only verifies non-zero output rather than independent reference pixels.
  • Several full-image, inverse-transform, entropy, and frame-header cases are disabled or commented out.
  • Existing bitstream, predictor, transform, and entropy unit tests are useful foundations, but many compare two in-tree implementations with the same assumptions.
  • There is no decode matrix covering bit depth, subsampling, range, matrix coefficients, alpha, grids, transformations, metadata, truncated data, or resource limits.
  • There is no cross-codec encode test in which libavif decodes ImageSharp output, or ImageSharp decodes independently encoded output.

Architectural direction

Public format identities, shared internal HEIF container

Expose HEIC and AVIF as distinct public format identities backed by one internal HEIF/ISO BMFF container implementation. Provide format-appropriate decoders, encoders, metadata, options, MIME types, extensions, and generated save methods. Generic .heif and .hif input must be dispatched by brands and primary item type to HEVC, AV1, or legacy JPEG rather than by extension. Avoid a broad internal namespace rename until the container model is stable.

Generate the standard sync and async SaveAsHeic and SaveAsAvif overloads through the existing format templates. If a generic SaveAsHeif API is retained, its encoder options must require an explicit supported payload codec rather than infer one from an extension. Public XML documentation must describe observable behavior and option effects only.

Parsed container model

Parse the meta box into an immutable logical model before decoding payloads:

  • dictionaries keyed by item ID, not collection position;
  • a one-based property table with explicit association records and essential flags;
  • resolved item references (dimg, auxl, thmb, and metadata relationships);
  • item locations represented as validated extents and construction methods;
  • typed properties for ispe, pixi, hvcC, av1C, colr, auxC, pasp, clap, irot, and imir;
  • bounded views over item payloads, independent of source box order; and
  • explicit primary, alpha, grid-tile, thumbnail, Exif, and XMP roles.

Use existing ImageSharp buffered stream and allocation abstractions. Do not copy the whole file into an unbounded byte array. Every length, offset, multiplication, allocation, tile count, and image dimension coming from the file is an external boundary and must be checked against the enclosing box and ImageSharp limits. Internal decode stages should rely on the validated model rather than repeat defensive checks.

Codec state and sample storage

Keep HEVC and AV1 bitstream state in separate codec implementations. Within each codec, separate parameter/sequence state, frame or picture headers, tile/slice entropy state, and reconstructed images. Still-image operations retain no reference pictures; bounded image-sequence operations retain only the reference frames required by their declared dependencies. Give each allocation one owner and a deterministic disposal point.

Represent 8-bit samples with bytes and high-bit-depth samples with unsigned 16-bit storage. Plane dimensions and strides must reflect monochrome and chroma subsampling instead of pretending every plane is full-resolution 4:4:4. Keep scalar reconstruction as the behavioral oracle for every SIMD implementation.

Decode directly into ImageSharp pixels

Perform chroma upsampling, range expansion, matrix conversion, alpha composition, and pixel packing into allocator-backed row buffers or the destination frame. Use PixelOperations<TPixel> and the existing packed pixel conversion paths instead of creating an intermediate Image<Rgb24>.

Do not reuse JPEG or WebP color constants merely because those codecs already contain vectorized YUV conversion. Reuse their vector dispatch, lane handling, row processing, and pixel packing patterns only when the signaled HEVC/AV1 range and matrix semantics are preserved.

Existing ImageSharp code to reuse

Need Reuse target Constraint
Stream parsing BufferedReadStream, existing endian readers, bounded decoder-core patterns HEIF box extents remain the source of truth.
Memory ownership MemoryAllocator, IMemoryOwner<T>, Buffer2D<T>, allocator-backed row buffers No unbounded file-sized arrays or per-block allocations.
Pixel output PixelOperations<TPixel>, Rgba32, Rgba64, existing packed conversion methods Preserve alpha and high-bit-depth precision.
SIMD utilities SimdUtils, Vector128_, Vector256_, Vector512_, Numerics helpers Add codec-specific math only where semantics differ.
SIMD structure JPEG color-converter factories and WebP YUV row converters Reuse dispatch/tail patterns, not incompatible coefficients.
Decode lifecycle JPEG/WebP decoder cores, ImageSharp cancellation and dimension-limit handling Identify must not reconstruct pixels.
Public format API PNG, JPEG, and WebP format/decoder/encoder/options/metadata patterns Keep HEIC, AVIF, and generic HEIF dispatch behavior explicit.
Generated save APIs _Formats.ttinclude and existing format templates Generate SaveAsHeic and SaveAsAvif; require an explicit codec for generic HEIF output.
Metadata Existing ICC, Exif, XMP, and CICP-related metadata structures where available Preserve payloads and apply only specified transforms.
Test infrastructure ReferenceCodec, image comparers, feature-test helpers, codec test base classes Include independent artifacts and cross-codec tests.
Performance tests Existing ImageSharp benchmark project and hardware-intrinsics test controls Benchmark real decode stages and representative images.

Implementation phases

Every phase exit gate is an internal prerequisite for the next phase. Only the Phase 9 exit gate together with a fully passing verification matrix marks the PR complete.

Phase 0: establish a reproducible baseline

Tasks:

  • Build the merged solution in Release and record compile errors and warnings attributable to the WIP.
  • Run only the existing HEIF/HEVC/AV1 tests first, then record disabled tests and unexecuted asset coverage.
  • Record the pinned official libaom, local libavif, HM, and Android libhevc revisions, then pin the independent HEIC container interoperability oracle and external fixture toolchain.
  • Create a provenance map from each WIP codec file to its specification section and exact upstream source. Preserve the current SVT-AV1 origins where applicable and identify which missing paths will use libaom.
  • Convert the completion boundary above into a feature matrix with unsupported, parses, decodes, encodes, and verified independently states.
  • Audit every source file under src/ImageSharp/Formats/Heif. Document every type and shared contract, and add technical comments wherever the code depends on non-obvious specification syntax, fixed-point arithmetic, transform staging, entropy state, buffer layout, or SIMD behavior. Keep public XML documentation limited to observable API behavior.

Exit gate:

  • The post-merge branch has a recorded Release baseline, every existing failure is classified, all upstream code origins are known, and the complete HEIF source tree passes the documentation audit before additional porting begins.

Phase 1: correct the format contract

Tasks:

  • Define distinct HEIC and AVIF public format types over the shared internal HEIF container and register the correct brands, MIME types, and extensions.
  • Add generated SaveAsHeic and SaveAsAvif APIs and format metadata integration through the same mechanisms as established codecs. Define generic SaveAsHeif only if its options require an explicit supported payload codec.
  • Define decoder options using existing DecoderOptions behavior, including target pixel type, metadata handling, cancellation, and image-size limits.
    • Generic decode selects the caller's pixel type and the non-generic entry point defaults to Rgba32. TargetSize and Sampler are applied once after HEIF presentation composition. MaxFrames bounds retained sequence samples. SkipMetadata, SegmentIntegrityHandling, ColorProfileHandling, Configuration, and cancellation flow through the container, item, sequence, and nested-codec boundaries.
    • Complete adversarial dimension and allocation-limit coverage for still items, grids, auxiliary images, and sequence tracks before closing this contract item. Large payload and image buffers already use the configured allocator, but the complete cross-product has not been verified.
  • Define codec-specific encoder options with observable semantics for quality, speed/effort, lossless mode, chroma subsampling, bit depth, alpha quality, and metadata handling. Avoid exposing internal HEVC or AV1 tuning knobs without a stable user-facing meaning.
  • Make Rgba32 the default 8-bit decode output so alpha is not silently lost.
  • Remove unsupported JPEG 2000, JPEG-XR, JPEG-XS, and AVC capability claims unless those payload codecs are added to the completion matrix. Retain legacy JPEG as an explicit supported HEIF image-item codec.

Exit gate:

  • API review confirms that names and documented behavior match existing ImageSharp patterns and promise only the completed HEVC, AV1, and legacy JPEG HEIF payload paths. Capabilities must not be registered before their implementation reaches the final PR gate.

Phase 2: rebuild the bounded HEIF still-image reader around validated items

Tasks:

  • Implement a bounded box reader supporting 32-bit, 64-bit, and to-end box sizes where allowed, with overflow-safe arithmetic and correct parent bounds.
  • Accept the applicable HEIF, HEIC, and AVIF still-image brands through the major or compatible brand rules. Recognize and reject sequence/movie brands without implementing their track surface.
  • Skip unknown non-essential boxes and properties. Reject unknown essential properties attached to a decoded item.
  • Parse meta children independently of physical order and build the item/property/reference model described above.
  • Correct item ID lookup, one-based property indices, association flag masks, full-box versions, and large IDs/offsets.
  • Resolve idat and mdat item data, multiple extents, construction methods, and 64-bit offsets through bounded item streams.
  • Parse and associate HEVC and AV1 configuration, dimensions, plane information, ICC/CICP color, alpha auxiliary type, grids, transforms, Exif, XMP, and thumbnails.
  • Make Identify return dimensions, bit depth, color type, alpha presence, profiles, and metadata from the parsed model without decoding AV1 tiles.
  • Add malformed-container tests for every external length, offset, count, ID, association, extent, and relationship boundary.

Exit gate:

  • The parser resolves each current HEIC, HIF, and AVIF asset into a stable logical model, malformed inputs fail without escaping bounds or allocating attacker-controlled sizes, and Identify has reference-verified metadata and payload classification.

Phase 3: complete scalar AV1 still-image reconstruction

Implement and verify in dependency order:

  • OBU framing, sequence headers, frame headers, tile groups, byte alignment, and trailing bits.
  • One coherent decoder lifecycle that retains parsed frame and tile state and disposes all buffers deterministically.
  • Tile partitioning, mode information, segmentation, delta quantization, transform-size selection, coefficient token decode, inverse quantization, and inverse transforms.
  • Intra prediction, including every directional, smooth, Paeth, CFL, filter-intra, and palette case permitted by AV1.
  • Lossless and high-bit-depth reconstruction with correct clipping and intermediate precision.
  • Deblocking loop filter.
  • CDEF.
  • Super-resolution scaling.
  • Loop restoration.
  • Frame padding and film-grain synthesis when signaled.

For each item, first add a small scalar unit test against normative or independent vectors, then enable it in the frame pipeline. Remove constant feature-disable flags and unsupported branches only when their replacement is verified. Unsupported syntax must produce a codec-specific invalid-image error; it must never silently skip a normative reconstruction stage.

Exit gate:

  • Independently encoded, opaque, single-item AVIF files reconstruct correctly across all AVIF profiles, bit depths, subsampling modes, and normative still-image compression tools. Pixel comparisons are made after applying the same signaled color conversion in the reference path.

Phase 4: complete scalar HEVC still-image reconstruction

Implement and verify in dependency order:

  • HEIF item-local length-delimited NAL units, hvcC, VPS, SPS, PPS, and the parameter-set and slice-header syntax needed to decode the one independently coded picture carried by a supported still-image item. Do not add an Annex B byte-stream API or an access-unit/timeline abstraction.
  • One bounded decoder lifecycle that owns only the parameter sets, current-picture state, slice/tile entropy state, and reconstructed planes required by that image item.
  • CABAC arithmetic decoding and every required context transition.
  • Coding-tree, coding-unit, prediction-unit, and transform-unit traversal across all permitted sizes and partition modes.
  • Intra prediction for every luma and chroma mode, including strong intra smoothing and constrained prediction rules.
  • Scaling lists, inverse quantization, transform skip, every required inverse transform, range-extension precision, and lossless reconstruction.
  • Deblocking and sample-adaptive offset for every signaled luma/chroma and bit-depth path.
  • Tiles, wavefront entry points, dependent slices, and all other parallelization syntax permitted by the exposed still-image profiles.
  • Supplemental enhancement information that changes image presentation or metadata exposed by ImageSharp.

Each subsystem starts with scalar conformance vectors derived from the HEVC specification and the pinned implementation reference. No valid syntax in the exposed HEIC profiles may terminate in an unsupported branch or silently omit a normative reconstruction stage.

Exit gate:

  • Independently encoded HEIC files reconstruct correctly across every exposed HEVC profile, chroma format, bit depth, range-extension tool, and normative still-image compression path.

Phase 5: shared color conversion, alpha, grids, and presentation transforms

Tasks:

  • Implement monochrome, 4:2:0, 4:2:2, and 4:4:4 plane access with every valid signaled chroma sample position.
    • Implement SIMD AV1 4:2:0 and 4:2:2 chroma reconstruction and downsampling, including centered/colocated reconstruction and odd image edges.
    • Complete the HEVC plane paths and the exhaustive AV1/HEVC chroma-position matrix before closing this item.
  • Implement full- and limited-range expansion for 8, 10, and 12-bit samples across every supported plane layout.
    • Implement AV1 range expansion and quantization with SIMD-first byte/ushort loaders and storers and pooled planar rows.
    • Complete the equivalent HEVC paths and independently verify the full cross-product.
  • Implement every non-reserved HEVC/AV1 color-primary, transfer-characteristic, and matrix-coefficient signaling path, including identity conversion, with correct fixed-point rounding and clipping.
    • Implement the AV1 H.273 matrix and transfer-function operators in both directions, including identity, derived coefficients, constant luminance, SMPTE ST 2085, and PQ/HLG ICtCp.
    • Add independent conformance vectors for signaling combinations that libavif deliberately rejects, then complete and verify the HEVC operator set.
  • Make SIMD the default decode path for YUV range expansion, chroma reconstruction, H.273 matrix and transfer conversion, clamping, and 8/10/12-bit RGB packing. Use the established static operator pattern with Vector512, Vector256, and Vector128 dispatch followed by one scalar tail that remains the behavioral oracle.
    • AV1 uses the JPEG-style static operator contract and descending SIMD-width traversal for decode and encode, with scalar execution limited to the remaining lanes or hardware without vector support.
    • Complete the same default SIMD pipeline for HEVC and record end-to-end measurements for both codecs.
  • Apply embedded ICC profiles after codec YUV/CICP conversion and image composition through ImageSharp's existing color-profile converter. Cover preserve, convert, compact, and ignore behavior for primary items, grids, alpha-composited images, and every presented sequence frame; never substitute CICP or an assumed sRGB profile for an embedded ICC profile.
    • Primary AVIF verification passes four independently scoped cases: exact profile preservation, non-sRGB conversion with changed pixels and paired-source comparison, canonical-sRGB compaction without pixel changes, and metadata skipping. Grid, auxiliary-alpha, sequence, and HEVC cases remain open.
  • Decode alpha auxiliary items as monochrome planes, validate dimensions and bit depth, and compose them without losing precision. Define premultiplication behavior from AVIF signaling and ImageSharp's pixel contract.
  • Validate grid tile type, dimensions, properties, order, and canvas coverage. Compose directly into the destination frame, cropping only the permitted right and bottom tile overlap.
  • Apply clean aperture, rotation, mirroring, and pixel aspect ratio according to the container property order and ImageSharp metadata/processing conventions.
  • Preserve ICC, CICP, Exif, and XMP metadata using existing ImageSharp profile types. ICC preservation and ICC color conversion are separate contracts and must both be independently verified.

Exit gate:

  • The complete cross-product of valid HEVC/AV1 bit depths, subsampling modes, ranges, color signaling, alpha, grids, and transforms is covered by focused vectors and representative HEIC/AVIF integration files and matches independently decoded references within a documented conversion tolerance. Tests must prove ICC conversion changes pixel values according to an independent profile transform instead of checking profile retention alone. Integer identity/lossless cases must match exactly.

Phase 6: implement a real AV1 still-image encoder

Delete the JPEG payload path from the production encoder. Keep the synchronous ImageSharp encoder contract synchronous unless the established base contract provides an async implementation; never use async void.

Implement in vertical slices that always produce a decodable AV1 bitstream:

  • Convert RGB/RGBA to AV1 planes for every 8, 10, and 12-bit output, range, matrix, and monochrome/4:2:0/4:2:2/4:4:4 combination permitted by the selected AV1 profile.
    • Implement the pooled sequential RGB-to-planar pipeline, 8/10/12-bit quantization, every implemented H.273 operator, and SIMD 4:2:0/4:2:2 downsampling with odd-edge handling.
    • Complete the exhaustive format/range/matrix/transfer/primary cross-product and validate encoded planes against an independent implementation before closing this item.
  • Write sequence, frame, tile-group, and metadata OBUs for a reduced still picture.
  • Implement a temporary smallest-valid intra-only vertical slice using existing partition, prediction, transform, quantization, coefficient, and entropy structures.
  • Complete block geometry, neighbor/context updates, transform selection and forward transforms, quantization, coefficient tokenization, and range coding.
  • Add mode decision and rate/distortion selection in increasing effort levels. Reuse computed prediction and transform results instead of duplicating analysis across stages.
  • Add lossless mode and validate the exact lossless constraints rather than treating quality 100 as lossless.
  • Add in-loop filter decisions and signaling. A legal choice to disable an encoder feature is distinct from a decoder skipping a signaled feature.
  • Add alpha as an auxiliary AV1 item with independently controllable quality where the public option warrants it.
  • Write the AVIF item graph, extents, av1C, pixel information, color properties, metadata, and alpha relationships with correct brands.
  • Write embedded ICC and CICP color properties independently, preserve the source profile selected by the encoder options, and verify both metadata round-trip and independently converted output pixels.
  • Make output deterministic for identical pixels, metadata, options, and configuration.
  • Complete the partition, prediction, transform, quantization, entropy, filter, and rate/distortion choices needed for quality and effort settings to provide a genuine compression tradeoff rather than selecting from a fixed coding subset.
  • Exercise every encoder bit-depth, plane-layout, range, color, alpha, lossless/lossy, quality, and effort combination through independent decode.

Every vertical slice must be decoded by libavif before additional compression features are added. Self-round-trip tests are supplementary because matching encoder and decoder bugs can otherwise hide invalid bitstreams.

Exit gate:

  • libavif and another independent AV1 decoder accept ImageSharp output across the complete encoder matrix, and ImageSharp reconstructs the same files. Lossless output is pixel-exact; lossy output demonstrates effective quality/effort tradeoffs and meets recorded quality and size expectations without malformed or non-AV1 payloads.

Phase 7: implement a real HEVC still-image encoder

Implement in vertical slices that always produce a HEVC bitstream accepted by an independent HEIC decoder:

  • Convert ImageSharp pixels into every exposed HEVC bit depth, range, and monochrome/4:2:0/4:2:2/4:4:4 combination.
  • Write VPS, SPS, PPS, slice headers, parameter arrays in hvcC, and a smallest-valid intra picture as a temporary vertical slice.
  • Complete coding-tree partitioning, intra prediction selection, forward transforms, scaling/quantization, coefficient scanning, and CABAC encoding.
  • Add real rate/distortion mode decision, quality and effort controls, lossless mode, deblocking decisions, and sample-adaptive-offset decisions.
  • Add alpha auxiliary images, grids, metadata, color properties, image relationships, and correct HEIC brands.
  • Write embedded ICC and CICP color properties independently, preserve the source profile selected by the encoder options, and verify both metadata round-trip and independently converted output pixels.
  • Exercise every encoder profile, bit depth, chroma format, range, color, alpha, lossless/lossy, quality, and effort combination through independent decode.

Exit gate:

  • Independent HEIC decoders accept ImageSharp output across the complete encoder matrix. Lossless output is pixel-exact; lossy output demonstrates effective quality/effort tradeoffs and meets recorded quality and size expectations.

Phase 8: SIMD and allocation optimization

SIMD work begins after the corresponding scalar stage has independent correctness tests; it then proceeds alongside subsequent functional phases.

Tasks:

  • Remove known avoidable allocations first: per-transform arrays, the intermediate RGB image, repeated block scratch arrays, and file-sized buffering.
    • Complete the active AV1 forward and inverse transform-family checklist above.
  • Benchmark codec-specific costs for CABAC/range decode, inverse transforms, still-image prediction, deblocking, SAO, CDEF, restoration, chroma upsampling, color conversion, alpha packing, and grid copies.
  • Implement vector paths only for confirmed hot loops, using existing Vector128, Vector256, and Vector512 helper and dispatch patterns where supported.
  • Prioritize shared color conversion and pixel packing, chroma upsampling, inverse-transform add-and-clip, intra predictors, HEVC deblock/SAO, AV1 loop filter/CDEF/restoration, and contiguous grid copies.
  • Benchmark the complete decode color pipeline on representative 8/10/12-bit AVIF and HEIC images with and without embedded ICC profiles. Report absolute end-to-end timings and allocations in addition to the isolated YUV/CICP and ICC stage costs.
  • Port upstream SIMD algorithms only after mapping lane width, signedness, intermediate precision, rounding, saturation, edge extension, and high-bit-depth behavior to the scalar oracle.
  • Keep one scalar implementation as the specification-shaped reference. Vector paths must share tables and constants with it rather than duplicate codec policy.
  • Test scalar and each available hardware path with intrinsics explicitly enabled and disabled, including widths shorter than a vector, exact-vector widths, non-multiples, edges, maximum sample values, and high-bit-depth overflow cases.
  • Remove dead or commented SIMD experiments once a verified production path replaces them.

Exit gate:

  • Benchmarks show a material improvement on representative AVIF files, allocation measurements meet an agreed budget, and every vector path is behaviorally identical to the scalar path for integer reconstruction or within the documented color-conversion tolerance.

Phase 9: hardening, documentation, and release readiness

Tasks:

  • Fuzz the box parser, AV1 OBU parser, HEVC NAL/parser, entropy decoders, and dimension/allocation boundaries using the same safety expectations as established ImageSharp codecs.
  • Test seekable and non-seekable streams, short reads, cancellation, truncated data, unknown optional boxes, unknown essential properties, oversized dimensions, malicious counts, and offset arithmetic overflow.
  • Run the focused HEIF/HEIC/AVIF suite after every final codec edit, then the full ImageSharp suite in Release.
  • Build all supported target frameworks and run packaging/API compatibility checks used by the repository.
  • Update public documentation, format tables, MIME/extension lists, samples, and THIRD-PARTY-NOTICES.TXT.
  • Remove placeholder images, legacy JPEG-in-HEIF production paths, stale TODO-only code, disabled tests that now have coverage, and unsupported capability claims.

Exit gate:

  • The full Release build and test matrix passes, every valid HEVC/AV1 color/compression/bit-depth entry is implemented, independent HEIC and AVIF interoperability is recorded for the complete feature matrix, the public API has been reviewed, provenance and patent/license obligations are complete, and no advertised feature depends on a placeholder, narrow temporary subset, silent fallback, disabled normative stage, or unsupported valid syntax branch.

Verification matrix

Every valid combination in the HEVC and AV1 profiles exposed by the final public contract needs focused coverage and representative integration coverage. Each axis needs independently produced HEIC/AVIF inputs and ImageSharp-produced outputs; pairwise and targeted cross-product cases must cover interactions where exhaustive media fixtures would be redundant.

Status Area Required coverage
- [ ] Container/payload HEIC with HEVC, AVIF with AV1, and generic HEIF/HIF with supported HEVC, AV1, or legacy JPEG items; detection by brands/items.
- [ ] Bit depth Every bit depth permitted by the exposed HEVC and AV1 profiles, including 8, 10, and 12-bit decode and encode.
- [ ] Planes monochrome, 4:2:0, 4:2:2, and 4:4:4 decode and encode in every valid HEVC/AV1 profile and depth combination.
- [ ] Range full and limited decode and encode.
- [ ] Color every valid non-reserved color-primary, transfer-characteristic, matrix-coefficient, and chroma-position signaling path; identity RGB; ICC; CICP defaults and overrides.
- [ ] Alpha opaque, binary, gradient, different alpha quality, high bit depth, malformed relationship.
- [ ] Structure single item, multiple extents, idat, mdat, grids with cropped edge tiles, metadata items.
- [ ] Transform pasp, clap, irot, imir, and valid combinations.
- [ ] AV1 decode tools every normative transform type/size, still-image predictor, partition, palette, segmentation, quantization, entropy/context, lossless, deblock, CDEF, super-resolution, restoration, and film-grain path valid in independently decodable AVIF still-image items.
- [ ] AV1 encode compression real mode decision and rate/distortion selection across partitions, predictions, transforms, quantization, entropy coding, filters, lossless/lossy quality, and effort settings; no permanent fixed coding subset.
- [ ] HEVC decode tools every normative NAL/parameter/slice, CABAC, coding-tree, intra prediction, transform, quantization, range-extension, lossless, tile/wavefront, deblock, and SAO path valid in independently decodable still-image items for the exposed HEIC profiles.
- [ ] HEVC encode compression real coding-tree, prediction, transform, quantization, CABAC, filter, and rate/distortion decisions across lossless/lossy quality and effort settings; no permanent fixed coding subset.
- [ ] Streams file, memory, non-seekable, short-read wrapper, cancellation.
- [ ] Failure truncation at every box/OBU layer, invalid sizes/offsets/counts, unknown essential properties, unsupported profile.
- [ ] Interop libavif and libaom for AVIF/AV1; a pinned independent HEIC container and HEVC codec implementation for HEIC/HEVC.
- [ ] SIMD scalar, 128, 256, and 512-bit paths where supported; tails and edge blocks.

Reference outputs must be versioned artifacts or generated by a pinned reference command whose exact tool version and arguments are recorded. Do not use ImageSharp's own decoder to establish the expected pixels for its encoder, and do not replace final-image assertions with internal buffer or non-zero checks.

No valid HEVC or AV1 color, compression, or bit-depth row may remain unsupported, partially implemented, disabled, or deferred when the PR is marked complete.

Working rules for implementation

  • Keep changes vertical and reviewable. A slice should add one behavior, its focused tests, independent evidence, and any required notice update.
  • Inspect every owning method and upstream invariant before adding guards. Validate external file data at the parser/model boundary and rely on those established invariants internally.
  • Do not extract one-use helpers merely to label code. Extract shared primitives only when they have genuine reuse or remove substantial complexity.
  • Put comments at the points where HEVC/AV1 rounding, edge extension, context propagation, or SIMD lane behavior is not evident from the code. Comments should explain why the algorithm has that shape and identify the normative rule and pinned implementation reference.
  • Use observable behavior only in public API documentation.
  • Do not use reflection, built-assembly probing, native runtime fallbacks, fabricated images, or self-round-trip-only evidence.
  • Build and test in Release configuration.

The critical path is:

  • Baseline and provenance.
  • Public HEIC/AVIF boundaries and the shared parsed HEIF container model.
  • Scalar AV1 still decode, starting with 8-bit 4:2:0 and completing every AVIF profile/tool.
  • Scalar HEVC still decode, starting with 8-bit 4:2:0 and completing every exposed HEIC profile/tool.
  • Shared color, alpha, grids, metadata, and presentation transforms across all bit depths and chroma formats.
  • Real AV1 encoder and libavif/libaom cross-decode across the complete matrix.
  • Real HEVC encoder and independent HEIC cross-decode across the complete matrix.
  • Measured SIMD/allocation work integrated after each scalar subsystem stabilizes.
  • Hardening and release gates.

This sequence does not define partial PR completion. The early 8-bit 4:2:0 decoders and smallest-valid encoders are temporary vertical slices, but the PR remains incomplete until all nine steps and the complete still-image verification matrix pass.

Do not begin by optimizing the current end-to-end pipeline: it cannot yet produce a correct HEIC or AVIF image, and several current data structures encode 8-bit 4:4:4 assumptions. Establish the correct scalar storage and behavior first so the reused or ported AOM/HEVC SIMD algorithms have a trustworthy managed oracle.