Write still images, grid tiles, and sequence frames into their final regions,
combining presentation transforms, alpha, and ICC conversion. Add explicit
chroma upsampling selection and correct high-bit-depth sample normalization.
Remove JPEG item handling and restore shared Flip/Rotate implementations.
Update encoded versus decoded metadata assertions and reference-image output.
Preserve the existing allocator identity contract. Document native reference
provenance, accepted ICC baselines, and remaining encoder acceptance gaps.
Validation: Roslynk reports zero errors; Release/net11 builds successfully.
Visual Studio VSTest passed all 10,487 selected HEIF/AV1, ICC, Flip/Rotate,
and allocator cases. Temporary native tooling and unused Hadamard changes
are excluded from this commit.
The reference images were regenerated independently; no managed decoded pixels or weakened assertions were used.
Release/net11 VSTest heif-unified-references-r1 ran only the 19 previous failures: 12 passed, 7 failed, 14.5724 seconds.
Remaining pixel mismatches still require investigation. This is not decoder byte-exact acceptance or libaom parity.
## Goal
## Goal
Complete a production-quality, fully managed AV1 codec and its bounded AVIF/HEIF image container integration for ImageSharp. The finished work must decode and encode still images and bounded image sequences, preserve source precision, use ImageSharp memory ownership, and provide SIMD-first hot paths with one behaviorally identical scalar fallback.
Complete a production-quality, fully managed AV1 codec and its bounded AVIF/HEIF image container integration for ImageSharp. The finished work must decode and encode still images and bounded image sequences, preserve source precision, use ImageSharp memory ownership, and provide SIMD-first hot paths with one behaviorally identical scalar fallback.
@ -8,6 +164,127 @@ This plan is the authoritative delivery checklist. A source file, unit test, bui
Checkpoint handling: work stays in the existing checkout. Do not create or use worktrees; the user reported a crash.
Checkpoint handling: work stays in the existing checkout. Do not create or use worktrees; the user reported a crash.
Commit completed, verified features regularly, with native reference material and temporary integration excluded.
Commit completed, verified features regularly, with native reference material and temporary integration excluded.
Cleanup commit `e38989d63` restored three JPEG production files and the PNG encoder exactly to upstream/main.
Cleanup commit `93f48a480` restored DecoderOptions, ImageDecoderCore, IccProfile, Point, and PointTests,
included the PixelOperations formatting correction, and migrated dependent HEIF integrity, ICC serialization,
and coordinate-shift callers. Its 16-file scope excludes component-domain ICC and region conversion changes.
The restored Point implementation passed 55 focused Release/net11 tests (`point-upstream-cleanup-r1.trx`).
FlipProcessor and RotateProcessor restorations remain coupled to the decoder-region caller changes: committing
only their helper removals would break the committed HEIF decoder. The region/ICC checkpoint remains unresolved.
## Required final cleanup and deletion commit
2026-09-08 chroma-mode correction: HeifDecoder now follows the specialized-options API used by JPEG.
HeifDecoderOptions.ChromaUpsampling has three values: Auto (default), NearestNeighbor, and Bilinear.
Auto chooses nearest-neighbour for 8-bit source chroma and bilinear for higher source bit depths.
The selection reaches still items, grid children, and sequence presentation. Nearest-neighbour writes
directly into component rows without interpolation scratch; bilinear retains its existing SIMD sampling path.
Existing helper callers now pass the mode explicitly. The existing bilinear test explicitly selects Bilinear;
its expected pixels and every golden remain unchanged. Roslynk reported zero compiler errors.
generated comparison artifacts from the repository. Preserve needed local reference tooling outside the repository.
`tests/ImageSharp.Benchmarks/Codecs/Heif/Native/aom_benchmark.c` is already tracked from `a7f0fca6b`;
include its deletion and other tracked temporary reference material in the final cleanup commit.
- [ ] Delete stray task READMEs, redundant reports, temporary notes, and documentation for discarded approaches.
Retain only documentation needed for the delivered codec and its supported verification workflow.
- [ ] Delete obsolete or unnecessary tests, including tests for rejected implementations, unrequested features,
and implementation details that do not establish a required contract. Preserve independent acceptance coverage.
Do not delete or weaken a failing test merely to conceal an unresolved production or verification defect.
- [ ] Delete obsolete component benchmarks, including `Av1SequenceEncoderBenchmarks`, and their stale references.
Retained performance verification must measure full encode/decode with equivalent end-to-end boundaries.
- [ ] Inspect the final diff against upstream/main for unrelated codec changes, accidental files, and temporary
tooling. Inspect the staged changes before committing the reviewed deletions.
- [ ] Commit the deletions normally at the end. The user explicitly chose this approach: no history rewriting
and no force-push to remove earlier native-tooling commits.
## Earlier checkpoint evidence
The probability-storage checkpoint is `a658a2cb7`; adaptive syntax and retained palette tokens are `d8f6a1de3`.
The probability-storage checkpoint is `a658a2cb7`; adaptive syntax and retained palette tokens are `d8f6a1de3`.
The final supporting entropy, mode-grid, frame-buffer, and intra-copy run passed 2,085 tests with zero failures.
The final supporting entropy, mode-grid, frame-buffer, and intra-copy run passed 2,085 tests with zero failures.
The encoder deblocking comparison covered 102,390 samples in 12 streams at 8/10/12 bits and 400/420/422/444:
The encoder deblocking comparison covered 102,390 samples in 12 streams at 8/10/12 bits and 400/420/422/444:
@ -45,9 +322,78 @@ Acceptance criteria are separate for encoding and decoding:
- Historical decoder reports of zero samples exceeding one are insufficient by themselves. A recorded maximum
- Historical decoder reports of zero samples exceeding one are insufficient by themselves. A recorded maximum
error of zero establishes sample equality only for the stated comparison scope, not complete decoder correctness.
error of zero establishes sample equality only for the stated comparison scope, not complete decoder correctness.
## Active production milestone: encoder motion search
## JPEG scope correction: 2026-09-07
The persistent goal remains active. Complete the integrated encoder motion-search path, including configuration,
- Removed legacy JPEG item decoding and encoding, compression selection, container brands, and dedicated HEIF JPEG tests.
- Removed the single-value compression-method enum and its encoder, metadata, and item-decoder properties. Encoding selects AV1 directly; still output always writes AVIF brands.
- Restored all JPEG production files to upstream/main, including earlier branch changes to metadata writing and spectral pixel packing.
- Restored the PNG cICP writer to upstream/main. No codec-specific production changes remain outside HEIF/AV1 against upstream/main.
- Removed the added spectral pixel converter and HEIF JPEG adapter. The HEIF implementation has no JPEG codec dependency.
- General container tests retain their assertions and now generate an opaque AV1 item instead of a JPEG item.
- The decoder-region callers now compile: Roslynk reports zero compiler errors across the loaded solution.
- Release .NET 11 test assembly preparation passed. Runtime verification is in progress and the refactor is not yet a verified checkpoint.
- The Compact ICC regression is corrected: HEIF invokes ICC conversion only for Convert mode. The subsequent decoder run passed 71 cases.
- The earlier ICC implementation processed packed destination pixels using two scratch rows. It was rejected
and deleted. Component-domain ICC conversion is now implemented before pixel packing, as recorded below;
its exact-output verification remains unresolved. Earlier ICC test results do not verify the replacement.
- Deblocking transform sizes now reside in block metadata: one selected size and sixteen inline variable-transform entries.
The separate frame-sized luma/chroma maps and their ownership class are removed. Runtime verification after this change is pending.
- Historical JPEG-backed verification below does not establish acceptance for the remaining AV1-only implementation.
## Active production milestone: complete decoder-region refactor
Complete the still-image, grid, sequence, alpha, crop, rotation, and mirroring paths using exact destination regions.
Keep JPEG and other codec implementations unchanged against upstream/main. Retain the existing shared L16 SIMD implementation.
The generic RGB packer must accept exact-length regions, consistent with its existing scalar and SIMD implementations.
Complete existing caller migration and focused runtime verification before committing this checkpoint.
No new test infrastructure, extra decoder overloads, or component-test results substitute for native decoder parity.
### Transform and ICC integration: 2026-09-08, verification in progress
- The float/ICC output path now packs directly into the final region, removing its temporary TPixel row
and subsequent WriteRow copy. Destination origins and integer row/column increments are resolved once.
Contiguous output retains bulk pixel packing; reversed rows and columns currently use scalar final packing.
This is not completion of the optimized traversal: tiled/SIMD placement and end-to-end performance remain
unverified, and the outstanding ICC oracle mismatch below still prevents a verified region/ICC checkpoint.
Roslynk reported zero errors after this edit. Release/net11 preparation succeeded with 1,008 warnings and
zero errors; serialized VSTest passed 38 sequence cases (direct-region-sequences-r1.trx) and 70 public
encoder/grid cases (direct-region-grid-r1.trx). These runs do not verify the outstanding ICC mismatch.
cases (`transform-sequence-regions-r1.trx`), and 12 retained native-reference presentation cases at 8/10/12 bits
and monochrome/420/422/444 (`native-presented-regions-r1.trx`). The last group uses exact image comparison
across configured intrinsic paths. These passing groups do not clear the outstanding ICC failure.
- No benchmark or codec-wide parity claim follows from these edits. Earlier ICC results used the rejected
post-packing implementation and do not verify this implementation.
## Remaining encoder milestone: motion search
The overall codec goal remains unresolved. After the decoder refactor, complete the integrated encoder motion-search path, including configuration,
allocation geometry, rate costs, candidate and winner state, full-pixel search, fractional refinement, and production
allocation geometry, rate costs, candidate and winner state, full-pixel search, fractional refinement, and production
verification before advancing. The following dependency result does not close that milestone.
verification before advancing. The following dependency result does not close that milestone.
@ -2759,7 +3105,7 @@ Writer primitives are not an encoder. The public encoder remains incomplete unti
- [x] Use the existing PNG, TIFF, and JPEG encoders as the ImageSharp architecture reference: generic `Image<TPixel>` input, encoder options taking precedence over converted format metadata and codec defaults, allocator-owned temporary storage, and deterministic disposal.
- [x] Use the existing PNG, TIFF, and JPEG encoders as the ImageSharp architecture reference: generic `Image<TPixel>` input, encoder options taking precedence over converted format metadata and codec defaults, allocator-owned temporary storage, and deterministic disposal.
- [x] Treat source pixel type, source alpha representation, and decoded source bit depth as conversion inputs, never as output-eligibility checks. Do not pre-scan pixels before encoding.
- [x] Treat source pixel type, source alpha representation, and decoded source bit depth as conversion inputs, never as output-eligibility checks. Do not pre-scan pixels before encoding.
- [x] Resolve output configuration once from explicit encoder options, converted `HeifMetadata`, and AV1 defaults in that order. Sanitize only combinations that cannot describe a legal requested output, and never write resolved values back to source metadata.
- [x] Resolve output configuration once from explicit encoder options, converted `HeifMetadata`, and AV1 defaults in that order. Sanitize only combinations that cannot describe a legal requested output, and never write resolved values back to source metadata.
- [~] Finalize observable options for quality, effort, lossless mode, bit depth, chroma subsampling, alpha quality, metadata, and bounded sequences. Sequence repeat-count options override converted HEIF metadata like the existing animated encoders. Legacy JPEG treats the AV1-specific lossless and bit-depth options as inapplicable and continues with its native eight-bit encoding contract. AV1 sequences now follow the existing animated-image contract: the primary item reuses the first sync sample when the root is animated, while an excluded root is encoded once as the independent primary image and the sequence begins at frame index one. Final verification remains open.
- [~] Finalize observable options for quality, effort, lossless mode, bit depth, chroma subsampling, alpha quality, metadata, and bounded sequences. Sequence repeat-count options override converted HEIF metadata like the existing animated encoders. AV1 sequences now follow the existing animated-image contract: the primary item reuses the first sync sample when the root is animated, while an excluded root is encoded once as the independent primary image and the sequence begins at frame index one. Final verification remains open.
- [x] Preserve high-bit-depth source precision through 16-bit RGB and native 10/12-bit component planes.
- [x] Preserve high-bit-depth source precision through 16-bit RGB and native 10/12-bit component planes.
- [~] HEIF is registered through the default configuration module. Keep public AV1 capability claims limited to the paths covered by the encoder verification matrix until the remaining encoder work is complete.
- [~] HEIF is registered through the default configuration module. Keep public AV1 capability claims limited to the paths covered by the encoder verification matrix until the remaining encoder work is complete.
- [~] AV1 image properties now write `ispe`, `pixi`, `av1C`, `colr`, and `auxC` in current AVIF item order. Only `av1C` is essential; color and alpha items retain independent property sets and the registered alpha auxiliary type. The property container reacquires its span after nested expansion before patching `ipco`, removing the prior stale-buffer write, and selects compact or 15-bit `ipma` indices from the property count rather than the unrelated item count. A forced-growth color-plus-alpha case validates every property payload and association byte; a separate 43-item, 129-property case proves indices 127 through 129 and the extended essential bit. Both pass direct foreground net11 Release VSTest. Complete AVIF assembly remains open.
- [~] AV1 image properties now write `ispe`, `pixi`, `av1C`, `colr`, and `auxC` in current AVIF item order. Only `av1C` is essential; color and alpha items retain independent property sets and the registered alpha auxiliary type. The property container reacquires its span after nested expansion before patching `ipco`, removing the prior stale-buffer write, and selects compact or 15-bit `ipma` indices from the property count rather than the unrelated item count. A forced-growth color-plus-alpha case validates every property payload and association byte; a separate 43-item, 129-property case proves indices 127 through 129 and the extended essential bit. Both pass direct foreground net11 Release VSTest. Complete AVIF assembly remains open.
- [~] Explicit public AV1 encoding now writes a still-image AVIF with `avif` major brand, compatible `avif`, `mif1`, and `miaf` brands, one primary color item, an optional alpha auxiliary item, `auxl` from alpha to color, independent item properties, absolute version-one `iloc` extents, and a shared `mdat`. Quality uses current libaom's quantizer-to-qindex mapping with public quality 100 deliberately clamped from lossless qindex 0 to qindex 4. Effort controls the implemented search stages, and the resolved value is required explicitly by every internal frame, tile, and mode-decision operation rather than repeated as optional defaults. Encoder options take precedence over source metadata for 8-, 10-, and 12-bit monochrome, 4:2:0, 4:2:2, and 4:4:4 output. Alpha derives from the source pixel type without scanning pixels, and incompatible identity-matrix metadata is normalized without mutating the source image.
- [~] Explicit public AV1 encoding now writes a still-image AVIF with `avif` major brand, compatible `avif`, `mif1`, and `miaf` brands, one primary color item, an optional alpha auxiliary item, `auxl` from alpha to color, independent item properties, absolute version-one `iloc` extents, and a shared `mdat`. Quality uses current libaom's quantizer-to-qindex mapping with public quality 100 deliberately clamped from lossless qindex 0 to qindex 4. Effort controls the implemented search stages, and the resolved value is required explicitly by every internal frame, tile, and mode-decision operation rather than repeated as optional defaults. Encoder options take precedence over source metadata for 8-, 10-, and 12-bit monochrome, 4:2:0, 4:2:2, and 4:4:4 output. Alpha derives from the source pixel type without scanning pixels, and incompatible identity-matrix metadata is normalized without mutating the source image.
- [~] The production path writes color and alpha payloads sequentially through allocator-backed chunked storage, supports non-seekable and prefixed destinations, and does not materialize a complete file or payload copy. Uniform encoder-side `pixi` depth is written directly without allocating per-item channel-depth arrays; decoder-side non-uniform channel depths remain supported. The Release test project builds with zero errors, all 39 encoder cases pass, the complete non-HEVC HEIF namespace passes 9,277 of 9,277, and current official libaom accepts all 47 generated payloads.
- [~] The production path writes color and alpha payloads sequentially through allocator-backed chunked storage, supports non-seekable and prefixed destinations, and does not materialize a complete file or payload copy. Uniform encoder-side `pixi` depth is written directly without allocating per-item channel-depth arrays; decoder-side non-uniform channel depths remain supported. The Release test project builds with zero errors, all 39 encoder cases pass, the complete non-HEVC HEIF namespace passes 9,277 of 9,277, and current official libaom accepts all 47 generated payloads.
- [x] Still-image AVIF metadata preservation now writes an unrestricted ICC `colr/prof` property before the independent `colr/nclx` property, Exif and XMP as separate `mdat` items, and one `cdsc` relationship from each metadata item to the primary color item. Exif stores the exact big-endian TIFF-header offset required by the HEIF item syntax; XMP uses the `mime` item type and `application/rdf+xml` content type. Existing ICC and XMP storage is read synchronously and copied once into final encoder storage rather than cloned into an intermediate array. `SkipMetadata` suppresses all three profile types while retaining the CICP values required to describe the encoded planes. The same option now reaches legacy JPEG payloads, whose encoder no longer writes application profiles or comments when metadata is disabled.
- [x] Still-image AVIF metadata preservation now writes an unrestricted ICC `colr/prof` property before the independent `colr/nclx` property, Exif and XMP as separate `mdat` items, and one `cdsc` relationship from each metadata item to the primary color item. Exif stores the exact big-endian TIFF-header offset required by the HEIF item syntax; XMP uses the `mime` item type and `application/rdf+xml` content type. Existing ICC and XMP storage is read synchronously and copied once into final encoder storage rather than cloned into an intermediate array. `SkipMetadata` suppresses all three profile types while retaining the CICP values required to describe the encoded planes.
- [x] Exact container tests verify every emitted item declaration, name, MIME content type, `cdsc` relationship, Exif offset and payload, XMP payload, ICC/CICP property order, compact association byte, propertyless metadata exclusion, decoded profile value, and both `SkipMetadata` branches. The final HEIF encoder set passes 44 of 44 and the complete JPEG encoder set passes 257 of 257 through direct foreground net11 Release VSTest. The complete non-HEVC HEIF namespace passes 9,282 of 9,282 with no failure, crash, or detached test host, and current official libaom accepts all 47 current generated AV1 payloads.
- [x] Exact container tests verify every emitted item declaration, name, MIME content type, `cdsc` relationship, Exif offset and payload, XMP payload, ICC/CICP property order, compact association byte, propertyless metadata exclusion, decoded profile value, and both `SkipMetadata` branches. The final HEIF encoder set passes 44 of 44 and the complete JPEG encoder set passes 257 of 257 through direct foreground net11 Release VSTest. The complete non-HEVC HEIF namespace passes 9,282 of 9,282 with no failure, crash, or detached test host, and current official libaom accepts all 47 current generated AV1 payloads.
- [x] A code-wide production HEIF/AV1 stack-storage audit, excluding HEVC, removed every block-sized, variable-length, or repeatedly nested scratch buffer. Spatial luma and chroma, filter-intra, chroma-from-luma, luma and chroma palette selection, and K-means iteration now use typed views over 642 signed-integer elements, about 2.51 KiB, at the start of the shared inter-prediction region. Those searches are sequential for one block, so the block-workspace owner does not grow and no rent, copy, or additional lifetime is introduced. CDEF directions, variances, and its 64-entry block list now append 1 KiB to the existing bounded operation owner instead of occupying hidden inline or explicit stack arrays. No remaining `stackalloc` depends on block dimensions, sample count, or runtime length; the largest remaining individual span is 128 bytes, and the remaining sites are fixed syntax, SIMD-lane, filter-tap, plane-metadata, or small candidate storage. The exact-owner test now proves the mode, palette, and reference-prediction views share one allocation. Roslynk reports zero compiler errors and no diagnostics in the changed files, the Release test-project build completes with the established 1,992 warnings and zero errors, 81 of 81 focused cases pass, and the complete non-HEVC HEIF/AV1 namespace passes 9,282 of 9,282 through one foreground net11 VSTest run.
- [x] A code-wide production HEIF/AV1 stack-storage audit, excluding HEVC, removed every block-sized, variable-length, or repeatedly nested scratch buffer. Spatial luma and chroma, filter-intra, chroma-from-luma, luma and chroma palette selection, and K-means iteration now use typed views over 642 signed-integer elements, about 2.51 KiB, at the start of the shared inter-prediction region. Those searches are sequential for one block, so the block-workspace owner does not grow and no rent, copy, or additional lifetime is introduced. CDEF directions, variances, and its 64-entry block list now append 1 KiB to the existing bounded operation owner instead of occupying hidden inline or explicit stack arrays. No remaining `stackalloc` depends on block dimensions, sample count, or runtime length; the largest remaining individual span is 128 bytes, and the remaining sites are fixed syntax, SIMD-lane, filter-tap, plane-metadata, or small candidate storage. The exact-owner test now proves the mode, palette, and reference-prediction views share one allocation. Roslynk reports zero compiler errors and no diagnostics in the changed files, the Release test-project build completes with the established 1,992 warnings and zero errors, 81 of 81 focused cases pass, and the complete non-HEVC HEIF/AV1 namespace passes 9,282 of 9,282 through one foreground net11 VSTest run.
- [~] Bounded public image-sequence output now emits an `avis` movie with version-one movie, track, and media headers; AV1 visual sample entries; exact run-length-compressed timing; per-sample sizes; 64-bit chunk offsets; and an explicit sync-sample table. The file type includes the required `miaf` compatibility brand, and every sequence now has the MIAF primary image item emitted by current libavif: normal sequences share the first sync-sample extent without another encode or copy, while separate-root sequences retain the still root as the primary image and begin timed samples at frame index one. The decoder allocates one final `Image<TPixel>`: the root is either the first timed sample or the separately decoded primary item, and each visible timed sample is decoded directly into a frame owned by that image. Exact quarter-turn presentation uses one frame-sized reusable pre-rotation buffer rather than a second image or a separately built frame collection. Color and optional auxiliary alpha use independently configured AV1 tracks linked by `auxl`. Lossless samples remain independently decodable key pictures and repeat the sequence header required for random access. Lossy continuation samples use LAST_FRAME inter prediction through the existing SIMD translational predictor; one track-scoped encoder session reuses its source allocation, packed-to-planar row storage and color converter, frame-sized coefficient storage, fixed-geometry picture and frame-header syntax state, tile/superblock/entropy cursors, block arithmetic workspace, complete probability graph, bounded tile-output owner, and OBU-header owner, and swaps two complete reconstruction buffers so the preceding decoded frame becomes the next reference without a plane copy. Sequence samples signal `still_picture=0` and use the complete non-reduced sequence and frame-header prefixes required for a multi-frame coded sequence. Frame payloads are written once into contiguous allocator-backed chunks per track. One compact managed table retains only offset, length, and duration for both tracks, and the bounded `moov` owner is patched once after its final size is known, so prefixed and non-seekable destinations require neither seeking nor a file-sized copy. The media timescale uses the exact representable least common multiple of animated frame-delay denominators and a documented microsecond fallback; zero delays become the smallest legal positive duration. Public lossless color-and-alpha round trips preserve all frames, distinct 24, 25, and 30 fps delays, finite or infinite repetition, ICC, Exif, and XMP metadata, while a separate case proves prefixed non-seekable output. Per-tile CDEF preset, preceding-quantizer state, and payload bounds now occupy one aligned region in the reusable allocator-owned picture buffer rather than separate managed arrays for every frame. The last verified net11 Release checkpoint completed with the established 1,005 warnings and zero errors, all 48 HEIF encoder cases passed, and the complete non-HEVC HEIF/AV1 namespace passed 9,317 of 9,317 through foreground VSTest. Current official libaom `main` at `d565eec60f084421fa34fc0534b760c6452b6a6c` accepted all 66 raw AV1 payloads regenerated by that suite. Verification of the current primary-item, separate-root, non-reduced sequence-header, retained-reference continuation, grid implementation, block-local motion search, conversion-row, probability, picture, frame-header, tile-cursor, tile-output, and OBU-header reuse, and multi-tile output is pending. Additional reference roles and compound prediction remain open.
- [~] Bounded public image-sequence output now emits an `avis` movie with version-one movie, track, and media headers; AV1 visual sample entries; exact run-length-compressed timing; per-sample sizes; 64-bit chunk offsets; and an explicit sync-sample table. The file type includes the required `miaf` compatibility brand, and every sequence now has the MIAF primary image item emitted by current libavif: normal sequences share the first sync-sample extent without another encode or copy, while separate-root sequences retain the still root as the primary image and begin timed samples at frame index one. The decoder allocates one final `Image<TPixel>`: the root is either the first timed sample or the separately decoded primary item, and each visible timed sample is decoded directly into a frame owned by that image. Exact quarter-turn presentation uses one frame-sized reusable pre-rotation buffer rather than a second image or a separately built frame collection. Color and optional auxiliary alpha use independently configured AV1 tracks linked by `auxl`. Lossless samples remain independently decodable key pictures and repeat the sequence header required for random access. Lossy continuation samples use LAST_FRAME inter prediction through the existing SIMD translational predictor; one track-scoped encoder session reuses its source allocation, packed-to-planar row storage and color converter, frame-sized coefficient storage, fixed-geometry picture and frame-header syntax state, tile/superblock/entropy cursors, block arithmetic workspace, complete probability graph, bounded tile-output owner, and OBU-header owner, and swaps two complete reconstruction buffers so the preceding decoded frame becomes the next reference without a plane copy. Sequence samples signal `still_picture=0` and use the complete non-reduced sequence and frame-header prefixes required for a multi-frame coded sequence. Frame payloads are written once into contiguous allocator-backed chunks per track. One compact managed table retains only offset, length, and duration for both tracks, and the bounded `moov` owner is patched once after its final size is known, so prefixed and non-seekable destinations require neither seeking nor a file-sized copy. The media timescale uses the exact representable least common multiple of animated frame-delay denominators and a documented microsecond fallback; zero delays become the smallest legal positive duration. Public lossless color-and-alpha round trips preserve all frames, distinct 24, 25, and 30 fps delays, finite or infinite repetition, ICC, Exif, and XMP metadata, while a separate case proves prefixed non-seekable output. Per-tile CDEF preset, preceding-quantizer state, and payload bounds now occupy one aligned region in the reusable allocator-owned picture buffer rather than separate managed arrays for every frame. The last verified net11 Release checkpoint completed with the established 1,005 warnings and zero errors, all 48 HEIF encoder cases passed, and the complete non-HEVC HEIF/AV1 namespace passed 9,317 of 9,317 through foreground VSTest. Current official libaom `main` at `d565eec60f084421fa34fc0534b760c6452b6a6c` accepted all 66 raw AV1 payloads regenerated by that suite. Verification of the current primary-item, separate-root, non-reduced sequence-header, retained-reference continuation, grid implementation, block-local motion search, conversion-row, probability, picture, frame-header, tile-cursor, tile-output, and OBU-header reuse, and multi-tile output is pending. Additional reference roles and compound prediction remain open.
The 2026-09-05 measurements below are historical results from earlier trees. The takeover source audit has not
accepted encoder or decoder completeness. The first decoder comparison following that audit's corrections ran
on 2026-09-07; its current results and limitations are recorded in the implementation plan.
Encoder comparisons require identical source samples and explicitly reconciled settings, with a maximum
component difference of one unit. Report component maxima and counts exceeding one.
Decoder comparisons for identical bitstreams must be byte exact: maximum error zero and zero differing samples.
Historical Y/U/V maxima of 40/30/53 fail that requirement; their exceedance counts were not recorded.
Agreement between two decoders on the same bitstream is a separate claim from agreement between two encoders.
`Av1SequenceEncoderBenchmarks` compares the managed AV1 sequence encoder with an optimized build of official libaom `main`.
`Av1SequenceEncoderBenchmarks` compares the managed AV1 sequence encoder with an optimized build of official libaom `main`.
The native adapter belongs only to this benchmark project. ImageSharp production code remains fully managed.
The native adapter is temporary local comparison tooling already present in this project. It is excluded from the
codec deliverable and future checkpoint commits. ImageSharp production code remains fully managed.
`Av1DecoderBenchmarks` compares complete AV1 elementary-stream decoding to `Rgb48` against that same optimized native build.
It includes decoder construction, parsing, reconstruction, output allocation, color conversion, and disposal on both sides.
File reads and correctness checks are setup work. HEIF container parsing is outside this elementary-stream comparison.
Native output planes are borrowed only during synchronous conversion through the existing HEIF converter; no plane copy or per-row interop call is added.
Setup compares every packed RGB sample with ImageSharp and fails before timing on any disagreement.
## Measurement boundary
## Measurement boundary
@ -15,6 +31,7 @@ MemoryStream capacity is retained between operations for both methods. These are
Do not compare them with `aomenc`'s internal encode-time report, which excludes the conversion boundary.
Do not compare them with `aomenc`'s internal encode-time report, which excludes the conversion boundary.
The source is the existing `TestImages.Png.Bike` photograph. Frames one and two are independently translated by half a pixel and one pixel on both axes.
The source is the existing `TestImages.Png.Bike` photograph. Frames one and two are independently translated by half a pixel and one pixel on both axes.
Encoder setup also checks all three encoded photographic frames with independent ImageSharp and libaom decoder contexts, including their retained references.
The parameter matrix contains 256x256 and 512x512 frames at ImageSharp efforts seven, eight, and nine.
The parameter matrix contains 256x256 and 512x512 frames at ImageSharp efforts seven, eight, and nine.
Input is full-range BT.601, eight-bit 4:2:0; each sequence contains one key frame and two dependent frames.
Input is full-range BT.601, eight-bit 4:2:0; each sequence contains one key frame and two dependent frames.
Both codecs use one coding thread. Libaom uses good-quality mode, no lookahead, disabled automatic key frames, and `cpu-used=6`.
Both codecs use one coding thread. Libaom uses good-quality mode, no lookahead, disabled automatic key frames, and `cpu-used=6`.
@ -135,3 +152,100 @@ Final OBU SHA-256 values:
The final Release benchmark build has zero errors and 39 existing benchmark warnings outside the new files.
The final Release benchmark build has zero errors and 39 existing benchmark warnings outside the new files.
Roslyn compiler and analyzer diagnostics contain no errors or warnings in the new benchmark files.
Roslyn compiler and analyzer diagnostics contain no errors or warnings in the new benchmark files.
No production file changed in this benchmark checkpoint; the preceding 2,524-case encoder/entropy verification remains the latest production test run.
No production file changed in this benchmark checkpoint; the preceding 2,524-case encoder/entropy verification remains the latest production test run.
## Decoder baseline and sparse-transform checkpoint: 2026-09-05
Run the two existing conformance inputs with the same in-process toolchain:
Both complete `Rgb48` outputs match libaom exactly. Initial warmed measurements were:
| Input | ImageSharp | Libaom |
| --- | ---: | ---: |
| `libavif-kodim23-8b.bit` | 15.130 ms | 5.423 ms |
| `libavif-cosmos1650-10b.bit` | 27.655 ms | 10.246 ms |
The lossy inverse dispatcher previously ignored EOB and transformed every coefficient, including DC-only blocks.
The first sparse path now evaluates the two DC cosine stages once, preserving rectangular normalization, axis rounding, input clamps, and final clipping.
The existing reconstruction output operators handle Vector512, Vector256, Vector128, and scalar stores without an extra sample buffer.
After this change the short measurements were 14.658/5.416 ms for the eight-bit pair and 26.722/9.992 ms for the ten-bit pair.
These small changes are not a statistically established decoder speedup; the approximately 2.7x gap remains open.
The encoder's measured output at this checkpoint remains byte-identical to the baseline hash above.
Evidence: `artifacts/BenchmarkDotNet/av1-decoder-rgb-short-20260905` and `artifacts/BenchmarkDotNet/av1-dc-short-20260905/20260905-134350`.
DC regression cases compare the sparse and full transforms for every size, both destination layouts, signed rounding boundaries, clipping, and untouched row padding.
The final focused screening/DC/native-profile/moving-color set passes 25 cases in each of the normal, AVX512-disabled, AVX-disabled, and all-intrinsics-disabled VSTest processes.
Reports are under `artifacts/TestResults/av1-screen-20260905`.
The matching optimized current-main decoder also reproduces every native Y/U/V sample in all twelve two-frame moving-color streams regenerated by this tree.
## Historical intra screening experiment: 2026-09-05
The current decoder-only comparison after source-led corrections is recorded at the end of this document.
The experimental fixed 8x8 luma search imported libaom's unnormalized Hadamard magnitude and 1.5x threshold
without its full candidate ordering, top-ranked list, neighbor/quantizer policy, or surrounding search controller.
The implementation reuses block scratch and existing vectorized tensor/transpose APIs, retains Int32 precision for twelve-bit residuals, and omits coefficient permutations irrelevant to SATD.
Lossless search is unchanged. Screening for larger blocks, top-ranked candidate pruning, transform bounds, and winner refinement remain open.
The first three-iteration run measured 1,437.53 ms versus 82.34 ms, with considerable run variance.
The managed output is 11.577 KiB at Y/U/V PSNR 38.942/38.613/32.755 dB and aggregate 37.069 dB.
Libaom remains 8.272 KiB at aggregate 38.942 dB. Both decode to three complete frames.
This experiment is not an accepted improvement: aggregate PSNR fell 0.055 dB and the payload grew slightly.
Neither its isolated primitive tests nor these timing results validate the imported pruning policy.
The exact photographic sequence also passes independent packed-RGB decoder comparison before either timed method runs.
A repeat with five warmup iterations and ten measurement iterations gives 1,363.09 ms for ImageSharp and 71.73 ms for libaom.
The payload and quality are unchanged from the screening result above. The native baseline is stable relative to the original run;
the managed reduction is about 34%, but the remaining encoding gap is still approximately 19x.
James Jackson-South
3 weeks ago
