37 KiB
AVIF and AV1 implementation plan
Goal
Complete a production-quality, fully managed AV1 codec and its bounded AVIF/HEIF image container integration for ImageSharp. The finished work must decode and encode still images and bounded image sequences, preserve source precision, use ImageSharp memory ownership, and provide SIMD-first hot paths with one behaviorally identical scalar fallback.
This plan is the authoritative delivery checklist. A source file, unit test, build, self-roundtrip, or local implementation is not completion evidence by itself.
Source authority
- AV1 codec syntax, tables, fixed-point arithmetic, prediction, transforms, entropy behavior, filters, encoder decisions, and lifecycle behavior must be ported and checked only against the current
mainbranch of the official libaom checkout atD:\GitHub\AOMediaCodec\aom. - Libaom is the sole external codec implementation source. Do not use HM, libheif, FFmpeg, GPAC, SVT-AV1, dav1d, libgav1, or any other codec implementation as an algorithm, arithmetic, output, or architecture reference.
- Existing ImageSharp and JPEG code is authoritative only for ImageSharp architecture, allocator ownership, SIMD dispatch, pixel conversion, and test API patterns. It is not an alternate AV1 algorithm source.
- Production code must not load, invoke, install, or fall back to a native codec.
- Existing independent container files may be used only as interoperability inputs. Native AV1 expected output must be generated by the current libaom
maincheckout, and no independent decoder output may substitute for it.
Reference checkout evidence on 2026-08-31:
D:\GitHub\AOMediaCodec\aomis attached tomain, clean, and aligned withorigin/mainafter a fresh fetch.- Both
HEADandorigin/mainresolved to441c439b9916474cac15d2822af47a9ad70674a8. This records the tree audited on that date; it is not a pin and must not prevent later work from updating to the then-currentmain.
Status notation
- Verified: the current behavior has exact evidence from the current libaom
maintree and the evidence proves the production contract. - [~] Locally implemented, checkpoint open: production source exists, but current-tree verification is missing or a known audit issue invalidates the checkpoint.
- Remaining: the production behavior is absent, incomplete, or has not reached its required implementation boundary.
Current source reconciliation
Reconciled with the worktree on 2026-08-31.
- [~] The bounded container reader, still-image path, sequence parser, AV1 decoder, color pipeline, presentation pipeline, and broad AV1 test suite exist locally.
- [~] The inter-frame decoder contains implementations for single-reference prediction, compound references, inter-intra prediction, selectable compound blending, OBMC, scaled references, local warped motion, and global motion. These downstream paths must not be called verified until the single-reference checkpoint below is corrected and rerun.
- [~] Loop filtering, CDEF, super-resolution, restoration, film grain, layered presentation, alpha composition, and color conversion exist locally. Shared-source cleanup changed the current tree, so final production-path verification is open.
- [~] AV1 writer primitives, forward transforms, symbol encoding, and tile-writing source exist locally, but they are not connected to the public encoder.
- The public AV1 encoder is not implemented. HeifEncoderCore.Encode throws NotSupportedException when AV1 is selected.
- Patented codec production code, registrations, tests, benchmarks, fixtures, reference outputs, and notices were manually deleted and committed by
78a74d448. - Remaining task-created HM, HEVC, libheif, GPAC, Nokia, FFmpeg, Pillow HEIF, libavif-build, and libjpeg-build directories were traced to their creation commands in the recovered Codex session history and deleted on 2026-08-31. The user-provided repositories and all libaom-only source, build, and reference data were left untouched.
- [~] A PNG metadata-suppression fix and three HEIF/AV1 diagnostic-save call-site corrections are implemented in the current worktree. The exact 34 cases that failed in the net11.0 ARM CI job now pass in Release, but these changes are not yet committed and are not decoder or encoder completion evidence.
- The complete decoder and encoder release matrix is not complete.
Immediate execution queue
Work must proceed in this order. Do not skip to a later item while an earlier checkpoint is open.
1. Finish and verify the AV1-only cleanup
- Remove production types, registrations, constants, parser branches, properties, tests, benchmarks, fixtures, reference outputs, notices, and documentation for removed codec work.
- Remove downloaded non-libaom reference source, tools, generated outputs, and local installations.
- Retain the official current-main libaom checkout and libaom-only build artifacts required for AV1 verification.
- Retain user-supplied AV1 fixtures and their recorded expected outputs.
- Audit production source, tests, benchmarks, assets, project files, notices, and documentation for stale removed-code references.
- [~] The prior cleanup tree built in Release with restore disabled, build servers disabled, and one MSBuild node. A fresh current-worktree build remains required after the local PNG cICP fix.
- [~] The prior focused AV1/container set passed, but it missed the net11.0 PNG cICP diagnostic-save failures. The exact 34 CI failures now pass after the local fix; the final current-worktree checkpoint set remains required.
- [~] Roslynk currently reports zero compiler errors for the local fix, and
git diff --checkpasses. Scoped StyleCop and final current-worktree inspection remain required. - [~] Record the final current-worktree cleanup and cICP evidence after the checkpoint verification completes.
Historical cleanup evidence from 2026-08-30, retained with its limitation:
- Release source builds passed for net10.0 and net11.0 with zero warnings and zero errors. Both builds used
--no-restore,--disable-build-servers, and one MSBuild node. - The focused net10.0 HEIF decoder, encoder, metadata, sequence-parser, and AV1 reconstruction set passed 221 of 221 tests with zero failures and zero skips. It did not execute the net11.0 diagnostic-save path that later failed in CI.
- The Roslyn compiler and configured StyleCop analyzers accepted the changed production source. Roslynk's
open_solutionentry point was attempted separately but failed before returning a solution handle, so no Roslynk result is claimed. - The tracked-source text and filename audit found no removed-code references outside the unchanged repository and shared-infrastructure
.gitattributespatterns. A later history reconstruction found ignored task-created reference directories that this audit missed; those directories were deleted on 2026-08-31. git diff --checkpassed and neither.gitattributesfile changed.
Current cICP failure correction evidence from 2026-08-31:
- The failure was not decoded HEIF metadata.
PngEncoderCore.WriteCicpChunkignoredPngChunkFilter.ExcludeAll, so diagnostic PNG saves attempted to write a non-identity source matrix that PNG cannot represent. PngEncoderCorenow honors the existingSkipMetadatacontract for cICP, and the three affected HEIF/AV1 diagnostic saves explicitly usePngEncoder { SkipMetadata = true }. Actual comparisons and decoded-image metadata assertions remain unchanged.- The direct embedded-ICC case and every row of the 12-case profile matrix passed: 13 of 13 net11.0 Release cases.
- The exact 34 cases reported by CI passed: 34 of 34 net11.0 Release cases, with zero failures and zero skips.
- Roslynk reported zero compiler errors after the fix, and
git diff --checkpassed.
Recovered task-history evidence from 2026-08-31:
- The primary session beginning on 2026-08-24 was reopened from task ID
01a03239-831b-7831-84e7-7f6947279ccb: 96,777 records, 295 turn contexts, 211 compactions, 190 user messages, 1,920 assistant messages, and 13,671 tool calls. - The continuation beginning on 2026-08-27 was reopened from task ID
01a04314-f1c6-7133-b1bc-5c74a94dd714: 61,129 records at the audit point, 166 turn contexts, 96 compactions, 223 user messages, 1,113 assistant messages, and 8,942 tool calls. - The restored first session records the user selecting official AOM/libaom as the AV1 source after the ImageSharp discussion was inspected. It does not authorize another codec implementation as an AV1 source and does not authorize importing a patented codec.
- The restored tool calls identify the exact creation commands for the non-libaom source, tool, and output directories removed on 2026-08-31. No directory was selected for deletion from its name alone.
- The recovered Git sequence establishes that
78a74d448removed the patented codec implementation and92fa7a8camerged the later upstream ImageSharp changes. The current branch and worktree, not an older summary, remain authoritative.
2. Correct the single-reference inter-frame checkpoint
The checkpoint implementation is complete. The first two inherited audit findings were rechecked against current libaom main and did not require production changes; the remaining production boundary, reconstruction, and ownership work is now implemented and verified.
- Reconcile interpolation-filter syntax in
Av1TileReaderwith current libaommain.- Current libaom
av1_is_interp_neededcallsis_nontrans_global_motion, whose loop rejects onlyTRANSLATION. Identity GLOBALMV therefore omits switchable-filter symbols. - Current
Av1TileReaderuses the same non-Translation classification. The existing Identity test leaves sentinel filter symbols unread, while the Translation test consumes them. - No production change is required. The stale test comment was changed from “pinned” to “current” libaom.
- Current libaom
- Reconcile both spatial single-reference extension loops in
Av1ReferenceMotionVectorswith current libaommain.- Current libaom
setup_ref_mv_liststops both loops atMAX_MV_REF_CANDIDATES, which is two.MAX_REF_MV_STACK_SIZE, which is eight, is the stack capacity used by the earlier direct and temporal candidate collection; it is not the stop condition for these two extension loops. - Current
Av1ReferenceMotionVectorsuses the same two-entry stop condition and retains an eight-entry stack for earlier candidates and DRL selection. - No production change is required. This remains spatial single-reference extension, not temporal extension.
- Current libaom
- Establish and enforce the contiguous frame-plane invariant used by
Av1FrameBufferand inter reconstruction.- Every frame plane is allocated with
preferContiguosImageBuffers: true, so a constrained allocator cannot split a representable padded plane into normal memory groups. Av1FrameBuffernow rejects an external frame geometry whose padded plane reaches theint.MaxValuefallback boundary before any allocation. This makes every directDangerousGetSingleSpancall an enforced owner invariant rather than a memory-group accident.ConstructorRequestsContiguousPaddedPlanesproves that a plane larger than the allocator's group capacity is one group.ConstructorRejectsPaddedPlaneThatCannotBeContiguousproves that an unrepresentable plane is rejected before allocation.- The production path performs no plane copy and no per-block, per-row, or per-scanline allocation.
- Every frame plane is allocated with
- Prove the real
Av1BlockDecoder.DecodeBlockinter-reconstruction branch.- Decode the progressive dependent-frame fixture through the complete public production path.
- Compare the final frame's native Y, Cb, and Cr planes exactly with current-main libaom output.
- Compare the final presented image through the established ImageSharp reference-image comparison API.
- Do not substitute an internal helper test, fake tile reader, non-zero assertion, custom pixel loop, or tolerant comparison.
- Prove motion-field ownership and lifetime.
- Track initialization, retained-slot aliases, failure unwinding, presentation ownership, decoder-result ownership, and final disposal.
- Every allocator-owned object must be returned exactly once.
- Correct stale documentation.
- Av1InterFrameModeInfoTests must describe the behavior it actually proves.
- Do not claim production reconstruction, constrained allocation, ownership, or reference-stack coverage unless the test executes that contract.
Checkpoint gate:
- Default Identity-omission and Translation-consumption GLOBALMV syntax cases pass in the focused current-tree run.
- Two-entry spatial single-reference extension passes; current-main source inspection confirms the separate eight-entry overall stack capacity and DRL access.
- The exact dependent-frame native-plane comparison passes.
- The established exact presentation comparison passes.
- Normal, AVX-512-disabled, AVX-disabled, and scalar FeatureTestRunner configurations pass where supported.
- Constrained allocation preserves the enforced single-group plane invariant without copying or per-block allocation.
- Motion-field allocation tracking is balanced across success and failure.
- Release builds for net10.0 and net11.0 pass with zero errors.
- Focused Release tests pass with zero failures or skips.
- Scoped semantic, StyleCop, whitespace, and git diff checks pass.
- The completed checkpoint was committed as
54bb6cbe59bd113058854a3ee31448cf61f462cawith author and committerJames Jackson-South <james_south@hotmail.com>.
Verified single-reference checkpoint evidence on 2026-08-31:
- The current-main
aomdecwas rebuilt directly fromD:\GitHub\AOMediaCodec\aomand identified itself as3.15.0-13-g441c439b99. - Decoding the 72-byte progressive payload with
--all-layers, one thread, and row multithreading disabled produced 2,178 YUV444 color samples. All samples in both layers match the first three planes of the stored YUV444-alpha reference exactly. DecodeProgressiveSingleReferenceMatchesPinnedReferencesexecutes the production decoder through FeatureTestRunner and compares the complete presentedRgba32image withCompareToReferenceOutput(ImageComparer.Exact, provider). The redundant manual alpha loop was removed.DecodeProgressiveSingleReferenceWithConstrainedAllocatorexecutes the same production reconstruction with a 1,024-byte allocator group capacity and verifies that every allocation is returned exactly once.MotionFieldsFollowReferenceAliasesAndPresentationOwnership,MotionFieldAllocationFailureUnwindsTileReaderOwnership,DecodeProgressiveSingleReferenceTracksMotionFieldResultOwnership, and the reference-store replacement, reset, and transfer tests cover initialization, aliases, presentation ownership, decoder-result ownership, failure unwinding, repeated disposal, and exactly-once final returns.- The focused Release set passed 17 of 17 tests on net10.0 and 17 of 17 tests on net11.0, with zero failures and zero skips. This includes both GLOBALMV syntax cases, spatial extension, plane invariants, production reconstruction, FeatureTestRunner dispatch, and ownership.
- Release source builds passed for net10.0 and net11.0 with zero warnings and zero errors.
- Roslynk reported zero compiler errors. The scoped changed-file analyzer inspection reported no StyleCop diagnostics attributable to this checkpoint; its only remaining match is the pre-existing xUnit cancellation warning in an unrelated
HeifDecoderTestsmethod. git diff --checkpassed, and neither.gitattributesfile changed.
Exact verification commands, run directly in the foreground from D:\GitHub\ynse01\ImageSharp:
$env:MSBUILDUSESERVER = '0'
$env:DOTNET_CLI_USE_MSBUILD_SERVER = '0'
$env:DOTNET_CLI_HOME = 'D:\GitHub\ynse01\ImageSharp\.dotnet'
$env:DOTNET_SKIP_FIRST_TIME_EXPERIENCE = '1'
$env:DOTNET_CLI_TELEMETRY_OPTOUT = '1'
$env:DOTNET_DbgEnableMiniDump = '0'
$env:COMPlus_DbgEnableMiniDump = '0'
$env:DOTNET_EnableCrashReport = '0'
$env:COMPlus_EnableCrashReport = '0'
$heifCheckpointFilter = 'FullyQualifiedName~Av1InterFrameModeInfoTests.ReadInterFrameModeInfoReadsInterpolationFilters|FullyQualifiedName~Av1InterFrameModeInfoTests.ReadInterFrameModeInfoOmitsInterpolationFiltersForIdentityGlobalMotion|FullyQualifiedName~Av1ReferenceMotionVectorsTests.BuildReversesOppositeDirectionExtensionCandidate|FullyQualifiedName~Av1FrameBufferTests|FullyQualifiedName~Av1ReferenceFrameStoreTests.MotionFieldsFollowReferenceAliasesAndPresentationOwnership|FullyQualifiedName~Av1ReferenceFrameStoreTests.MotionFieldAllocationFailureUnwindsTileReaderOwnership|FullyQualifiedName~Av1ReferenceFrameStoreTests.PartialReplacementPreservesSharedOwner|FullyQualifiedName~Av1ReferenceFrameStoreTests.FinalReplacementReleasesDisplacedOwner|FullyQualifiedName~Av1ReferenceFrameStoreTests.ResetReleasesUniqueOwnersAndClearsSlots|FullyQualifiedName~Av1ReferenceFrameStoreTests.TakeOutputTransfersPlanesAndReleasesOtherReferences|FullyQualifiedName~Av1ReconstructionConformanceTests.DecodeProgressiveSingleReferenceMatchesPinnedReferences|FullyQualifiedName~Av1ReconstructionConformanceTests.DecodeProgressiveSingleReferenceWithConstrainedAllocator|FullyQualifiedName~Av1ReconstructionConformanceTests.DecodeProgressiveSingleReferenceTracksMotionFieldResultOwnership'
dotnet build src\ImageSharp\ImageSharp.csproj -c Release -f net10.0 --no-restore --disable-build-servers -m:1 --no-incremental --nologo --verbosity:minimal
dotnet build src\ImageSharp\ImageSharp.csproj -c Release -f net11.0 --no-restore --disable-build-servers -m:1 --no-incremental --nologo --verbosity:minimal
dotnet test tests\ImageSharp.Tests\ImageSharp.Tests.csproj -c Release -f net10.0 --no-restore --disable-build-servers -m:1 --filter $heifCheckpointFilter --logger 'console;verbosity=minimal'
dotnet test tests\ImageSharp.Tests\ImageSharp.Tests.csproj -c Release -f net11.0 --no-restore --disable-build-servers -m:1 --filter $heifCheckpointFilter --logger 'console;verbosity=minimal'
$aomVcVars = 'C:\Program Files (x86)\Microsoft Visual Studio\18\BuildTools\VC\Auxiliary\Build\vcvars64.bat'
$aomCmake = 'C:\Program Files (x86)\Microsoft Visual Studio\18\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe'
$aomEnvironment = & cmd.exe /d /s /c "`"$aomVcVars`" >nul && set"
foreach ($aomEntry in $aomEnvironment)
{
$aomParts = $aomEntry -split '=', 2
if ($aomParts.Length -eq 2)
{
[Environment]::SetEnvironmentVariable($aomParts[0], $aomParts[1], 'Process')
}
}
& $aomCmake --build artifacts\reference\aom-generic --target aomdec --config Release --parallel 1
& 'artifacts\reference\aom-generic\aomdec.exe' --codec=av1 --rawvideo --all-layers --threads=1 --row-mt=0 --output='artifacts\reference\aom-generic\progressive-current-main-all-layers.yuv' 'tests\Images\Input\Heif\Av1\Conformance\libavif-progressive-draw-points-8b.bit'
3. Reverify downstream inter prediction in recorded order
The single-reference syntax, buffer, reconstruction, and ownership foundation is verified by 54bb6cbe59bd113058854a3ee31448cf61f462ca. Reverify the existing downstream implementations in this exact order, treating each as locally implemented but unverified until its current-main evidence is recorded.
- Compound reference selection, paired reference-MV derivation, and equal averaging.
- Inter-intra prediction.
- Distance-weighted compound prediction.
- [~] Wedge compound prediction. Current item.
- [~] Difference-weighted compound prediction.
- [~] OBMC.
- [~] Scaled-reference prediction.
- [~] Local warped prediction.
- [~] Non-translational global prediction.
- [~] Inter deblocking decisions and reference/mode deltas.
Verified equal-average compound checkpoint evidence on 2026-08-31:
- Refreshed the clean official libaom
maincheckout and audited the observed revision441c439b9916474cac15d2822af47a9ad70674a8. Reference selection and compound mode syntax matchread_comp_reference_typeandread_ref_framesinav1/decoder/decodemv.c; contexts matchav1/common/pred_common.c; paired reference-MV construction and eight-entry extension matchprocess_compound_ref_mv_candidateandsetup_ref_mv_listinav1/common/mvref_common.c. - Audited equal-average reconstruction against
av1/common/convolve.candav1/common/convolve.h. Corrected the unscaled 10/12-bit translational path so both references retain libaom's no-round compound intermediates until the sole final average and clipping step, including the larger first-round shift required for 12-bit horizontal intermediates. - Added descending Vector512, Vector256, Vector128, and scalar high-bit-depth traversal to the existing semantic compound-prediction operator families. No per-block, per-row, or per-scanline allocation or copy was added.
- Added FeatureTestRunner coverage for 10/12-bit copy, horizontal, vertical, and separable subpixel prediction at widths 9, 17, 33, and 65, with an independent no-round bilinear oracle, row-padding sentinels, and explicit scalar comparison.
- Added a complete
Av1BlockDecoder.DecodeBlock10/12-bit half-sample regression whose expected result comes from the scalar no-round pipeline. The selected vector differs by one sample from the obsolete round-each-reference behavior, so the test proves the production branch selection. - Refreshed the official libaom
mainremote immediately before verification and decoded the fixture's 5,465-byte AV1mdatpayload with currentaomdec, one thread and row threading disabled. All 19 frames decoded; the final 19,200 YUV444 samples have SHA-256E79D2F49C260B1AC9B1B9BBBB2D611126AFD3B241DA389EB9E7BD4EA0ED42080and match the retained native reference with zero differing samples. - The real 19-frame production sequence requires decoded equal-average compound blocks, compares the final native Y, U, and V planes exactly, compares final RGBA presentation through ImageSharp's established reference-output API, and repeats the complete decode with a 1,024-byte constrained tracked allocator and exactly-once return checks.
- The focused Release checkpoint set passes 31/31 on net10.0 and 31/31 on net11.0, with zero
failures or skips. Scoped analyzer and whitespace verification pass for every changed C# file,
Roslynk reports zero compiler errors and no diagnostics in the changed files, and
git diff --checkpasses..gitattributesis unchanged. - The completed checkpoint was committed as
4075a0844836e863a93cb2e2f3ca42d202c7df1bwith author and committerJames Jackson-South <james_south@hotmail.com>.
Verified inter-intra checkpoint evidence on 2026-08-31:
- Audited syntax against current libaom
av1/decoder/decodemv.candav1/common/blockd.h. ImageSharp applies the same sequence enable, skip-mode, block-size, and single-reference gates, reads the same four-mode CDF, and reads wedge syntax only within libaom's wedge-supportedBLOCK_8X8throughBLOCK_32X32range. - Audited reconstruction against
ii_weights1d,ii_size_scales,build_smooth_interintra_mask, andcombine_interintrain currentav1/common/reconinter.c. The ImageSharp weights, plane-size scaling, smooth-mask direction, complemented destination orientation, wedge sign, subsampling, and final 6-bit blend match. No production change was required. - The mask tests cover all four inter-intra modes, complemented orientation, row-padding
sentinels, and the 32-wide curve. FeatureTestRunner covers byte and high-bit-depth selectable
blending under SIMD and scalar dispatch, and complete
Av1BlockDecoder.DecodeBlocktests execute smooth inter-intra reconstruction at 8, 10, and 12 bits. - Extracted the fixture's 5,327-byte AV1
mdatpayload and decoded it with the refreshed current libaomaomdec, using one thread with row threading disabled. All 19 frames decoded. The final 19,200 YUV444 samples have SHA-256E8B776C2751DC30CA838931A4B74535FC6E681179568A1278747A38CFF2E5BFAand match the retained native reference with zero differing samples. - The real production sequence requires both smooth and wedge inter-intra blocks, compares the final native Y, Cb, and Cr planes exactly, and compares final RGBA presentation through ImageSharp's established reference-output API. Its constrained 1,024-byte tracked-allocator run proves motion-field allocation and exactly one return for every allocation.
- The focused Release checkpoint set passes 50/50 on net10.0 and 50/50 on net11.0, with zero
failures or skips. Scoped analyzer and whitespace verification pass for both changed C# files.
Roslynk reports zero compiler errors and no diagnostics in the changed files,
git diff --checkpasses, and.gitattributesis unchanged. - The completed checkpoint was committed as
18b1c881271a3494489ca6f410ab140544902e2dwith author and committerJames Jackson-South <james_south@hotmail.com>.
Verified distance-weighted compound checkpoint evidence on 2026-08-31:
- Audited reference-distance quantization against
quant_dist_weightandquant_dist_lookup_tablein current libaomav1/common/common_data.h, and audited order-hint distance selection and forward/backward reference assignment againstav1_dist_wtd_comp_weight_assigninav1/common/reconinter.c. - Audited reconstruction against current libaom
av1/common/convolve.c. Corrected the production 10/12-bit subpixel path, which incorrectly finalized its two no-round compound intermediates with an equal average instead of the signaled distance weights. The fixed path applies libaom's 4-bit weighted shift before bias removal, final rounding, and clipping. - Added descending Vector512, Vector256, Vector128, and scalar traversal to the existing semantic distance-weighted intermediate predictor family. Unsigned widening preserves the biased 12-bit intermediate range. No per-block, per-row, or per-scanline allocation or copy was added.
- Added FeatureTestRunner coverage for every current-libaom distance-weight class in both reference orders, and for 10/12-bit copy, horizontal, vertical, and separable subpixel prediction at widths 9, 17, 33, and 65, with an independent no-round oracle and row-padding sentinels.
- Added a complete
Av1BlockDecoder.DecodeBlock10/12-bit half-sample regression that selects the 13:3 distance weights through real order hints. Its first reconstructed sample differs from the old equal-average result, so the test proves the corrected production branch is executed. - Extracted the fixture's 5,372-byte AV1
mdatpayload and decoded it with the refreshed current libaomaomdec, using one thread with row threading disabled. All 19 frames decoded. The final 19,200 YUV444 samples have SHA-256E8CAA650F1571C5B9CACAF8C06E1DDF5F5D2ED35F65F1C34377076C573425899and match the retained native reference with zero differing samples. - The real 19-frame production sequence requires decoded distance-weighted compound blocks, compares the final native Y, Cb, and Cr planes exactly, compares final RGBA presentation through ImageSharp's established reference-output API, and repeats the complete decode with a 1,024-byte constrained tracked allocator and exactly-once return checks.
- The focused Release checkpoint set passes 44/44 on net10.0 and 44/44 on net11.0, with zero failures
or skips. Scoped analyzer and whitespace verification pass for every changed C# file. Roslynk reports
zero compiler errors and no diagnostics in the changed files,
git diff --checkpasses, and.gitattributesis unchanged.
For every item:
- Trace syntax and arithmetic to the current libaom
maintree. - Execute the real production decoder path.
- Compare native planes exactly.
- Compare presentation through the established reference-image API.
- Run constrained allocator and exactly-once ownership coverage.
- Run FeatureTestRunner for SIMD and scalar dispatch when the implementation has SIMD.
- Record focused Release evidence before marking the item verified.
4. Close AV1 decoder coverage
Previously verified algorithm checkpoints remain valuable evidence, but the final decoder gate requires a fresh current-tree run after the inter and cleanup corrections.
- [~] Bounded OBU framing, sequence headers, frame headers, tile groups, alignment, and trailing-bit parsing have historical checkpoint evidence against an obsolete pinned tree. Re-audit the current libaom
mainimplementation before restoring verified status. - [~] Partition traversal, mode information, segmentation, delta quantization, transform-size selection, coefficient decoding, inverse quantization, and inverse transforms have historical checkpoint evidence against an obsolete pinned tree. Re-audit the current libaom
mainimplementation before restoring verified status. - Intra prediction covers directional, DC, smooth, Paeth, chroma-from-luma, filter-intra, and palette families with the established operator architecture.
- Intra-block copy has exact native reconstruction and feature-isolated SIMD evidence.
- Lossless inverse transform, loop filtering, CDEF, super-resolution, restoration, and film grain have focused checkpoint evidence.
- [~] Retained references, CDF snapshots, segmentation maps, global motion, temporal motion fields, and dependent-frame lifecycle exist locally and require current-tree re-verification.
- [~] All-intra and dependent-frame profile fixtures exist for 8, 10, and 12-bit monochrome, 4:2:0, 4:2:2, and 4:4:4 paths.
- Re-run the exact current-tree native-plane matrix through the production decoder.
- Re-run the exact current-tree presentation matrix through ImageSharp's established comparison API.
- Verify malformed/truncated data, frame IDs, reference slots, tile bounds, allocation limits, cancellation, and failure unwinding.
- Verify still items and bounded sequences from file, memory, non-seekable, and short-read streams.
- Verify ICC, CICP, alpha, grids, pixel aspect ratio, clean aperture, rotation, mirroring, metadata, and every presented sequence frame.
- Complete the public AVIF format/API review so registered capabilities match implemented behavior.
- Remove or reject every valid in-scope AV1 syntax branch that remains silently ignored or unsupported.
Decoder exit gate:
- Every supported native format and AV1 tool has exact current-main libaom production-path evidence.
- Every supported presentation behavior has established reference-image evidence at the correct output precision.
- No decoder path relies on a native codec, copied plane, per-block allocation, or contiguous memory-group accident.
- All allocator ownership is deterministic and exactly once.
- Full focused Release verification is recorded with no false coverage claims.
AV1 encoder implementation
Writer primitives are not an encoder. The public encoder remains incomplete until it produces independently decodable AV1 payloads and AVIF containers for every exposed option.
5. Define and enforce the encoder contract
- Finalize observable options for quality, effort, lossless mode, bit depth, chroma subsampling, alpha quality, metadata, and bounded sequences.
- Preserve high-bit-depth source precision through 16-bit RGB and native 10/12-bit component planes.
- Reject unsupported combinations at the public boundary before writing output.
- Register only capabilities that the completed encoder proves.
6. Build the complete AV1 frame encoder
- [~] SIMD-first RGB-to-native-plane conversion exists locally.
- [~] Forward transform families and transform workspace exist locally.
- [~] Symbol writer, coefficient writer, and tile writer fragments exist locally.
- Connect a frame-owned encoder lifecycle using ImageSharp allocators and pools.
- Write compliant temporal delimiter, sequence header, frame header, tile group, metadata, and padding OBUs as required.
- Implement superblock and partition analysis for every permitted block size and partition.
- Implement intra mode search, chroma mode search, palette, filter intra, chroma-from-luma, and intra-block copy decisions.
- Implement inter mode search for bounded sequences, including reference selection and the decoder-supported inter tools.
- Implement transform-size/type search, forward transform, quantization, coefficient optimization, and lossless behavior.
- Implement real rate-distortion selection and make quality and effort change work, size, and output quality.
- Implement tile-local entropy coding and CDF update behavior.
- Implement legal deblocking, CDEF, restoration, super-resolution, and film-grain signaling decisions.
- Remove per-transform and per-block managed allocations from active encoder paths.
- Use descending SIMD dispatch: Vector512, Vector256, Vector128, then scalar.
- Verify every SIMD operator with FeatureTestRunner and an independent scalar oracle shaped from the same current-main libaom behavior.
7. Write complete AVIF output
- Write the correct AVIF file type, item information, locations, references, properties, AV1 configuration, dimensions, color, alpha, metadata, and media data.
- Support single images, alpha auxiliary images, grids, multiple extents, and bounded image sequences in the final public scope.
- Preserve ICC, Exif, and XMP according to encoder options.
- Write CICP, range, chroma position, bit depth, and subsampling values that match the encoded planes.
- Apply orientation and clean-aperture behavior consistently with ImageSharp encoder conventions.
- Stream output through allocator-backed chunked storage without file-sized copies or ToArray materialization.
Encoder exit gate:
- Current-main libaom accepts every produced AV1 payload.
- Lossless output is exact at native-plane and final-pixel precision.
- Lossy output demonstrates recorded quality and effort tradeoffs with absolute size, quality, timing, and allocation evidence.
- 8, 10, and 12-bit monochrome, 4:2:0, 4:2:2, and 4:4:4 outputs pass.
- Alpha, grids, metadata, color profiles, transforms, and bounded sequences pass.
- ImageSharp decode of its own output is supplemental coverage only, never the sole oracle.
- Public encoding no longer throws for a supported AV1 request.
- Focused Release and FeatureTestRunner verification passes with exact recorded evidence.
Architecture rules
- Follow the JPEG color-converter operator architecture exactly.
- Each distinct prediction traversal owns a family-named predictor type.
- The family .Operator.cs file defines the nested static operator contract.
- Each semantic readonly struct belongs to that owner and implements concrete scalar, Vector128, Vector256, and Vector512 arithmetic for the shared traversal.
- Do not place a distinct predictor beneath a broad Av1IntraPredictor or Av1InterPredictor.
- Do not create semantic forwarding wrappers, top-level operator types, hardware-width-named operator types, CRTP contracts, or one file containing unrelated semantic operators.
- Forward transforms belong to Av1ForwardTransformer and its semantic operator files.
- Inverse axis transforms belong to Av1Inverse2dTransformer and its semantic operator files.
- Reconstruction output operators belong to Av1InverseTransformer.
- Shared lane primitives belong only in explicitly named Operations types.
- Dispatch from widest to narrowest supported SIMD width, then execute one scalar tail.
- Keep codec execution sequential. Do not add parallel execution inside the codec.
- Do not allocate per row, block, transform, scanline, or SIMD tail.
- Use ImageSharp allocators and pools. Do not use ToArray to cross an ownership boundary.
- On internal types, use public members when other types consume them; reserve private members for type-local behavior.
- Use established ImageSharp test data, allocator tracking, FeatureTestRunner, and reference-image comparison APIs. Do not build custom substitutes.
- Public XML documentation describes observable behavior only.
- Inline comments explain the current-libaom numerical rule, ownership boundary, edge extension, entropy ordering, or SIMD shape at technically complex points.
- Every multiline statement or declaration is followed by vertical whitespace.
- Do not edit .gitattributes directly.
- Do not install or download tools without explicit permission.
Final verification matrix
- Release source build: net10.0, zero errors.
- Release source build: net11.0, zero errors.
- Scoped semantic inspection: zero compiler errors attributable to this work.
- Focused decoder syntax, reconstruction, ownership, presentation, and malformed-input tests.
- Focused encoder syntax, payload, container, precision, ownership, and option tests.
- FeatureTestRunner coverage for normal, narrower SIMD tiers, and scalar fallback.
- Constrained multi-group allocator coverage with balanced exactly-once returns.
- Exact native-plane comparisons against current-main libaom.
- Established final-presentation comparisons at the target pixel precision.
- Scoped StyleCop and vertical-whitespace inspection.
- No stale unsupported capability claims or removed-code references.
- No restore-source failures, background test hosts, detached processes, or crash-report popups.
- .gitattributes unchanged.
- git diff --check clean.
- Documentation records exact commands, counts, current-main reference revision evidence, and results.
- Commit only after the relevant checkpoint is genuinely complete.
- Do not push.