Browse Source

Fix AV1 decoder model parsing

pull/2633/head
James Jackson-South 2 months ago
parent
commit
80c59dd6d2
  1. 1
      HEIF_IMPLEMENTATION_PLAN.md
  2. 18
      src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuReader.cs

1
HEIF_IMPLEMENTATION_PLAN.md

@ -69,6 +69,7 @@ This snapshot pins or classifies the available references and failures; it does
| --- | --- | --- | --- |
| `Av1YuvConverter.ConvertToRgb`, `ConvertFromRgb`, scalar row conversion, chroma reconstruction, and chroma downsampling | ITU-T H.273 (V4) equations 14-16 and 27-84, including limited/full-range scaling, chromaticity-derived equations 39-47, YCgCo equations 51-57, constant-luminance equations 66-75, and the PQ/HLG ICtCp matrices; AV1 section 6.4.2 chroma sample positions | Official ITU-T H.273 (V4) (07/2024); libavif `src/reformat.c` and `src/colr.c` at `092276ce89098ead06db80975173191e5fee1826`; libaom `aom/aom_image.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Scalar behavioral oracle for full/limited-range conversion at 8, 10, and 12 bits. Decode and encode cover identity, YCgCo, coefficient-based NCL, both fixed and chromaticity-derived constant/non-constant-luminance systems, SMPTE ST 2085, and ICtCp across monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with AV1 chroma sample positioning. Limited-range YCgCo retains the 219-code scale inherited from its R/G/B inputs instead of applying YCbCr's unrelated 224-code chroma range. Chromaticity derivation uses every defined H.273 primary and matches libavif's BT.709 fallback for unspecified or reserved primaries. The ICtCp inverse is derived from the exact H.273 integer matrices rather than an unrelated display conversion. Later SIMD paths must preserve this scalar behavior. |
| `Av1TransferFunctions` | ITU-T H.273 (V4) Table 3 transfer characteristics 1-18 | Official ITU-T H.273 (V4) (07/2024); libavif `src/colr.c` at `092276ce89098ead06db80975173191e5fee1826` | Apply every AV1-signallable transfer function required by constant-luminance and ICtCp color conversion. Retain the H.273 normalized PQ and HLG definitions; do not import libavif's display-oriented 203-nit scaling or HLG OOTF into codec sample interpretation. Use libavif's midpoint convention only for the non-bijective zero code of the two logarithmic curves. |
| `ObuReader.ReadSequenceHeader`, decoder-model parsing, and operating-parameter consumption | AV1 sections 5.5.2 through 5.5.4 sequence header, timing info, decoder model info, and operating parameters | libaom `av1/decoder/decodeframe.c` functions `av1_read_decoder_model_info` and `av1_read_op_parameters_info`, plus `common/av1_config.c`, at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Read the normative 32-bit decoding-tick field and consume the two variable-width delays plus low-delay flag for every signaled operating point so a non-reduced still-image sequence remains aligned. Retain no scheduling or playback behavior from those values and introduce no ISO BMFF timing, track, sample-table, or sequence surface. |
| `Av1FrameBuffer` high-bit-depth sample layout and `Av1YuvConverter` 10/12-bit packed-pixel conversion | AV1 section 6.4.1 bit depth and H.273 sample-range scaling | libaom `aom_scale/yv12config.h`, `av1/common/idct.c`, and `av1/common/reconintra.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f`; libavif `src/avif.c` and `src/reformat.c` at `092276ce89098ead06db80975173191e5fee1826` | Establish two-byte native sample storage with sample-unit strides for 10/12-bit reconstruction and use ImageSharp's existing `Rgb48` pixel-operation paths in both directions so packed-pixel staging does not reduce high-bit-depth samples to eight bits. |
| `Av1PredictionDecoder`, `Av1HighBitDepthPredictor`, `Av1ChromaFromLumaContext`, `Av1PartitionInfo`, and the scalar DC, directional, Paeth, smooth, filter-intra, and chroma-from-luma predictors | AV1 sections 7.11.2 and 7.11.2.3 intra prediction | libaom `aom_dsp/intrapred.c`, `av1/common/reconintra.c`, `av1/common/av1_common_int.h`, `av1/common/blockd.h`, `av1/common/cfl.c`, and `av1/common/cfl.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Behavioral oracle for luma/chroma mode-neighbor addressing, directional upsampling, Paeth selection, smooth normalization, filter-intra taps, high-bit-depth clipping, chroma-from-luma storage/subsampling, and chroma-from-luma row strides. Existing managed scalar tables and predictors remain the implementation base. The WIP rectangular byte-pipeline smooth digest expectations encode width/height-swapped weights and must be replaced only from an independently generated oracle, not regenerated from this implementation. |
| `Av1TileReader` palette mode/color-map parsing, `Av1SymbolDecoder` palette distributions, `Av1BlockModeInfo` palette state, and `Av1PredictionDecoder` palette reconstruction | AV1 sections 5.11.46, 5.11.49, and 7.11.2 palette prediction | libaom `av1/decoder/decodemv.c`, `av1/decoder/detokenize.c`, `av1/decoder/decoder.h`, `av1/common/pred_common.c`, `av1/common/pred_common.h`, and `av1/common/entropymode.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Reuse the existing tile range decoder and frame-owned mode map while porting the normative palette cache merge, high-bit-depth color deltas, tile-adaptive mode/size/index distributions, diagonal color-map traversal, edge padding, and direct palette-sample reconstruction. This is AV1 still-image compression syntax and does not add retained video reference state or any ISO BMFF surface. |

18
src/ImageSharp/Formats/Heif/Av1/OpenBitstreamUnit/ObuReader.cs

@ -327,7 +327,9 @@ internal class ObuReader
sequenceHeader.OperatingPoint[i].IsDecoderModelInfoPresent = reader.ReadBoolean();
if (sequenceHeader.OperatingPoint[i].IsDecoderModelInfoPresent)
{
// TODO: operating_parameters_info( i )
// Operating-point delays affect scheduling rather than still-image reconstruction, but their
// syntax must be consumed so the following image dimensions remain bit aligned.
ReadOperatingParametersInfo(ref reader, (int)sequenceHeader.DecoderModelInfo!.BufferDelayLength);
}
}
else
@ -551,11 +553,23 @@ internal class ObuReader
private static void ReadDecoderModelInfo(ref Av1BitStreamReader reader, ObuSequenceHeader sequenceHeader) => sequenceHeader.DecoderModelInfo = new ObuDecoderModelInfo
{
BufferDelayLength = reader.ReadLiteral(5) + 1,
NumUnitsInDecodingTick = reader.ReadLiteral(16),
NumUnitsInDecodingTick = reader.ReadLiteral(32),
BufferRemovalTimeLength = reader.ReadLiteral(5) + 1,
FramePresentationTimeLength = reader.ReadLiteral(5) + 1
};
/// <summary>
/// Consumes operating-point buffer parameters that do not affect still-image reconstruction.
/// </summary>
/// <param name="reader">The reader positioned at the operating-point parameters.</param>
/// <param name="bufferDelayLength">The bit width of each encoded buffer delay.</param>
private static void ReadOperatingParametersInfo(ref Av1BitStreamReader reader, int bufferDelayLength)
{
_ = reader.ReadLiteral(bufferDelayLength);
_ = reader.ReadLiteral(bufferDelayLength);
_ = reader.ReadBoolean();
}
/// <summary>
/// Reads the sequence timing information.
/// </summary>

Loading…
Cancel
Save