Browse Source

Track HEVC still picture coding units

pull/2633/head
James Jackson-South 1 week ago
parent
commit
8bb10e3813
  1. 1
      HEIF_IMPLEMENTATION_PLAN.md
  2. 168
      src/ImageSharp/Formats/Heif/Hevc/HevcCodingTreeState.cs

1
HEIF_IMPLEMENTATION_PLAN.md

@ -121,6 +121,7 @@ This snapshot pins or classifies the available references and failures; it does
| `HevcCabacContexts` | HEVC section 9.3.2.2 initialization of context variables for an intra-coded slice and Range Extensions context additions | HM `source/Lib/TLibCommon/ContextTables.h` and `source/Lib/TLibDecoder/TDecSbac.cpp` function `resetEntropy` at `9c1f298659ab0cee9dc13d23d0304221575410b9`; Android `libhevc` `common/ihevc_cabac_tables.c` and `decoder/ihevcd_cabac.c` at `c83a76b084498d55f252f48b2e3786804cdf24b7` | Initialize only the context groups reachable from the independently coded intra-picture syntax, including transform skip, chroma quantization adjustment, and cross-component prediction. Keep one contiguous mutable context set per entropy substream so tile and wavefront synchronization can copy the normative adaptive state without retaining reference-picture or playback state. Omit skip, merge, motion, reference-index, and other inter-picture-only groups. |
| `HevcCabacSyntaxReader` | HEVC sections 9.3.3 through 9.3.4 CABAC binarization and context selection for intra coding units, transform trees, residual coefficients, sample-adaptive offset, and Range Extensions tools | HM `source/Lib/TLibDecoder/TDecSbac.cpp`, `source/Lib/TLibCommon/TComChromaFormat.h`, and `source/Lib/TLibCommon/TComTrQuant.cpp` at `9c1f298659ab0cee9dc13d23d0304221575410b9`; Android `libhevc` `decoder/ihevcd_cabac.c`, `decoder/ihevcd_parse_slice.c`, and `decoder/ihevcd_parse_residual.c` at `c83a76b084498d55f252f48b2e3786804cdf24b7` | Decode only syntax values reachable while reconstructing the selected independently coded still picture: intra partition and mode selectors, transform splitting and coded-block flags, quantization adjustments, transform skip, coefficient significance and levels, cross-component prediction, SAO, bypass alignment, and termination. Keep neighbor-dependent context selection in the owning coding-tree/reconstruction layer and expose no inter prediction, reference-picture, access-unit, track, timing, or playback syntax. |
| `HevcPlane` and `HevcPictureBuffer` | HEVC sections 6.2 and 6.3 source and decoded picture sample-array dimensions for monochrome, 4:2:0, 4:2:2, 4:4:4, and separate-color-plane coding | HM `source/Lib/TLibCommon/TComPicYuv.cpp` at `9c1f298659ab0cee9dc13d23d0304221575410b9`; Android `libhevc` decoded-picture buffer layout paths at `c83a76b084498d55f252f48b2e3786804cdf24b7`; ImageSharp `MemoryAllocator` and `Buffer2D<T>` | Allocate one allocator-owned native `ushort` plane set for the selected still picture so the same reconstruction path preserves every supported 8-through-16-bit sample without byte-backed reinterpretation. Derive chroma dimensions by ceiling division for each sampling layout and treat separate color planes as full-resolution independently coded arrays. The buffer contains one picture only and introduces no decoded-picture buffer, reference lifetime, frame queue, or playback state. |
| `HevcCodingTreeState` | HEVC sections 6.4 coding-tree block and coding-block availability plus section 9.3 split-flag context derivation | HM `source/Lib/TLibCommon/TComDataCU.cpp` function `getCtxSplitFlag` and `source/Lib/TLibDecoder/TDecCu.cpp` coding-tree traversal at `9c1f298659ab0cee9dc13d23d0304221575410b9`; Android `libhevc` `decoder/ihevcd_parse_slice.c` coding-tree paths at `c83a76b084498d55f252f48b2e3786804cdf24b7`; ImageSharp `MemoryAllocator` and `Buffer2D<T>` | Store only leaf depth, effective luma QP, transquant-bypass, and PCM state at minimum-coding-block resolution for the selected picture. Derive split contexts from caller-approved left and above availability so slice and tile boundaries remain owned by traversal rather than hidden in a general block graph. Clip edge writes to the coded still-picture state and add no prediction units, motion fields, references, decoded-picture queue, or sequence lifetime. |
| `HeifContentColorVolume`, `HeifItem.ContentColorVolume`, and `HeifDecoderCore` content color-volume parsing and presentation | HEIF content color-volume item property; AVIF 1.2 content color-volume requirements; ITU-T H.274 (V4) content colour volume syntax and semantics | libavif `src/read.c` function `avifSkipContentColourVolume` at `092276ce89098ead06db80975173191e5fee1826`; official ITU-T H.274 (V4), January 2026 | Decode only the bounded per-image `cclv` property: require zero cancellation, persistence, and reserved bits; preserve optional signed G/B/R primary coordinates and normalized minimum, maximum, and average luminance values; and validate their registered ranges and ordering. Expose the effective grid-or-tile still-image value through `HeifMetadata`. Do not add SEI persistence, retained video state, tracks, samples, timing, or a generic ISO BMFF color-volume box model. |
| `HeifAmbientViewingEnvironment`, `HeifReferenceViewingEnvironment`, `HeifNominalDiffuseWhite`, and their per-item presentation metadata | ISOBMFF ambient viewing environment; ITU-T H.274 (V4) section 8.13; HEIF Amendment 1 sections 6.5.44 and 6.5.45; AVIF 1.2 image-item box requirements | libavif `src/read.c` functions `avifSkipAmbientViewingEnvironment`, `avifSkipReferenceViewingEnvironment`, and `avifSkipNominalDiffuseWhite` at `092276ce89098ead06db80975173191e5fee1826`; official ITU-T H.274 (V4), January 2026 | Decode the fixed-size `amve`, version-zero `reve`, and version-zero `ndwt` properties only when associated with the presented still-image item. Preserve physical illuminance and luminance units, distinct surround/periphery chromaticities, and the coded zero that requests the standard nominal diffuse-white default. Validate registered coordinate ranges and duplicate associations. Retain no video-SEI persistence, visual sample entry, display pipeline, track, timing, or generic viewing-environment box model. |
| `GridHeifItemDecoder` and `HeifDecoderCore` grid/thumbnail selection | ISO/IEC 23008-12 section 6.6.2.3 image-grid syntax and MIAF grid-cell constraints | libavif `src/read.c` functions `avifParseImageGridBox`, `avifDecoderDataAllocateImagePlanes`, and `avifDecoderDataCopyTileToImage` at `092276ce89098ead06db80975173191e5fee1826` | Parse version-zero 16-bit and 32-bit grid descriptors, preserve row-major `dimg` order, require the declared tile count and one coding format, validate canvas coverage and edge overlap, and crop only the rightmost column and bottom row while copying through ImageSharp pixel buffers. A primary grid whose tile codec is unavailable may use only a decodable thumbnail that explicitly references that grid. |

168
src/ImageSharp/Formats/Heif/Hevc/HevcCodingTreeState.cs

@ -0,0 +1,168 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Hevc;
/// <summary>
/// Stores the spatial coding-unit state required by later HEVC still-picture syntax and reconstruction stages.
/// </summary>
internal sealed class HevcCodingTreeState : IDisposable
{
/// <summary>The coding-unit flag indicating transform and quantization bypass.</summary>
private const byte TransquantBypassFlag = 1 << 0;
/// <summary>The coding-unit flag indicating pulse-code-modulated samples.</summary>
private const byte PcmFlag = 1 << 1;
/// <summary>The decoded coding-unit depth at minimum-coding-block resolution.</summary>
private readonly Buffer2D<byte> depths;
/// <summary>The effective luma quantization parameter at minimum-coding-block resolution.</summary>
private readonly Buffer2D<sbyte> quantizationParameters;
/// <summary>The packed bypass and PCM flags at minimum-coding-block resolution.</summary>
private readonly Buffer2D<byte> flags;
/// <summary>
/// Initializes a new instance of the <see cref="HevcCodingTreeState"/> class.
/// </summary>
/// <param name="configuration">The configuration providing the image memory allocator.</param>
/// <param name="sequenceParameterSet">The coded picture and minimum coding-block geometry.</param>
public HevcCodingTreeState(Configuration configuration, HevcSequenceParameterSet sequenceParameterSet)
{
this.MinCodingBlockLog2 = sequenceParameterSet.MinCodingBlockLog2;
this.WidthInMinCodingBlocks = DivideCeilingByPowerOfTwo(
sequenceParameterSet.Width,
this.MinCodingBlockLog2);
this.HeightInMinCodingBlocks = DivideCeilingByPowerOfTwo(
sequenceParameterSet.Height,
this.MinCodingBlockLog2);
this.depths = configuration.MemoryAllocator.Allocate2D<byte>(
this.WidthInMinCodingBlocks,
this.HeightInMinCodingBlocks);
this.quantizationParameters = configuration.MemoryAllocator.Allocate2D<sbyte>(
this.WidthInMinCodingBlocks,
this.HeightInMinCodingBlocks);
this.flags = configuration.MemoryAllocator.Allocate2D<byte>(
this.WidthInMinCodingBlocks,
this.HeightInMinCodingBlocks);
}
/// <summary>Gets the base-two logarithm of the state map's luma sample unit.</summary>
public int MinCodingBlockLog2 { get; }
/// <summary>Gets the state-map width in minimum coding blocks.</summary>
public int WidthInMinCodingBlocks { get; }
/// <summary>Gets the state-map height in minimum coding blocks.</summary>
public int HeightInMinCodingBlocks { get; }
/// <summary>Gets the split-flag context derived from available left and above coding units.</summary>
/// <param name="x">The current coding-unit left coordinate in luma samples.</param>
/// <param name="y">The current coding-unit top coordinate in luma samples.</param>
/// <param name="depth">The current coding-tree depth.</param>
/// <param name="leftAvailable">A value indicating whether the left coding unit is available for prediction.</param>
/// <param name="aboveAvailable">A value indicating whether the above coding unit is available for prediction.</param>
/// <returns>The split context in the inclusive range zero through two.</returns>
public int GetSplitContext(int x, int y, int depth, bool leftAvailable, bool aboveAvailable)
{
int unitX = x >> this.MinCodingBlockLog2;
int unitY = y >> this.MinCodingBlockLog2;
int context = 0;
if (leftAvailable && this.depths.DangerousGetRowSpan(unitY)[unitX - 1] > depth)
{
context++;
}
if (aboveAvailable && this.depths.DangerousGetRowSpan(unitY - 1)[unitX] > depth)
{
context++;
}
return context;
}
/// <summary>Records the state shared by every minimum coding block covered by one leaf coding unit.</summary>
/// <param name="x">The coding-unit left coordinate in luma samples.</param>
/// <param name="y">The coding-unit top coordinate in luma samples.</param>
/// <param name="log2Size">The base-two logarithm of the square coding-unit size.</param>
/// <param name="depth">The coding-tree depth.</param>
/// <param name="quantizationParameter">The effective luma quantization parameter.</param>
/// <param name="transquantBypass">A value indicating whether transform and quantization are bypassed.</param>
/// <param name="pcm">A value indicating whether the coding unit contains pulse-code-modulated samples.</param>
public void SetCodingUnit(
int x,
int y,
int log2Size,
int depth,
int quantizationParameter,
bool transquantBypass,
bool pcm)
{
int unitX = x >> this.MinCodingBlockLog2;
int unitY = y >> this.MinCodingBlockLog2;
int unitCount = 1 << (log2Size - this.MinCodingBlockLog2);
int endX = Math.Min(unitX + unitCount, this.WidthInMinCodingBlocks);
int endY = Math.Min(unitY + unitCount, this.HeightInMinCodingBlocks);
byte packedFlags = (byte)((transquantBypass ? TransquantBypassFlag : 0) | (pcm ? PcmFlag : 0));
// Edge coding units still cover a complete power-of-two block in syntax, but the state map contains only
// displayed picture coordinates. Clipping here keeps later neighbor lookup within the owned picture state.
for (int row = unitY; row < endY; row++)
{
this.depths.DangerousGetRowSpan(row)[unitX..endX].Fill((byte)depth);
this.quantizationParameters.DangerousGetRowSpan(row)[unitX..endX].Fill((sbyte)quantizationParameter);
this.flags.DangerousGetRowSpan(row)[unitX..endX].Fill(packedFlags);
}
}
/// <summary>Gets the recorded coding-tree depth at a luma sample coordinate.</summary>
/// <param name="x">The luma sample X coordinate.</param>
/// <param name="y">The luma sample Y coordinate.</param>
/// <returns>The leaf coding-unit depth.</returns>
public int GetDepth(int x, int y)
=> this.depths.DangerousGetRowSpan(y >> this.MinCodingBlockLog2)[x >> this.MinCodingBlockLog2];
/// <summary>Gets the effective luma quantization parameter at a luma sample coordinate.</summary>
/// <param name="x">The luma sample X coordinate.</param>
/// <param name="y">The luma sample Y coordinate.</param>
/// <returns>The effective luma quantization parameter.</returns>
public int GetQuantizationParameter(int x, int y)
=> this.quantizationParameters.DangerousGetRowSpan(y >> this.MinCodingBlockLog2)[x >> this.MinCodingBlockLog2];
/// <summary>Gets a value indicating whether the coding unit at a luma sample coordinate bypasses transform and quantization.</summary>
/// <param name="x">The luma sample X coordinate.</param>
/// <param name="y">The luma sample Y coordinate.</param>
/// <returns><see langword="true"/> when bypass is enabled; otherwise, <see langword="false"/>.</returns>
public bool IsTransquantBypass(int x, int y)
=> (this.flags.DangerousGetRowSpan(y >> this.MinCodingBlockLog2)[x >> this.MinCodingBlockLog2]
& TransquantBypassFlag) != 0;
/// <summary>Gets a value indicating whether the coding unit at a luma sample coordinate contains PCM samples.</summary>
/// <param name="x">The luma sample X coordinate.</param>
/// <param name="y">The luma sample Y coordinate.</param>
/// <returns><see langword="true"/> for pulse-code-modulated samples; otherwise, <see langword="false"/>.</returns>
public bool IsPcm(int x, int y)
=> (this.flags.DangerousGetRowSpan(y >> this.MinCodingBlockLog2)[x >> this.MinCodingBlockLog2]
& PcmFlag) != 0;
/// <summary>Releases the owned coding-tree state maps.</summary>
public void Dispose()
{
this.depths.Dispose();
this.quantizationParameters.Dispose();
this.flags.Dispose();
}
/// <summary>Divides a nonnegative sample count by a power of two with upward rounding.</summary>
/// <param name="value">The sample count.</param>
/// <param name="shift">The base-two divisor logarithm.</param>
/// <returns>The upward-rounded quotient.</returns>
private static int DivideCeilingByPowerOfTwo(int value, int shift) => (value + (1 << shift) - 1) >> shift;
}
Loading…
Cancel
Save