Browse Source

Allocate HEVC still picture planes

pull/2633/head
James Jackson-South 1 week ago
parent
commit
a5f5dd154d
  1. 3
      HEIF_IMPLEMENTATION_PLAN.md
  2. 119
      src/ImageSharp/Formats/Heif/Hevc/HevcPictureBuffer.cs
  3. 19
      src/ImageSharp/Formats/Heif/Hevc/HevcPlane.cs

3
HEIF_IMPLEMENTATION_PLAN.md

@ -120,6 +120,7 @@ This snapshot pins or classifies the available references and failures; it does
| `HevcCabacContext` and `HevcCabacDecoder` | HEVC sections 9.3.3.2 through 9.3.3.4 context initialization, arithmetic decoding, bypass decoding, and termination | HM `source/Lib/TLibCommon/ContextModel.cpp`, `source/Lib/TLibCommon/TComCABACTables.cpp`, and `source/Lib/TLibDecoder/TDecBinCoderCABAC.cpp` at `9c1f298659ab0cee9dc13d23d0304221575410b9`; Android `libhevc` `common/ihevc_cabac_tables.c`, `common/ihevc_cabac_tables.h`, `decoder/ihevcd_cabac.c`, and `decoder/ihevcd_cabac.h` at `c83a76b084498d55f252f48b2e3786804cdf24b7` | Implement the bit-exact scalar probability-state transitions, range normalization, bounded byte refills, bypass modes, termination, and alignment validation for one slice tile or wavefront substream. CABAC decisions are serial and adaptive, so this primitive is the scalar oracle rather than a SIMD target; later prediction, transform, filtering, and color-conversion stages must reuse ImageSharp's compatible SIMD kernels. The decoder owns no container boxes, access units, reference pictures, tracks, timing, playback, or general-purpose bitstream surface. |
| `HevcCabacContexts` | HEVC section 9.3.2.2 initialization of context variables for an intra-coded slice and Range Extensions context additions | HM `source/Lib/TLibCommon/ContextTables.h` and `source/Lib/TLibDecoder/TDecSbac.cpp` function `resetEntropy` at `9c1f298659ab0cee9dc13d23d0304221575410b9`; Android `libhevc` `common/ihevc_cabac_tables.c` and `decoder/ihevcd_cabac.c` at `c83a76b084498d55f252f48b2e3786804cdf24b7` | Initialize only the context groups reachable from the independently coded intra-picture syntax, including transform skip, chroma quantization adjustment, and cross-component prediction. Keep one contiguous mutable context set per entropy substream so tile and wavefront synchronization can copy the normative adaptive state without retaining reference-picture or playback state. Omit skip, merge, motion, reference-index, and other inter-picture-only groups. |
| `HevcCabacSyntaxReader` | HEVC sections 9.3.3 through 9.3.4 CABAC binarization and context selection for intra coding units, transform trees, residual coefficients, sample-adaptive offset, and Range Extensions tools | HM `source/Lib/TLibDecoder/TDecSbac.cpp`, `source/Lib/TLibCommon/TComChromaFormat.h`, and `source/Lib/TLibCommon/TComTrQuant.cpp` at `9c1f298659ab0cee9dc13d23d0304221575410b9`; Android `libhevc` `decoder/ihevcd_cabac.c`, `decoder/ihevcd_parse_slice.c`, and `decoder/ihevcd_parse_residual.c` at `c83a76b084498d55f252f48b2e3786804cdf24b7` | Decode only syntax values reachable while reconstructing the selected independently coded still picture: intra partition and mode selectors, transform splitting and coded-block flags, quantization adjustments, transform skip, coefficient significance and levels, cross-component prediction, SAO, bypass alignment, and termination. Keep neighbor-dependent context selection in the owning coding-tree/reconstruction layer and expose no inter prediction, reference-picture, access-unit, track, timing, or playback syntax. |
| `HevcPlane` and `HevcPictureBuffer` | HEVC sections 6.2 and 6.3 source and decoded picture sample-array dimensions for monochrome, 4:2:0, 4:2:2, 4:4:4, and separate-color-plane coding | HM `source/Lib/TLibCommon/TComPicYuv.cpp` at `9c1f298659ab0cee9dc13d23d0304221575410b9`; Android `libhevc` decoded-picture buffer layout paths at `c83a76b084498d55f252f48b2e3786804cdf24b7`; ImageSharp `MemoryAllocator` and `Buffer2D<T>` | Allocate one allocator-owned native `ushort` plane set for the selected still picture so the same reconstruction path preserves every supported 8-through-16-bit sample without byte-backed reinterpretation. Derive chroma dimensions by ceiling division for each sampling layout and treat separate color planes as full-resolution independently coded arrays. The buffer contains one picture only and introduces no decoded-picture buffer, reference lifetime, frame queue, or playback state. |
| `HeifContentColorVolume`, `HeifItem.ContentColorVolume`, and `HeifDecoderCore` content color-volume parsing and presentation | HEIF content color-volume item property; AVIF 1.2 content color-volume requirements; ITU-T H.274 (V4) content colour volume syntax and semantics | libavif `src/read.c` function `avifSkipContentColourVolume` at `092276ce89098ead06db80975173191e5fee1826`; official ITU-T H.274 (V4), January 2026 | Decode only the bounded per-image `cclv` property: require zero cancellation, persistence, and reserved bits; preserve optional signed G/B/R primary coordinates and normalized minimum, maximum, and average luminance values; and validate their registered ranges and ordering. Expose the effective grid-or-tile still-image value through `HeifMetadata`. Do not add SEI persistence, retained video state, tracks, samples, timing, or a generic ISO BMFF color-volume box model. |
| `HeifAmbientViewingEnvironment`, `HeifReferenceViewingEnvironment`, `HeifNominalDiffuseWhite`, and their per-item presentation metadata | ISOBMFF ambient viewing environment; ITU-T H.274 (V4) section 8.13; HEIF Amendment 1 sections 6.5.44 and 6.5.45; AVIF 1.2 image-item box requirements | libavif `src/read.c` functions `avifSkipAmbientViewingEnvironment`, `avifSkipReferenceViewingEnvironment`, and `avifSkipNominalDiffuseWhite` at `092276ce89098ead06db80975173191e5fee1826`; official ITU-T H.274 (V4), January 2026 | Decode the fixed-size `amve`, version-zero `reve`, and version-zero `ndwt` properties only when associated with the presented still-image item. Preserve physical illuminance and luminance units, distinct surround/periphery chromaticities, and the coded zero that requests the standard nominal diffuse-white default. Validate registered coordinate ranges and duplicate associations. Retain no video-SEI persistence, visual sample entry, display pipeline, track, timing, or generic viewing-environment box model. |
| `GridHeifItemDecoder` and `HeifDecoderCore` grid/thumbnail selection | ISO/IEC 23008-12 section 6.6.2.3 image-grid syntax and MIAF grid-cell constraints | libavif `src/read.c` functions `avifParseImageGridBox`, `avifDecoderDataAllocateImagePlanes`, and `avifDecoderDataCopyTileToImage` at `092276ce89098ead06db80975173191e5fee1826` | Parse version-zero 16-bit and 32-bit grid descriptors, preserve row-major `dimg` order, require the declared tile count and one coding format, validate canvas coverage and edge overlap, and crop only the rightmost column and bottom row while copying through ImageSharp pixel buffers. A primary grid whose tile codec is unavailable may use only a decodable thumbnail that explicitly references that grid. |
@ -160,7 +161,7 @@ This assessment is based on the current source after the upstream ImageSharp mer
### HEVC decoder and encoder
- `Heif4CharCode` recognizes `hvc1` image items, `HevcCodecConfiguration` validates and associates the bounded `hvcC` property, and Identify reports its HEVC precision and monochrome shape, but `HeifCompressionFactory` has no HEVC item decoder.
- The HEVC path now validates NAL headers and RBSP escaping, parses and links `hvcC` VPS/SPS/PPS structures through Range Extensions, and parses the selected image item's length-delimited base-layer IDR slice headers and entropy payload boundaries. Its scalar CABAC primitive implements the exact adaptive state transitions, arithmetic range normalization, bypass modes, bounded refills, termination, and stop-bit validation from the pinned HM reference. The one-allocation context owner initializes every intra-picture and Range Extensions context that still-image reconstruction can select while omitting inter-picture-only groups. The codec-local syntax reader decodes the intra coding-unit, transform, coefficient, SAO, and Range Extensions binarizations, but coding-tree traversal and neighbor-derived context selection are not yet wired. The path is deliberately not registered as an item decoder until coding-tree reconstruction, still-image intra prediction, inverse quantization/transform, deblocking, sample-adaptive offset, color conversion, and high-bit-depth pixel output are implemented.
- The HEVC path now validates NAL headers and RBSP escaping, parses and links `hvcC` VPS/SPS/PPS structures through Range Extensions, and parses the selected image item's length-delimited base-layer IDR slice headers and entropy payload boundaries. Its scalar CABAC primitive implements the exact adaptive state transitions, arithmetic range normalization, bypass modes, bounded refills, termination, and stop-bit validation from the pinned HM reference. The one-allocation context owner initializes every intra-picture and Range Extensions context that still-image reconstruction can select while omitting inter-picture-only groups. The codec-local syntax reader decodes the intra coding-unit, transform, coefficient, SAO, and Range Extensions binarizations. Allocator-owned native `ushort` picture planes cover monochrome, 4:2:0, 4:2:2, 4:4:4, separate-color-plane, and 8-through-16-bit reconstruction without a decoded-picture queue. Coding-tree traversal and neighbor-derived context selection are not yet wired. The path is deliberately not registered as an item decoder until coding-tree reconstruction, still-image intra prediction, inverse quantization/transform, deblocking, sample-adaptive offset, color conversion, and pixel output are implemented.
- There is no HEVC encoder. The current HEIC-branded encoder writes a legacy JPEG payload and therefore cannot provide HEIC output.
- Existing HEVC tests prove container identification only; they do not decode or compare HEIC pixels.

119
src/ImageSharp/Formats/Heif/Hevc/HevcPictureBuffer.cs

@ -0,0 +1,119 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Hevc;
/// <summary>
/// Owns the native-precision luma and chroma sample planes for one reconstructed HEVC still picture.
/// </summary>
internal sealed class HevcPictureBuffer : IDisposable
{
/// <summary>The horizontal chroma subsampling shift.</summary>
private readonly int chromaSubsamplingX;
/// <summary>The vertical chroma subsampling shift.</summary>
private readonly int chromaSubsamplingY;
/// <summary>
/// Initializes a new instance of the <see cref="HevcPictureBuffer"/> class.
/// </summary>
/// <param name="configuration">The configuration providing the image memory allocator.</param>
/// <param name="sequenceParameterSet">The coded dimensions, precision, and chroma layout.</param>
public HevcPictureBuffer(Configuration configuration, HevcSequenceParameterSet sequenceParameterSet)
{
this.Width = sequenceParameterSet.Width;
this.Height = sequenceParameterSet.Height;
this.BitDepthLuma = sequenceParameterSet.BitDepthLuma;
this.BitDepthChroma = sequenceParameterSet.BitDepthChroma;
this.ChromaFormat = sequenceParameterSet.ChromaFormat;
this.SeparateColorPlane = sequenceParameterSet.SeparateColorPlaneFlag;
// Separate color planes are independently coded at full resolution even though chroma_format_idc is 4:4:4.
this.chromaSubsamplingX = !this.SeparateColorPlane && this.ChromaFormat is 1 or 2 ? 1 : 0;
this.chromaSubsamplingY = !this.SeparateColorPlane && this.ChromaFormat == 1 ? 1 : 0;
this.Luma = configuration.MemoryAllocator.Allocate2D<ushort>(this.Width, this.Height);
if (this.ChromaFormat != 0)
{
int chromaWidth = DivideCeilingByPowerOfTwo(this.Width, this.chromaSubsamplingX);
int chromaHeight = DivideCeilingByPowerOfTwo(this.Height, this.chromaSubsamplingY);
this.ChromaBlue = configuration.MemoryAllocator.Allocate2D<ushort>(chromaWidth, chromaHeight);
this.ChromaRed = configuration.MemoryAllocator.Allocate2D<ushort>(chromaWidth, chromaHeight);
}
}
/// <summary>Gets the coded luma width in samples.</summary>
public int Width { get; }
/// <summary>Gets the coded luma height in samples.</summary>
public int Height { get; }
/// <summary>Gets the luma sample precision in bits.</summary>
public int BitDepthLuma { get; }
/// <summary>Gets the chroma sample precision in bits.</summary>
public int BitDepthChroma { get; }
/// <summary>Gets the HEVC chroma-format identifier.</summary>
public byte ChromaFormat { get; }
/// <summary>Gets a value indicating whether the three planes are coded as independent full-resolution color planes.</summary>
public bool SeparateColorPlane { get; }
/// <summary>Gets the luma or first separate-color-plane allocation.</summary>
public Buffer2D<ushort> Luma { get; }
/// <summary>Gets the blue-difference chroma or second separate-color-plane allocation.</summary>
public Buffer2D<ushort>? ChromaBlue { get; }
/// <summary>Gets the red-difference chroma or third separate-color-plane allocation.</summary>
public Buffer2D<ushort>? ChromaRed { get; }
/// <summary>Gets the horizontal chroma subsampling shift for the selected plane.</summary>
/// <param name="plane">The reconstruction plane.</param>
/// <returns>Zero for luma and full-resolution planes; otherwise, the chroma shift.</returns>
public int GetSubsamplingX(HevcPlane plane) => plane == HevcPlane.Y ? 0 : this.chromaSubsamplingX;
/// <summary>Gets the vertical chroma subsampling shift for the selected plane.</summary>
/// <param name="plane">The reconstruction plane.</param>
/// <returns>Zero for luma and full-resolution planes; otherwise, the chroma shift.</returns>
public int GetSubsamplingY(HevcPlane plane) => plane == HevcPlane.Y ? 0 : this.chromaSubsamplingY;
/// <summary>Gets the selected plane width in samples.</summary>
/// <param name="plane">The reconstruction plane.</param>
/// <returns>The coded plane width.</returns>
public int GetWidth(HevcPlane plane) => DivideCeilingByPowerOfTwo(this.Width, this.GetSubsamplingX(plane));
/// <summary>Gets the selected plane height in samples.</summary>
/// <param name="plane">The reconstruction plane.</param>
/// <returns>The coded plane height.</returns>
public int GetHeight(HevcPlane plane) => DivideCeilingByPowerOfTwo(this.Height, this.GetSubsamplingY(plane));
/// <summary>Gets one coded row from the selected reconstruction plane.</summary>
/// <param name="plane">The reconstruction plane.</param>
/// <param name="row">The zero-based row index in plane samples.</param>
/// <returns>The complete coded plane row.</returns>
public Span<ushort> GetRowSpan(HevcPlane plane, int row)
=> plane switch
{
HevcPlane.Y => this.Luma.DangerousGetRowSpan(row),
HevcPlane.Cb => this.ChromaBlue!.DangerousGetRowSpan(row),
_ => this.ChromaRed!.DangerousGetRowSpan(row),
};
/// <summary>Releases the owned luma and chroma plane allocations.</summary>
public void Dispose()
{
this.Luma.Dispose();
this.ChromaBlue?.Dispose();
this.ChromaRed?.Dispose();
}
/// <summary>Divides a nonnegative sample count by a power of two with upward rounding.</summary>
/// <param name="value">The sample count.</param>
/// <param name="shift">The base-two divisor logarithm.</param>
/// <returns>The upward-rounded quotient.</returns>
private static int DivideCeilingByPowerOfTwo(int value, int shift) => (value + (1 << shift) - 1) >> shift;
}

19
src/ImageSharp/Formats/Heif/Hevc/HevcPlane.cs

@ -0,0 +1,19 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
namespace SixLabors.ImageSharp.Formats.Heif.Hevc;
/// <summary>
/// Identifies an HEVC luma or chroma reconstruction plane.
/// </summary>
internal enum HevcPlane
{
/// <summary>The luma or first separate-color plane.</summary>
Y = 0,
/// <summary>The blue-difference chroma or second separate-color plane.</summary>
Cb = 1,
/// <summary>The red-difference chroma or third separate-color plane.</summary>
Cr = 2,
}
Loading…
Cancel
Save