Browse Source

Verify bounded HEVC decoder ownership

pull/2633/head
James Jackson-South 4 days ago
parent
commit
b6ba2dbf2f
  1. 14
      HEIF_IMPLEMENTATION_PLAN.md
  2. 53
      src/ImageSharp/Formats/Heif/Hevc/HevcCodingTreeState.cs
  3. 25
      src/ImageSharp/Formats/Heif/Hevc/HevcDeblockingState.cs
  4. 35
      src/ImageSharp/Formats/Heif/Hevc/HevcIntraPredictionState.cs
  5. 31
      src/ImageSharp/Formats/Heif/Hevc/HevcPictureBuffer.cs
  6. 116
      src/ImageSharp/Formats/Heif/Hevc/HevcPictureDecoder.cs
  7. 27
      src/ImageSharp/Formats/Heif/Hevc/HevcReconstructionState.cs
  8. 31
      src/ImageSharp/Formats/Heif/Hevc/HevcSampleAdaptiveOffsetParameters.cs
  9. 120
      tests/ImageSharp.Tests/Formats/Heif/Hevc/HevcPictureDecoderTests.cs

14
HEIF_IMPLEMENTATION_PLAN.md

@ -69,15 +69,15 @@ Status meanings:
- **Not started:** supporting primitives may exist, but the production format path is absent. - **Not started:** supporting primitives may exist, but the production format path is absent.
- **Current:** the only work item that should be advanced before taking the next queued item. - **Current:** the only work item that should be advanced before taking the next queued item.
Current development stage: **Stage 4 — complete HEVC still-image decoding.** The AV1 still-image exit gate is complete across the exact 12-profile matrix, layered reference-dependent items, every connected normative reconstruction tool, official quantizer and size boundaries, constrained allocation, and pinned native and presentation output. HEVC reconstruction already reaches the recorded HM/libheif fixtures; the remaining work starts with the bounded item-local `hvcC`, VPS, SPS, PPS, and slice-header surface before completing the still-image profile and Range Extensions matrix. Neither AV1 nor HEVC production encoding is implemented. Current development stage: **Stage 4 — complete HEVC still-image decoding.** The AV1 still-image exit gate is complete across the exact 12-profile matrix, layered reference-dependent items, every connected normative reconstruction tool, official quantizer and size boundaries, constrained allocation, and pinned native and presentation output. The bounded HEVC item-local syntax and allocator-backed decoder lifecycle are implemented and verified through official native-plane, real HEIC payload, constrained-allocation, constructor-failure, and public presentation cases. The next dependency is complete CABAC arithmetic and context-transition verification. Neither AV1 nor HEVC production encoding is implemented.
Immediate checkpoint: **remaining — implement the Phase 4 HEVC item-local bitstream and parameter-set syntax.** Follow the existing bounded HEIF item model and JPEG-style operator architecture, implement only the still-image `hvcC`, VPS, SPS, PPS, and slice-header surface required by independently encoded HEIC fixtures, and verify each completed dependency before advancing. Immediate checkpoint: **current — verify Phase 4 HEVC CABAC arithmetic and every required context transition.** Reconcile the existing syntax reader and context tables with the pinned HM implementation, close any valid still-image gaps, and retain focused bit-exact and complete-slice evidence before advancing to traversal.
| Order | Delivery stage | State | Delivered state | Gate that remains open | | Order | Delivery stage | State | Delivered state | Gate that remains open |
| --- | --- | --- | --- | --- | | --- | --- | --- | --- | --- |
| 1 | Baseline, provenance, documentation, and public contract | In progress | Pinned codec references, a bounded image-only scope, encoder options, typed bit depth, decoder-option propagation, and extensive HEIF documentation exist. | Complete the all-file documentation audit, record a fresh Release baseline, finish distinct public HEIC/AVIF save boundaries, and close API review. | | 1 | Baseline, provenance, documentation, and public contract | In progress | Pinned codec references, a bounded image-only scope, encoder options, typed bit depth, decoder-option propagation, and extensive HEIF documentation exist. | Complete the all-file documentation audit, record a fresh Release baseline, finish distinct public HEIC/AVIF save boundaries, and close API review. |
| 2 | Bounded HEIF item and image-sequence container | In progress | Still-item parsing, grids, auxiliary alpha, metadata properties, bounded image-sequence tracks, Identify, and all-sync AV1 sequence presentation are connected. | Complete adversarial boundary coverage, remaining item/property behavior, reference-dependent sequence reconstruction, and the bounded sequence writer. | | 2 | Bounded HEIF item and image-sequence container | In progress | Still-item parsing, grids, auxiliary alpha, metadata properties, bounded image-sequence tracks, Identify, and all-sync AV1 sequence presentation are connected. | Complete adversarial boundary coverage, remaining item/property behavior, reference-dependent sequence reconstruction, and the bounded sequence writer. |
| 3 | Still-image AV1 and HEVC decoding | **Current** | AV1 still-image decoding has passed its complete exact native/presentation profile, layered-item, normative-tool, official-sequence, constrained-allocation, and SIMD-fallback exit gate. HEVC reconstruction reaches exact HM/libheif fixtures across the recorded 8/10/12-bit and chroma cases. | Complete the remaining HEVC item-local syntax, profile, and Range Extensions matrix. | | 3 | Still-image AV1 and HEVC decoding | **Current** | AV1 still-image decoding has passed its complete exact native/presentation profile, layered-item, normative-tool, official-sequence, constrained-allocation, and SIMD-fallback exit gate. HEVC item-local syntax and reconstruction pass exact HM/libheif fixtures across the recorded 8/10/12-bit and chroma cases. | Complete HEVC lifecycle verification, then the remaining normative-tool, profile, and Range Extensions matrix in dependency order. |
| 4 | Complete decoded presentation and animation | In progress | Shared SIMD-first AV1/HEVC color conversion, ICC application, grids, transforms, direct planar alpha composition, frame metadata, repetition, and independently decodable AV1 sequence samples exist. | Close the full color/ICC cross-product, HEVC sequence decoding, AV1/HEVC reference-dependent samples, frame-local metadata/alpha behavior, and independent animated decode vectors. | | 4 | Complete decoded presentation and animation | In progress | Shared SIMD-first AV1/HEVC color conversion, ICC application, grids, transforms, direct planar alpha composition, frame metadata, repetition, and independently decodable AV1 sequence samples exist. | Close the full color/ICC cross-product, HEVC sequence decoding, AV1/HEVC reference-dependent samples, frame-local metadata/alpha behavior, and independent animated decode vectors. |
| 5 | AV1/AVIF encoding | Not started | RGB-to-planar conversion, forward transforms, OBU writer foundations, options, and container-writing infrastructure exist. | `HeifEncoderCore` still rejects AV1. Implement a real independently decodable lossy/lossless AV1 payload and the complete AVIF item/metadata matrix. | | 5 | AV1/AVIF encoding | Not started | RGB-to-planar conversion, forward transforms, OBU writer foundations, options, and container-writing infrastructure exist. | `HeifEncoderCore` still rejects AV1. Implement a real independently decodable lossy/lossless AV1 payload and the complete AVIF item/metadata matrix. |
| 6 | HEVC/HEIC encoding | Not started | Shared input color conversion, options, and HEIF writer infrastructure exist. | `HeifEncoderCore` still rejects HEVC. Implement a real independently decodable lossy/lossless HEVC payload and the complete HEIC item/metadata matrix. | | 6 | HEVC/HEIC encoding | Not started | Shared input color conversion, options, and HEIF writer infrastructure exist. | `HeifEncoderCore` still rejects HEVC. Implement a real independently decodable lossy/lossless HEVC payload and the complete HEIC item/metadata matrix. |
@ -134,7 +134,7 @@ Immediate checkpoint: **remaining — implement the Phase 4 HEVC item-local bits
- [x] Add the missing official ten-bit sequence evidence from libaom's pinned test-data manifest. All ten active-film-grain YUV420 frames and all ten monochrome frames match the pinned generic libaom native Y4M samples exactly under normal and scalar `FeatureTestRunner` dispatch, and both complete sequences pass through a 2 KiB constrained tracked allocator with balanced exactly-once returns. The three new focused cases and all 20 official-sequence regression cases pass without failures or skips, the scoped Release test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and `git diff --check` is clean. The existing production decoder required no correction. Commit `deefeba2340932909aa3e27516d965a294cf6faa` records the checkpoint. - [x] Add the missing official ten-bit sequence evidence from libaom's pinned test-data manifest. All ten active-film-grain YUV420 frames and all ten monochrome frames match the pinned generic libaom native Y4M samples exactly under normal and scalar `FeatureTestRunner` dispatch, and both complete sequences pass through a 2 KiB constrained tracked allocator with balanced exactly-once returns. The three new focused cases and all 20 official-sequence regression cases pass without failures or skips, the scoped Release test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and `git diff --check` is clean. The existing production decoder required no correction. Commit `deefeba2340932909aa3e27516d965a294cf6faa` records the checkpoint.
- [x] Add official minimum- and maximum-quantizer dependent-frame evidence for the eight- and ten-bit libaom matrices. The four retained streams and pinned generic-libaom Y4M references pass exact native comparison under normal and scalar `FeatureTestRunner` dispatch, and the combined sequences pass through a 2,560-byte row-aligned constrained tracked allocator with balanced exactly-once returns. All three new focused cases and all 23 official-sequence regression cases pass without failures or skips, the scoped Release test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and `git diff --check` is clean. The existing production decoder required no correction. Commit `68cbe2fb9483215db36977a06a2dcac04e2d90bc` records the checkpoint. - [x] Add official minimum- and maximum-quantizer dependent-frame evidence for the eight- and ten-bit libaom matrices. The four retained streams and pinned generic-libaom Y4M references pass exact native comparison under normal and scalar `FeatureTestRunner` dispatch, and the combined sequences pass through a 2,560-byte row-aligned constrained tracked allocator with balanced exactly-once returns. All three new focused cases and all 23 official-sequence regression cases pass without failures or skips, the scoped Release test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and `git diff --check` is clean. The existing production decoder required no correction. Commit `68cbe2fb9483215db36977a06a2dcac04e2d90bc` records the checkpoint.
- [x] Add exact official frame-size evidence at all four corners of libaom's 196-to-226-pixel matrix. Both frames from each retained stream match the pinned generic-libaom native Y4M samples under normal and scalar `FeatureTestRunner` dispatch, and all four sequences pass through a 1 KiB constrained tracked allocator with balanced exactly-once returns. Both new focused cases and all 25 official-sequence regression cases pass without failures or skips, the scoped Release test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and `git diff --check` is clean. The existing production decoder required no correction. Commit `ccdcddc92e3f7c45eb0a964569a9c8d2756c0526` records the checkpoint. - [x] Add exact official frame-size evidence at all four corners of libaom's 196-to-226-pixel matrix. Both frames from each retained stream match the pinned generic-libaom native Y4M samples under normal and scalar `FeatureTestRunner` dispatch, and all four sequences pass through a 1 KiB constrained tracked allocator with balanced exactly-once returns. Both new focused cases and all 25 official-sequence regression cases pass without failures or skips, the scoped Release test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and `git diff --check` is clean. The existing production decoder required no correction. Commit `ccdcddc92e3f7c45eb0a964569a9c8d2756c0526` records the checkpoint.
- [ ] **Current:** complete the remaining HEVC still-image profile and Range Extensions matrix with exact independent native-plane and presentation evidence. - [ ] **Current:** verify HEVC CABAC arithmetic and every required context transition, then complete the remaining HEVC still-image dependencies and Range Extensions matrix in Phase 4 order.
- [ ] Close shared decoded presentation, ICC, alpha, grid, transform, metadata, and animated AV1/HEVC decode gates. - [ ] Close shared decoded presentation, ICC, alpha, grid, transform, metadata, and animated AV1/HEVC decode gates.
- [ ] Implement and independently verify real AV1/AVIF still encoding. - [ ] Implement and independently verify real AV1/AVIF still encoding.
- [ ] Implement and independently verify real HEVC/HEIC still encoding. - [ ] Implement and independently verify real HEVC/HEIC still encoding.
@ -664,14 +664,14 @@ For each SIMD-suitable item, design the data layout, operator contract, scratch
Exit gate: Exit gate:
- [x] Independently encoded opaque AVIF still images, including reference-dependent layered items, reconstruct correctly across all exposed AV1 profiles, bit depths, subsampling modes, and normative image compression tools. Native planes are compared exactly with pinned libaom output, and presented pixels are compared after applying the same signaled color conversion in the reference path. The exact 12-profile native and presentation matrix passes all four normal/fallback cases; five real progressive layered native, presentation, constrained-allocation, and ownership cases pass; and all 25 official-sequence cases pass without failures or skips. Both Release source targets remain zero-warning and zero-error, the scoped Release test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and `git diff --check` is clean. - [x] Independently encoded opaque AVIF still images, including reference-dependent layered items, reconstruct correctly across all exposed AV1 profiles, bit depths, subsampling modes, and normative image compression tools. Native planes are compared exactly with pinned libaom output, and presented pixels are compared after applying the same signaled color conversion in the reference path. The exact 12-profile native and presentation matrix passes all four normal/fallback cases; five real progressive layered native, presentation, constrained-allocation, and ownership cases pass; and all 25 official-sequence cases pass without failures or skips. Both Release source targets remain zero-warning and zero-error, the scoped Release test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and `git diff --check` is clean. Commit `2219edbc91edb6a2d905f1221a45207eb2719abc` closes Phase 3.
### Phase 4: complete HEVC still-image reconstruction ### Phase 4: complete HEVC still-image reconstruction
Implement and verify in dependency order: Implement and verify in dependency order:
- [ ] HEIF item-local length-delimited NAL units, `hvcC`, VPS, SPS, PPS, and the parameter-set and slice-header syntax needed to decode the one independently coded picture carried by a supported still-image item. Do not add an Annex B byte-stream API or an access-unit/timeline abstraction. - [x] HEIF item-local length-delimited NAL units, `hvcC`, VPS, SPS, PPS, and the parameter-set and slice-header syntax needed to decode the one independently coded picture carried by a supported still-image item. The production path remains item-local and exposes no Annex B byte-stream API or access-unit/timeline abstraction. Ten official Sony GENERAL Range Extensions first pictures, five real HEIC item payloads, and five public HEIC presentation cases pass exactly in Release: 20 cases with no failures or skips.
- [ ] One bounded decoder lifecycle that owns only the parameter sets, current-picture state, slice/tile entropy state, and reconstructed planes required by that image item. - [x] One bounded decoder lifecycle owns only the parameter sets, current-picture state, slice/tile entropy state, and reconstructed planes required by that image item. Every allocator-backed child constructor and the owning picture decoder now unwind unpublished rents in reverse order. Focused tests exercise every constructor allocation-failure position and a complete real HEIC reconstruction through 2 KiB allocator groups with balanced exactly-once returns; all 22 official native-plane, real-item, ownership, constrained-allocation, and public presentation cases pass without failures or skips. Both Release source targets build with zero warnings and errors, the scoped Release test-project analyzer build completes with zero errors and 1,011 repository warnings, Roslynk reports zero compiler errors, and `git diff --check` is clean.
- [ ] CABAC arithmetic decoding and every required context transition. - [ ] CABAC arithmetic decoding and every required context transition.
- [ ] Coding-tree, coding-unit, prediction-unit, and transform-unit traversal across all permitted sizes and partition modes. - [ ] Coding-tree, coding-unit, prediction-unit, and transform-unit traversal across all permitted sizes and partition modes.
- [ ] Intra prediction for every luma and chroma mode, including strong intra smoothing and constrained prediction rules. - [ ] Intra prediction for every luma and chroma mode, including strong intra smoothing and constrained prediction rules.

53
src/ImageSharp/Formats/Heif/Hevc/HevcCodingTreeState.cs

@ -61,25 +61,48 @@ internal sealed class HevcCodingTreeState : IDisposable
sequenceParameterSet.Height, sequenceParameterSet.Height,
this.MinCodingBlockLog2); this.MinCodingBlockLog2);
this.depths = configuration.MemoryAllocator.Allocate2D<byte>( Buffer2D<byte>? depths = null;
this.WidthInMinCodingBlocks, Buffer2D<sbyte>? quantizationParameters = null;
this.HeightInMinCodingBlocks); Buffer2D<sbyte>? chromaBlueQuantizationOffsets = null;
Buffer2D<sbyte>? chromaRedQuantizationOffsets = null;
Buffer2D<byte>? flags = null;
try
{
depths = configuration.MemoryAllocator.Allocate2D<byte>(
this.WidthInMinCodingBlocks,
this.HeightInMinCodingBlocks);
quantizationParameters = configuration.MemoryAllocator.Allocate2D<sbyte>(
this.WidthInMinCodingBlocks,
this.HeightInMinCodingBlocks);
this.quantizationParameters = configuration.MemoryAllocator.Allocate2D<sbyte>( chromaBlueQuantizationOffsets = configuration.MemoryAllocator.Allocate2D<sbyte>(
this.WidthInMinCodingBlocks, this.WidthInMinCodingBlocks,
this.HeightInMinCodingBlocks); this.HeightInMinCodingBlocks);
this.chromaBlueQuantizationOffsets = configuration.MemoryAllocator.Allocate2D<sbyte>( chromaRedQuantizationOffsets = configuration.MemoryAllocator.Allocate2D<sbyte>(
this.WidthInMinCodingBlocks, this.WidthInMinCodingBlocks,
this.HeightInMinCodingBlocks); this.HeightInMinCodingBlocks);
this.chromaRedQuantizationOffsets = configuration.MemoryAllocator.Allocate2D<sbyte>( flags = configuration.MemoryAllocator.Allocate2D<byte>(
this.WidthInMinCodingBlocks, this.WidthInMinCodingBlocks,
this.HeightInMinCodingBlocks); this.HeightInMinCodingBlocks);
this.flags = configuration.MemoryAllocator.Allocate2D<byte>( this.depths = depths;
this.WidthInMinCodingBlocks, this.quantizationParameters = quantizationParameters;
this.HeightInMinCodingBlocks); this.chromaBlueQuantizationOffsets = chromaBlueQuantizationOffsets;
this.chromaRedQuantizationOffsets = chromaRedQuantizationOffsets;
this.flags = flags;
}
catch
{
flags?.Dispose();
chromaRedQuantizationOffsets?.Dispose();
chromaBlueQuantizationOffsets?.Dispose();
quantizationParameters?.Dispose();
depths?.Dispose();
throw;
}
} }
/// <summary> /// <summary>

25
src/ImageSharp/Formats/Heif/Hevc/HevcDeblockingState.cs

@ -40,13 +40,24 @@ internal sealed class HevcDeblockingState : IDisposable
int width = DivideCeilingByPowerOfTwo(sequenceParameterSet.Width, UnitLog2); int width = DivideCeilingByPowerOfTwo(sequenceParameterSet.Width, UnitLog2);
int height = DivideCeilingByPowerOfTwo(sequenceParameterSet.Height, UnitLog2); int height = DivideCeilingByPowerOfTwo(sequenceParameterSet.Height, UnitLog2);
// MarkBlock combines sparse edge flags with existing values, so zero initialization is part of the state contract. Buffer2D<byte>? lumaBoundaries = null;
this.boundaries = Buffer2D<byte>? chromaBlueBoundaries = null;
[ Buffer2D<byte>? chromaRedBoundaries = null;
configuration.MemoryAllocator.Allocate2D<byte>(width, height, AllocationOptions.Clean), try
configuration.MemoryAllocator.Allocate2D<byte>(width, height, AllocationOptions.Clean), {
configuration.MemoryAllocator.Allocate2D<byte>(width, height, AllocationOptions.Clean), // MarkBlock combines sparse edge flags with existing values, so zero initialization is part of the state contract.
]; lumaBoundaries = configuration.MemoryAllocator.Allocate2D<byte>(width, height, AllocationOptions.Clean);
chromaBlueBoundaries = configuration.MemoryAllocator.Allocate2D<byte>(width, height, AllocationOptions.Clean);
chromaRedBoundaries = configuration.MemoryAllocator.Allocate2D<byte>(width, height, AllocationOptions.Clean);
this.boundaries = [lumaBoundaries, chromaBlueBoundaries, chromaRedBoundaries];
}
catch
{
chromaRedBoundaries?.Dispose();
chromaBlueBoundaries?.Dispose();
lumaBoundaries?.Dispose();
throw;
}
} }
/// <summary> /// <summary>

35
src/ImageSharp/Formats/Heif/Hevc/HevcIntraPredictionState.cs

@ -85,17 +85,34 @@ internal sealed class HevcIntraPredictionState : IDisposable
sequenceParameterSet.Height, sequenceParameterSet.Height,
MinPredictionBlockLog2); MinPredictionBlockLog2);
this.lumaModes = configuration.MemoryAllocator.Allocate2D<byte>( Buffer2D<byte>? lumaModes = null;
this.WidthInMinPredictionBlocks, Buffer2D<byte>? chromaModes = null;
this.HeightInMinPredictionBlocks); Buffer2D<byte>? effectiveChromaModes = null;
try
{
lumaModes = configuration.MemoryAllocator.Allocate2D<byte>(
this.WidthInMinPredictionBlocks,
this.HeightInMinPredictionBlocks);
chromaModes = configuration.MemoryAllocator.Allocate2D<byte>(
this.WidthInMinPredictionBlocks,
this.HeightInMinPredictionBlocks);
this.chromaModes = configuration.MemoryAllocator.Allocate2D<byte>( effectiveChromaModes = configuration.MemoryAllocator.Allocate2D<byte>(
this.WidthInMinPredictionBlocks, this.WidthInMinPredictionBlocks,
this.HeightInMinPredictionBlocks); this.HeightInMinPredictionBlocks);
this.effectiveChromaModes = configuration.MemoryAllocator.Allocate2D<byte>( this.lumaModes = lumaModes;
this.WidthInMinPredictionBlocks, this.chromaModes = chromaModes;
this.HeightInMinPredictionBlocks); this.effectiveChromaModes = effectiveChromaModes;
}
catch
{
effectiveChromaModes?.Dispose();
chromaModes?.Dispose();
lumaModes?.Dispose();
throw;
}
this.derivedChromaUsesColocatedLuma = sequenceParameterSet.ChromaFormat == 3; this.derivedChromaUsesColocatedLuma = sequenceParameterSet.ChromaFormat == 3;
this.codingTreeBlockMask = (1 << sequenceParameterSet.CodingTreeBlockLog2) - 1; this.codingTreeBlockMask = (1 << sequenceParameterSet.CodingTreeBlockLog2) - 1;

31
src/ImageSharp/Formats/Heif/Hevc/HevcPictureBuffer.cs

@ -72,14 +72,33 @@ internal sealed class HevcPictureBuffer : IDisposable
this.chromaSubsamplingY = !this.SeparateColorPlane && this.ChromaFormat == 1 ? 1 : 0; this.chromaSubsamplingY = !this.SeparateColorPlane && this.ChromaFormat == 1 ? 1 : 0;
int storageWidth = DivideCeilingByPowerOfTwo(this.Width, BitOperations.Log2((uint)storageAlignment)) * storageAlignment; int storageWidth = DivideCeilingByPowerOfTwo(this.Width, BitOperations.Log2((uint)storageAlignment)) * storageAlignment;
int storageHeight = DivideCeilingByPowerOfTwo(this.Height, BitOperations.Log2((uint)storageAlignment)) * storageAlignment; int storageHeight = DivideCeilingByPowerOfTwo(this.Height, BitOperations.Log2((uint)storageAlignment)) * storageAlignment;
this.Luma = configuration.MemoryAllocator.Allocate2D<ushort>(storageWidth, storageHeight); Buffer2D<ushort>? luma = null;
if (this.ChromaFormat != 0) Buffer2D<ushort>? chromaBlue = null;
Buffer2D<ushort>? chromaRed = null;
try
{ {
int chromaWidth = DivideCeilingByPowerOfTwo(storageWidth, this.chromaSubsamplingX); luma = configuration.MemoryAllocator.Allocate2D<ushort>(storageWidth, storageHeight);
int chromaHeight = DivideCeilingByPowerOfTwo(storageHeight, this.chromaSubsamplingY); if (this.ChromaFormat != 0)
{
int chromaWidth = DivideCeilingByPowerOfTwo(storageWidth, this.chromaSubsamplingX);
int chromaHeight = DivideCeilingByPowerOfTwo(storageHeight, this.chromaSubsamplingY);
chromaBlue = configuration.MemoryAllocator.Allocate2D<ushort>(chromaWidth, chromaHeight);
chromaRed = configuration.MemoryAllocator.Allocate2D<ushort>(chromaWidth, chromaHeight);
}
this.ChromaBlue = configuration.MemoryAllocator.Allocate2D<ushort>(chromaWidth, chromaHeight); this.Luma = luma;
this.ChromaRed = configuration.MemoryAllocator.Allocate2D<ushort>(chromaWidth, chromaHeight); this.ChromaBlue = chromaBlue;
this.ChromaRed = chromaRed;
}
catch
{
// Construction transfers no plane ownership when a later rent fails, so unwind the unpublished owners
// here instead of relying on Dispose being reachable through a fully constructed picture buffer.
chromaRed?.Dispose();
chromaBlue?.Dispose();
luma?.Dispose();
throw;
} }
} }

116
src/ImageSharp/Formats/Heif/Hevc/HevcPictureDecoder.cs

@ -151,42 +151,96 @@ internal sealed partial class HevcPictureDecoder : IDisposable
this.configuration = configuration; this.configuration = configuration;
this.pictureParameterSet = pictureParameterSet; this.pictureParameterSet = pictureParameterSet;
this.sequenceParameterSet = pictureParameterSet.SequenceParameterSet; this.sequenceParameterSet = pictureParameterSet.SequenceParameterSet;
this.Picture = new HevcPictureBuffer(configuration, this.sequenceParameterSet); HevcPictureBuffer? picture = null;
int codingTreeStateCount = this.sequenceParameterSet.SeparateColorPlaneFlag ? 3 : 1; HevcCodingTreeState[]? codingTreeStates = null;
this.codingTreeStates = new HevcCodingTreeState[codingTreeStateCount]; HevcIntraPredictionState[]? intraPredictionStates = null;
for (int index = 0; index < this.codingTreeStates.Length; index++) HevcReconstructionState? reconstructionState = null;
HevcCoefficientDecoder? coefficientDecoder = null;
HevcSampleAdaptiveOffsetState? sampleAdaptiveOffsetState = null;
HevcDeblockingState? deblockingState = null;
IMemoryOwner<int>? integerScratch = null;
IMemoryOwner<ushort>? predictionScratch = null;
IMemoryOwner<bool>? availabilityScratch = null;
try
{ {
this.codingTreeStates[index] = new HevcCodingTreeState(configuration, this.sequenceParameterSet); picture = new HevcPictureBuffer(configuration, this.sequenceParameterSet);
} int codingTreeStateCount = this.sequenceParameterSet.SeparateColorPlaneFlag ? 3 : 1;
codingTreeStates = new HevcCodingTreeState[codingTreeStateCount];
int intraPredictionStateCount = this.sequenceParameterSet.SeparateColorPlaneFlag ? 3 : 1; for (int index = 0; index < codingTreeStates.Length; index++)
this.intraPredictionStates = new HevcIntraPredictionState[intraPredictionStateCount]; {
for (int index = 0; index < this.intraPredictionStates.Length; index++) codingTreeStates[index] = new HevcCodingTreeState(configuration, this.sequenceParameterSet);
{ }
this.intraPredictionStates[index] = new HevcIntraPredictionState(configuration, this.sequenceParameterSet);
}
this.reconstructionState = new HevcReconstructionState(configuration, this.sequenceParameterSet); int intraPredictionStateCount = this.sequenceParameterSet.SeparateColorPlaneFlag ? 3 : 1;
this.coefficientDecoder = new HevcCoefficientDecoder(configuration); intraPredictionStates = new HevcIntraPredictionState[intraPredictionStateCount];
int codingTreeBlockCount = HevcParameterSetSyntax.GetCodingTreeBlockCount( for (int index = 0; index < intraPredictionStates.Length; index++)
this.sequenceParameterSet.Width, {
this.sequenceParameterSet.CodingTreeBlockLog2) intraPredictionStates[index] = new HevcIntraPredictionState(configuration, this.sequenceParameterSet);
* HevcParameterSetSyntax.GetCodingTreeBlockCount( }
this.sequenceParameterSet.Height,
this.sequenceParameterSet.CodingTreeBlockLog2);
this.sampleAdaptiveOffsetState = new HevcSampleAdaptiveOffsetState(configuration, codingTreeBlockCount); reconstructionState = new HevcReconstructionState(configuration, this.sequenceParameterSet);
this.deblockingState = new HevcDeblockingState(configuration, this.sequenceParameterSet); coefficientDecoder = new HevcCoefficientDecoder(configuration);
int codingTreeBlockCount = HevcParameterSetSyntax.GetCodingTreeBlockCount(
this.sequenceParameterSet.Width,
this.sequenceParameterSet.CodingTreeBlockLog2)
* HevcParameterSetSyntax.GetCodingTreeBlockCount(
this.sequenceParameterSet.Height,
this.sequenceParameterSet.CodingTreeBlockLog2);
sampleAdaptiveOffsetState = new HevcSampleAdaptiveOffsetState(configuration, codingTreeBlockCount);
deblockingState = new HevcDeblockingState(configuration, this.sequenceParameterSet);
// Six transform-sized integer regions retain quantized, dequantized, reconstructed, cross-component, and
// two-pass inverse-transform data without allocating in coding-unit or transform-unit loops.
integerScratch = configuration.MemoryAllocator.Allocate<int>(MaximumTransformSampleCount * 6);
int maximumPredictionScratch = HevcIntraPredictor.GetScratchLength(5);
int maximumReferenceScratch = HevcIntraPredictor.GetReferenceScratchLength(5, 4);
predictionScratch = configuration.MemoryAllocator.Allocate<ushort>(
MaximumTransformSampleCount + maximumPredictionScratch + maximumReferenceScratch + (MaximumReferenceLength * 4));
availabilityScratch = configuration.MemoryAllocator.Allocate<bool>((4 * 32 / 2) + 1);
this.Picture = picture;
this.codingTreeStates = codingTreeStates;
this.intraPredictionStates = intraPredictionStates;
this.reconstructionState = reconstructionState;
this.coefficientDecoder = coefficientDecoder;
this.sampleAdaptiveOffsetState = sampleAdaptiveOffsetState;
this.deblockingState = deblockingState;
this.integerScratch = integerScratch;
this.predictionScratch = predictionScratch;
this.availabilityScratch = availabilityScratch;
}
catch
{
// No decoder ownership is published when construction fails. Unwind every completed child owner in reverse
// order because the caller cannot dispose an object whose constructor did not return.
availabilityScratch?.Dispose();
predictionScratch?.Dispose();
integerScratch?.Dispose();
deblockingState?.Dispose();
sampleAdaptiveOffsetState?.Dispose();
coefficientDecoder?.Dispose();
reconstructionState?.Dispose();
if (intraPredictionStates is not null)
{
for (int index = intraPredictionStates.Length - 1; index >= 0; index--)
{
intraPredictionStates[index]?.Dispose();
}
}
// Six transform-sized integer regions retain quantized, dequantized, reconstructed, cross-component, and if (codingTreeStates is not null)
// two-pass inverse-transform data without allocating in coding-unit or transform-unit loops. {
this.integerScratch = configuration.MemoryAllocator.Allocate<int>(MaximumTransformSampleCount * 6); for (int index = codingTreeStates.Length - 1; index >= 0; index--)
int maximumPredictionScratch = HevcIntraPredictor.GetScratchLength(5); {
int maximumReferenceScratch = HevcIntraPredictor.GetReferenceScratchLength(5, 4); codingTreeStates[index]?.Dispose();
this.predictionScratch = configuration.MemoryAllocator.Allocate<ushort>( }
MaximumTransformSampleCount + maximumPredictionScratch + maximumReferenceScratch + (MaximumReferenceLength * 4)); }
this.availabilityScratch = configuration.MemoryAllocator.Allocate<bool>((4 * 32 / 2) + 1); picture?.Dispose();
throw;
}
} }
/// <summary> /// <summary>

27
src/ImageSharp/Formats/Heif/Hevc/HevcReconstructionState.cs

@ -54,14 +54,25 @@ internal sealed class HevcReconstructionState : IDisposable
this.chromaSubsamplingX = !sequenceParameterSet.SeparateColorPlaneFlag && sequenceParameterSet.ChromaFormat is 1 or 2 ? 1 : 0; this.chromaSubsamplingX = !sequenceParameterSet.SeparateColorPlaneFlag && sequenceParameterSet.ChromaFormat is 1 or 2 ? 1 : 0;
this.chromaSubsamplingY = !sequenceParameterSet.SeparateColorPlaneFlag && sequenceParameterSet.ChromaFormat == 1 ? 1 : 0; this.chromaSubsamplingY = !sequenceParameterSet.SeparateColorPlaneFlag && sequenceParameterSet.ChromaFormat == 1 ? 1 : 0;
// Region identifiers gate every reconstructed-neighbor read. A stale pooled identifier can match the first Buffer2D<int>? lumaRegions = null;
// region of a later picture, so these maps must begin at the reserved unavailable value zero. Buffer2D<int>? chromaBlueRegions = null;
this.regions = Buffer2D<int>? chromaRedRegions = null;
[ try
configuration.MemoryAllocator.Allocate2D<int>(widthInUnits, heightInUnits, AllocationOptions.Clean), {
configuration.MemoryAllocator.Allocate2D<int>(widthInUnits, heightInUnits, AllocationOptions.Clean), // Region identifiers gate every reconstructed-neighbor read. A stale pooled identifier can match the first
configuration.MemoryAllocator.Allocate2D<int>(widthInUnits, heightInUnits, AllocationOptions.Clean), // region of a later picture, so these maps must begin at the reserved unavailable value zero.
]; lumaRegions = configuration.MemoryAllocator.Allocate2D<int>(widthInUnits, heightInUnits, AllocationOptions.Clean);
chromaBlueRegions = configuration.MemoryAllocator.Allocate2D<int>(widthInUnits, heightInUnits, AllocationOptions.Clean);
chromaRedRegions = configuration.MemoryAllocator.Allocate2D<int>(widthInUnits, heightInUnits, AllocationOptions.Clean);
this.regions = [lumaRegions, chromaBlueRegions, chromaRedRegions];
}
catch
{
chromaRedRegions?.Dispose();
chromaBlueRegions?.Dispose();
lumaRegions?.Dispose();
throw;
}
} }
/// <summary> /// <summary>

31
src/ImageSharp/Formats/Heif/Hevc/HevcSampleAdaptiveOffsetParameters.cs

@ -281,13 +281,30 @@ internal sealed class HevcSampleAdaptiveOffsetState : IDisposable
/// <param name="codingTreeBlockCount">The raster-ordered coding-tree-block count.</param> /// <param name="codingTreeBlockCount">The raster-ordered coding-tree-block count.</param>
public HevcSampleAdaptiveOffsetState(Configuration configuration, int codingTreeBlockCount) public HevcSampleAdaptiveOffsetState(Configuration configuration, int codingTreeBlockCount)
{ {
this.parameters = configuration.MemoryAllocator.Allocate<HevcSampleAdaptiveOffsetParameters>(codingTreeBlockCount * 3); IMemoryOwner<HevcSampleAdaptiveOffsetParameters>? parameters = null;
IMemoryOwner<int>? regions = null;
// Slice headers can disable SAO independently for luma and chroma. Initialize every component record to Off IMemoryOwner<HevcLoopFilterRegion>? loopFilterRegions = null;
// so an enabled component never causes untouched records from pooled memory to enter the picture-level pass. try
this.parameters.Memory.Span.Clear(); {
this.regions = configuration.MemoryAllocator.Allocate<int>(codingTreeBlockCount * 3); parameters = configuration.MemoryAllocator.Allocate<HevcSampleAdaptiveOffsetParameters>(codingTreeBlockCount * 3);
this.loopFilterRegions = configuration.MemoryAllocator.Allocate<HevcLoopFilterRegion>(codingTreeBlockCount * 3);
// Slice headers can disable SAO independently for luma and chroma. Initialize every component record to Off
// so an enabled component never causes untouched records from pooled memory to enter the picture-level pass.
parameters.Memory.Span.Clear();
regions = configuration.MemoryAllocator.Allocate<int>(codingTreeBlockCount * 3);
loopFilterRegions = configuration.MemoryAllocator.Allocate<HevcLoopFilterRegion>(codingTreeBlockCount * 3);
this.parameters = parameters;
this.regions = regions;
this.loopFilterRegions = loopFilterRegions;
}
catch
{
loopFilterRegions?.Dispose();
regions?.Dispose();
parameters?.Dispose();
throw;
}
} }
/// <summary> /// <summary>

120
tests/ImageSharp.Tests/Formats/Heif/Hevc/HevcPictureDecoderTests.cs

@ -4,11 +4,13 @@
using System.Buffers.Binary; using System.Buffers.Binary;
using System.Security.Cryptography; using System.Security.Cryptography;
using SixLabors.ImageSharp.Formats.Heif.Hevc; using SixLabors.ImageSharp.Formats.Heif.Hevc;
using SixLabors.ImageSharp.Memory;
using SixLabors.ImageSharp.Tests.Memory;
namespace SixLabors.ImageSharp.Tests.Formats.Heif.Hevc; namespace SixLabors.ImageSharp.Tests.Formats.Heif.Hevc;
/// <summary> /// <summary>
/// Validates complete HEVC still-picture reconstruction against independently decoded samples. /// Validates complete HEVC still-picture reconstruction and decoder ownership against independent results.
/// </summary> /// </summary>
[Trait("Format", "Heif")] [Trait("Format", "Heif")]
public class HevcPictureDecoderTests public class HevcPictureDecoderTests
@ -122,6 +124,67 @@ public class HevcPictureDecoderTests
Assert.Equal(expectedYuv.Length, offset); Assert.Equal(expectedYuv.Length, offset);
} }
/// <summary>
/// Verifies that every possible allocator failure during decoder construction releases all earlier owners.
/// </summary>
[Fact]
public void ConstructorFailureReleasesEveryEarlierAllocation()
{
byte[] configurationData = TestFile.Create(TestImages.Heif.Image1TileHvcConfiguration).Bytes;
byte[] itemData = TestFile.Create(TestImages.Heif.Image1Tile1Payload).Bytes;
HevcCodecConfiguration codecConfiguration = new(configurationData);
HevcImageItemBitstream bitstream = new(itemData, codecConfiguration);
HevcPictureParameterSet pictureParameterSet = bitstream.SliceSegments[0].PictureParameterSet;
FailingTestMemoryAllocator successfulAllocator = new(int.MaxValue);
Configuration successfulConfiguration = Configuration.Default.Clone();
successfulConfiguration.MemoryAllocator = successfulAllocator;
using (new HevcPictureDecoder(successfulConfiguration, pictureParameterSet))
{
}
int allocationCount = successfulAllocator.AllocationAttemptCount;
Assert.True(allocationCount > 0);
AssertBalancedAllocations(successfulAllocator);
for (int failureAllocationNumber = 1; failureAllocationNumber <= allocationCount; failureAllocationNumber++)
{
FailingTestMemoryAllocator allocator = new(failureAllocationNumber);
Configuration configuration = Configuration.Default.Clone();
configuration.MemoryAllocator = allocator;
Assert.Throws<InvalidMemoryOperationException>(
() => new HevcPictureDecoder(configuration, pictureParameterSet));
Assert.Equal(failureAllocationNumber, allocator.AllocationAttemptCount);
Assert.Equal(failureAllocationNumber - 1, allocator.AllocationLog.Count);
AssertBalancedAllocations(allocator);
}
}
/// <summary>
/// Verifies successful production reconstruction with split allocator groups and balanced final disposal.
/// </summary>
[Fact]
public void DecodeWithConstrainedAllocatorReleasesEveryAllocation()
{
byte[] configurationData = TestFile.Create(TestImages.Heif.Image1TileHvcConfiguration).Bytes;
byte[] itemData = TestFile.Create(TestImages.Heif.Image1Tile1Payload).Bytes;
HevcCodecConfiguration codecConfiguration = new(configurationData);
HevcImageItemBitstream bitstream = new(itemData, codecConfiguration);
TestMemoryAllocator allocator = new() { BufferCapacityInBytes = 2_048 };
allocator.EnableNonThreadSafeLogging();
Configuration configuration = Configuration.Default.Clone();
configuration.MemoryAllocator = allocator;
using (HevcPictureDecoder decoder = new(configuration, bitstream.SliceSegments[0].PictureParameterSet))
{
decoder.Decode(bitstream);
}
Assert.NotEmpty(allocator.AllocationLog);
AssertBalancedAllocations(allocator);
}
/// <summary> /// <summary>
/// Compares one decoded component plane with its planar reference samples. /// Compares one decoded component plane with its planar reference samples.
/// </summary> /// </summary>
@ -517,4 +580,59 @@ public class HevcPictureDecoderTests
return Convert.ToHexString(hash.GetHashAndReset()).ToLowerInvariant(); return Convert.ToHexString(hash.GetHashAndReset()).ToLowerInvariant();
} }
/// <summary>
/// Verifies that every tracked allocation was returned exactly once.
/// </summary>
/// <param name="allocator">The allocator whose ownership log is complete.</param>
private static void AssertBalancedAllocations(TestMemoryAllocator allocator)
{
Assert.Equal(allocator.AllocationLog.Count, allocator.ReturnLog.Count);
foreach (TestMemoryAllocator.AllocationRequest allocation in allocator.AllocationLog)
{
Assert.Single(
allocator.ReturnLog,
returned => returned.AllocationId == allocation.AllocationId);
}
}
/// <summary>
/// Provides tracked owners until the configured allocation attempt fails.
/// </summary>
private sealed class FailingTestMemoryAllocator : TestMemoryAllocator
{
private readonly int failureAllocationNumber;
private int allocationAttemptCount;
/// <summary>
/// Initializes a new instance of the <see cref="FailingTestMemoryAllocator"/> class.
/// </summary>
/// <param name="failureAllocationNumber">The one-based allocation attempt that must fail.</param>
public FailingTestMemoryAllocator(int failureAllocationNumber)
{
this.failureAllocationNumber = failureAllocationNumber;
this.EnableNonThreadSafeLogging();
}
/// <summary>
/// Gets the number of backing-owner allocation attempts.
/// </summary>
public int AllocationAttemptCount => this.allocationAttemptCount;
/// <inheritdoc/>
protected override AllocationTrackedMemoryManager<T> AllocateCore<T>(
int length,
AllocationOptions options = AllocationOptions.None)
{
this.allocationAttemptCount++;
if (this.allocationAttemptCount == this.failureAllocationNumber)
{
// Fail before delegation so the failed attempt never creates an owner that needs rollback.
throw new InvalidMemoryOperationException("The configured HEVC allocation failed.");
}
return base.AllocateCore<T>(length, options);
}
}
} }

Loading…
Cancel
Save