Browse Source

Strengthen HEIF sequence parser coverage

pull/2633/head
James Jackson-South 1 week ago
parent
commit
43bc4ef1c5
  1. 10
      HEIF_IMPLEMENTATION_PLAN.md
  2. 46
      src/ImageSharp/Formats/Heif/HeifSequenceParser.cs
  3. 1197
      tests/ImageSharp.Tests/Formats/Heif/HeifSequenceParserTests.cs
  4. 6
      tests/ImageSharp.Tests/TestImages.cs
  5. 3
      tests/Images/Input/Heif/colors-animated-12bpc-keyframes-0-2-3.avif
  6. 3
      tests/Images/Input/Heif/colors-animated-8bpc-alpha-exif-xmp.avif
  7. 3
      tests/Images/Input/Heif/colors-animated-8bpc-audio.avif
  8. 3
      tests/Images/Input/Heif/colors-animated-8bpc.avif

10
HEIF_IMPLEMENTATION_PLAN.md

@ -50,9 +50,9 @@ Checkboxes may be marked complete only when the implementation and the verificat
- The bounded syntax matrix below is derived from ISO/IEC 23008-12:2022 clauses 7 through 10, the codec-specific HEVC Annex B, AVIF 1.2 section 3, AV1-ISOBMFF 1.2 section 2, and the pinned libavif `read.c` and `write.c` sequence paths.
- [ ] Parse and write only that bounded image-sequence syntax; do not add audio, arbitrary media, editing, fragmentation, streaming, or general presentation APIs.
- [x] Share one bounded 32/64-bit box-header reader between the existing item path and the image-sequence path, including short-read handling and parent-boundary validation.
- [x] Select one enabled `pict` master track without materializing unrelated tracks, then parse its `mvhd`/`tkhd`/`mdhd`/`hdlr`, self-contained `dref`, `av01` or `hvc1` sample entry, codec configuration, mandatory `ccst`, and bounded repetition edit.
- [x] Select one enabled `pict` master track without materializing unrelated tracks, accepting conforming image-sequence tracks that omit `track_in_movie`, then parse its `mvhd`/`tkhd`/`mdhd`/`hdlr`, self-contained `dref`, `av01` or `hvc1` sample entry, codec configuration, mandatory `ccst`, and bounded repetition edit.
- [x] Resolve `stsc`, `stco`/`co64`, `stsz`/`stz2`, `stts`, and `stss` into one exact value-type descriptor array capped by `DecoderOptions.MaxFrames`; validate complete run/count syntax through one allocator-owned sequential scratch buffer.
- Release verification passes the libavif-shaped two-sample parser fixture, the one-frame retention boundary, and a sample offset/length beyond the file. The parser performs no per-entry allocation and does not buffer `moov`, `mdat`, or complete attacker-sized tables.
- Release verification passes the libavif-shaped two-sample parser fixture, the one-frame retention boundary, and a sample offset/length beyond the file. Pinned upstream libavif files independently verify 8-bit and 12-bit sequence identification, alpha-track and Exif/XMP retention, infinite and finite repetition, and safe omission of an unrelated audio track. The parser performs no per-entry allocation and does not buffer `moov`, `mdat`, unrelated track payloads, or complete attacker-sized tables.
- [x] Parse HEVC composition offsets and hidden samples from `ctts`/`cslg`, while rejecting the `ctts` box prohibited for AV1 tracks.
- Release verification covers signed and unsigned composition-offset syntax, hidden-sample visibility, composition-time calculation, required `cslg` and edit-list signaling, complete run counts, and the AV1 prohibition without buffering either table.
- [x] Parse optional direct dependencies from `refs` sample groups into a compact retained-sample dependency graph.
@ -68,7 +68,7 @@ Checkboxes may be marked complete only when the implementation and the verificat
- [x] Match auxiliary alpha samples by exact decode duration, visibility, and presentation time, and validate premultiplication track identity.
- [x] Require unity movie and track matrices so image presentation remains on the optimized `clap`/`irot`/`imir` path without a movie compositor.
- [x] Apply the shared `DecoderOptions` contract consistently to still items, nested payload codecs, grids, metadata properties, and sequence samples.
- `Strict` rejects recoverable ancillary and image-data errors, `IgnoreAncillary` suppresses only ancillary failures, and `IgnoreImageData` additionally permits failed image properties or samples to be omitted. `SkipMetadata` avoids optional property and item-payload validation, while cancellation and the caller configuration flow into nested JPEG and AV1 decoders. Target scaling and ICC conversion remain presentation-level operations after item or grid composition. The focused Release matrix passes all 19 new still-image policy cases, all 3 new sequence-sample cases, and the complete 32-test sequence-parser suite; the Release test-project build completes with zero errors.
- `Strict` rejects recoverable ancillary and image-data errors, `IgnoreAncillary` suppresses only ancillary failures, and `IgnoreImageData` additionally permits failed image properties or samples to be omitted. `SkipMetadata` avoids optional property and item-payload validation, while cancellation and the caller configuration flow into nested JPEG and AV1 decoders. Target scaling and ICC conversion remain presentation-level operations after item or grid composition. The focused Release matrix passes all 19 new still-image policy cases, all 3 new sequence-sample cases, and the complete 37-test sequence-parser suite; the Release test-project build completes with zero errors.
- [ ] Complete reference-dependent AV1 and HEVC sample reconstruction and independent sequence vectors.
- [ ] Write the same bounded movie, track, sample-description, location, dependency, timing, repetition, alpha, and metadata syntax from ImageSharp frames.
- [ ] Decode frame dependencies, durations, repetition, frame-local auxiliary images, and frame-local metadata into the existing ImageSharp multi-frame model.
@ -126,7 +126,7 @@ The sequence reader and writer may retain only the following syntax and the reso
| Syntax | Required image behavior |
| --- | --- |
| `ftyp` sequence and structural brands | Recognize `avis` AV1 sequences and the non-layered `hevc`/`hevx` HEVC sequence profiles. `avio` can additionally signal an all-sync AV1 sequence. Layered `hevm`/`hevs`, JPEG `jpgs`, arbitrary video brands, and brands for unimplemented codecs remain unsupported until their image payload and presentation requirements are implemented. |
| `moov`/`mvhd`, `trak`/`tkhd`, and `mdia`/`mdhd`/`hdlr` | Select an enabled, in-movie `pict` master image-sequence track; retain its displayed dimensions, media time scale, media duration, and movie-time-scale track duration. Require unity movie and track matrices because arbitrary movie-canvas composition is outside image-format scope; use `clap`/`irot`/`imir` for supported image presentation. Ignore unrelated tracks rather than exposing them. |
| `moov`/`mvhd`, `trak`/`tkhd`, and `mdia`/`mdhd`/`hdlr` | Select an enabled `pict` master image-sequence track without requiring the optional `track_in_movie` flag; retain its displayed dimensions, media time scale, media duration, and movie-time-scale track duration. Require unity movie and track matrices because arbitrary movie-canvas composition is outside image-format scope; use `clap`/`irot`/`imir` for supported image presentation. Inspect unrelated tracks only far enough to identify and skip their non-image handlers. |
| `minf`/`dinf`/`dref` and `stbl` | Accept only self-contained sample data references and own the bounded sample-table state for one selected image sequence plus an optional linked auxiliary-alpha sequence. No reusable data-reference, media-information, or sample-table API is created. |
| `stsd` and one supported visual sample entry | Require exactly one `av01` entry for AVIF or one non-layered `hvc1` entry for HEIC. Retain only its dimensions, codec configuration (`av1C` or `hvcC`), image presentation/color/HDR properties, and mandatory version-zero `ccst` coding constraints. Reject an unsupported essential configuration rather than treating it as generic video. |
| `stsc`, `stco`/`co64`, and `stsz`/`stz2` | Resolve each declared image sample directly to a validated file offset and length. Expand run tables once into a compact frame-owned descriptor array bounded by `DecoderOptions.MaxFrames`; never buffer the movie or complete `mdat`. |
@ -285,7 +285,7 @@ This assessment is based on the current source after the upstream ImageSharp mer
### Tests
- The repository contains HEIC, HIF, and AVIF assets, but only the legacy JPEG HIF path reaches a full reference-image comparison.
- The repository contains HEIC, HIF, and AVIF assets, but only the legacy JPEG HIF path reaches a full reference-image comparison. Pinned upstream libavif animations now verify real 8-bit, 12-bit, alpha, Exif/XMP, repetition, and unrelated-audio sequence parsing without claiming inter-frame pixel reconstruction.
- HEVC fixtures are identified but not decoded, and there are no HEVC algorithm tests.
- The strongest AV1 integration test now drives the single-still decoder through tile parsing, reconstruction, and pixel conversion, but only verifies non-zero output rather than independent reference pixels.
- Several full-image, inverse-transform, entropy, and frame-header cases are disabled or commented out.

46
src/ImageSharp/Formats/Heif/HeifSequenceParser.cs

@ -89,7 +89,7 @@ internal sealed class HeifSequenceParser
{
TrackIdentity identity = ScanTrackIdentity(stream, childLength, scratch);
if (colorTrackId == 0
&& identity.IsEnabledInMovie
&& identity.IsEnabled
&& identity.HandlerType == Heif4CharCode.Pict
&& identity.AuxiliaryForTrackId == 0)
{
@ -201,9 +201,25 @@ internal sealed class HeifSequenceParser
stream.Position = checked(childStart + childLength);
}
if (!trackHeader.IsPresent || !media.IsPresent)
if (!media.IsPresent)
{
throw new InvalidImageContentException("A HEIF track is missing its media box.");
}
stream.Position = media.Offset;
Heif4CharCode handlerType = ScanMediaHandler(stream, media.Length, scratch);
if (handlerType is not Heif4CharCode.Pict and not Heif4CharCode.Auxv)
{
throw new InvalidImageContentException("A HEIF image-sequence track is missing its header or media box.");
// Non-image tracks are outside this parser's retained ISOBMFF surface. Do not impose image dimensions,
// matrices, or reference semantics on audio and other unrelated media carried by a valid image file.
TrackIdentity unrelatedIdentity = default;
unrelatedIdentity.HandlerType = handlerType;
return unrelatedIdentity;
}
if (!trackHeader.IsPresent)
{
throw new InvalidImageContentException("A HEIF image-sequence track is missing its track header.");
}
stream.Position = trackHeader.Offset;
@ -214,8 +230,7 @@ internal sealed class HeifSequenceParser
ParseTrackReferences(stream, trackReferences.Length, ref identity, scratch);
}
stream.Position = media.Offset;
identity.HandlerType = ScanMediaHandler(stream, media.Length, scratch);
identity.HandlerType = handlerType;
return identity;
}
@ -351,7 +366,6 @@ internal sealed class HeifSequenceParser
const int dimensionsLength = 2 * sizeof(uint);
const int fixedPointFractionalBits = 16;
const uint trackEnabledFlag = 1 << 0;
const uint trackInMovieFlag = 1 << 1;
ReadOnlySpan<byte> prefix = ReadPrefix(stream, boxLength, scratch, fullBoxHeaderLength, "track header");
byte version = prefix[0];
@ -418,13 +432,11 @@ internal sealed class HeifSequenceParser
throw new NotSupportedException("The HEIF image-sequence track requires an unsupported movie presentation matrix.");
}
// Bits 0 and 1 are track_enabled and track_in_movie respectively. A primary image-sequence candidate must
// participate in movie playback as well as being enabled; preview-only and disabled tracks remain available
// for explicit references but are not selected as the primary color track.
uint enabledInMovieFlags = trackEnabledFlag | trackInMovieFlag;
bool isEnabledInMovie = (flags & enabledInMovieFlags) == enabledInMovieFlags;
// ISO/IEC 14496-12 Section 8.3.2 assigns bit zero to track_enabled. Image sequences, including files written
// by libavif, do not require track_in_movie to be set, so only the enabled bit participates in selection.
bool isEnabled = (flags & trackEnabledFlag) != 0;
return new TrackIdentity(id, isEnabledInMovie, width, height, duration);
return new TrackIdentity(id, isEnabled, width, height, duration);
}
/// <summary>
@ -2384,14 +2396,14 @@ internal sealed class HeifSequenceParser
/// Initializes a new instance of the <see cref="TrackIdentity"/> struct.
/// </summary>
/// <param name="id">The file-defined track identifier.</param>
/// <param name="isEnabledInMovie">Whether the track is enabled and used in the movie presentation.</param>
/// <param name="isEnabled">Whether the track is enabled.</param>
/// <param name="width">The displayed track width.</param>
/// <param name="height">The displayed track height.</param>
/// <param name="trackDuration">The track duration in movie-time-scale units.</param>
public TrackIdentity(uint id, bool isEnabledInMovie, int width, int height, ulong trackDuration)
public TrackIdentity(uint id, bool isEnabled, int width, int height, ulong trackDuration)
{
this.Id = id;
this.IsEnabledInMovie = isEnabledInMovie;
this.IsEnabled = isEnabled;
this.Width = width;
this.Height = height;
this.TrackDuration = trackDuration;
@ -2406,9 +2418,9 @@ internal sealed class HeifSequenceParser
public uint Id { get; }
/// <summary>
/// Gets a value indicating whether the track is enabled and used in the movie presentation.
/// Gets a value indicating whether the track is enabled.
/// </summary>
public bool IsEnabledInMovie { get; }
public bool IsEnabled { get; }
/// <summary>
/// Gets the displayed track width.

1197
tests/ImageSharp.Tests/Formats/Heif/HeifSequenceParserTests.cs

File diff suppressed because it is too large

6
tests/ImageSharp.Tests/TestImages.cs

@ -1295,6 +1295,12 @@ public static class TestImages
// Extremely small image, 4x4 pixels with a single solid color.
public const string Orange4x4 = "Heif/Orange4x4.avif";
// Sourced from libavif/tests/data under libavif's BSD-2-Clause license.
public const string Animated8Bit = "Heif/colors-animated-8bpc.avif";
public const string Animated8BitWithAudio = "Heif/colors-animated-8bpc-audio.avif";
public const string Animated8BitWithAlphaExifXmp = "Heif/colors-animated-8bpc-alpha-exif-xmp.avif";
public const string Animated12BitWithKeyframes = "Heif/colors-animated-12bpc-keyframes-0-2-3.avif";
}
public static class Ico

3
tests/Images/Input/Heif/colors-animated-12bpc-keyframes-0-2-3.avif

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3bf9f91da471749e7df639ba7945d4d94c1c3e3968c26f3619fbbcfc92790576
size 2267

3
tests/Images/Input/Heif/colors-animated-8bpc-alpha-exif-xmp.avif

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:c2e38681057c15009c4b76ea08cea68cdde80806abd41d42a646f697bf5aabb2
size 7506

3
tests/Images/Input/Heif/colors-animated-8bpc-audio.avif

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:624f3bfe78b6bd75e9e12fe9b36c6132e3effaf82aa2a443f1b2a207a7d3561b
size 3505

3
tests/Images/Input/Heif/colors-animated-8bpc.avif

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2f8683d21725261f37f86e115f0c212cc52d0fefd3a2ddfcc4fa648c1859906d
size 1235
Loading…
Cancel
Save