diff --git a/HEIF_IMPLEMENTATION_PLAN.md b/HEIF_IMPLEMENTATION_PLAN.md index e9c627885..3df12c180 100644 --- a/HEIF_IMPLEMENTATION_PLAN.md +++ b/HEIF_IMPLEMENTATION_PLAN.md @@ -59,7 +59,8 @@ Checkboxes may be marked complete only when the implementation and the verificat - The parser validates complete `sbgp` runs and `sgpd` entry lengths, versions, indices, positive identifier uniqueness, decode order, sync-sample independence, and `ccst` reference limits. It uses pooled value-type sort indexes plus one exact flat reference array, with no per-sample collections or attacker-sized table materialization. - [x] Parse track presentation, color, and HDR properties through the same typed property parser used by still-image items. - Selected visual sample entries now retain ICC and CICP color profiles, `pasp`, `clap`, `irot`, `imir`, `clli`, `mdcv`, `cclv`, `amve`, `reve`, and `ndwt`. Fixed-size properties use the parser's existing scratch buffer; only retained ICC bytes use bounded pooled staging before entering ImageSharp's existing profile model. - - [ ] Parse bounded sequence metadata items. + - [x] Parse bounded sequence metadata items. + - Selected image tracks now retain implicitly associated Exif and unencoded `application/rdf+xml` XMP items from bounded `iinf`/`infe` and `iloc` declarations, including file-relative and `idat`-relative extents. Item identifiers and extent descriptors use allocator-owned temporary storage; only the exact retained profile payloads survive parsing. `DecoderOptions.SkipMetadata` bypasses metadata parsing, `SegmentIntegrityHandling` follows the shared ancillary/image-data policy, and structural errors that prevent safe track parsing remain fatal. The focused Release sequence-parser suite passes all 15 tests. - [ ] Connect the parsed sequence index to HEIF detection, Identify, frame decode, alpha matching, and frame metadata without changing still-image source selection. - [ ] Write the same bounded movie, track, sample-description, location, dependency, timing, repetition, alpha, and metadata syntax from ImageSharp frames. - [ ] Decode frame dependencies, durations, repetition, frame-local auxiliary images, and frame-local metadata into the existing ImageSharp multi-frame model. @@ -184,7 +185,7 @@ This snapshot pins or classifies the available references and failures; it does | `HeifDecoderCore` box extension handling and `HeifDecoderCore`/`HeifEncoderCore` item-property associations | ISO/IEC 14496-12 box extensibility and section 8.11.14 item properties and `ipma` syntax | libavif `src/read.c` and `src/write.c` at `092276ce89098ead06db80975173191e5fee1826` | Skip unrecognized top-level and metadata child boxes, preserve the position of every property in `ipco`, reject an unrecognized property only when its item association marks it essential, associate properties by item ID, and read or write the essential bit plus one-based 7-bit or 15-bit property index according to the full-box flags. Independent HEIC, HIF, and AVIF fixtures provide the reader oracle; container-level identification of encoded output guards the writer independently of pixel roundtripping. | | `HeifCleanAperture`, `HeifItem` presentation state, and `HeifDecoderCore` transformative-property parsing and application | ISO/IEC 14496-12 section 12.1.4 clean aperture; HEIF image rotation and mirror properties; MIAF section 7.3.6.7 presentation order and section 7.3.9 essential transformative properties | libavif `src/avif.c` clean-aperture conversion, `src/read.c` property parsers and alpha-property validation, and `apps/shared/avifutil.c` transform application at `092276ce89098ead06db80975173191e5fee1826` | Resolve fractional clean-aperture dimensions and center offsets to exact bounded integer pixels, validate the registered rotation/mirror reserved bits, require essential associations, crop after auxiliary-alpha composition, map counter-clockwise HEIF quarter turns to ImageSharp's optimized clockwise rotate modes, then mirror around the signaled axis. Reuse ImageSharp's existing crop, rotation, and flip processors for every pixel type. Retain only the three image-item property values; do not add a generic transform-box or ISO BMFF model. | | `HeifConstants.IsSupportedFileType`, `HeifImageFormatDetector`, and `HeifDecoderCore.CheckFileTypeBox` | ISO/IEC 14496-12 `FileTypeBox` syntax and the MP4 Registration Authority HEIF/AVIF still-image and sequence brand registrations | libavif `src/read.c` functions `avifParseFileTypeBox`, `avifFileTypeHasBrand`, and `avifFileTypeIsCompatible` at `092276ce89098ead06db80975173191e5fee1826` | Apply one rule to the major and compatible brands, accept implemented still-image brands and the bounded `avis`, `hevc`, and `hevx` image-sequence brands, and distinguish item and sequence presentation before parsing their payloads. Layered HEVC and JPEG sequence brands remain unsupported. The decoder validates the complete `ftyp` payload; the fixed-size format detector inspects the available prefix. | -| Bounded HEIF image-sequence parser and writer | ISO/IEC 23008-12:2022 clauses 7 through 10 and HEVC Annex B; ISO/IEC 14496-12 sample tables and normal edit lists; AVIF 1.2 section 3; AV1-ISOBMFF 1.2 section 2 | libavif `src/read.c` track/sample-table paths and `src/write.c` sequence path at `092276ce89098ead06db80975173191e5fee1826`; independently produced HEIC sequence fixtures remain required | Resolve only `pict` master tracks and linked image alpha/metadata state into validated sample descriptors, exact frame durations, sync/dependency state, repetition, presentation transforms, color/HDR properties, and frame metadata. Consume the required container boxes listed in the bounded syntax matrix without exposing a movie, track, edit, or sample-table API. | +| `HeifSequenceParser`, `HeifTrackMetadataParser`, and the bounded HEIF image-sequence writer | ISO/IEC 23008-12:2022 clauses 7 through 10 and HEVC Annex B; ISO/IEC 14496-12 sample tables, normal edit lists, and track-level metadata; AVIF 1.2 section 3; AV1-ISOBMFF 1.2 section 2 | libavif `src/read.c` track/sample-table and track-metadata paths and `src/write.c` sequence path at `092276ce89098ead06db80975173191e5fee1826`; independently produced HEIC sequence fixtures remain required | Resolve only `pict` master tracks and linked image alpha/metadata state into validated sample descriptors, exact frame durations, sync/dependency state, repetition, presentation transforms, color/HDR properties, and frame metadata. Track metadata is limited to implicitly associated Exif and unencoded XMP items declared by bounded `iinf`/`infe`, `iloc`, and optional `idat` syntax. `DecoderOptions` controls metadata retention and recoverable ancillary/image-data errors through the same rules as other decoders. Consume the required container boxes listed in the bounded syntax matrix without exposing a movie, track, edit, or sample-table API. | | `HeifDecoderCore.ReadBoxHeader` and `HeifDecoderCore.ParseBoxHeader` | ISO/IEC 14496-12 section 4.2.2 basic box syntax | libavif `src/stream.c` functions `avifROStreamReadBoxHeaderPartial` and `avifROStreamReadBoxHeader` at `092276ce89098ead06db80975173191e5fee1826` | Resolve 32-bit, 64-bit, UUID, and top-level size-zero boxes into content lengths only after validating the complete variable-sized header and the remaining parent boundary. Nested size-zero boxes are invalid; large skips retain 64-bit offsets. | | `HeifDecoderCore.ParseMetadata` | ISO/IEC 14496-12 `MetaBox` and HEIF item declarations, locations, properties, and associations | libavif `src/read.c` functions `avifParseMetaBox`, `avifMetaFindOrCreateItem`, `avifParseItemLocationBox`, and `avifParseItemPropertiesBox` at `092276ce89098ead06db80975173191e5fee1826` | Index unique recognized metadata children by type and payload location, then parse them in dependency order so physical placement does not control item lookup or property association. Duplicate unique children and truncated full-box headers are invalid. | | `HeifDecoderCore.ApplyAssociatedMetadata` | HEIF Annex A Exif item data, MIME metadata items, and `cdsc` item references | libavif `src/read.c` function `avifDecoderFindMetadata`, `src/exif.c` function `avifGetExifTiffHeaderOffset`, and the Exif/XMP item writing paths in `src/write.c` at `092276ce89098ead06db80975173191e5fee1826` | Resolve only metadata items whose `cdsc` reference identifies the decoded primary image, validate the Exif TIFF-header offset, and attach Exif or `application/rdf+xml` XMP through ImageSharp's existing profile types. This is a bounded still-image metadata path; it does not introduce a generic ISO BMFF metadata, media, or track model. | diff --git a/src/ImageSharp/Formats/Heif/HeifBoxPayloadReader.cs b/src/ImageSharp/Formats/Heif/HeifBoxPayloadReader.cs new file mode 100644 index 000000000..7e598e9be --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifBoxPayloadReader.cs @@ -0,0 +1,211 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers.Binary; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Reads fixed-width values and null-terminated strings from one bounded HEIF box payload. +/// +internal ref struct HeifBoxPayloadReader +{ + /// + /// The source stream shared by the container parser. + /// + private readonly Stream stream; + + /// + /// The caller-owned buffer reused for sequential values. + /// + private readonly Span buffer; + + /// + /// The payload name used in malformed-image diagnostics. + /// + private readonly string name; + + /// + /// The number of bytes not yet loaded from the bounded payload. + /// + private long remaining; + + /// + /// The next unread byte in . + /// + private int offset; + + /// + /// The number of valid bytes currently stored in . + /// + private int count; + + /// + /// Initializes a new instance of the struct. + /// + /// The stream positioned at the bounded payload. + /// The exact number of payload bytes. + /// The caller-owned reusable buffer. + /// The payload name used in malformed-image diagnostics. + public HeifBoxPayloadReader(Stream stream, long length, Span buffer, string name) + { + this.stream = stream; + this.buffer = buffer; + this.name = name; + this.remaining = length; + this.offset = 0; + this.count = 0; + } + + /// + /// Gets the number of unread bytes in the bounded payload. + /// + public readonly long Remaining => this.remaining + this.count - this.offset; + + /// + /// Gets a value indicating whether the complete bounded payload has been consumed. + /// + public readonly bool IsComplete => this.Remaining == 0; + + /// + /// Reads one unsigned byte from the bounded payload. + /// + /// The next byte. + public byte ReadByte() + { + this.Ensure(1); + return this.buffer[this.offset++]; + } + + /// + /// Reads one big-endian unsigned 16-bit value from the bounded payload. + /// + /// The next 16-bit value. + public ushort ReadUInt16() + { + this.Ensure(2); + ushort value = BinaryPrimitives.ReadUInt16BigEndian(this.buffer[this.offset..]); + this.offset += 2; + return value; + } + + /// + /// Reads one big-endian unsigned 32-bit value from the bounded payload. + /// + /// The next 32-bit value. + public uint ReadUInt32() + { + this.Ensure(4); + uint value = BinaryPrimitives.ReadUInt32BigEndian(this.buffer[this.offset..]); + this.offset += 4; + return value; + } + + /// + /// Reads one big-endian unsigned 64-bit value from the bounded payload. + /// + /// The next 64-bit value. + public ulong ReadUInt64() + { + this.Ensure(8); + ulong value = BinaryPrimitives.ReadUInt64BigEndian(this.buffer[this.offset..]); + this.offset += 8; + return value; + } + + /// + /// Reads a zero-width, 32-bit, or 64-bit unsigned field. + /// + /// The field width in bytes. + /// The decoded unsigned value. + public ulong ReadVariableUInt(int size) => size switch + { + 0 => 0, + 4 => this.ReadUInt32(), + 8 => this.ReadUInt64(), + _ => throw new InvalidImageContentException($"The {this.name} payload uses an unsupported integer field size.") + }; + + /// + /// Consumes one null-terminated byte string without materializing it. + /// + public void SkipNullTerminatedString() + { + while (this.Remaining > 0) + { + if (this.ReadByte() == 0) + { + return; + } + } + + throw new InvalidImageContentException($"The {this.name} payload contains an unterminated string."); + } + + /// + /// Consumes one null-terminated byte string and compares it with an expected ASCII value. + /// + /// The expected ASCII bytes without a null terminator. + /// when the complete string matches . + public bool ReadNullTerminatedStringEquals(ReadOnlySpan expected) + { + int index = 0; + bool equals = true; + while (this.Remaining > 0) + { + byte value = this.ReadByte(); + if (value == 0) + { + return equals && index == expected.Length; + } + + if ((uint)index >= (uint)expected.Length || value != expected[index]) + { + equals = false; + } + + index++; + } + + throw new InvalidImageContentException($"The {this.name} payload contains an unterminated string."); + } + + /// + /// Refills the reusable buffer without reading beyond the bounded payload. + /// + /// The number of contiguous bytes required by the next value. + private void Ensure(int required) + { + int buffered = this.count - this.offset; + if (buffered >= required) + { + return; + } + + if (buffered > 0) + { + // Preserve an incomplete fixed-width value at the start of the buffer before the next read. + this.buffer.Slice(this.offset, buffered).CopyTo(this.buffer); + } + + this.offset = 0; + this.count = buffered; + while (this.count < required && this.remaining > 0) + { + int requested = (int)Math.Min(this.buffer.Length - this.count, this.remaining); + int read = this.stream.Read(this.buffer.Slice(this.count, requested)); + if (read == 0) + { + throw new InvalidImageContentException($"The {this.name} payload is truncated."); + } + + this.count += read; + this.remaining -= read; + } + + if (this.count < required) + { + throw new InvalidImageContentException($"The {this.name} payload is truncated."); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifSequenceMetadata.cs b/src/ImageSharp/Formats/Heif/HeifSequenceMetadata.cs new file mode 100644 index 000000000..e192d2c96 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifSequenceMetadata.cs @@ -0,0 +1,31 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Contains the bounded Exif and XMP item payloads implicitly associated with a HEIF image-sequence track. +/// +internal sealed class HeifSequenceMetadata +{ + /// + /// Initializes a new instance of the class. + /// + /// The complete HEIF Exif item payload, or . + /// The complete XMP packet, or . + public HeifSequenceMetadata(byte[]? exifData, byte[]? xmpData) + { + this.ExifData = exifData; + this.XmpData = xmpData; + } + + /// + /// Gets the complete HEIF Exif item payload, including its TIFF-header offset field. + /// + public byte[]? ExifData { get; } + + /// + /// Gets the raw UTF-8 XMP packet. + /// + public byte[]? XmpData { get; } +} diff --git a/src/ImageSharp/Formats/Heif/HeifSequenceParser.cs b/src/ImageSharp/Formats/Heif/HeifSequenceParser.cs index 97e650b1c..6d19166a2 100644 --- a/src/ImageSharp/Formats/Heif/HeifSequenceParser.cs +++ b/src/ImageSharp/Formats/Heif/HeifSequenceParser.cs @@ -6,6 +6,7 @@ using System.Buffers.Binary; using SixLabors.ImageSharp.Formats.Heif.Av1; using SixLabors.ImageSharp.Formats.Heif.Hevc; using SixLabors.ImageSharp.Memory; +using SixLabors.ImageSharp.Metadata.Profiles.Icc; namespace SixLabors.ImageSharp.Formats.Heif; @@ -29,21 +30,32 @@ internal sealed class HeifSequenceParser /// private readonly HeifBoxReader boxReader; + /// + /// The bounded parser for Exif and XMP items embedded in selected image tracks. + /// + private readonly HeifTrackMetadataParser metadataParser; + /// /// The maximum number of sample descriptors retained for decoding or identification. /// private readonly int maxFrames; + /// + /// The general decoder options controlling frame limits, metadata loading, and recoverable segment errors. + /// + private readonly DecoderOptions options; + /// /// Initializes a new instance of the class. /// - /// The configured ImageSharp memory allocator. - /// The maximum number of image-sequence frames to retain. - public HeifSequenceParser(MemoryAllocator allocator, uint maxFrames) + /// The general decoder options. + public HeifSequenceParser(DecoderOptions options) { - this.allocator = allocator; - this.boxReader = new HeifBoxReader(allocator); - this.maxFrames = (int)maxFrames; + this.options = options; + this.allocator = options.Configuration.MemoryAllocator; + this.boxReader = new HeifBoxReader(this.allocator); + this.metadataParser = new HeifTrackMetadataParser(this.allocator); + this.maxFrames = (int)options.MaxFrames; } /// @@ -208,6 +220,7 @@ internal sealed class HeifSequenceParser { long trackEnd = checked(stream.Position + boxLength); BoxReference edit = default; + BoxReference metadata = default; BoxReference media = default; while (stream.Position < trackEnd) @@ -218,6 +231,10 @@ internal sealed class HeifSequenceParser { SetUnique(ref edit, childStart, childLength, "track", childType); } + else if (childType == Heif4CharCode.Meta) + { + SetUnique(ref metadata, childStart, childLength, "track", childType); + } else if (childType == Heif4CharCode.Mdia) { SetUnique(ref media, childStart, childLength, "track", childType); @@ -249,6 +266,19 @@ internal sealed class HeifSequenceParser ParseEdit(stream, edit.Length, track, scratch); } + if (metadata.IsPresent && !this.options.SkipMetadata) + { + try + { + stream.Position = metadata.Offset; + track.Metadata = this.metadataParser.Parse(stream, metadata.Length, scratch); + } + catch (Exception ex) when (this.ShouldIgnoreAncillarySegmentError(ex)) + { + // The validated parent range lets decoding continue safely without this optional metadata box. + } + } + stream.Position = media.Offset; this.ParseMedia(stream, media.Length, track, scratch); if (track.Matrix.HasPerspective) @@ -888,18 +918,41 @@ internal sealed class HeifSequenceParser track.IsAlpha = this.ParseAuxiliaryType(stream, childLength); auxiliaryTypeSeen = true; break; - case Heif4CharCode.Pasp: case Heif4CharCode.Colr: + this.ParseTrackColorInformation(stream, childLength, track, scratch); + break; + case Heif4CharCode.Pasp: case Heif4CharCode.Clli: case Heif4CharCode.Mdcv: case Heif4CharCode.Cclv: case Heif4CharCode.Amve: case Heif4CharCode.Reve: case Heif4CharCode.Ndwt: + if (!this.options.SkipMetadata) + { + try + { + ParseTrackImageProperty(stream, childLength, childType, track, scratch); + } + catch (Exception ex) when (this.ShouldIgnoreAncillarySegmentError(ex)) + { + // The complete child range remains known, so optional metadata can be discarded safely. + } + } + + break; case Heif4CharCode.Clap: case Heif4CharCode.Irot: case Heif4CharCode.Imir: - this.ParseTrackImageProperty(stream, childLength, childType, track, scratch); + try + { + ParseTrackImageProperty(stream, childLength, childType, track, scratch); + } + catch (Exception ex) when (this.ShouldIgnoreImageDataSegmentError(ex)) + { + // IgnoreImageData permits a recoverable presentation property to be omitted. + } + break; } @@ -920,19 +973,13 @@ internal sealed class HeifSequenceParser /// The registered image property type. /// The selected image track receiving the property. /// The parser-owned reusable scratch span. - private void ParseTrackImageProperty( + private static void ParseTrackImageProperty( Stream stream, long boxLength, Heif4CharCode boxType, HeifSequenceTrack track, Span scratch) { - if (boxType == Heif4CharCode.Colr) - { - this.ParseTrackColorInformation(stream, boxLength, track, scratch); - return; - } - ReadOnlySpan data = ReadPropertyPayload(stream, boxLength, scratch, boxType); switch (boxType) { @@ -1034,35 +1081,49 @@ internal sealed class HeifSequenceParser Heif4CharCode profileType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(prefix); if (profileType == Heif4CharCode.Nclx) { - if (track.CicpProfile is not null) + try { - throw new InvalidImageContentException("The image-sequence sample entry has duplicate CICP color properties."); - } + if (track.CicpProfile is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate CICP color properties."); + } + + if (boxLength != 11) + { + throw new InvalidImageContentException("The CICP color-information property has an invalid length."); + } - if (boxLength != 11) + prefix = ReadPrefixFromStart(stream, boxLength, scratch, 11, "color information"); + track.CicpProfile = HeifPropertyParser.ParseCicpProfile(prefix[4..]); + } + catch (Exception ex) when (this.ShouldIgnoreImageDataSegmentError(ex)) { - throw new InvalidImageContentException("The CICP color-information property has an invalid length."); + // IgnoreImageData permits the decoder to fall back to the coded sequence's color description. } - - prefix = ReadPrefixFromStart(stream, boxLength, scratch, 11, "color information"); - track.CicpProfile = HeifPropertyParser.ParseCicpProfile(prefix[4..]); } - else if (profileType is Heif4CharCode.RICC or Heif4CharCode.Prof) + else if ((profileType is Heif4CharCode.RICC or Heif4CharCode.Prof) && !this.options.SkipMetadata) { - if (track.IccProfile is not null) + try { - throw new InvalidImageContentException("The image-sequence sample entry has duplicate ICC color properties."); - } + if (track.IccProfile is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate ICC color properties."); + } - if (boxLength <= 4 || boxLength > int.MaxValue) + if (boxLength <= 4 || boxLength > int.MaxValue) + { + throw new InvalidImageContentException("The ICC color-information property is empty or too large."); + } + + stream.Position -= 4; + using IMemoryOwner payload = this.boxReader.ReadPayload(stream, boxLength); + byte[] profileData = payload.GetSpan()[4..].ToArray(); + track.IccProfile = HeifPropertyParser.ParseIccProfile(profileData); + } + catch (Exception ex) when (this.ShouldIgnoreAncillarySegmentError(ex)) { - throw new InvalidImageContentException("The ICC color-information property is empty or too large."); + // A malformed optional ICC profile does not invalidate the coded image outside strict mode. } - - stream.Position -= 4; - using IMemoryOwner payload = this.boxReader.ReadPayload(stream, boxLength); - byte[] profileData = payload.GetSpan()[4..].ToArray(); - track.IccProfile = HeifPropertyParser.ParseIccProfile(profileData); } } @@ -1197,7 +1258,7 @@ internal sealed class HeifSequenceParser return; } - TableReader reader = new(stream, entryBytes, scratch, "sample sizes"); + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "sample sizes"); for (uint i = 0; i < sampleCount; i++) { int size = ValidateSampleSize(reader.ReadUInt32()); @@ -1240,7 +1301,7 @@ internal sealed class HeifSequenceParser int retainedCount = (int)Math.Min(sampleCount, (uint)this.maxFrames); track.TotalSampleCount = sampleCount; track.Samples = new HeifSequenceSample[retainedCount]; - TableReader reader = new(stream, entryBytes, scratch, "compact sample sizes"); + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "compact sample sizes"); for (uint i = 0; i < sampleCount; i++) { uint size; @@ -1298,7 +1359,7 @@ internal sealed class HeifSequenceParser throw new InvalidImageContentException("The image-sequence timing table is empty or has an invalid length."); } - TableReader reader = new(stream, entryBytes, scratch, "sample timing"); + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "sample timing"); ulong describedSamples = 0; ulong decodedDuration = 0; int retainedOffset = 0; @@ -1383,7 +1444,7 @@ internal sealed class HeifSequenceParser int retainedCapacity = (int)Math.Min(entryCount, (uint)track.Samples.Length); IMemoryOwner owner = this.allocator.Allocate(retainedCapacity); Span retainedEntries = owner.GetSpan(); - TableReader reader = new(stream, entryBytes, scratch, "sample-to-chunk"); + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "sample-to-chunk"); uint previousFirstChunk = 0; uint previousSamplesPerChunk = 0; ulong describedSamples = 0; @@ -1452,7 +1513,7 @@ internal sealed class HeifSequenceParser { _ = ReadChunkCount(stream, boxLength, boxType, scratch); int entrySize = boxType == Heif4CharCode.Co64 ? 8 : 4; - TableReader reader = new(stream, checked((long)chunkCount * entrySize), scratch, "chunk offsets"); + HeifBoxPayloadReader reader = new(stream, checked((long)chunkCount * entrySize), scratch, "chunk offsets"); int retainedSample = 0; int runIndex = 0; for (uint chunkIndex = 0; chunkIndex < chunkCount; chunkIndex++) @@ -1509,7 +1570,7 @@ internal sealed class HeifSequenceParser throw new InvalidImageContentException("The sync-sample table is empty or has an invalid length."); } - TableReader reader = new(stream, entryBytes, scratch, "sync samples"); + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "sync samples"); uint previousSample = 0; for (uint i = 0; i < entryCount; i++) { @@ -1661,7 +1722,7 @@ internal sealed class HeifSequenceParser throw new InvalidImageContentException("The direct-reference sample map is empty or has an invalid length."); } - TableReader reader = new(stream, entryBytes, scratch, "direct-reference sample map"); + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "direct-reference sample map"); ulong describedSamples = 0; int retainedOffset = 0; uint greatestGroupIndex = 0; @@ -1737,7 +1798,7 @@ internal sealed class HeifSequenceParser } long entryBytes = boxLength - headerLength; - TableReader reader = new(stream, entryBytes, scratch, "direct-reference descriptions"); + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "direct-reference descriptions"); long consumedBytes = 0; int assignmentOffset = 0; int directReferenceCount = 0; @@ -1898,7 +1959,7 @@ internal sealed class HeifSequenceParser throw new InvalidImageContentException("The composition-offset table is empty or has an invalid length."); } - TableReader reader = new(stream, entryBytes, scratch, "composition offsets"); + HeifBoxPayloadReader reader = new(stream, entryBytes, scratch, "composition offsets"); ulong describedSamples = 0; int retainedOffset = 0; long leastOffset = long.MaxValue; @@ -2142,6 +2203,34 @@ internal sealed class HeifSequenceParser return (int)size; } + /// + /// Determines whether a recoverable ancillary-segment error should be ignored by the configured decoder policy. + /// + /// The exception raised while parsing the ancillary segment. + /// when decoding may continue without the segment. + private bool ShouldIgnoreAncillarySegmentError(Exception exception) + => this.options.SegmentIntegrityHandling is not SegmentIntegrityHandling.Strict && IsRecoverableSegmentError(exception); + + /// + /// Determines whether a recoverable image-data-segment error should be ignored by the configured decoder policy. + /// + /// The exception raised while parsing the image-data segment. + /// when decoding may continue without the segment. + private bool ShouldIgnoreImageDataSegmentError(Exception exception) + => this.options.SegmentIntegrityHandling is SegmentIntegrityHandling.IgnoreImageData && IsRecoverableSegmentError(exception); + + /// + /// Identifies the recoverable exception types handled by . + /// + /// The exception raised while parsing a segment. + /// when the exception represents a recoverable segment error. + private static bool IsRecoverableSegmentError(Exception exception) + => exception is ImageFormatException + or InvalidIccProfileException + or InvalidImageContentException + or InvalidOperationException + or NotSupportedException; + /// /// Records one unique child box while retaining only its stream range. /// @@ -2476,141 +2565,4 @@ internal sealed class HeifSequenceParser /// public bool HasHiddenSamples { get; } } - - /// - /// Reads fixed-width sample-table values through one bounded reusable buffer. - /// - private ref struct TableReader - { - /// - /// The source stream shared by the container parser. - /// - private readonly Stream stream; - - /// - /// The parser-owned buffer reused for sequential values. - /// - private readonly Span buffer; - - /// - /// The table name used in malformed-image diagnostics. - /// - private readonly string name; - - /// - /// The number of bytes not yet loaded from the bounded table payload. - /// - private long remaining; - - /// - /// The next unread byte in . - /// - private int offset; - - /// - /// The number of valid bytes currently stored in . - /// - private int count; - - /// - /// Initializes a new instance of the struct. - /// - /// The stream positioned at the table entries. - /// The exact number of bounded entry bytes. - /// The parser-owned reusable buffer. - /// The table name used in malformed-image diagnostics. - public TableReader(Stream stream, long length, Span buffer, string name) - { - this.stream = stream; - this.buffer = buffer; - this.name = name; - this.remaining = length; - this.offset = 0; - this.count = 0; - } - - /// - /// Reads one unsigned byte from the bounded table. - /// - /// The next byte. - public byte ReadByte() - { - this.Ensure(1); - return this.buffer[this.offset++]; - } - - /// - /// Reads one big-endian unsigned 16-bit value from the bounded table. - /// - /// The next 16-bit value. - public ushort ReadUInt16() - { - this.Ensure(2); - ushort value = BinaryPrimitives.ReadUInt16BigEndian(this.buffer[this.offset..]); - this.offset += 2; - return value; - } - - /// - /// Reads one big-endian unsigned 32-bit value from the bounded table. - /// - /// The next 32-bit value. - public uint ReadUInt32() - { - this.Ensure(4); - uint value = BinaryPrimitives.ReadUInt32BigEndian(this.buffer[this.offset..]); - this.offset += 4; - return value; - } - - /// - /// Reads one big-endian unsigned 64-bit value from the bounded table. - /// - /// The next 64-bit value. - public ulong ReadUInt64() - { - this.Ensure(8); - ulong value = BinaryPrimitives.ReadUInt64BigEndian(this.buffer[this.offset..]); - this.offset += 8; - return value; - } - - /// - /// Refills the reusable buffer without reading beyond the bounded table payload. - /// - /// The number of contiguous bytes required by the next value. - private void Ensure(int required) - { - int buffered = this.count - this.offset; - if (buffered >= required) - { - return; - } - - if (buffered > 0) - { - this.buffer.Slice(this.offset, buffered).CopyTo(this.buffer); - } - - this.offset = 0; - this.count = buffered; - while (this.count < required && this.remaining > 0) - { - int requested = (int)Math.Min(this.buffer.Length - this.count, this.remaining); - int read = this.stream.Read(this.buffer.Slice(this.count, requested)); - if (read == 0) - { - throw new InvalidImageContentException($"The {this.name} table is truncated."); - } - - this.count += read; - this.remaining -= read; - } - - if (this.count < required) - { - throw new InvalidImageContentException($"The {this.name} table is truncated."); - } - } - } } diff --git a/src/ImageSharp/Formats/Heif/HeifSequenceTrack.cs b/src/ImageSharp/Formats/Heif/HeifSequenceTrack.cs index 1d3bcbe48..a8748f493 100644 --- a/src/ImageSharp/Formats/Heif/HeifSequenceTrack.cs +++ b/src/ImageSharp/Formats/Heif/HeifSequenceTrack.cs @@ -140,6 +140,11 @@ internal sealed class HeifSequenceTrack /// public byte? MirrorAxis { get; set; } + /// + /// Gets or sets the Exif and XMP item payloads implicitly associated with the image-sequence track. + /// + public HeifSequenceMetadata? Metadata { get; set; } + /// /// Gets or sets the retained sample descriptors in decode order. /// diff --git a/src/ImageSharp/Formats/Heif/HeifTrackMetadataParser.cs b/src/ImageSharp/Formats/Heif/HeifTrackMetadataParser.cs new file mode 100644 index 000000000..def4dbb77 --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifTrackMetadataParser.cs @@ -0,0 +1,718 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers; +using System.Buffers.Binary; +using SixLabors.ImageSharp.Memory; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Parses the Exif and XMP items implicitly associated with one HEIF image-sequence track. +/// +internal sealed class HeifTrackMetadataParser +{ + /// + /// The allocator used for temporary item identifiers and extent descriptors. + /// + private readonly MemoryAllocator allocator; + + /// + /// Initializes a new instance of the class. + /// + /// The allocator used for bounded temporary parser state. + public HeifTrackMetadataParser(MemoryAllocator allocator) => this.allocator = allocator; + + /// + /// Gets the registered MIME content type for XMP metadata without allocating a managed byte array. + /// + private static ReadOnlySpan XmpContentType => "application/rdf+xml"u8; + + /// + /// Parses the bounded metadata items from one track-level metadata box. + /// + /// The seekable HEIF stream positioned at the metadata full-box header. + /// The validated metadata payload length. + /// The caller-owned reusable parser scratch. + /// The retained Exif and XMP payloads. + public HeifSequenceMetadata Parse(Stream stream, long boxLength, Span scratch) + { + long metadataStart = stream.Position; + long metadataEnd = checked(metadataStart + boxLength); + HeifBoxReader.EnsureInsideParent(boxLength, stream.Length - metadataStart); + ReadOnlySpan fullBoxHeader = ReadPrefix(stream, boxLength, scratch, 4, "track metadata"); + if (BinaryPrimitives.ReadUInt32BigEndian(fullBoxHeader) != 0) + { + throw new InvalidImageContentException("The track metadata box has unsupported version or flags."); + } + + BoxReference handler = default; + BoxReference itemInformation = default; + BoxReference itemLocations = default; + BoxReference itemData = default; + bool firstChild = true; + while (stream.Position < metadataEnd) + { + long childLength = HeifBoxReader.ReadHeader(stream, metadataEnd, scratch, out Heif4CharCode childType); + long childStart = stream.Position; + if (firstChild && childType != Heif4CharCode.Hdlr) + { + throw new InvalidImageContentException("The track metadata box does not begin with its picture handler."); + } + + switch (childType) + { + case Heif4CharCode.Hdlr: + SetUnique(ref handler, childStart, childLength, childType); + break; + case Heif4CharCode.Iinf: + SetUnique(ref itemInformation, childStart, childLength, childType); + break; + case Heif4CharCode.Iloc: + SetUnique(ref itemLocations, childStart, childLength, childType); + break; + case Heif4CharCode.Idat: + SetUnique(ref itemData, childStart, childLength, childType); + break; + } + + firstChild = false; + stream.Position = checked(childStart + childLength); + } + + if (!handler.IsPresent) + { + throw new InvalidImageContentException("The track metadata box has no picture handler."); + } + + stream.Position = handler.Offset; + if (ParseHandler(stream, handler.Length, scratch) != Heif4CharCode.Pict) + { + throw new InvalidImageContentException("The track metadata box does not use the picture handler."); + } + + if (!itemInformation.IsPresent && !itemLocations.IsPresent) + { + return new HeifSequenceMetadata(null, null); + } + + if (!itemInformation.IsPresent || !itemLocations.IsPresent) + { + throw new InvalidImageContentException("The track metadata box has incomplete item declarations or locations."); + } + + stream.Position = itemInformation.Offset; + MetadataItemIds itemIds = this.ParseItemInformation(stream, itemInformation.Length, scratch); + byte[]? exifData = itemIds.ExifItemId == 0 + ? null + : this.ReadItemPayload(stream, itemLocations, itemData, itemIds.ExifItemId, scratch); + + byte[]? xmpData = itemIds.XmpItemId == 0 + ? null + : this.ReadItemPayload(stream, itemLocations, itemData, itemIds.XmpItemId, scratch); + + return new HeifSequenceMetadata(exifData, xmpData); + } + + /// + /// Parses item declarations while retaining only the identifiers for Exif and unencoded XMP items. + /// + /// The stream positioned at the item-information full-box header. + /// The validated item-information payload length. + /// The caller-owned reusable parser scratch. + /// The recognized metadata item identifiers. + private MetadataItemIds ParseItemInformation(Stream stream, long boxLength, Span scratch) + { + long itemInformationStart = stream.Position; + long itemInformationEnd = checked(itemInformationStart + boxLength); + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 4, "track item information"); + byte version = prefix[0]; + int headerLength = version switch + { + 0 => 6, + 1 => 8, + _ => throw new InvalidImageContentException($"The track item-information box has unsupported version {version}.") + }; + + prefix = ReadPrefixFromStart(stream, boxLength, scratch, headerLength, "track item information"); + if ((BinaryPrimitives.ReadUInt32BigEndian(prefix) & 0x00FFFFFF) != 0) + { + throw new InvalidImageContentException("The track item-information box has unsupported flags."); + } + + uint entryCount = version == 0 + ? BinaryPrimitives.ReadUInt16BigEndian(prefix[4..]) + : BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]); + + long entryBytes = itemInformationEnd - stream.Position; + if (entryCount > int.MaxValue || entryCount > (ulong)(entryBytes / 8)) + { + throw new InvalidImageContentException("The track item-information entry count exceeds its bounded payload."); + } + + if (entryCount == 0) + { + if (entryBytes != 0) + { + throw new InvalidImageContentException("The empty track item-information box contains trailing data."); + } + + return default; + } + + using IMemoryOwner identifierOwner = this.allocator.Allocate((int)entryCount); + Span identifiers = identifierOwner.GetSpan()[..(int)entryCount]; + MetadataItemIds result = default; + for (int i = 0; i < identifiers.Length; i++) + { + if (stream.Position >= itemInformationEnd) + { + throw new InvalidImageContentException("The track item-information entry count exceeds its bounded payload."); + } + + long entryLength = HeifBoxReader.ReadHeader(stream, itemInformationEnd, scratch, out Heif4CharCode entryType); + if (entryType != Heif4CharCode.Infe) + { + throw new InvalidImageContentException($"The track item-information box contains unexpected child '{entryType}'."); + } + + uint itemId = ParseItemInformationEntry(stream, entryLength, scratch, out Heif4CharCode itemType, out bool isXmp); + identifiers[i] = itemId; + if (itemType == Heif4CharCode.Exif) + { + if (result.ExifItemId != 0) + { + throw new InvalidImageContentException("The image-sequence track declares more than one Exif metadata item."); + } + + result.ExifItemId = itemId; + } + else if (isXmp) + { + if (result.XmpItemId != 0) + { + throw new InvalidImageContentException("The image-sequence track declares more than one XMP metadata item."); + } + + result.XmpItemId = itemId; + } + } + + if (stream.Position != itemInformationEnd) + { + throw new InvalidImageContentException("The track item-information entry count does not consume its bounded payload."); + } + + identifiers.Sort(); + if (identifiers[0] == 0) + { + throw new InvalidImageContentException("The track item-information box declares item identifier zero."); + } + + for (int i = 1; i < identifiers.Length; i++) + { + if (identifiers[i] == identifiers[i - 1]) + { + throw new InvalidImageContentException($"The track item-information box contains duplicate item ID {identifiers[i]}."); + } + } + + return result; + } + + /// + /// Parses one item-information entry without materializing its name or MIME strings. + /// + /// The stream positioned at the item-information-entry full-box header. + /// The validated item-information-entry payload length. + /// The caller-owned reusable parser scratch. + /// Receives the explicit item type for version two or three entries. + /// Receives whether the entry declares an unencoded XMP MIME item. + /// The positive item identifier. + private static uint ParseItemInformationEntry( + Stream stream, + long boxLength, + Span scratch, + out Heif4CharCode itemType, + out bool isXmp) + { + long entryStart = stream.Position; + long entryEnd = checked(entryStart + boxLength); + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 4, "track item-information entry"); + byte version = prefix[0]; + int fixedLength = version switch + { + 0 or 1 => 8, + 2 => 12, + 3 => 14, + _ => throw new InvalidImageContentException($"The track item-information entry has unsupported version {version}.") + }; + + prefix = ReadPrefixFromStart(stream, boxLength, scratch, fixedLength, "track item-information entry"); + uint flags = BinaryPrimitives.ReadUInt32BigEndian(prefix) & 0x00FFFFFF; + if ((flags & ~1U) != 0) + { + throw new InvalidImageContentException("The track item-information entry has unsupported flags."); + } + + int itemIdOffset = 4; + uint itemId = version == 3 + ? BinaryPrimitives.ReadUInt32BigEndian(prefix[itemIdOffset..]) + : BinaryPrimitives.ReadUInt16BigEndian(prefix[itemIdOffset..]); + + int protectionOffset = version == 3 ? 8 : 6; + if (BinaryPrimitives.ReadUInt16BigEndian(prefix[protectionOffset..]) != 0) + { + throw new InvalidImageContentException($"Track metadata item {itemId} uses unsupported item protection."); + } + + itemType = version >= 2 + ? (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(prefix[(protectionOffset + 2)..]) + : default; + + isXmp = false; + if (version < 2 || itemType is not Heif4CharCode.Exif and not Heif4CharCode.Mime) + { + stream.Position = entryEnd; + return itemId; + } + + HeifBoxPayloadReader reader = new(stream, entryEnd - stream.Position, scratch, "track item-information entry"); + reader.SkipNullTerminatedString(); + if (itemType == Heif4CharCode.Mime) + { + isXmp = reader.ReadNullTerminatedStringEquals(XmpContentType); + if (reader.Remaining > 0) + { + // ImageSharp cannot decode an encoded MIME payload. Retain XMP only when the optional encoding is empty. + isXmp &= reader.ReadNullTerminatedStringEquals(ReadOnlySpan.Empty); + } + } + + if (!reader.IsComplete) + { + throw new InvalidImageContentException($"Track metadata item {itemId} contains unexpected trailing data."); + } + + return itemId; + } + + /// + /// Resolves and reads one recognized metadata item while pooling only its transient extent descriptors. + /// + /// The seekable HEIF stream. + /// The item-location payload range. + /// The optional item-data payload range. + /// The recognized metadata item identifier. + /// The caller-owned reusable parser scratch. + /// The exact retained item payload, or for an item without data. + private byte[]? ReadItemPayload( + Stream stream, + BoxReference itemLocations, + BoxReference itemData, + uint itemId, + Span scratch) + { + stream.Position = itemLocations.Offset; + ItemLocationSummary summary = ParseItemLocation( + stream, + itemLocations.Length, + itemData, + itemId, + Span.Empty, + scratch); + + if (summary.ExtentCount == 0 || summary.TotalLength == 0) + { + return null; + } + + using IMemoryOwner extentOwner = this.allocator.Allocate(summary.ExtentCount); + Span extents = extentOwner.GetSpan()[..summary.ExtentCount]; + stream.Position = itemLocations.Offset; + ItemLocationSummary verifiedSummary = ParseItemLocation(stream, itemLocations.Length, itemData, itemId, extents, scratch); + if (verifiedSummary.TotalLength != summary.TotalLength) + { + throw new InvalidImageContentException($"Track metadata item {itemId} changed between location parser passes."); + } + + // The payload survives parser disposal and is handed directly to ImageSharp's metadata profile model. + byte[] data = new byte[summary.TotalLength]; + int destinationOffset = 0; + foreach (MetadataExtent extent in extents) + { + stream.Position = extent.Offset; + HeifBoxReader.ReadExactly( + stream, + data.AsSpan(destinationOffset, extent.Length), + $"Track metadata item {itemId} has a truncated extent."); + + destinationOffset += extent.Length; + } + + return data; + } + + /// + /// Parses one item-location box for a selected metadata item. + /// + /// The stream positioned at the item-location full-box header. + /// The validated item-location payload length. + /// The optional item-data payload range. + /// The metadata item identifier whose extents are retained. + /// The exact target extent span, or an empty span for the counting pass. + /// The caller-owned reusable parser scratch. + /// The target item's extent count and total payload length. + private static ItemLocationSummary ParseItemLocation( + Stream stream, + long boxLength, + BoxReference itemData, + uint targetItemId, + Span targetExtents, + Span scratch) + { + HeifBoxPayloadReader reader = new(stream, boxLength, scratch, "track item location"); + uint versionAndFlags = reader.ReadUInt32(); + byte version = (byte)(versionAndFlags >> 24); + if (version > 2 || (versionAndFlags & 0x00FFFFFF) != 0) + { + throw new InvalidImageContentException("The track item-location box has unsupported version or flags."); + } + + byte offsetAndLengthSizes = reader.ReadByte(); + byte baseAndIndexSizes = reader.ReadByte(); + int offsetSize = offsetAndLengthSizes >> 4; + int lengthSize = offsetAndLengthSizes & 15; + int baseOffsetSize = baseAndIndexSizes >> 4; + int indexSize = version is 1 or 2 ? baseAndIndexSizes & 15 : 0; + if (!IsSupportedFieldSize(offsetSize) + || !IsSupportedFieldSize(lengthSize) + || !IsSupportedFieldSize(baseOffsetSize) + || !IsSupportedFieldSize(indexSize)) + { + throw new InvalidImageContentException("The track item-location box uses an unsupported integer field size."); + } + + uint itemCount = version == 2 ? reader.ReadUInt32() : reader.ReadUInt16(); + int minimumItemLength = version == 0 ? 6 : 8; + if (itemCount > (ulong)(reader.Remaining / minimumItemLength)) + { + throw new InvalidImageContentException("The track item-location count exceeds its bounded payload."); + } + + bool targetFound = false; + int targetExtentIndex = 0; + int targetLength = 0; + for (uint i = 0; i < itemCount; i++) + { + uint itemId = version == 2 ? reader.ReadUInt32() : reader.ReadUInt16(); + int constructionMethod = 0; + if (version is 1 or 2) + { + ushort constructionField = reader.ReadUInt16(); + if ((constructionField & 0xFFF0) != 0) + { + throw new InvalidImageContentException("The track item-location box has nonzero reserved construction bits."); + } + + constructionMethod = constructionField & 15; + } + + if (constructionMethod is not 0 and not 1) + { + throw new InvalidImageContentException($"The track item-location box uses unsupported construction method {constructionMethod}."); + } + + if (reader.ReadUInt16() != 0) + { + throw new InvalidImageContentException("External track metadata data references are not supported."); + } + + ulong baseOffset = reader.ReadVariableUInt(baseOffsetSize); + int extentCount = reader.ReadUInt16(); + bool isTarget = itemId == targetItemId; + if (isTarget) + { + if (targetFound) + { + throw new InvalidImageContentException($"The track item-location box contains duplicate locations for item ID {itemId}."); + } + + if (!targetExtents.IsEmpty && targetExtents.Length != extentCount) + { + throw new InvalidImageContentException($"Track metadata item {itemId} changed between location parser passes."); + } + + targetFound = true; + } + + for (int j = 0; j < extentCount; j++) + { + if (version is 1 or 2 && indexSize > 0) + { + _ = reader.ReadVariableUInt(indexSize); + } + + ulong extentOffset = reader.ReadVariableUInt(offsetSize); + ulong extentLength = reader.ReadVariableUInt(lengthSize); + if (!isTarget) + { + continue; + } + + MetadataExtent extent = ResolveExtent(stream, itemData, constructionMethod, baseOffset, extentOffset, extentLength, itemId); + if (!targetExtents.IsEmpty) + { + targetExtents[targetExtentIndex] = extent; + } + + targetExtentIndex++; + if (extent.Length > int.MaxValue - targetLength) + { + throw new InvalidImageContentException($"Track metadata item {itemId} has an unsupported combined length."); + } + + targetLength += extent.Length; + } + } + + if (!reader.IsComplete) + { + throw new InvalidImageContentException("The track item-location box contains unexpected trailing data."); + } + + return targetFound ? new ItemLocationSummary(targetExtentIndex, targetLength) : default; + } + + /// + /// Resolves one file-relative or item-data-relative extent into an absolute stream range. + /// + /// The complete seekable HEIF stream. + /// The optional item-data payload range. + /// The item-location construction method. + /// The item-location base offset. + /// The extent offset relative to the base offset. + /// The extent length in bytes. + /// The metadata item identifier used in malformed-image diagnostics. + /// The validated absolute extent. + private static MetadataExtent ResolveExtent( + Stream stream, + BoxReference itemData, + int constructionMethod, + ulong baseOffset, + ulong extentOffset, + ulong extentLength, + uint itemId) + { + if (baseOffset > ulong.MaxValue - extentOffset || extentLength > int.MaxValue) + { + throw new InvalidImageContentException($"Track metadata item {itemId} has an unsupported extent range."); + } + + ulong relativeOffset = baseOffset + extentOffset; + ulong origin = 0; + if (constructionMethod == 1) + { + if (!itemData.IsPresent || relativeOffset > (ulong)itemData.Length || extentLength > (ulong)itemData.Length - relativeOffset) + { + throw new InvalidImageContentException($"Track metadata item {itemId} has an extent outside its item-data box."); + } + + origin = (ulong)itemData.Offset; + } + + if (relativeOffset > ulong.MaxValue - origin) + { + throw new InvalidImageContentException($"Track metadata item {itemId} has an unsupported extent offset."); + } + + ulong absoluteOffset = origin + relativeOffset; + if (absoluteOffset > (ulong)stream.Length || extentLength > (ulong)stream.Length - absoluteOffset) + { + throw new InvalidImageContentException($"Track metadata item {itemId} has an extent outside the HEIF stream."); + } + + return new MetadataExtent((long)absoluteOffset, (int)extentLength); + } + + /// + /// Parses one picture handler and returns its registered handler type. + /// + /// The stream positioned at the handler full-box header. + /// The validated handler payload length. + /// The caller-owned reusable parser scratch. + /// The registered handler type. + private static Heif4CharCode ParseHandler(Stream stream, long boxLength, Span scratch) + { + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 24, "track metadata handler"); + if (BinaryPrimitives.ReadUInt32BigEndian(prefix) != 0 || BinaryPrimitives.ReadUInt32BigEndian(prefix[4..]) != 0) + { + throw new InvalidImageContentException("The track metadata handler has unsupported fields."); + } + + return (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(prefix[8..]); + } + + /// + /// Determines whether an item-location integer width is supported by the bounded reader. + /// + /// The width in bytes. + /// for zero-width, 32-bit, or 64-bit fields. + private static bool IsSupportedFieldSize(int size) => size is 0 or 4 or 8; + + /// + /// Reads a fixed prefix from the stream's current position. + /// + /// The source stream. + /// The validated enclosing payload length. + /// The caller-owned reusable parser scratch. + /// The required prefix length. + /// The payload name used in malformed-image diagnostics. + /// The requested prefix within . + private static ReadOnlySpan ReadPrefix(Stream stream, long boxLength, Span scratch, int length, string name) + { + if (boxLength < length) + { + throw new InvalidImageContentException($"The {name} payload is truncated."); + } + + Span destination = scratch[..length]; + HeifBoxReader.ReadExactly(stream, destination, $"The {name} payload is truncated."); + return destination; + } + + /// + /// Rewinds to the start of a partially read payload and reads a larger fixed prefix. + /// + /// The source stream positioned after a four-byte prefix. + /// The validated enclosing payload length. + /// The caller-owned reusable parser scratch. + /// The required prefix length. + /// The payload name used in malformed-image diagnostics. + /// The requested prefix within . + private static ReadOnlySpan ReadPrefixFromStart(Stream stream, long boxLength, Span scratch, int length, string name) + { + stream.Position -= 4; + return ReadPrefix(stream, boxLength, scratch, length, name); + } + + /// + /// Records one unique metadata child while retaining only its stream range. + /// + /// The child reference owned by the metadata parser. + /// The absolute payload offset. + /// The validated payload length. + /// The unique child box type. + private static void SetUnique(ref BoxReference reference, long offset, long length, Heif4CharCode boxType) + { + if (reference.IsPresent) + { + throw new InvalidImageContentException($"The track metadata box contains duplicate '{boxType}' boxes."); + } + + reference = new BoxReference(offset, length); + } + + /// + /// Retains the recognized item identifiers from one track metadata box. + /// + private struct MetadataItemIds + { + /// + /// Gets or sets the Exif item identifier, or zero when absent. + /// + public uint ExifItemId { get; set; } + + /// + /// Gets or sets the XMP item identifier, or zero when absent. + /// + public uint XmpItemId { get; set; } + } + + /// + /// Retains one unique child payload range without creating a generic metadata box model. + /// + private readonly struct BoxReference + { + /// + /// Initializes a new instance of the struct. + /// + /// The absolute payload offset. + /// The validated payload length. + public BoxReference(long offset, long length) + { + this.Offset = offset; + this.Length = length; + this.IsPresent = true; + } + + /// + /// Gets the absolute payload offset. + /// + public long Offset { get; } + + /// + /// Gets the validated payload length. + /// + public long Length { get; } + + /// + /// Gets a value indicating whether the child was present. + /// + public bool IsPresent { get; } + } + + /// + /// Describes one absolute metadata item extent. + /// + private readonly struct MetadataExtent + { + /// + /// Initializes a new instance of the struct. + /// + /// The absolute stream offset. + /// The extent length in bytes. + public MetadataExtent(long offset, int length) + { + this.Offset = offset; + this.Length = length; + } + + /// + /// Gets the absolute stream offset. + /// + public long Offset { get; } + + /// + /// Gets the extent length in bytes. + /// + public int Length { get; } + } + + /// + /// Summarizes the retained extents for one metadata item. + /// + private readonly struct ItemLocationSummary + { + /// + /// Initializes a new instance of the struct. + /// + /// The number of retained extents. + /// The combined payload length. + public ItemLocationSummary(int extentCount, int totalLength) + { + this.ExtentCount = extentCount; + this.TotalLength = totalLength; + } + + /// + /// Gets the number of retained extents. + /// + public int ExtentCount { get; } + + /// + /// Gets the combined payload length. + /// + public int TotalLength { get; } + } +} diff --git a/tests/ImageSharp.Tests/Formats/Heif/HeifSequenceParserTests.cs b/tests/ImageSharp.Tests/Formats/Heif/HeifSequenceParserTests.cs index 60d8ba3fa..844bcaa58 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/HeifSequenceParserTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/HeifSequenceParserTests.cs @@ -3,6 +3,7 @@ using System.Buffers.Binary; using System.Text; +using SixLabors.ImageSharp.Formats; using SixLabors.ImageSharp.Formats.Heif; namespace SixLabors.ImageSharp.Tests.Formats.Heif; @@ -11,12 +12,19 @@ namespace SixLabors.ImageSharp.Tests.Formats.Heif; [ValidateDisposedMemoryAllocations] public class HeifSequenceParserTests { + private const int TrackExifOffset = 1800; + private const int TrackXmpOffset = 1840; + + private static ReadOnlySpan TrackExifData => [0, 0, 0, 0, 0x49, 0x49, 0x2A, 0, 8, 0, 0, 0]; + + private static ReadOnlySpan TrackXmpData => ""u8; + [Fact] public void ParseResolvesLibavifShapedSampleTable() { byte[] data = CreateSequenceFile(1024); using MemoryStream stream = new(data, false); - HeifSequenceParser parser = new(Configuration.Default.MemoryAllocator, 2); + HeifSequenceParser parser = CreateParser(2); stream.Position = 8; HeifSequence sequence = parser.Parse(stream, GetMoviePayloadLength(data)); @@ -56,7 +64,7 @@ public class HeifSequenceParserTests { byte[] data = CreateSequenceFile(1024); using MemoryStream stream = new(data, false); - HeifSequenceParser parser = new(Configuration.Default.MemoryAllocator, 1); + HeifSequenceParser parser = CreateParser(1); stream.Position = 8; HeifSequence sequence = parser.Parse(stream, GetMoviePayloadLength(data)); @@ -73,7 +81,7 @@ public class HeifSequenceParserTests { byte[] data = CreateSequenceFile(2040); using MemoryStream stream = new(data, false); - HeifSequenceParser parser = new(Configuration.Default.MemoryAllocator, 2); + HeifSequenceParser parser = CreateParser(2); stream.Position = 8; Assert.Throws(() => parser.Parse(stream, GetMoviePayloadLength(data))); @@ -84,7 +92,7 @@ public class HeifSequenceParserTests { byte[] data = CreateSequenceFile(1024, hevc: true, compositionOffsets: true); using MemoryStream stream = new(data, false); - HeifSequenceParser parser = new(Configuration.Default.MemoryAllocator, 2); + HeifSequenceParser parser = CreateParser(2); stream.Position = 8; HeifSequence sequence = parser.Parse(stream, GetMoviePayloadLength(data)); @@ -102,7 +110,7 @@ public class HeifSequenceParserTests { byte[] data = CreateSequenceFile(1024, compositionOffsets: true); using MemoryStream stream = new(data, false); - HeifSequenceParser parser = new(Configuration.Default.MemoryAllocator, 2); + HeifSequenceParser parser = CreateParser(2); stream.Position = 8; Assert.Throws(() => parser.Parse(stream, GetMoviePayloadLength(data))); @@ -113,7 +121,7 @@ public class HeifSequenceParserTests { byte[] data = CreateSequenceFile(1024, directReferences: true); using MemoryStream stream = new(data, false); - HeifSequenceParser parser = new(Configuration.Default.MemoryAllocator, 2); + HeifSequenceParser parser = CreateParser(2); stream.Position = 8; HeifSequence sequence = parser.Parse(stream, GetMoviePayloadLength(data)); @@ -131,7 +139,7 @@ public class HeifSequenceParserTests { byte[] data = CreateSequenceFile(1024, directReferences: true, directReferenceSampleId: 2); using MemoryStream stream = new(data, false); - HeifSequenceParser parser = new(Configuration.Default.MemoryAllocator, 2); + HeifSequenceParser parser = CreateParser(2); stream.Position = 8; Assert.Throws(() => parser.Parse(stream, GetMoviePayloadLength(data))); @@ -142,7 +150,7 @@ public class HeifSequenceParserTests { byte[] data = CreateSequenceFile(1024, trackProperties: true); using MemoryStream stream = new(data, false); - HeifSequenceParser parser = new(Configuration.Default.MemoryAllocator, 2); + HeifSequenceParser parser = CreateParser(2); stream.Position = 8; HeifSequenceTrack track = parser.Parse(stream, GetMoviePayloadLength(data)).ColorTrack; @@ -161,13 +169,96 @@ public class HeifSequenceParserTests Assert.NotNull(track.NominalDiffuseWhite); } + [Theory] + [InlineData(false)] + [InlineData(true)] + public void ParseRetainsBoundedTrackMetadata(bool useItemData) + { + byte[] data = CreateSequenceFile(1024, trackMetadata: true, metadataInItemData: useItemData); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(2); + stream.Position = 8; + + HeifSequenceTrack track = parser.Parse(stream, GetMoviePayloadLength(data)).ColorTrack; + + Assert.NotNull(track.Metadata); + Assert.Equal(TrackExifData.ToArray(), track.Metadata.ExifData); + Assert.Equal(TrackXmpData.ToArray(), track.Metadata.XmpData); + } + + [Fact] + public void ParseDoesNotValidateOrRetainSkippedTrackMetadata() + { + byte[] data = CreateSequenceFile(1024, trackMetadata: true, invalidTrackMetadata: true); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(2, skipMetadata: true); + stream.Position = 8; + + HeifSequenceTrack track = parser.Parse(stream, GetMoviePayloadLength(data)).ColorTrack; + + Assert.Null(track.Metadata); + Assert.Equal(2, track.Samples.Length); + } + + [Fact] + public void ParseUsesAncillaryIntegrityPolicyForTrackMetadata() + { + byte[] data = CreateSequenceFile(1024, trackMetadata: true, invalidTrackMetadata: true); + using MemoryStream strictStream = new(data, false); + HeifSequenceParser strictParser = CreateParser(2); + strictStream.Position = 8; + + Assert.Throws(() => strictParser.Parse(strictStream, GetMoviePayloadLength(data))); + + using MemoryStream tolerantStream = new(data, false); + HeifSequenceParser tolerantParser = CreateParser(2, segmentIntegrityHandling: SegmentIntegrityHandling.IgnoreAncillary); + tolerantStream.Position = 8; + + HeifSequenceTrack track = tolerantParser.Parse(tolerantStream, GetMoviePayloadLength(data)).ColorTrack; + + Assert.Null(track.Metadata); + Assert.Equal(2, track.Samples.Length); + } + + [Theory] + [InlineData(SegmentIntegrityHandling.Strict)] + [InlineData(SegmentIntegrityHandling.IgnoreAncillary)] + public void ParseRejectsInvalidPresentationPropertyUnlessImageDataErrorsAreIgnored(SegmentIntegrityHandling handling) + { + byte[] data = CreateSequenceFile(1024, trackProperties: true, invalidRotation: true); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(2, segmentIntegrityHandling: handling); + stream.Position = 8; + + Assert.Throws(() => parser.Parse(stream, GetMoviePayloadLength(data))); + } + + [Fact] + public void ParseOmitsInvalidPresentationPropertyWhenImageDataErrorsAreIgnored() + { + byte[] data = CreateSequenceFile(1024, trackProperties: true, invalidRotation: true); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = CreateParser(2, segmentIntegrityHandling: SegmentIntegrityHandling.IgnoreImageData); + stream.Position = 8; + + HeifSequenceTrack track = parser.Parse(stream, GetMoviePayloadLength(data)).ColorTrack; + + Assert.Null(track.RotationAngle); + Assert.NotNull(track.PixelAspectRatio); + Assert.Equal(2, track.Samples.Length); + } + private static byte[] CreateSequenceFile( uint chunkOffset, bool hevc = false, bool compositionOffsets = false, bool directReferences = false, uint directReferenceSampleId = 1, - bool trackProperties = false) + bool trackProperties = false, + bool trackMetadata = false, + bool metadataInItemData = false, + bool invalidTrackMetadata = false, + bool invalidRotation = false) { using MemoryStream stream = new(); using BinaryWriter writer = new(stream, Encoding.UTF8, true); @@ -185,6 +276,10 @@ public class HeifSequenceParserTests long track = BeginBox(writer, Heif4CharCode.Trak); WriteTrackHeader(writer); WriteEditList(writer); + if (trackMetadata) + { + WriteTrackMetadata(writer, metadataInItemData, invalidTrackMetadata); + } long media = BeginBox(writer, Heif4CharCode.Mdia); WriteMediaHeader(writer); @@ -192,7 +287,16 @@ public class HeifSequenceParserTests long mediaInformation = BeginBox(writer, Heif4CharCode.Minf); WriteDataInformation(writer); - WriteSampleTable(writer, chunkOffset, hevc, compositionOffsets, directReferences, directReferenceSampleId, trackProperties); + WriteSampleTable( + writer, + chunkOffset, + hevc, + compositionOffsets, + directReferences, + directReferenceSampleId, + trackProperties, + invalidRotation); + EndBox(writer, mediaInformation); EndBox(writer, media); EndBox(writer, track); @@ -202,6 +306,12 @@ public class HeifSequenceParserTests byte[] file = new byte[2048]; movieBytes.CopyTo(file, 0); + if (trackMetadata && !metadataInItemData) + { + TrackExifData.CopyTo(file.AsSpan(TrackExifOffset)); + TrackXmpData.CopyTo(file.AsSpan(TrackXmpOffset)); + } + return file; } @@ -280,6 +390,76 @@ public class HeifSequenceParserTests EndBox(writer, dataInformation); } + private static void WriteTrackMetadata(BinaryWriter writer, bool useItemData, bool invalidHandler) + { + long metadata = BeginBox(writer, Heif4CharCode.Meta); + WriteFullBoxHeader(writer, 0, 0); + WriteHandler(writer, invalidHandler ? Heif4CharCode.Vide : Heif4CharCode.Pict); + + long itemLocations = BeginBox(writer, Heif4CharCode.Iloc); + WriteFullBoxHeader(writer, useItemData ? (byte)1 : (byte)0, 0); + writer.Write((byte)0x44); + writer.Write((byte)0); + WriteUInt16(writer, 2); + WriteTrackMetadataLocation(writer, 1, useItemData, useItemData ? 0U : TrackExifOffset, (uint)TrackExifData.Length); + WriteTrackMetadataLocation( + writer, + 2, + useItemData, + useItemData ? (uint)TrackExifData.Length : TrackXmpOffset, + (uint)TrackXmpData.Length); + + EndBox(writer, itemLocations); + + long itemInformation = BeginBox(writer, Heif4CharCode.Iinf); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt16(writer, 2); + WriteTrackMetadataItem(writer, 1, Heif4CharCode.Exif); + WriteTrackMetadataItem(writer, 2, Heif4CharCode.Mime); + EndBox(writer, itemInformation); + + if (useItemData) + { + long itemData = BeginBox(writer, Heif4CharCode.Idat); + writer.Write(TrackExifData); + writer.Write(TrackXmpData); + EndBox(writer, itemData); + } + + EndBox(writer, metadata); + } + + private static void WriteTrackMetadataLocation(BinaryWriter writer, ushort itemId, bool useItemData, uint offset, uint length) + { + WriteUInt16(writer, itemId); + if (useItemData) + { + WriteUInt16(writer, 1); + } + + WriteUInt16(writer, 0); + WriteUInt16(writer, 1); + WriteUInt32(writer, offset); + WriteUInt32(writer, length); + } + + private static void WriteTrackMetadataItem(BinaryWriter writer, ushort itemId, Heif4CharCode itemType) + { + long itemInformationEntry = BeginBox(writer, Heif4CharCode.Infe); + WriteFullBoxHeader(writer, 2, 0); + WriteUInt16(writer, itemId); + WriteUInt16(writer, 0); + WriteUInt32(writer, (uint)itemType); + writer.Write((byte)0); + if (itemType == Heif4CharCode.Mime) + { + writer.Write("application/rdf+xml"u8); + writer.Write((byte)0); + } + + EndBox(writer, itemInformationEntry); + } + private static void WriteSampleTable( BinaryWriter writer, uint chunkOffset, @@ -287,10 +467,11 @@ public class HeifSequenceParserTests bool compositionOffsets, bool directReferences, uint directReferenceSampleId, - bool trackProperties) + bool trackProperties, + bool invalidRotation) { long sampleTable = BeginBox(writer, Heif4CharCode.Stbl); - WriteSampleDescription(writer, hevc, trackProperties); + WriteSampleDescription(writer, hevc, trackProperties, invalidRotation); long timing = BeginBox(writer, Heif4CharCode.Stts); WriteFullBoxHeader(writer, 0, 0); @@ -383,7 +564,7 @@ public class HeifSequenceParserTests EndBox(writer, sampleMap); } - private static void WriteSampleDescription(BinaryWriter writer, bool hevc, bool trackProperties) + private static void WriteSampleDescription(BinaryWriter writer, bool hevc, bool trackProperties, bool invalidRotation) { long description = BeginBox(writer, Heif4CharCode.Stsd); WriteFullBoxHeader(writer, 0, 0); @@ -415,7 +596,7 @@ public class HeifSequenceParserTests if (trackProperties) { - WriteTrackImageProperties(writer); + WriteTrackImageProperties(writer, invalidRotation); } long codingConstraints = BeginBox(writer, Heif4CharCode.Ccst); @@ -426,7 +607,7 @@ public class HeifSequenceParserTests EndBox(writer, description); } - private static void WriteTrackImageProperties(BinaryWriter writer) + private static void WriteTrackImageProperties(BinaryWriter writer, bool invalidRotation) { long color = BeginBox(writer, Heif4CharCode.Colr); WriteUInt32(writer, (uint)Heif4CharCode.Nclx); @@ -453,7 +634,7 @@ public class HeifSequenceParserTests EndBox(writer, cleanAperture); long rotation = BeginBox(writer, Heif4CharCode.Irot); - writer.Write((byte)1); + writer.Write(invalidRotation ? (byte)0xFC : (byte)1); EndBox(writer, rotation); long mirror = BeginBox(writer, Heif4CharCode.Imir); @@ -555,4 +736,15 @@ public class HeifSequenceParserTests private static int GetMoviePayloadLength(byte[] data) => checked((int)BinaryPrimitives.ReadUInt32BigEndian(data) - 8); + + private static HeifSequenceParser CreateParser( + uint maxFrames, + bool skipMetadata = false, + SegmentIntegrityHandling segmentIntegrityHandling = SegmentIntegrityHandling.Strict) + => new(new DecoderOptions + { + MaxFrames = maxFrames, + SkipMetadata = skipMetadata, + SegmentIntegrityHandling = segmentIntegrityHandling + }); }