From 5ddd1bbc05479e1ff643b968c9db04c823a1cd7b Mon Sep 17 00:00:00 2001 From: James Jackson-South Date: Tue, 25 Aug 2026 15:07:15 +1000 Subject: [PATCH] Parse HEIF sequence image properties --- HEIF_IMPLEMENTATION_PLAN.md | 8 +- .../Formats/Heif/HeifDecoderCore.cs | 365 ++--------------- .../Formats/Heif/HeifPixelAspectRatio.cs | 2 +- .../Formats/Heif/HeifPropertyParser.cs | 369 ++++++++++++++++++ .../Formats/Heif/HeifSequenceParser.cs | 185 +++++++++ .../Formats/Heif/HeifSequenceTrack.cs | 62 +++ .../Formats/Heif/HeifSequenceParserTests.cs | 122 +++++- 7 files changed, 761 insertions(+), 352 deletions(-) create mode 100644 src/ImageSharp/Formats/Heif/HeifPropertyParser.cs diff --git a/HEIF_IMPLEMENTATION_PLAN.md b/HEIF_IMPLEMENTATION_PLAN.md index be5948ef0..e9c627885 100644 --- a/HEIF_IMPLEMENTATION_PLAN.md +++ b/HEIF_IMPLEMENTATION_PLAN.md @@ -57,7 +57,9 @@ Checkboxes may be marked complete only when the implementation and the verificat - Release verification covers signed and unsigned composition-offset syntax, hidden-sample visibility, composition-time calculation, required `cslg` and edit-list signaling, complete run counts, and the AV1 prohibition without buffering either table. - [x] Parse optional direct dependencies from `refs` sample groups into a compact retained-sample dependency graph. - The parser validates complete `sbgp` runs and `sgpd` entry lengths, versions, indices, positive identifier uniqueness, decode order, sync-sample independence, and `ccst` reference limits. It uses pooled value-type sort indexes plus one exact flat reference array, with no per-sample collections or attacker-sized table materialization. - - [ ] Parse track presentation/color/HDR properties and bounded sequence metadata items. + - [x] Parse track presentation, color, and HDR properties through the same typed property parser used by still-image items. + - Selected visual sample entries now retain ICC and CICP color profiles, `pasp`, `clap`, `irot`, `imir`, `clli`, `mdcv`, `cclv`, `amve`, `reve`, and `ndwt`. Fixed-size properties use the parser's existing scratch buffer; only retained ICC bytes use bounded pooled staging before entering ImageSharp's existing profile model. + - [ ] Parse bounded sequence metadata items. - [ ] Connect the parsed sequence index to HEIF detection, Identify, frame decode, alpha matching, and frame metadata without changing still-image source selection. - [ ] Write the same bounded movie, track, sample-description, location, dependency, timing, repetition, alpha, and metadata syntax from ImageSharp frames. - [ ] Decode frame dependencies, durations, repetition, frame-local auxiliary images, and frame-local metadata into the existing ImageSharp multi-frame model. @@ -186,8 +188,8 @@ This snapshot pins or classifies the available references and failures; it does | `HeifDecoderCore.ReadBoxHeader` and `HeifDecoderCore.ParseBoxHeader` | ISO/IEC 14496-12 section 4.2.2 basic box syntax | libavif `src/stream.c` functions `avifROStreamReadBoxHeaderPartial` and `avifROStreamReadBoxHeader` at `092276ce89098ead06db80975173191e5fee1826` | Resolve 32-bit, 64-bit, UUID, and top-level size-zero boxes into content lengths only after validating the complete variable-sized header and the remaining parent boundary. Nested size-zero boxes are invalid; large skips retain 64-bit offsets. | | `HeifDecoderCore.ParseMetadata` | ISO/IEC 14496-12 `MetaBox` and HEIF item declarations, locations, properties, and associations | libavif `src/read.c` functions `avifParseMetaBox`, `avifMetaFindOrCreateItem`, `avifParseItemLocationBox`, and `avifParseItemPropertiesBox` at `092276ce89098ead06db80975173191e5fee1826` | Index unique recognized metadata children by type and payload location, then parse them in dependency order so physical placement does not control item lookup or property association. Duplicate unique children and truncated full-box headers are invalid. | | `HeifDecoderCore.ApplyAssociatedMetadata` | HEIF Annex A Exif item data, MIME metadata items, and `cdsc` item references | libavif `src/read.c` function `avifDecoderFindMetadata`, `src/exif.c` function `avifGetExifTiffHeaderOffset`, and the Exif/XMP item writing paths in `src/write.c` at `092276ce89098ead06db80975173191e5fee1826` | Resolve only metadata items whose `cdsc` reference identifies the decoded primary image, validate the Exif TIFF-header offset, and attach Exif or `application/rdf+xml` XMP through ImageSharp's existing profile types. This is a bounded still-image metadata path; it does not introduce a generic ISO BMFF metadata, media, or track model. | -| `HeifDecoderCore` color-property parsing/association, `HeifItem` color profiles, and `Av1Decoder` effective color validation | ISO/IEC 14496-12 section 12.1.5 color information; HEIF section 6.5.5.1 color-information properties; AV1-ISOBMFF section 2.3.4 configuration semantics | libavif `src/read.c` functions `avifParseColourInformationBox`, `avifReadColorNclxProperty`, and `avifReadColorProperties`, plus `src/write.c` function `avifEncoderWriteColorProperties`, at `092276ce89098ead06db80975173191e5fee1826` | Associate at most one ICC and one `nclx` property with each presented color image item, validate ICC payloads and CICP reserved bits, expose them through ImageSharp's existing profile types, and inherit a grid's CICP description only for tiles that do not declare one. For AV1 items, require every specified sequence-header CICP field and the range flag to match `nclx`; use the item property only to supply sequence-header fields coded as unspecified before color conversion, then expose the effective CICP values even when no item property is present. Retain only the two image color profiles; do not add a reusable color-box, sample-entry, track, or media model. | -| `HeifPixelAspectRatio`, `HeifItem.PixelAspectRatio`, and `HeifDecoderCore.ApplyItemPixelAspectRatioMetadata` | ISO/IEC 14496-12 section 12.1.4.3 pixel aspect ratio | libavif `src/read.c` function `avifParsePixelAspectRatioBox`, `src/write.c` function `avifEncoderWritePaspProperty`, and the presented-image property selection in `src/read.c` at `092276ce89098ead06db80975173191e5fee1826` | Preserve the two unsigned 32-bit relative spacings on the associated image item, reject zero or duplicate ratios, and map the displayed pixel width-to-height ratio into ImageSharp's existing unitless resolution metadata. Exchange the metadata axes after a quarter-turn presentation rotation and fall back from a derived grid to its first decodable tile only when the grid does not declare `pasp`. This remains one still-image presentation property and introduces no generic transform, sample-entry, or display model. | +| `HeifPropertyParser`, `HeifDecoderCore` color-property association, `HeifItem` and `HeifSequenceTrack` color profiles, and `Av1Decoder` effective color validation | ISO/IEC 14496-12 section 12.1.5 color information; HEIF section 6.5.5.1 color-information properties; AV1-ISOBMFF section 2.3.4 configuration semantics | libavif `src/read.c` functions `avifParseColourInformationBox`, `avifReadColorNclxProperty`, and `avifReadColorProperties`, plus `src/write.c` function `avifEncoderWriteColorProperties`, at `092276ce89098ead06db80975173191e5fee1826` | Associate at most one ICC and one `nclx` property with each presented color image item or selected image-sequence sample entry, validate ICC payloads and CICP reserved bits, expose them through ImageSharp's existing profile types, and inherit a grid's CICP description only for tiles that do not declare one. For AV1 images, require every specified sequence-header CICP field and the range flag to match `nclx`; use the container property only to supply sequence-header fields coded as unspecified before color conversion, then expose the effective CICP values even when no property is present. Retain only the two image color profiles; do not add a reusable color-box or media model. | +| `HeifPropertyParser`, `HeifPixelAspectRatio`, item/track presentation state, and `HeifDecoderCore.ApplyItemPixelAspectRatioMetadata` | ISO/IEC 14496-12 section 12.1.4.3 pixel aspect ratio | libavif `src/read.c` function `avifParsePixelAspectRatioBox`, `src/write.c` functions `avifEncoderWritePaspProperty` and `avifEncoderWriteTransformativeProperties`, and the presented-image property selection in `src/read.c` at `092276ce89098ead06db80975173191e5fee1826` | Preserve the registered `pasp`, `clap`, `irot`, and `imir` presentation properties on an associated image item or selected image-sequence sample entry, reject invalid lengths, reserved bits, duplicates, and non-integral clean apertures, and map the displayed pixel ratio and transforms through ImageSharp's existing metadata and processing model. No generic transform or display model is introduced. | | `Av1CodecConfiguration`, `HeifItem.Av1CodecConfiguration`, `Av1HeifItemDecoder`, and AV1 grid configuration checks | AV1-ISOBMFF sections 2.3.3 and 2.3.4 codec-configuration record syntax and semantics; AVIF sections 2.1, 2.2.1, and 2.2.3 AV1 image-item, item-configuration, and HDR metadata constraints; AV1 sections 5.8.3, 5.8.4, 6.7.3, and 6.7.4 HDR metadata syntax and semantics; ISOBMFF mastering-display and content-light image properties; ITU-T H.274 section 8.9 mastering-display field semantics; MIAF section 7.3.11.4.1 grid input constraints | libavif `src/read.c` functions `avifParseCodecConfiguration`, `avifDecoderItemValidateProperties`, `avifReadCodecConfigProperty`, `avifParseContentLightLevelInformation`, and `avifSkipMasteringDisplayColourVolume` at `092276ce89098ead06db80975173191e5fee1826`; libaom `av1/decoder/obu.c` functions `read_metadata`, `read_metadata_hdr_cll`, and `read_metadata_hdr_mdcv` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Associate exactly one `av1C` property with each decoded `av01` image item, validate the fixed record and its bit depth/chroma fields against the item's AV1 sequence header and optional `pixi` channel depths, require matching configurations across grid tiles, and report the encoded image precision and monochrome shape through `HeifMetadata`. Validate low-overhead OBU framing, require exactly one sequence header in the image item, allow at most one first-position sequence header in `configOBUs`, and compare a repeated header's extension and payload exactly while ignoring only its legal size-field representation. Decode `clli` and `mdcv` as still-image item properties, validate matching HDR CLL and HDR MDCV metadata OBUs from the combined configuration/item sequence, and account for the different primary order and fixed-point precision of the ISOBMFF and AV1 MDCV representations. Expose the effective HDR values without adding sample groups, tracks, or media metadata. Related still-image HDR properties remain required. Consume but do not retain presentation-delay syntax, and introduce no sample entry, sample description, track, timing, or generic decoder-configuration model. | | `HevcCodecConfiguration`, `HevcNalUnitArray`, and `HeifItem.HevcCodecConfiguration` | ISO/IEC 14496-15 `HEVCDecoderConfigurationRecord`, HEVC NAL-unit header syntax, and HEVC image-item configuration requirements | HM `source/Lib/TLibDecoder/NALread.cpp` and parameter-header paths at `9c1f298659ab0cee9dc13d23d0304221575410b9`; Android `libhevc` v1.6.0 decoder NAL and parameter-header paths at `c83a76b084498d55f252f48b2e3786804cdf24b7`; independently produced HEIC fixtures for `hvcC` record layout | Associate exactly one `hvcC` property with each `hvc1` image item; validate the fixed profile, level, chroma, bit-depth, reserved, length-size, array, and NAL-header fields within the property boundary; and retain only the image description plus bounded configuration NAL units required to decode that item. Consume but do not retain average-frame-rate or temporal-layer presentation fields. Do not add visual sample entries, sample descriptions, tracks, timing, access-unit timelines, or a generic decoder-configuration model. BSD and Apache-2.0 notice preservation and separate HEVC patent/release clearance remain final integration gates. | | `HevcNalUnit`, `HevcNalUnitHeader`, `HevcRbspDecoder`, and `HevcBitReader` | HEVC sections 7.3.1.1 and 7.3.1.2 NAL-unit and header syntax, section 7.3.2.11 RBSP trailing bits, and section 9.1 Exp-Golomb parsing | HM `source/Lib/TLibDecoder/NALread.cpp` and `SyntaxElementParser.cpp` at `9c1f298659ab0cee9dc13d23d0304221575410b9`; Android `libhevc` NAL extraction and bitstream paths at `c83a76b084498d55f252f48b2e3786804cdf24b7` | Validate the fixed NAL header, remove only legal emulation-prevention bytes, and provide bounded fixed-width, flag, unsigned/signed Exp-Golomb, and RBSP-trailing-bit reads. Configuration arrays immediately use the shared parser. The primitives consume only NAL units delivered by a supported HEIF image item or its `hvcC` property and introduce no Annex B API, access-unit model, timeline, track, sample table, or generic ISO BMFF surface. | diff --git a/src/ImageSharp/Formats/Heif/HeifDecoderCore.cs b/src/ImageSharp/Formats/Heif/HeifDecoderCore.cs index 722b93834..647270f47 100644 --- a/src/ImageSharp/Formats/Heif/HeifDecoderCore.cs +++ b/src/ImageSharp/Formats/Heif/HeifDecoderCore.cs @@ -4,7 +4,6 @@ using System.Buffers; using System.Buffers.Binary; using System.Text; -using SixLabors.ImageSharp.ColorProfiles; using SixLabors.ImageSharp.Common.Helpers; using SixLabors.ImageSharp.Formats.Heif.Av1; using SixLabors.ImageSharp.Formats.Heif.Hevc; @@ -727,18 +726,10 @@ internal sealed class HeifDecoderCore : ImageDecoderCore properties.Add(new KeyValuePair(Heif4CharCode.Ispe, new Size((int)width, (int)height))); break; case Heif4CharCode.Pasp: - EnsureBufferRemaining(boxBuffer, 0, 8, "pixel aspect ratio"); - uint horizontalSpacing = BinaryPrimitives.ReadUInt32BigEndian(boxBuffer); - uint verticalSpacing = BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[4..]); - if (horizontalSpacing == 0 || verticalSpacing == 0) - { - throw new InvalidImageContentException("The pixel aspect ratio property has zero spacing."); - } - properties.Add( new KeyValuePair( Heif4CharCode.Pasp, - new HeifPixelAspectRatio(horizontalSpacing, verticalSpacing))); + HeifPropertyParser.ParsePixelAspectRatio(boxBuffer))); break; case Heif4CharCode.Pixi: @@ -795,16 +786,7 @@ internal sealed class HeifDecoderCore : ImageDecoderCore EnsureBufferRemaining(boxBuffer, 4, 1, "ICC color information"); byte[] iccData = boxBuffer[4..].ToArray(); IccProfile? iccProfile = null; - this.ExecuteAncillarySegmentAction(() => - { - IccProfile candidate = new(iccData); - if (!candidate.CheckIsValid()) - { - throw new InvalidIccProfileException("Invalid HEIF ICC profile."); - } - - iccProfile = candidate; - }); + this.ExecuteAncillarySegmentAction(() => iccProfile = HeifPropertyParser.ParseIccProfile(iccData)); // A malformed ancillary profile can be ignored by policy while the physical property still // occupies its ipco index and remains understood for essential-association handling. @@ -812,339 +794,52 @@ internal sealed class HeifDecoderCore : ImageDecoderCore } else if (profileType == Heif4CharCode.Nclx) { - EnsureBufferRemaining(boxBuffer, 4, 7, "CICP color information"); - ushort colorPrimaries = BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[4..]); - ushort transferCharacteristics = BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[6..]); - ushort matrixCoefficients = BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[8..]); - byte rangeAndReserved = boxBuffer[10]; - if ((rangeAndReserved & 0x7F) != 0) - { - throw new InvalidImageContentException("The HEIF CICP color property has nonzero reserved bits."); - } - - // The box fields are 16-bit so future registrations remain representable. ImageSharp's CICP - // profile exposes the currently registered byte-sized H.273 values and maps others to unspecified. - byte colorPrimariesValue = colorPrimaries <= byte.MaxValue - ? (byte)colorPrimaries - : (byte)CicpColorPrimaries.Unspecified; - - byte transferCharacteristicsValue = transferCharacteristics <= byte.MaxValue - ? (byte)transferCharacteristics - : (byte)CicpTransferCharacteristics.Unspecified; - - byte matrixCoefficientsValue = matrixCoefficients <= byte.MaxValue - ? (byte)matrixCoefficients - : (byte)CicpMatrixCoefficients.Unspecified; - - colorInformation = new CicpProfile( - colorPrimariesValue, - transferCharacteristicsValue, - matrixCoefficientsValue, - (rangeAndReserved & 0x80) != 0); + colorInformation = HeifPropertyParser.ParseCicpProfile(boxBuffer[4..]); } properties.Add(new KeyValuePair(Heif4CharCode.Colr, colorInformation)); break; case Heif4CharCode.Clli: - EnsureBufferRemaining(boxBuffer, 0, 4, "content light level information"); - if (boxBuffer.Length != 4) - { - throw new InvalidImageContentException("The content light level property has an invalid length."); - } - properties.Add( new KeyValuePair( Heif4CharCode.Clli, - new HeifContentLightLevel( - BinaryPrimitives.ReadUInt16BigEndian(boxBuffer), - BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[2..])))); + HeifPropertyParser.ParseContentLightLevel(boxBuffer))); break; case Heif4CharCode.Mdcv: - EnsureBufferRemaining(boxBuffer, 0, 24, "mastering display color volume"); - if (boxBuffer.Length != 24) - { - throw new InvalidImageContentException("The mastering display color-volume property has an invalid length."); - } - - const float chromaticityScale = 1F / 50000F; - const double luminanceScale = 1D / 10000D; - - // The registered mastering-display payload inherits the G, B, R primary order used by its - // mastering-display source syntax. Reorder it into ImageSharp's existing RGB coordinate type. - CieXyChromaticityCoordinates greenPrimary = new( - BinaryPrimitives.ReadUInt16BigEndian(boxBuffer) * chromaticityScale, - BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[2..]) * chromaticityScale); - - CieXyChromaticityCoordinates bluePrimary = new( - BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[4..]) * chromaticityScale, - BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[6..]) * chromaticityScale); - - CieXyChromaticityCoordinates redPrimary = new( - BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[8..]) * chromaticityScale, - BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[10..]) * chromaticityScale); - properties.Add( new KeyValuePair( Heif4CharCode.Mdcv, - new HeifMasteringDisplayColorVolume( - new RgbPrimariesChromaticityCoordinates(redPrimary, greenPrimary, bluePrimary), - new CieXyChromaticityCoordinates( - BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[12..]) * chromaticityScale, - BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[14..]) * chromaticityScale), - BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[16..]) * luminanceScale, - BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[20..]) * luminanceScale))); + HeifPropertyParser.ParseMasteringDisplayColorVolume(boxBuffer))); break; case Heif4CharCode.Cclv: - EnsureBufferRemaining(boxBuffer, 0, 1, "content color volume"); - byte contentColorVolumeFlags = boxBuffer[0]; - if ((contentColorVolumeFlags & 0xC3) != 0) - { - throw new InvalidImageContentException("The content color-volume property has nonzero reserved flags."); - } - - bool contentPrimariesPresent = (contentColorVolumeFlags & 0x20) != 0; - bool minimumLuminancePresent = (contentColorVolumeFlags & 0x10) != 0; - bool maximumLuminancePresent = (contentColorVolumeFlags & 0x08) != 0; - bool averageLuminancePresent = (contentColorVolumeFlags & 0x04) != 0; - if (!contentPrimariesPresent - && !minimumLuminancePresent - && !maximumLuminancePresent - && !averageLuminancePresent) - { - throw new InvalidImageContentException("The content color-volume property does not describe any values."); - } - - int expectedContentColorVolumeLength = 1 - + (contentPrimariesPresent ? 24 : 0) - + (minimumLuminancePresent ? 4 : 0) - + (maximumLuminancePresent ? 4 : 0) - + (averageLuminancePresent ? 4 : 0); - - if (boxBuffer.Length != expectedContentColorVolumeLength) - { - throw new InvalidImageContentException("The content color-volume property has an invalid length."); - } - - int contentColorVolumeOffset = 1; - RgbPrimariesChromaticityCoordinates? contentPrimaries = null; - if (contentPrimariesPresent) - { - int greenPrimaryX = BinaryPrimitives.ReadInt32BigEndian(boxBuffer[contentColorVolumeOffset..]); - int greenPrimaryY = BinaryPrimitives.ReadInt32BigEndian(boxBuffer[(contentColorVolumeOffset + 4)..]); - int bluePrimaryX = BinaryPrimitives.ReadInt32BigEndian(boxBuffer[(contentColorVolumeOffset + 8)..]); - int bluePrimaryY = BinaryPrimitives.ReadInt32BigEndian(boxBuffer[(contentColorVolumeOffset + 12)..]); - int redPrimaryX = BinaryPrimitives.ReadInt32BigEndian(boxBuffer[(contentColorVolumeOffset + 16)..]); - int redPrimaryY = BinaryPrimitives.ReadInt32BigEndian(boxBuffer[(contentColorVolumeOffset + 20)..]); - - const int maximumContentChromaticityValue = 5_000_000; - if (greenPrimaryX < -maximumContentChromaticityValue - || greenPrimaryX > maximumContentChromaticityValue - || greenPrimaryY < -maximumContentChromaticityValue - || greenPrimaryY > maximumContentChromaticityValue - || bluePrimaryX < -maximumContentChromaticityValue - || bluePrimaryX > maximumContentChromaticityValue - || bluePrimaryY < -maximumContentChromaticityValue - || bluePrimaryY > maximumContentChromaticityValue - || redPrimaryX < -maximumContentChromaticityValue - || redPrimaryX > maximumContentChromaticityValue - || redPrimaryY < -maximumContentChromaticityValue - || redPrimaryY > maximumContentChromaticityValue) - { - throw new InvalidImageContentException("The content color-volume property has an out-of-range primary coordinate."); - } - - const float contentChromaticityScale = 1F / 50000F; - - // Content-color-volume syntax stores signed coordinates in G, B, R order. Reorder the - // optional primaries into ImageSharp's existing RGB coordinate representation. - contentPrimaries = new RgbPrimariesChromaticityCoordinates( - new CieXyChromaticityCoordinates( - redPrimaryX * contentChromaticityScale, - redPrimaryY * contentChromaticityScale), - new CieXyChromaticityCoordinates( - greenPrimaryX * contentChromaticityScale, - greenPrimaryY * contentChromaticityScale), - new CieXyChromaticityCoordinates( - bluePrimaryX * contentChromaticityScale, - bluePrimaryY * contentChromaticityScale)); - - contentColorVolumeOffset += 24; - } - - uint? minimumLuminanceValue = null; - if (minimumLuminancePresent) - { - minimumLuminanceValue = BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[contentColorVolumeOffset..]); - contentColorVolumeOffset += 4; - } - - uint? maximumLuminanceValue = null; - if (maximumLuminancePresent) - { - maximumLuminanceValue = BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[contentColorVolumeOffset..]); - contentColorVolumeOffset += 4; - } - - uint? averageLuminanceValue = null; - if (averageLuminancePresent) - { - averageLuminanceValue = BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[contentColorVolumeOffset..]); - } - - if (minimumLuminanceValue is not null - && averageLuminanceValue is not null - && minimumLuminanceValue.Value > averageLuminanceValue.Value) - { - throw new InvalidImageContentException("The content color-volume minimum luminance exceeds its average luminance."); - } - - if (averageLuminanceValue is not null - && maximumLuminanceValue is not null - && averageLuminanceValue.Value > maximumLuminanceValue.Value) - { - throw new InvalidImageContentException("The content color-volume average luminance exceeds its maximum luminance."); - } - - if (minimumLuminanceValue is not null - && maximumLuminanceValue is not null - && minimumLuminanceValue.Value > maximumLuminanceValue.Value) - { - throw new InvalidImageContentException("The content color-volume minimum luminance exceeds its maximum luminance."); - } - - const double contentLuminanceScale = 1D / 10000000D; - - // These values are normalized according to the signaled transfer characteristics. Preserve - // that unitless meaning instead of presenting them as physical display luminance. - double? minimumContentLuminance = minimumLuminanceValue is not null - ? minimumLuminanceValue.Value * contentLuminanceScale - : null; - - double? maximumContentLuminance = maximumLuminanceValue is not null - ? maximumLuminanceValue.Value * contentLuminanceScale - : null; - - double? averageContentLuminance = averageLuminanceValue is not null - ? averageLuminanceValue.Value * contentLuminanceScale - : null; - properties.Add( new KeyValuePair( Heif4CharCode.Cclv, - new HeifContentColorVolume( - contentPrimaries, - minimumContentLuminance, - maximumContentLuminance, - averageContentLuminance))); + HeifPropertyParser.ParseContentColorVolume(boxBuffer))); break; case Heif4CharCode.Amve: - EnsureBufferRemaining(boxBuffer, 0, 8, "ambient viewing environment"); - if (boxBuffer.Length != 8) - { - throw new InvalidImageContentException("The ambient viewing-environment property has an invalid length."); - } - - uint ambientIlluminanceValue = BinaryPrimitives.ReadUInt32BigEndian(boxBuffer); - ushort ambientLightX = BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[4..]); - ushort ambientLightY = BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[6..]); - if (ambientIlluminanceValue == 0) - { - throw new InvalidImageContentException("The ambient viewing-environment property has zero illuminance."); - } - - if (ambientLightX > 50000 || ambientLightY > 50000) - { - throw new InvalidImageContentException("The ambient viewing-environment property has an out-of-range chromaticity coordinate."); - } - - const double ambientIlluminanceScale = 1D / 10000D; - const float ambientChromaticityScale = 1F / 50000F; - - // The item property inherits H.274's fixed-point units: 0.0001 lux for illuminance and - // 0.00002 for each normalized CIE chromaticity coordinate. properties.Add( new KeyValuePair( Heif4CharCode.Amve, - new HeifAmbientViewingEnvironment( - ambientIlluminanceValue * ambientIlluminanceScale, - new CieXyChromaticityCoordinates( - ambientLightX * ambientChromaticityScale, - ambientLightY * ambientChromaticityScale)))); + HeifPropertyParser.ParseAmbientViewingEnvironment(boxBuffer))); break; case Heif4CharCode.Reve: - EnsureBufferRemaining(boxBuffer, 0, 20, "reference viewing environment"); - if (boxBuffer.Length != 20) - { - throw new InvalidImageContentException("The reference viewing-environment property has an invalid length."); - } - - if (BinaryPrimitives.ReadUInt32BigEndian(boxBuffer) != 0) - { - throw new InvalidImageContentException("The reference viewing-environment property has an unsupported version or flags."); - } - - ushort surroundLightX = BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[8..]); - ushort surroundLightY = BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[10..]); - ushort peripheryLightX = BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[16..]); - ushort peripheryLightY = BinaryPrimitives.ReadUInt16BigEndian(boxBuffer[18..]); - if (surroundLightX > 10000 - || surroundLightY > 10000 - || peripheryLightX > 10000 - || peripheryLightY > 10000) - { - throw new InvalidImageContentException("The reference viewing-environment property has an out-of-range chromaticity coordinate."); - } - - const double viewingEnvironmentLuminanceScale = 1D / 10000D; - const float referenceChromaticityScale = 1F / 10000F; - - // The full-box header is followed by the display surround and then the wider periphery. - // Both field groups use 0.0001 increments, but luminance is physical cd/m2 while the CIE - // coordinates are normalized. Keep the regions distinct because they affect different areas. properties.Add( new KeyValuePair( Heif4CharCode.Reve, - new HeifReferenceViewingEnvironment( - BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[4..]) * viewingEnvironmentLuminanceScale, - new CieXyChromaticityCoordinates( - surroundLightX * referenceChromaticityScale, - surroundLightY * referenceChromaticityScale), - BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[12..]) * viewingEnvironmentLuminanceScale, - new CieXyChromaticityCoordinates( - peripheryLightX * referenceChromaticityScale, - peripheryLightY * referenceChromaticityScale)))); + HeifPropertyParser.ParseReferenceViewingEnvironment(boxBuffer))); break; case Heif4CharCode.Ndwt: - EnsureBufferRemaining(boxBuffer, 0, 8, "nominal diffuse white"); - if (boxBuffer.Length != 8) - { - throw new InvalidImageContentException("The nominal diffuse-white property has an invalid length."); - } - - if (BinaryPrimitives.ReadUInt32BigEndian(boxBuffer) != 0) - { - throw new InvalidImageContentException("The nominal diffuse-white property has an unsupported version or flags."); - } - - uint diffuseWhiteLuminanceValue = BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[4..]); - const double diffuseWhiteLuminanceScale = 1D / 10000D; - - // A zero coded value requests the standard-defined default; it does not describe a black - // diffuse white. Preserve that distinction separately from an absent item property. - double? diffuseWhiteLuminance = diffuseWhiteLuminanceValue == 0 - ? null - : diffuseWhiteLuminanceValue * diffuseWhiteLuminanceScale; - properties.Add( new KeyValuePair( Heif4CharCode.Ndwt, - new HeifNominalDiffuseWhite(diffuseWhiteLuminance))); + HeifPropertyParser.ParseNominalDiffuseWhite(boxBuffer))); break; case Heif4CharCode.Av1C: @@ -1163,36 +858,17 @@ internal sealed class HeifDecoderCore : ImageDecoderCore break; case Heif4CharCode.Clap: - EnsureBufferRemaining(boxBuffer, 0, 32, "clean aperture"); - HeifCleanAperture cleanAperture = new( - unchecked((int)BinaryPrimitives.ReadUInt32BigEndian(boxBuffer)), - unchecked((int)BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[4..])), - unchecked((int)BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[8..])), - unchecked((int)BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[12..])), - unchecked((int)BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[16..])), - unchecked((int)BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[20..])), - unchecked((int)BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[24..])), - unchecked((int)BinaryPrimitives.ReadUInt32BigEndian(boxBuffer[28..]))); - - properties.Add(new KeyValuePair(Heif4CharCode.Clap, cleanAperture)); + properties.Add( + new KeyValuePair( + Heif4CharCode.Clap, + HeifPropertyParser.ParseCleanAperture(boxBuffer))); + break; case Heif4CharCode.Irot: - EnsureBufferRemaining(boxBuffer, 0, 1, "image rotation"); - if ((boxBuffer[0] & 0xFC) != 0) - { - throw new InvalidImageContentException("The image rotation property has nonzero reserved bits."); - } - - properties.Add(new KeyValuePair(Heif4CharCode.Irot, (byte)(boxBuffer[0] & 3))); + properties.Add(new KeyValuePair(Heif4CharCode.Irot, HeifPropertyParser.ParseRotation(boxBuffer))); break; case Heif4CharCode.Imir: - EnsureBufferRemaining(boxBuffer, 0, 1, "image mirror"); - if ((boxBuffer[0] & 0xFE) != 0) - { - throw new InvalidImageContentException("The image mirror property has nonzero reserved bits."); - } - - properties.Add(new KeyValuePair(Heif4CharCode.Imir, (byte)(boxBuffer[0] & 1))); + properties.Add(new KeyValuePair(Heif4CharCode.Imir, HeifPropertyParser.ParseMirrorAxis(boxBuffer))); break; case Heif4CharCode.Altt: case Heif4CharCode.Iscl: @@ -1278,7 +954,8 @@ internal sealed class HeifDecoderCore : ImageDecoderCore propertyIndex--; if (propertyIndex >= properties.Count) { - throw new InvalidImageContentException($"Item {itemId} references property index {propertyIndex + 1}, but only {properties.Count} properties exist."); + throw new InvalidImageContentException( + $"Item {itemId} references property index {propertyIndex + 1}, but only {properties.Count} properties exist."); } KeyValuePair prop = properties[(int)propertyIndex]; @@ -1325,7 +1002,8 @@ internal sealed class HeifDecoderCore : ImageDecoderCore case Heif4CharCode.Av1C: if (item.Type != Heif4CharCode.Av01) { - throw new InvalidImageContentException($"Item {itemId} associates an AV1 codec configuration with non-AV1 item type '{item.Type}'."); + throw new InvalidImageContentException( + $"Item {itemId} associates an AV1 codec configuration with non-AV1 item type '{item.Type}'."); } if (item.Av1CodecConfiguration is not null) @@ -1338,7 +1016,8 @@ internal sealed class HeifDecoderCore : ImageDecoderCore case Heif4CharCode.HvcC: if (item.Type != Heif4CharCode.Hvc1) { - throw new InvalidImageContentException($"Item {itemId} associates an HEVC codec configuration with non-HEVC item type '{item.Type}'."); + throw new InvalidImageContentException( + $"Item {itemId} associates an HEVC codec configuration with non-HEVC item type '{item.Type}'."); } if (item.HevcCodecConfiguration is not null) diff --git a/src/ImageSharp/Formats/Heif/HeifPixelAspectRatio.cs b/src/ImageSharp/Formats/Heif/HeifPixelAspectRatio.cs index e72b0d3ed..a276fba57 100644 --- a/src/ImageSharp/Formats/Heif/HeifPixelAspectRatio.cs +++ b/src/ImageSharp/Formats/Heif/HeifPixelAspectRatio.cs @@ -4,7 +4,7 @@ namespace SixLabors.ImageSharp.Formats.Heif; /// -/// Describes the relative horizontal and vertical spacing of pixels in a HEIF still-image item. +/// Describes the relative horizontal and vertical spacing of pixels in a HEIF image. /// /// The relative horizontal pixel spacing. /// The relative vertical pixel spacing. diff --git a/src/ImageSharp/Formats/Heif/HeifPropertyParser.cs b/src/ImageSharp/Formats/Heif/HeifPropertyParser.cs new file mode 100644 index 000000000..96464c56b --- /dev/null +++ b/src/ImageSharp/Formats/Heif/HeifPropertyParser.cs @@ -0,0 +1,369 @@ +// Copyright (c) Six Labors. +// Licensed under the Six Labors Split License. + +using System.Buffers.Binary; +using SixLabors.ImageSharp.ColorProfiles; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.Metadata.Profiles.Icc; + +namespace SixLabors.ImageSharp.Formats.Heif; + +/// +/// Parses the image properties shared by HEIF items and image-sequence sample entries. +/// +internal static class HeifPropertyParser +{ + /// + /// Parses relative horizontal and vertical pixel spacing. + /// + /// The complete pixel-aspect-ratio payload. + /// The validated relative pixel spacing. + public static HeifPixelAspectRatio ParsePixelAspectRatio(ReadOnlySpan data) + { + EnsureExactLength(data, 8, "pixel aspect ratio"); + uint horizontalSpacing = BinaryPrimitives.ReadUInt32BigEndian(data); + uint verticalSpacing = BinaryPrimitives.ReadUInt32BigEndian(data[4..]); + if (horizontalSpacing == 0 || verticalSpacing == 0) + { + throw new InvalidImageContentException("The pixel aspect ratio property has zero spacing."); + } + + return new HeifPixelAspectRatio(horizontalSpacing, verticalSpacing); + } + + /// + /// Parses and validates an embedded ICC profile. + /// + /// The complete ICC profile bytes. + /// The validated ICC profile. + public static IccProfile ParseIccProfile(byte[] data) + { + if (data.Length == 0) + { + throw new InvalidImageContentException("The HEIF ICC color property contains an empty profile."); + } + + IccProfile profile = new(data); + if (!profile.CheckIsValid()) + { + throw new InvalidIccProfileException("Invalid HEIF ICC profile."); + } + + return profile; + } + + /// + /// Parses a CICP color description from an nclx color-information payload. + /// + /// The complete payload following the nclx color type. + /// The CICP color description. + public static CicpProfile ParseCicpProfile(ReadOnlySpan data) + { + EnsureExactLength(data, 7, "CICP color information"); + ushort colorPrimaries = BinaryPrimitives.ReadUInt16BigEndian(data); + ushort transferCharacteristics = BinaryPrimitives.ReadUInt16BigEndian(data[2..]); + ushort matrixCoefficients = BinaryPrimitives.ReadUInt16BigEndian(data[4..]); + byte rangeAndReserved = data[6]; + if ((rangeAndReserved & 0x7F) != 0) + { + throw new InvalidImageContentException("The HEIF CICP color property has nonzero reserved bits."); + } + + // The box fields are 16-bit so future registrations remain representable. ImageSharp's CICP profile exposes + // the currently registered byte-sized H.273 values and maps larger future values to unspecified. + byte colorPrimariesValue = colorPrimaries <= byte.MaxValue ? (byte)colorPrimaries : (byte)CicpColorPrimaries.Unspecified; + byte transferCharacteristicsValue = transferCharacteristics <= byte.MaxValue + ? (byte)transferCharacteristics + : (byte)CicpTransferCharacteristics.Unspecified; + + byte matrixCoefficientsValue = matrixCoefficients <= byte.MaxValue + ? (byte)matrixCoefficients + : (byte)CicpMatrixCoefficients.Unspecified; + + return new CicpProfile(colorPrimariesValue, transferCharacteristicsValue, matrixCoefficientsValue, (rangeAndReserved & 0x80) != 0); + } + + /// + /// Parses content light-level information. + /// + /// The complete content-light-level payload. + /// The content light-level information. + public static HeifContentLightLevel ParseContentLightLevel(ReadOnlySpan data) + { + EnsureExactLength(data, 4, "content light level"); + return new HeifContentLightLevel(BinaryPrimitives.ReadUInt16BigEndian(data), BinaryPrimitives.ReadUInt16BigEndian(data[2..])); + } + + /// + /// Parses mastering-display color-volume information. + /// + /// The complete mastering-display color-volume payload. + /// The mastering-display color-volume information. + public static HeifMasteringDisplayColorVolume ParseMasteringDisplayColorVolume(ReadOnlySpan data) + { + EnsureExactLength(data, 24, "mastering display color volume"); + const float chromaticityScale = 1F / 50000F; + const double luminanceScale = 1D / 10000D; + + // The registered payload inherits the G, B, R primary order used by its mastering-display source syntax. + // Reorder it into ImageSharp's existing RGB coordinate type at the shared container boundary. + CieXyChromaticityCoordinates greenPrimary = new( + BinaryPrimitives.ReadUInt16BigEndian(data) * chromaticityScale, + BinaryPrimitives.ReadUInt16BigEndian(data[2..]) * chromaticityScale); + + CieXyChromaticityCoordinates bluePrimary = new( + BinaryPrimitives.ReadUInt16BigEndian(data[4..]) * chromaticityScale, + BinaryPrimitives.ReadUInt16BigEndian(data[6..]) * chromaticityScale); + + CieXyChromaticityCoordinates redPrimary = new( + BinaryPrimitives.ReadUInt16BigEndian(data[8..]) * chromaticityScale, + BinaryPrimitives.ReadUInt16BigEndian(data[10..]) * chromaticityScale); + + return new HeifMasteringDisplayColorVolume( + new RgbPrimariesChromaticityCoordinates(redPrimary, greenPrimary, bluePrimary), + new CieXyChromaticityCoordinates( + BinaryPrimitives.ReadUInt16BigEndian(data[12..]) * chromaticityScale, + BinaryPrimitives.ReadUInt16BigEndian(data[14..]) * chromaticityScale), + BinaryPrimitives.ReadUInt32BigEndian(data[16..]) * luminanceScale, + BinaryPrimitives.ReadUInt32BigEndian(data[20..]) * luminanceScale); + } + + /// + /// Parses content color-volume information. + /// + /// The complete content color-volume payload. + /// The content color-volume information. + public static HeifContentColorVolume ParseContentColorVolume(ReadOnlySpan data) + { + if (data.IsEmpty) + { + throw new InvalidImageContentException("The content color-volume property is truncated."); + } + + byte flags = data[0]; + if ((flags & 0xC3) != 0) + { + throw new InvalidImageContentException("The content color-volume property has nonzero reserved flags."); + } + + bool primariesPresent = (flags & 0x20) != 0; + bool minimumLuminancePresent = (flags & 0x10) != 0; + bool maximumLuminancePresent = (flags & 0x08) != 0; + bool averageLuminancePresent = (flags & 0x04) != 0; + if (!primariesPresent && !minimumLuminancePresent && !maximumLuminancePresent && !averageLuminancePresent) + { + throw new InvalidImageContentException("The content color-volume property does not describe any values."); + } + + int expectedLength = 1 + + (primariesPresent ? 24 : 0) + + (minimumLuminancePresent ? 4 : 0) + + (maximumLuminancePresent ? 4 : 0) + + (averageLuminancePresent ? 4 : 0); + + EnsureExactLength(data, expectedLength, "content color volume"); + int offset = 1; + RgbPrimariesChromaticityCoordinates? primaries = null; + if (primariesPresent) + { + int greenX = BinaryPrimitives.ReadInt32BigEndian(data[offset..]); + int greenY = BinaryPrimitives.ReadInt32BigEndian(data[(offset + 4)..]); + int blueX = BinaryPrimitives.ReadInt32BigEndian(data[(offset + 8)..]); + int blueY = BinaryPrimitives.ReadInt32BigEndian(data[(offset + 12)..]); + int redX = BinaryPrimitives.ReadInt32BigEndian(data[(offset + 16)..]); + int redY = BinaryPrimitives.ReadInt32BigEndian(data[(offset + 20)..]); + const int maximumChromaticityValue = 5_000_000; + if (greenX is < -maximumChromaticityValue or > maximumChromaticityValue + || greenY is < -maximumChromaticityValue or > maximumChromaticityValue + || blueX is < -maximumChromaticityValue or > maximumChromaticityValue + || blueY is < -maximumChromaticityValue or > maximumChromaticityValue + || redX is < -maximumChromaticityValue or > maximumChromaticityValue + || redY is < -maximumChromaticityValue or > maximumChromaticityValue) + { + throw new InvalidImageContentException("The content color-volume property has an out-of-range primary coordinate."); + } + + const float chromaticityScale = 1F / 50000F; + + // Content-color-volume syntax also stores signed coordinates in G, B, R order. + primaries = new RgbPrimariesChromaticityCoordinates( + new CieXyChromaticityCoordinates(redX * chromaticityScale, redY * chromaticityScale), + new CieXyChromaticityCoordinates(greenX * chromaticityScale, greenY * chromaticityScale), + new CieXyChromaticityCoordinates(blueX * chromaticityScale, blueY * chromaticityScale)); + + offset += 24; + } + + uint? minimumLuminance = minimumLuminancePresent ? BinaryPrimitives.ReadUInt32BigEndian(data[offset..]) : null; + offset += minimumLuminancePresent ? 4 : 0; + uint? maximumLuminance = maximumLuminancePresent ? BinaryPrimitives.ReadUInt32BigEndian(data[offset..]) : null; + offset += maximumLuminancePresent ? 4 : 0; + uint? averageLuminance = averageLuminancePresent ? BinaryPrimitives.ReadUInt32BigEndian(data[offset..]) : null; + if ((minimumLuminance is not null && averageLuminance is not null && minimumLuminance.Value > averageLuminance.Value) + || (averageLuminance is not null && maximumLuminance is not null && averageLuminance.Value > maximumLuminance.Value) + || (minimumLuminance is not null && maximumLuminance is not null && minimumLuminance.Value > maximumLuminance.Value)) + { + throw new InvalidImageContentException("The content color-volume luminance values are not in ascending order."); + } + + const double luminanceScale = 1D / 10000000D; + + // These values are normalized according to the signaled transfer characteristics. Preserve that + // unitless meaning instead of presenting them as physical display luminance. + return new HeifContentColorVolume( + primaries, + minimumLuminance * luminanceScale, + maximumLuminance * luminanceScale, + averageLuminance * luminanceScale); + } + + /// + /// Parses an ambient viewing environment. + /// + /// The complete ambient viewing-environment payload. + /// The ambient viewing environment. + public static HeifAmbientViewingEnvironment ParseAmbientViewingEnvironment(ReadOnlySpan data) + { + EnsureExactLength(data, 8, "ambient viewing environment"); + uint illuminance = BinaryPrimitives.ReadUInt32BigEndian(data); + ushort lightX = BinaryPrimitives.ReadUInt16BigEndian(data[4..]); + ushort lightY = BinaryPrimitives.ReadUInt16BigEndian(data[6..]); + if (illuminance == 0) + { + throw new InvalidImageContentException("The ambient viewing-environment property has zero illuminance."); + } + + if (lightX > 50000 || lightY > 50000) + { + throw new InvalidImageContentException("The ambient viewing-environment property has an out-of-range chromaticity coordinate."); + } + + const double illuminanceScale = 1D / 10000D; + const float chromaticityScale = 1F / 50000F; + + // The property inherits H.274's fixed-point units: 0.0001 lux for illuminance and 0.00002 for each + // normalized CIE chromaticity coordinate. + return new HeifAmbientViewingEnvironment( + illuminance * illuminanceScale, + new CieXyChromaticityCoordinates(lightX * chromaticityScale, lightY * chromaticityScale)); + } + + /// + /// Parses a reference viewing environment. + /// + /// The complete reference viewing-environment payload. + /// The reference viewing environment. + public static HeifReferenceViewingEnvironment ParseReferenceViewingEnvironment(ReadOnlySpan data) + { + EnsureExactLength(data, 20, "reference viewing environment"); + if (BinaryPrimitives.ReadUInt32BigEndian(data) != 0) + { + throw new InvalidImageContentException("The reference viewing-environment property has an unsupported version or flags."); + } + + ushort surroundX = BinaryPrimitives.ReadUInt16BigEndian(data[8..]); + ushort surroundY = BinaryPrimitives.ReadUInt16BigEndian(data[10..]); + ushort peripheryX = BinaryPrimitives.ReadUInt16BigEndian(data[16..]); + ushort peripheryY = BinaryPrimitives.ReadUInt16BigEndian(data[18..]); + if (surroundX > 10000 || surroundY > 10000 || peripheryX > 10000 || peripheryY > 10000) + { + throw new InvalidImageContentException("The reference viewing-environment property has an out-of-range chromaticity coordinate."); + } + + const double luminanceScale = 1D / 10000D; + const float chromaticityScale = 1F / 10000F; + + // The full-box header is followed by display-surround and wider-periphery fields. Both groups use + // 0.0001 increments, but luminance is physical cd/m2 while the CIE coordinates are normalized. + return new HeifReferenceViewingEnvironment( + BinaryPrimitives.ReadUInt32BigEndian(data[4..]) * luminanceScale, + new CieXyChromaticityCoordinates(surroundX * chromaticityScale, surroundY * chromaticityScale), + BinaryPrimitives.ReadUInt32BigEndian(data[12..]) * luminanceScale, + new CieXyChromaticityCoordinates(peripheryX * chromaticityScale, peripheryY * chromaticityScale)); + } + + /// + /// Parses nominal diffuse-white information. + /// + /// The complete nominal diffuse-white payload. + /// The nominal diffuse-white information. + public static HeifNominalDiffuseWhite ParseNominalDiffuseWhite(ReadOnlySpan data) + { + EnsureExactLength(data, 8, "nominal diffuse white"); + if (BinaryPrimitives.ReadUInt32BigEndian(data) != 0) + { + throw new InvalidImageContentException("The nominal diffuse-white property has an unsupported version or flags."); + } + + uint luminance = BinaryPrimitives.ReadUInt32BigEndian(data[4..]); + const double luminanceScale = 1D / 10000D; + + // A zero coded value requests the standard-defined default rather than describing black diffuse white. + return new HeifNominalDiffuseWhite(luminance == 0 ? null : luminance * luminanceScale); + } + + /// + /// Parses a clean-aperture crop description. + /// + /// The complete clean-aperture payload. + /// The clean-aperture crop description. + public static HeifCleanAperture ParseCleanAperture(ReadOnlySpan data) + { + EnsureExactLength(data, 32, "clean aperture"); + return new HeifCleanAperture( + BinaryPrimitives.ReadInt32BigEndian(data), + BinaryPrimitives.ReadInt32BigEndian(data[4..]), + BinaryPrimitives.ReadInt32BigEndian(data[8..]), + BinaryPrimitives.ReadInt32BigEndian(data[12..]), + BinaryPrimitives.ReadInt32BigEndian(data[16..]), + BinaryPrimitives.ReadInt32BigEndian(data[20..]), + BinaryPrimitives.ReadInt32BigEndian(data[24..]), + BinaryPrimitives.ReadInt32BigEndian(data[28..])); + } + + /// + /// Parses the number of counter-clockwise quarter turns applied to an image. + /// + /// The complete image-rotation payload. + /// The number of counter-clockwise quarter turns. + public static byte ParseRotation(ReadOnlySpan data) + { + EnsureExactLength(data, 1, "image rotation"); + if ((data[0] & 0xFC) != 0) + { + throw new InvalidImageContentException("The image rotation property has nonzero reserved bits."); + } + + return (byte)(data[0] & 3); + } + + /// + /// Parses the horizontal or vertical image-mirror axis. + /// + /// The complete image-mirror payload. + /// Zero for the horizontal axis or one for the vertical axis. + public static byte ParseMirrorAxis(ReadOnlySpan data) + { + EnsureExactLength(data, 1, "image mirror"); + if ((data[0] & 0xFE) != 0) + { + throw new InvalidImageContentException("The image mirror property has nonzero reserved bits."); + } + + return (byte)(data[0] & 1); + } + + /// + /// Requires a fixed-size image property to contain exactly its registered payload length. + /// + /// The complete property payload. + /// The registered payload length. + /// The property name used in malformed-image diagnostics. + private static void EnsureExactLength(ReadOnlySpan data, int length, string name) + { + if (data.Length != length) + { + throw new InvalidImageContentException($"The {name} property has an invalid length."); + } + } +} diff --git a/src/ImageSharp/Formats/Heif/HeifSequenceParser.cs b/src/ImageSharp/Formats/Heif/HeifSequenceParser.cs index 592f6a25b..97e650b1c 100644 --- a/src/ImageSharp/Formats/Heif/HeifSequenceParser.cs +++ b/src/ImageSharp/Formats/Heif/HeifSequenceParser.cs @@ -888,6 +888,19 @@ internal sealed class HeifSequenceParser track.IsAlpha = this.ParseAuxiliaryType(stream, childLength); auxiliaryTypeSeen = true; break; + case Heif4CharCode.Pasp: + case Heif4CharCode.Colr: + case Heif4CharCode.Clli: + case Heif4CharCode.Mdcv: + case Heif4CharCode.Cclv: + case Heif4CharCode.Amve: + case Heif4CharCode.Reve: + case Heif4CharCode.Ndwt: + case Heif4CharCode.Clap: + case Heif4CharCode.Irot: + case Heif4CharCode.Imir: + this.ParseTrackImageProperty(stream, childLength, childType, track, scratch); + break; } stream.Position = checked(childStart + childLength); @@ -899,6 +912,178 @@ internal sealed class HeifSequenceParser } } + /// + /// Parses one presentation, color, or HDR property carried by a visual sample entry. + /// + /// The stream positioned at the property payload. + /// The validated property payload length. + /// The registered image property type. + /// The selected image track receiving the property. + /// The parser-owned reusable scratch span. + private void ParseTrackImageProperty( + Stream stream, + long boxLength, + Heif4CharCode boxType, + HeifSequenceTrack track, + Span scratch) + { + if (boxType == Heif4CharCode.Colr) + { + this.ParseTrackColorInformation(stream, boxLength, track, scratch); + return; + } + + ReadOnlySpan data = ReadPropertyPayload(stream, boxLength, scratch, boxType); + switch (boxType) + { + case Heif4CharCode.Pasp: + if (track.PixelAspectRatio is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate pixel-aspect-ratio properties."); + } + + track.PixelAspectRatio = HeifPropertyParser.ParsePixelAspectRatio(data); + break; + case Heif4CharCode.Clli: + if (track.ContentLightLevel is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate content-light-level properties."); + } + + track.ContentLightLevel = HeifPropertyParser.ParseContentLightLevel(data); + break; + case Heif4CharCode.Mdcv: + if (track.MasteringDisplayColorVolume is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate mastering-display properties."); + } + + track.MasteringDisplayColorVolume = HeifPropertyParser.ParseMasteringDisplayColorVolume(data); + break; + case Heif4CharCode.Cclv: + if (track.ContentColorVolume is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate content-color-volume properties."); + } + + track.ContentColorVolume = HeifPropertyParser.ParseContentColorVolume(data); + break; + case Heif4CharCode.Amve: + if (track.AmbientViewingEnvironment is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate ambient-viewing properties."); + } + + track.AmbientViewingEnvironment = HeifPropertyParser.ParseAmbientViewingEnvironment(data); + break; + case Heif4CharCode.Reve: + if (track.ReferenceViewingEnvironment is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate reference-viewing properties."); + } + + track.ReferenceViewingEnvironment = HeifPropertyParser.ParseReferenceViewingEnvironment(data); + break; + case Heif4CharCode.Ndwt: + if (track.NominalDiffuseWhite is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate nominal-diffuse-white properties."); + } + + track.NominalDiffuseWhite = HeifPropertyParser.ParseNominalDiffuseWhite(data); + break; + case Heif4CharCode.Clap: + if (track.CleanAperture is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate clean-aperture properties."); + } + + HeifCleanAperture cleanAperture = HeifPropertyParser.ParseCleanAperture(data); + _ = cleanAperture.ToRectangle(new Size(track.CodedWidth, track.CodedHeight)); + track.CleanAperture = cleanAperture; + break; + case Heif4CharCode.Irot: + if (track.RotationAngle is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate rotation properties."); + } + + track.RotationAngle = HeifPropertyParser.ParseRotation(data); + break; + case Heif4CharCode.Imir: + if (track.MirrorAxis is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate mirror properties."); + } + + track.MirrorAxis = HeifPropertyParser.ParseMirrorAxis(data); + break; + } + } + + /// + /// Parses one ICC or CICP color-information property from a visual sample entry. + /// + /// The stream positioned at the color-information payload. + /// The validated color-information payload length. + /// The selected image track receiving the color description. + /// The parser-owned reusable scratch span. + private void ParseTrackColorInformation(Stream stream, long boxLength, HeifSequenceTrack track, Span scratch) + { + ReadOnlySpan prefix = ReadPrefix(stream, boxLength, scratch, 4, "color information"); + Heif4CharCode profileType = (Heif4CharCode)BinaryPrimitives.ReadUInt32BigEndian(prefix); + if (profileType == Heif4CharCode.Nclx) + { + if (track.CicpProfile is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate CICP color properties."); + } + + if (boxLength != 11) + { + throw new InvalidImageContentException("The CICP color-information property has an invalid length."); + } + + prefix = ReadPrefixFromStart(stream, boxLength, scratch, 11, "color information"); + track.CicpProfile = HeifPropertyParser.ParseCicpProfile(prefix[4..]); + } + else if (profileType is Heif4CharCode.RICC or Heif4CharCode.Prof) + { + if (track.IccProfile is not null) + { + throw new InvalidImageContentException("The image-sequence sample entry has duplicate ICC color properties."); + } + + if (boxLength <= 4 || boxLength > int.MaxValue) + { + throw new InvalidImageContentException("The ICC color-information property is empty or too large."); + } + + stream.Position -= 4; + using IMemoryOwner payload = this.boxReader.ReadPayload(stream, boxLength); + byte[] profileData = payload.GetSpan()[4..].ToArray(); + track.IccProfile = HeifPropertyParser.ParseIccProfile(profileData); + } + } + + /// + /// Reads a bounded fixed-size image property through the parser's reusable scratch buffer. + /// + /// The stream positioned at the property payload. + /// The validated property payload length. + /// The parser-owned reusable scratch span. + /// The property type used in malformed-image diagnostics. + /// The complete property payload within . + private static ReadOnlySpan ReadPropertyPayload(Stream stream, long boxLength, Span scratch, Heif4CharCode boxType) + { + if (boxLength > scratch.Length) + { + throw new InvalidImageContentException($"The '{boxType}' image-sequence property exceeds its registered bounded size."); + } + + return ReadPrefix(stream, boxLength, scratch, (int)boxLength, $"{boxType} image-sequence property"); + } + /// /// Parses coding constraints that bound inter-picture references for an image sequence. /// diff --git a/src/ImageSharp/Formats/Heif/HeifSequenceTrack.cs b/src/ImageSharp/Formats/Heif/HeifSequenceTrack.cs index f762fc745..1d3bcbe48 100644 --- a/src/ImageSharp/Formats/Heif/HeifSequenceTrack.cs +++ b/src/ImageSharp/Formats/Heif/HeifSequenceTrack.cs @@ -3,6 +3,8 @@ using SixLabors.ImageSharp.Formats.Heif.Av1; using SixLabors.ImageSharp.Formats.Heif.Hevc; +using SixLabors.ImageSharp.Metadata.Profiles.Cicp; +using SixLabors.ImageSharp.Metadata.Profiles.Icc; namespace SixLabors.ImageSharp.Formats.Heif; @@ -78,6 +80,66 @@ internal sealed class HeifSequenceTrack /// public HevcCodecConfiguration? HevcCodecConfiguration { get; set; } + /// + /// Gets or sets the ICC profile associated with the image sequence. + /// + public IccProfile? IccProfile { get; set; } + + /// + /// Gets or sets the CICP color description associated with the image sequence. + /// + public CicpProfile? CicpProfile { get; set; } + + /// + /// Gets or sets the content light-level information associated with the image sequence. + /// + public HeifContentLightLevel? ContentLightLevel { get; set; } + + /// + /// Gets or sets the mastering-display color volume associated with the image sequence. + /// + public HeifMasteringDisplayColorVolume? MasteringDisplayColorVolume { get; set; } + + /// + /// Gets or sets the content color volume associated with the image sequence. + /// + public HeifContentColorVolume? ContentColorVolume { get; set; } + + /// + /// Gets or sets the nominal ambient viewing environment associated with the image sequence. + /// + public HeifAmbientViewingEnvironment? AmbientViewingEnvironment { get; set; } + + /// + /// Gets or sets the reference mastering environment associated with the image sequence. + /// + public HeifReferenceViewingEnvironment? ReferenceViewingEnvironment { get; set; } + + /// + /// Gets or sets the nominal diffuse-white description associated with the image sequence. + /// + public HeifNominalDiffuseWhite? NominalDiffuseWhite { get; set; } + + /// + /// Gets or sets the relative pixel spacing associated with the image sequence. + /// + public HeifPixelAspectRatio? PixelAspectRatio { get; set; } + + /// + /// Gets or sets the clean-aperture crop applied to each decoded sample. + /// + public HeifCleanAperture? CleanAperture { get; set; } + + /// + /// Gets or sets the number of 90-degree counter-clockwise rotations applied to each decoded sample. + /// + public byte? RotationAngle { get; set; } + + /// + /// Gets or sets the image-mirror axis, where zero is horizontal and one is vertical. + /// + public byte? MirrorAxis { get; set; } + /// /// Gets or sets the retained sample descriptors in decode order. /// diff --git a/tests/ImageSharp.Tests/Formats/Heif/HeifSequenceParserTests.cs b/tests/ImageSharp.Tests/Formats/Heif/HeifSequenceParserTests.cs index 78cfc197c..60d8ba3fa 100644 --- a/tests/ImageSharp.Tests/Formats/Heif/HeifSequenceParserTests.cs +++ b/tests/ImageSharp.Tests/Formats/Heif/HeifSequenceParserTests.cs @@ -137,12 +137,37 @@ public class HeifSequenceParserTests Assert.Throws(() => parser.Parse(stream, GetMoviePayloadLength(data))); } + [Fact] + public void ParseRetainsTrackImageProperties() + { + byte[] data = CreateSequenceFile(1024, trackProperties: true); + using MemoryStream stream = new(data, false); + HeifSequenceParser parser = new(Configuration.Default.MemoryAllocator, 2); + stream.Position = 8; + + HeifSequenceTrack track = parser.Parse(stream, GetMoviePayloadLength(data)).ColorTrack; + + Assert.NotNull(track.CicpProfile); + Assert.Equal(4U, track.PixelAspectRatio!.HorizontalSpacing); + Assert.Equal(3U, track.PixelAspectRatio.VerticalSpacing); + Assert.Equal(new Rectangle(0, 0, 320, 240), track.CleanAperture!.Value.ToRectangle(new Size(320, 240))); + Assert.Equal((byte)1, track.RotationAngle); + Assert.Equal((byte)1, track.MirrorAxis); + Assert.Equal((ushort)1000, track.ContentLightLevel!.Value.MaximumContentLightLevel); + Assert.NotNull(track.MasteringDisplayColorVolume); + Assert.NotNull(track.ContentColorVolume); + Assert.NotNull(track.AmbientViewingEnvironment); + Assert.NotNull(track.ReferenceViewingEnvironment); + Assert.NotNull(track.NominalDiffuseWhite); + } + private static byte[] CreateSequenceFile( uint chunkOffset, bool hevc = false, bool compositionOffsets = false, bool directReferences = false, - uint directReferenceSampleId = 1) + uint directReferenceSampleId = 1, + bool trackProperties = false) { using MemoryStream stream = new(); using BinaryWriter writer = new(stream, Encoding.UTF8, true); @@ -167,7 +192,7 @@ public class HeifSequenceParserTests long mediaInformation = BeginBox(writer, Heif4CharCode.Minf); WriteDataInformation(writer); - WriteSampleTable(writer, chunkOffset, hevc, compositionOffsets, directReferences, directReferenceSampleId); + WriteSampleTable(writer, chunkOffset, hevc, compositionOffsets, directReferences, directReferenceSampleId, trackProperties); EndBox(writer, mediaInformation); EndBox(writer, media); EndBox(writer, track); @@ -261,10 +286,11 @@ public class HeifSequenceParserTests bool hevc, bool compositionOffsets, bool directReferences, - uint directReferenceSampleId) + uint directReferenceSampleId, + bool trackProperties) { long sampleTable = BeginBox(writer, Heif4CharCode.Stbl); - WriteSampleDescription(writer, hevc); + WriteSampleDescription(writer, hevc, trackProperties); long timing = BeginBox(writer, Heif4CharCode.Stts); WriteFullBoxHeader(writer, 0, 0); @@ -357,7 +383,7 @@ public class HeifSequenceParserTests EndBox(writer, sampleMap); } - private static void WriteSampleDescription(BinaryWriter writer, bool hevc) + private static void WriteSampleDescription(BinaryWriter writer, bool hevc, bool trackProperties) { long description = BeginBox(writer, Heif4CharCode.Stsd); WriteFullBoxHeader(writer, 0, 0); @@ -387,6 +413,11 @@ public class HeifSequenceParserTests EndBox(writer, configuration); } + if (trackProperties) + { + WriteTrackImageProperties(writer); + } + long codingConstraints = BeginBox(writer, Heif4CharCode.Ccst); WriteFullBoxHeader(writer, 0, 0); WriteUInt32(writer, 0x7C000000); @@ -395,6 +426,87 @@ public class HeifSequenceParserTests EndBox(writer, description); } + private static void WriteTrackImageProperties(BinaryWriter writer) + { + long color = BeginBox(writer, Heif4CharCode.Colr); + WriteUInt32(writer, (uint)Heif4CharCode.Nclx); + WriteUInt16(writer, 1); + WriteUInt16(writer, 13); + WriteUInt16(writer, 6); + writer.Write((byte)0x80); + EndBox(writer, color); + + long pixelAspectRatio = BeginBox(writer, Heif4CharCode.Pasp); + WriteUInt32(writer, 4); + WriteUInt32(writer, 3); + EndBox(writer, pixelAspectRatio); + + long cleanAperture = BeginBox(writer, Heif4CharCode.Clap); + WriteUInt32(writer, 320); + WriteUInt32(writer, 1); + WriteUInt32(writer, 240); + WriteUInt32(writer, 1); + WriteUInt32(writer, 0); + WriteUInt32(writer, 1); + WriteUInt32(writer, 0); + WriteUInt32(writer, 1); + EndBox(writer, cleanAperture); + + long rotation = BeginBox(writer, Heif4CharCode.Irot); + writer.Write((byte)1); + EndBox(writer, rotation); + + long mirror = BeginBox(writer, Heif4CharCode.Imir); + writer.Write((byte)1); + EndBox(writer, mirror); + + long contentLightLevel = BeginBox(writer, Heif4CharCode.Clli); + WriteUInt16(writer, 1000); + WriteUInt16(writer, 400); + EndBox(writer, contentLightLevel); + + long masteringDisplay = BeginBox(writer, Heif4CharCode.Mdcv); + WriteUInt16(writer, 15000); + WriteUInt16(writer, 30000); + WriteUInt16(writer, 7500); + WriteUInt16(writer, 3000); + WriteUInt16(writer, 34000); + WriteUInt16(writer, 16000); + WriteUInt16(writer, 15635); + WriteUInt16(writer, 16450); + WriteUInt32(writer, 10_000_000); + WriteUInt32(writer, 50); + EndBox(writer, masteringDisplay); + + long contentColorVolume = BeginBox(writer, Heif4CharCode.Cclv); + writer.Write((byte)0x1C); + WriteUInt32(writer, 1_000_000); + WriteUInt32(writer, 10_000_000); + WriteUInt32(writer, 5_000_000); + EndBox(writer, contentColorVolume); + + long ambientViewing = BeginBox(writer, Heif4CharCode.Amve); + WriteUInt32(writer, 10_000); + WriteUInt16(writer, 15_635); + WriteUInt16(writer, 16_450); + EndBox(writer, ambientViewing); + + long referenceViewing = BeginBox(writer, Heif4CharCode.Reve); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, 10_000); + WriteUInt16(writer, 3_127); + WriteUInt16(writer, 3_290); + WriteUInt32(writer, 5_000); + WriteUInt16(writer, 3_127); + WriteUInt16(writer, 3_290); + EndBox(writer, referenceViewing); + + long nominalDiffuseWhite = BeginBox(writer, Heif4CharCode.Ndwt); + WriteFullBoxHeader(writer, 0, 0); + WriteUInt32(writer, 2_030_000); + EndBox(writer, nominalDiffuseWhite); + } + private static void WriteHevcConfiguration(BinaryWriter writer) { long configuration = BeginBox(writer, Heif4CharCode.HvcC);