Browse Source

Unify HEIF planar color conversion

pull/2633/head
James Jackson-South 2 months ago
parent
commit
1098109df3
  1. 5
      HEIF_IMPLEMENTATION_PLAN.md
  2. 1
      src/ImageSharp/Formats/Heif/Av1/Av1Decoder.cs
  3. 455
      src/ImageSharp/Formats/Heif/Av1/Av1YuvConverter.RowConverters.cs
  4. 174
      src/ImageSharp/Formats/Heif/Av1/Av1YuvConverter.cs
  5. 133
      src/ImageSharp/Formats/Heif/Av1/Color/Av1PlanarSampleBuffer.cs
  6. 109
      src/ImageSharp/Formats/Heif/Av1/Color/Av1YuvConverter.cs
  7. 18
      src/ImageSharp/Formats/Heif/Color/HeifColorConversionParameters.cs
  8. 738
      src/ImageSharp/Formats/Heif/Color/HeifPlanarColorConverter.cs
  9. 18
      src/ImageSharp/Formats/Heif/Color/HeifYuv420ToRgb8Converter.cs
  10. 78
      src/ImageSharp/Formats/Heif/Color/IHeifPlanarSampleBuffer.cs
  11. 31
      src/ImageSharp/Formats/Heif/Color/IHeifYuv420ToRgb8Source.cs
  12. 81
      src/ImageSharp/Formats/Heif/Hevc/Color/HevcPlanarSampleBuffer.cs
  13. 32
      src/ImageSharp/Formats/Heif/Hevc/Color/HevcYuv420ToRgb8Source.cs
  14. 597
      src/ImageSharp/Formats/Heif/Hevc/Color/HevcYuvConverter.cs
  15. 1
      tests/ImageSharp.Benchmarks/Codecs/Heif/Av1ColorConversionBenchmarks.cs
  16. 35
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1YuvConverterTests.cs

5
HEIF_IMPLEMENTATION_PLAN.md

@ -41,7 +41,7 @@ Checkboxes may be marked complete only when the implementation and the verificat
- [x] Verify IPT-C2 against independently calculated matrix values and verify YCgCo-Re/YCgCo-Ro against exact 8/10/12-bit code values, limited-range black/white values, and SIMD-lane/scalar-tail parity. - [x] Verify IPT-C2 against independently calculated matrix values and verify YCgCo-Re/YCgCo-Ro against exact 8/10/12-bit code values, limited-range black/white values, and SIMD-lane/scalar-tail parity.
- [x] Implement SIMD 4:2:0 and 4:2:2 encoder downsampling with odd-width and odd-height tails, and verify the stored 8/12-bit chroma samples against independently encoded full-resolution planes. - [x] Implement SIMD 4:2:0 and 4:2:2 encoder downsampling with odd-width and odd-height tails, and verify the stored 8/12-bit chroma samples against independently encoded full-resolution planes.
- [x] Move the H.273 parameter resolver, closed color operators, transfer functions, sample loading/storage, chroma filtering, and RGB packing into one shared HEIF color pipeline used by both AV1 and HEVC rather than maintaining codec-specific arithmetic copies. - [x] Move the H.273 parameter resolver, closed color operators, transfer functions, sample loading/storage, chroma filtering, and RGB packing into one shared HEIF color pipeline used by both AV1 and HEVC rather than maintaining codec-specific arithmetic copies.
- [x] Keep optimized color arithmetic, fixed-point parameter storage, SIMD-width traversal, sample loading/storage, chroma filtering, and RGB packing primitives in the shared HEIF color namespace. AV1 and HEVC namespaces adapt their native planes, crop windows, and signaled chroma positions without owning parallel color arithmetic. - [x] Drive AV1 and HEVC through the same `IHeifPlanarSampleBuffer<TSample>` and `HeifPlanarColorConverter` architecture. Codec namespaces now adapt only native rows, sample precision, subsampling, crop windows, chroma positions, and codec color signaling; pooled row scheduling, chroma reconstruction/downsampling, fixed-point specialization, SIMD traversal, scalar tails, and RGB packing are selected by the shared HEIF driver.
- [x] Implement sequential pooled HEVC conversion in both directions for monochrome, 4:2:0, 4:2:2, and 4:4:4; independent luma/chroma precision; 8/10/12-bit full/limited ranges; and all six progressive 4:2:0 chroma sample locations. Eight-bit paths use JPEG's optimized RGB plane contracts, while high-bit-depth paths retain 16-bit packed pixels. - [x] Implement sequential pooled HEVC conversion in both directions for monochrome, 4:2:0, 4:2:2, and 4:4:4; independent luma/chroma precision; 8/10/12-bit full/limited ranges; and all six progressive 4:2:0 chroma sample locations. Eight-bit paths use JPEG's optimized RGB plane contracts, while high-bit-depth paths retain 16-bit packed pixels.
- [x] Implement allocation-free SIMD-first HEVC intra prediction for all 35 modes, 4/8/16/32 blocks, and 8/10/12-bit samples, including three-tap and strong-bilinear reference filtering, negative-angle reference extension, luma boundary filters, and SIMD horizontal transposition. Verify the normal and forced-scalar paths against one scalar oracle. - [x] Implement allocation-free SIMD-first HEVC intra prediction for all 35 modes, 4/8/16/32 blocks, and 8/10/12-bit samples, including three-tap and strong-bilinear reference filtering, negative-angle reference extension, luma boundary filters, and SIMD horizontal transposition. Verify the normal and forced-scalar paths against one scalar oracle.
- [x] Match HM's channel-specific smoothing rules: combined 4:4:4 chroma uses the normal three-tap filter when selected, while strong bilinear smoothing remains restricted to luma syntax. Exact official RExt reconstruction passes at 8, 10, and 12 bits. - [x] Match HM's channel-specific smoothing rules: combined 4:4:4 chroma uses the normal three-tap filter when selected, while strong bilinear smoothing remains restricted to luma syntax. Exact official RExt reconstruction passes at 8, 10, and 12 bits.
@ -63,6 +63,7 @@ Checkboxes may be marked complete only when the implementation and the verificat
- [x] Add permanent frame-wide `Av1ColorConversionBenchmarks` coverage for sequential 1920x1080 YUV 4:2:0 conversion in both directions at 8, 10, and 12 bits, including managed-allocation reporting. - [x] Add permanent frame-wide `Av1ColorConversionBenchmarks` coverage for sequential 1920x1080 YUV 4:2:0 conversion in both directions at 8, 10, and 12 bits, including managed-allocation reporting.
- [x] Record the .NET 10 short-run AV1 baseline on 2026-08-25. SIMD decode measured 4.025/2.803/2.742 ms and encode measured 3.750/2.555/2.450 ms at 8/10/12 bits with 488-744 B allocated per frame. Forced-scalar decode measured 31.15/29.47/28.40 ms and encode measured 19.19/15.77/15.14 ms, making the normal SIMD pipeline 7.7-10.5 times faster for decode and 5.1-6.2 times faster for encode on this machine. - [x] Record the .NET 10 short-run AV1 baseline on 2026-08-25. SIMD decode measured 4.025/2.803/2.742 ms and encode measured 3.750/2.555/2.450 ms at 8/10/12 bits with 488-744 B allocated per frame. Forced-scalar decode measured 31.15/29.47/28.40 ms and encode measured 19.19/15.77/15.14 ms, making the normal SIMD pipeline 7.7-10.5 times faster for decode and 5.1-6.2 times faster for encode on this machine.
- [x] Add permanent frame-wide `HevcColorConversionBenchmarks` coverage and record the .NET 10 short-run plane-conversion baseline on 2026-08-26. SIMD decode measured 1.141/2.640/2.668 ms and encode measured 2.089/2.481/2.412 ms at 8/10/12 bits with 504-712 B allocated per frame. Forced-scalar decode measured 13.42/28.35/27.60 ms and encode measured 14.82/14.67/14.45 ms, making the normal SIMD pipeline 10.3-11.8 times faster for decode and 5.9-7.1 times faster for encode on this machine. - [x] Add permanent frame-wide `HevcColorConversionBenchmarks` coverage and record the .NET 10 short-run plane-conversion baseline on 2026-08-26. SIMD decode measured 1.141/2.640/2.668 ms and encode measured 2.089/2.481/2.412 ms at 8/10/12 bits with 504-712 B allocated per frame. Forced-scalar decode measured 13.42/28.35/27.60 ms and encode measured 14.82/14.67/14.45 ms, making the normal SIMD pipeline 10.3-11.8 times faster for decode and 5.9-7.1 times faster for encode on this machine.
- [x] Re-verify the unified AV1/HEVC driver with all 127 focused color and exact HEIC presentation cases under normal SIMD dispatch and with all hardware intrinsics disabled. Re-run all 12 permanent 1920x1080 plane-conversion benchmarks; BenchmarkDotNet could not hold the configured power plan and the CPU changed performance state during warmup, so do not replace the controlled baselines above with those throttled sustained means.
- [ ] Add representative ICC-enabled AVIF cases and the equivalent HEVC cases after the HEVC reconstruction path is complete. - [ ] Add representative ICC-enabled AVIF cases and the equivalent HEVC cases after the HEVC reconstruction path is complete.
- [x] **Completed:** remove production `ToArray()` materializations from the HEIF implementation and make every retained copy an explicit ownership decision. - [x] **Completed:** remove production `ToArray()` materializations from the HEIF implementation and make every retained copy an explicit ownership decision.
- [x] Store AV1 palette colors and HEVC short-term reference-picture state inline at their normative fixed bounds. - [x] Store AV1 palette colors and HEVC short-term reference-picture state inline at their normative fixed bounds.
@ -216,7 +217,7 @@ This snapshot pins or classifies the available references and failures; it does
| Managed implementation | Normative behavior | Reviewed implementation reference | Use | | Managed implementation | Normative behavior | Reviewed implementation reference | Use |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| `HeifColorConversionParameters`, `HeifColorConverter`, `Av1YuvConverter`, and `HevcYuvConverter` | ITU-T H.273 (V4) equations 14-16 and 27-84, including limited/full-range scaling, chromaticity-derived equations 39-47, YCgCo equations 51-57, constant-luminance equations 66-75, IPT-C2, YCgCo-Re/YCgCo-Ro, and the PQ/HLG ICtCp matrices; AV1 section 6.4.2 and HEVC Annex E chroma sample positions | Official ITU-T H.273 (V4) (07/2024); libavif `src/reformat.c` and `src/colr.c` at `092276ce89098ead06db80975173191e5fee1826`; libaom `aom/aom_image.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Scalar behavioral oracle for full/limited-range conversion at 8, 10, and 12 bits. Decode and encode cover every H.273 V4 matrix code point through 17: identity, YCgCo, coefficient-based NCL, both fixed and chromaticity-derived constant/non-constant-luminance systems, SMPTE ST 2085, ICtCp, IPT-C2, YCgCo-Re, and YCgCo-Ro. Shared paths cover monochrome, YUV 4:2:0, 4:2:2, and 4:4:4; AV1 centered/co-sited placement; and all six HEVC progressive 4:2:0 sample locations. Identity and reversible lifting enforce their full-chroma constraint. Limited-range YCgCo retains the 219-code scale inherited from its R/G/B inputs instead of applying YCbCr's unrelated 224-code chroma range. YCgCo-Re and YCgCo-Ro apply range adjustment at reduced-precision RGB code values before signed integer lifting. Chromaticity derivation uses every defined H.273 primary and matches libavif's BT.709 fallback for unspecified or reserved primaries. ICtCp and IPT-C2 inverses come from the exact H.273 matrices rather than unrelated display conversions. Closed operators provide `Vector512`, `Vector256`, and `Vector128` implementations with one scalar tail. Eight-bit codec wrappers use JPEG's RGB plane pack/unpack contracts; high-bit-depth wrappers retain native 16-bit packed pixels. | | `HeifColorConversionParameters`, `HeifColorConverter`, `IHeifPlanarSampleBuffer`, `HeifPlanarColorConverter`, `Av1YuvConverter`, and `HevcYuvConverter` | ITU-T H.273 (V4) equations 14-16 and 27-84, including limited/full-range scaling, chromaticity-derived equations 39-47, YCgCo equations 51-57, constant-luminance equations 66-75, IPT-C2, YCgCo-Re/YCgCo-Ro, and the PQ/HLG ICtCp matrices; AV1 section 6.4.2 and HEVC Annex E chroma sample positions | Official ITU-T H.273 (V4) (07/2024); libavif `src/reformat.c` and `src/colr.c` at `092276ce89098ead06db80975173191e5fee1826`; libaom `aom/aom_image.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Scalar behavioral oracle for full/limited-range conversion at 8, 10, and 12 bits. Decode and encode cover every H.273 V4 matrix code point through 17: identity, YCgCo, coefficient-based NCL, both fixed and chromaticity-derived constant/non-constant-luminance systems, SMPTE ST 2085, ICtCp, IPT-C2, YCgCo-Re, and YCgCo-Ro. One closed generic planar driver owns pooled sequential row traversal, chroma reconstruction/downsampling, fixed-point specialization, SIMD dispatch, scalar tails, and packed-pixel conversion for both AV1 and HEVC adapters. Shared paths cover monochrome, YUV 4:2:0, 4:2:2, and 4:4:4; AV1 centered/co-sited placement; and all six HEVC progressive 4:2:0 sample locations. Identity and reversible lifting enforce their full-chroma constraint. Limited-range YCgCo retains the 219-code scale inherited from its R/G/B inputs instead of applying YCbCr's unrelated 224-code chroma range. YCgCo-Re and YCgCo-Ro apply range adjustment at reduced-precision RGB code values before signed integer lifting. Chromaticity derivation uses every defined H.273 primary and matches libavif's BT.709 fallback for unspecified or reserved primaries. ICtCp and IPT-C2 inverses come from the exact H.273 matrices rather than unrelated display conversions. Closed operators provide `Vector512`, `Vector256`, and `Vector128` implementations with one scalar tail. Eight-bit conversion uses JPEG's RGB plane pack/unpack contracts; high-bit-depth conversion retains native 16-bit packed pixels. |
| `HeifTransferFunctions` | ITU-T H.273 (V4) Table 3 transfer characteristics 1-18 | Official ITU-T H.273 (V4) (07/2024); libavif `src/colr.c` at `092276ce89098ead06db80975173191e5fee1826` | Apply every HEVC/AV1-signallable transfer function required by constant-luminance, ICtCp, and IPT-C2 conversion. Retain the H.273 normalized PQ and HLG definitions; do not import libavif's display-oriented 203-nit scaling or HLG OOTF into codec sample interpretation. Use libavif's midpoint convention only for the non-bijective zero code of the two logarithmic curves. | | `HeifTransferFunctions` | ITU-T H.273 (V4) Table 3 transfer characteristics 1-18 | Official ITU-T H.273 (V4) (07/2024); libavif `src/colr.c` at `092276ce89098ead06db80975173191e5fee1826` | Apply every HEVC/AV1-signallable transfer function required by constant-luminance, ICtCp, and IPT-C2 conversion. Retain the H.273 normalized PQ and HLG definitions; do not import libavif's display-oriented 203-nit scaling or HLG OOTF into codec sample interpretation. Use libavif's midpoint convention only for the non-bijective zero code of the two logarithmic curves. |
| `ObuReader.ReadSequenceHeader`, `ReadUncompressedFrameHeader`, decoder-model parsing, and operating-parameter consumption | AV1 sections 5.5.2 through 5.5.4 sequence timing and decoder-model syntax, section 5.9.2 uncompressed frame-header syntax, and section 5.9.31 temporal-point syntax | libaom `av1/decoder/decodeframe.c` functions `av1_read_decoder_model_info`, `av1_read_op_parameters_info`, `read_temporal_point_info`, and `read_uncompressed_header`, plus `common/av1_config.c`, at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Read the normative 32-bit decoding-tick field; consume operating-point buffer delays and the layer-applicable frame removal times needed to keep a non-reduced still-image sequence aligned; and read presentation time only under the normative decoder-model condition. Retain no scheduling or playback behavior from those values and introduce no ISO BMFF timing, track, sample-table, or sequence surface. | | `ObuReader.ReadSequenceHeader`, `ReadUncompressedFrameHeader`, decoder-model parsing, and operating-parameter consumption | AV1 sections 5.5.2 through 5.5.4 sequence timing and decoder-model syntax, section 5.9.2 uncompressed frame-header syntax, and section 5.9.31 temporal-point syntax | libaom `av1/decoder/decodeframe.c` functions `av1_read_decoder_model_info`, `av1_read_op_parameters_info`, `read_temporal_point_info`, and `read_uncompressed_header`, plus `common/av1_config.c`, at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Read the normative 32-bit decoding-tick field; consume operating-point buffer delays and the layer-applicable frame removal times needed to keep a non-reduced still-image sequence aligned; and read presentation time only under the normative decoder-model condition. Retain no scheduling or playback behavior from those values and introduce no ISO BMFF timing, track, sample-table, or sequence surface. |
| `Av1FrameBuffer` high-bit-depth sample layout and `Av1YuvConverter` 10/12-bit packed-pixel conversion | AV1 section 6.4.1 bit depth and H.273 sample-range scaling | libaom `aom_scale/yv12config.h`, `av1/common/idct.c`, and `av1/common/reconintra.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f`; libavif `src/avif.c` and `src/reformat.c` at `092276ce89098ead06db80975173191e5fee1826` | Establish two-byte native sample storage with sample-unit strides for 10/12-bit reconstruction and use ImageSharp's existing `Rgb48` pixel-operation paths in both directions so packed-pixel staging does not reduce high-bit-depth samples to eight bits. | | `Av1FrameBuffer` high-bit-depth sample layout and `Av1YuvConverter` 10/12-bit packed-pixel conversion | AV1 section 6.4.1 bit depth and H.273 sample-range scaling | libaom `aom_scale/yv12config.h`, `av1/common/idct.c`, and `av1/common/reconintra.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f`; libavif `src/avif.c` and `src/reformat.c` at `092276ce89098ead06db80975173191e5fee1826` | Establish two-byte native sample storage with sample-unit strides for 10/12-bit reconstruction and use ImageSharp's existing `Rgb48` pixel-operation paths in both directions so packed-pixel staging does not reduce high-bit-depth samples to eight bits. |

1
src/ImageSharp/Formats/Heif/Av1/Av1Decoder.cs

@ -1,6 +1,7 @@
// Copyright (c) Six Labors. // Copyright (c) Six Labors.
// Licensed under the Six Labors Split License. // Licensed under the Six Labors Split License.
using SixLabors.ImageSharp.Formats.Heif.Av1.Color;
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline; using SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline;
using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling; using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;

455
src/ImageSharp/Formats/Heif/Av1/Av1YuvConverter.RowConverters.cs

@ -1,455 +0,0 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using System.Runtime.InteropServices;
using SixLabors.ImageSharp.Advanced;
using SixLabors.ImageSharp.Common.Helpers;
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Formats.Heif.Color;
using SixLabors.ImageSharp.Memory;
using SixLabors.ImageSharp.PixelFormats;
using static SixLabors.ImageSharp.Formats.Heif.Color.HeifColorConverterBase;
namespace SixLabors.ImageSharp.Formats.Heif.Av1;
/// <content>
/// Provides the pooled row converters used by AV1 color conversion.
/// </content>
internal static partial class Av1YuvConverter
{
/// <summary>
/// Converts one reconstructed AV1 row using frame-scoped pooled component, chroma, and packed-pixel storage.
/// </summary>
/// <typeparam name="TPixel">The destination pixel type.</typeparam>
/// <typeparam name="TSample">The reconstructed sample type.</typeparam>
/// <typeparam name="TLoader">The SIMD widening operations for the sample type.</typeparam>
private readonly struct YuvToRgbRowConverter<TPixel, TSample, TLoader>
where TPixel : unmanaged, IPixel<TPixel>
where TSample : unmanaged
where TLoader : struct, IHeifSampleLoader<TSample>
{
/// <summary>
/// The configuration used by bulk pixel conversion.
/// </summary>
private readonly Configuration configuration;
/// <summary>
/// The reconstructed AV1 frame containing the source planes.
/// </summary>
private readonly Av1FrameBuffer<byte> frameBuffer;
/// <summary>
/// The destination image frame.
/// </summary>
private readonly ImageFrame<TPixel> image;
/// <summary>
/// The full-resolution luma plane.
/// </summary>
private readonly Buffer2DRegion<byte> yPlane;
/// <summary>
/// The blue-difference plane when the frame contains chroma.
/// </summary>
private readonly Buffer2DRegion<byte> uPlane;
/// <summary>
/// The red-difference plane when the frame contains chroma.
/// </summary>
private readonly Buffer2DRegion<byte> vPlane;
/// <summary>
/// The frame-scoped color-model converter.
/// </summary>
private readonly HeifColorConverterBase colorConverter;
/// <summary>
/// The signaled chroma sample position used for reconstruction.
/// </summary>
private readonly ObuChromoSamplePosition chromaSamplePosition;
/// <summary>
/// Whether the source contains only a luma plane.
/// </summary>
private readonly bool isMonochrome;
/// <summary>
/// The horizontal chroma subsampling shift.
/// </summary>
private readonly int subX;
/// <summary>
/// The vertical chroma subsampling shift.
/// </summary>
private readonly int subY;
/// <summary>
/// Initializes a new instance of the <see cref="YuvToRgbRowConverter{TPixel, TSample, TLoader}"/> struct.
/// </summary>
/// <param name="configuration">The configuration used for packed-pixel conversion.</param>
/// <param name="frameBuffer">The reconstructed AV1 frame.</param>
/// <param name="image">The destination image frame.</param>
/// <param name="colorConverter">The selected H.273 color converter.</param>
public YuvToRgbRowConverter(
Configuration configuration,
Av1FrameBuffer<byte> frameBuffer,
ImageFrame<TPixel> image,
HeifColorConverterBase colorConverter)
{
this.configuration = configuration;
this.frameBuffer = frameBuffer;
this.image = image;
this.colorConverter = colorConverter;
this.isMonochrome = frameBuffer.ColorFormat == Av1ColorFormat.Yuv400;
this.subX = frameBuffer.ColorConfig.SubSamplingX ? 1 : 0;
this.subY = frameBuffer.ColorConfig.SubSamplingY ? 1 : 0;
this.chromaSamplePosition = frameBuffer.ColorConfig.ChromaSamplePosition;
this.yPlane = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0);
this.uPlane = this.isMonochrome ? default : frameBuffer.DeriveBlockPointer(Av1Plane.U, this.subX, this.subY);
this.vPlane = this.isMonochrome ? default : frameBuffer.DeriveBlockPointer(Av1Plane.V, this.subX, this.subY);
}
/// <summary>
/// Gets the number of float elements required by the reusable conversion buffer.
/// </summary>
public int BufferLength
{
get
{
// Three float component rows are converted in place. Color input adds two reusable chroma scratch
// rows, and the final one or two float-sized slots per pixel back RGB byte planes or an Rgba64 row.
int rowCount = this.isMonochrome ? 3 : 5;
int packedRowCount = typeof(TSample) == typeof(byte) ? 1 : 2;
return this.image.Width * (rowCount + packedRowCount);
}
}
/// <summary>
/// Converts one reconstructed AV1 row to packed pixels.
/// </summary>
/// <param name="y">The row index.</param>
/// <param name="span">The reusable conversion buffer.</param>
public void Convert(int y, Span<float> span)
{
int width = this.image.Width;
Span<float> red = span[..width];
Span<float> green = span.Slice(width, width);
Span<float> blue = span.Slice(width * 2, width);
ReadOnlySpan<TSample> ySource;
if (typeof(TSample) == typeof(byte))
{
ySource = MemoryMarshal.Cast<byte, TSample>(this.yPlane.DangerousGetRowSpan(y));
}
else
{
ySource = MemoryMarshal.Cast<ushort, TSample>(this.frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, y, 0, 0));
}
ConvertSamplesToFloat<TSample, TLoader>(ySource, red);
int packedOffset = width * 3;
if (!this.isMonochrome)
{
GetChromaCoordinates(
y,
this.subY,
this.subY != 0 && this.chromaSamplePosition != ObuChromoSamplePosition.Colocated,
this.uPlane.Height - 1,
out int y0,
out int y1,
out int y1Weight);
ReadOnlySpan<TSample> uRow0;
ReadOnlySpan<TSample> uRow1;
ReadOnlySpan<TSample> vRow0;
ReadOnlySpan<TSample> vRow1;
if (typeof(TSample) == typeof(byte))
{
uRow0 = MemoryMarshal.Cast<byte, TSample>(this.uPlane.DangerousGetRowSpan(y0));
uRow1 = MemoryMarshal.Cast<byte, TSample>(this.uPlane.DangerousGetRowSpan(y1));
vRow0 = MemoryMarshal.Cast<byte, TSample>(this.vPlane.DangerousGetRowSpan(y0));
vRow1 = MemoryMarshal.Cast<byte, TSample>(this.vPlane.DangerousGetRowSpan(y1));
}
else
{
uRow0 = MemoryMarshal.Cast<ushort, TSample>(this.frameBuffer.GetHighBitDepthRowSpan(Av1Plane.U, y0, this.subX, this.subY));
uRow1 = MemoryMarshal.Cast<ushort, TSample>(this.frameBuffer.GetHighBitDepthRowSpan(Av1Plane.U, y1, this.subX, this.subY));
vRow0 = MemoryMarshal.Cast<ushort, TSample>(this.frameBuffer.GetHighBitDepthRowSpan(Av1Plane.V, y0, this.subX, this.subY));
vRow1 = MemoryMarshal.Cast<ushort, TSample>(this.frameBuffer.GetHighBitDepthRowSpan(Av1Plane.V, y1, this.subX, this.subY));
}
Span<float> scratch0 = span.Slice(width * 3, width);
Span<float> scratch1 = span.Slice(width * 4, width);
bool isCenteredX = this.subX != 0 && (this.subY == 0 || this.chromaSamplePosition == ObuChromoSamplePosition.Unknown);
ReconstructChromaRow<TSample, TLoader>(uRow0, uRow1, y1Weight, this.subX, isCenteredX, green, scratch0, scratch1);
ReconstructChromaRow<TSample, TLoader>(vRow0, vRow1, y1Weight, this.subX, isCenteredX, blue, scratch0, scratch1);
packedOffset = width * 5;
}
this.colorConverter.ConvertToRgbInPlace(red, green, blue);
Span<float> packedStorage = span[packedOffset..];
Span<TPixel> destination = this.image.PixelBuffer.DangerousGetRowSpan(y);
if (typeof(TSample) == typeof(byte))
{
// Match JPEG's byte-plane packing contract so optimized pixel types use the existing RGB packer.
// The final float row provides enough byte storage for three tightly packed component planes.
Span<byte> byteStorage = MemoryMarshal.AsBytes(packedStorage)[..(width * 3)];
Span<byte> redBytes = byteStorage[..width];
Span<byte> greenBytes = byteStorage.Slice(width, width);
Span<byte> blueBytes = byteStorage.Slice(width * 2, width);
SimdUtils.NormalizedFloatToByteSaturate(red, redBytes);
SimdUtils.NormalizedFloatToByteSaturate(green, greenBytes);
SimdUtils.NormalizedFloatToByteSaturate(blue, blueBytes);
PixelOperations<TPixel>.Instance.PackFromRgbPlanes(redBytes, greenBytes, blueBytes, destination);
}
else
{
Span<Rgba64> packed = MemoryMarshal.Cast<float, Rgba64>(packedStorage)[..width];
PackRgba64(red, green, blue, packed);
PixelOperations<TPixel>.Instance.FromRgba64(this.configuration, packed, destination);
}
}
}
/// <summary>
/// Converts one source row, or one vertically subsampled row pair, using reusable pooled component storage.
/// </summary>
/// <typeparam name="TPixel">The source pixel type.</typeparam>
/// <typeparam name="TSample">The encoded sample type.</typeparam>
/// <typeparam name="TStorer">The SIMD narrowing and storage operations for the sample type.</typeparam>
private readonly struct RgbToYuvRowConverter<TPixel, TSample, TStorer>
where TPixel : unmanaged, IPixel<TPixel>
where TSample : unmanaged
where TStorer : struct, IHeifSampleStorer<TSample>
{
/// <summary>
/// The configuration used by bulk pixel conversion.
/// </summary>
private readonly Configuration configuration;
/// <summary>
/// The destination AV1 frame.
/// </summary>
private readonly Av1FrameBuffer<byte> frameBuffer;
/// <summary>
/// The source image frame.
/// </summary>
private readonly ImageFrame<TPixel> image;
/// <summary>
/// The full-resolution luma plane.
/// </summary>
private readonly Buffer2DRegion<byte> yPlane;
/// <summary>
/// The blue-difference plane when the frame contains chroma.
/// </summary>
private readonly Buffer2DRegion<byte> uPlane;
/// <summary>
/// The red-difference plane when the frame contains chroma.
/// </summary>
private readonly Buffer2DRegion<byte> vPlane;
/// <summary>
/// The frame-scoped color-model converter.
/// </summary>
private readonly HeifColorConverterBase colorConverter;
/// <summary>
/// The largest value represented by the encoded AV1 bit depth.
/// </summary>
private readonly float sampleMaximum;
/// <summary>
/// Whether the destination contains only a luma plane.
/// </summary>
private readonly bool isMonochrome;
/// <summary>
/// The horizontal chroma subsampling shift.
/// </summary>
private readonly int subX;
/// <summary>
/// The vertical chroma subsampling shift.
/// </summary>
private readonly int rowShift;
/// <summary>
/// Initializes a new instance of the <see cref="RgbToYuvRowConverter{TPixel, TSample, TStorer}"/> struct.
/// </summary>
/// <param name="configuration">The configuration used for packed-pixel conversion.</param>
/// <param name="frameBuffer">The destination AV1 frame.</param>
/// <param name="image">The source image frame.</param>
/// <param name="colorConverter">The selected H.273 color converter.</param>
/// <param name="sampleMaximum">The largest encoded sample value.</param>
public RgbToYuvRowConverter(
Configuration configuration,
Av1FrameBuffer<byte> frameBuffer,
ImageFrame<TPixel> image,
HeifColorConverterBase colorConverter,
float sampleMaximum)
{
this.configuration = configuration;
this.frameBuffer = frameBuffer;
this.image = image;
this.colorConverter = colorConverter;
this.sampleMaximum = sampleMaximum;
this.isMonochrome = frameBuffer.ColorFormat == Av1ColorFormat.Yuv400;
this.subX = frameBuffer.ColorConfig.SubSamplingX ? 1 : 0;
this.rowShift = !this.isMonochrome && frameBuffer.ColorConfig.SubSamplingY ? 1 : 0;
this.yPlane = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0);
this.uPlane = this.isMonochrome ? default : frameBuffer.DeriveBlockPointer(Av1Plane.U, this.subX, this.rowShift);
this.vPlane = this.isMonochrome ? default : frameBuffer.DeriveBlockPointer(Av1Plane.V, this.subX, this.rowShift);
}
/// <summary>
/// Gets the number of float elements required by the reusable conversion buffer.
/// </summary>
public int ComponentBufferLength
{
get
{
// Three planar rows hold the converted YUV values, and 4:2:0 keeps a second set until its
// chroma has been averaged with the first.
int componentRowCount = this.rowShift == 0 ? 3 : 6;
return this.image.Width * componentRowCount;
}
}
/// <summary>
/// Converts one source row, or one vertically subsampled row pair, to AV1 planes.
/// </summary>
/// <param name="y">The conversion iteration index.</param>
/// <param name="packed">The reusable high-bit-depth RGB staging row.</param>
/// <param name="components">The reusable planar component buffer.</param>
public void Convert(int y, Span<Rgb48> packed, Span<float> components)
{
int width = this.image.Width;
int sourceY = y << this.rowShift;
Span<float> yRow0 = components[..width];
Span<float> cbRow0 = components.Slice(width, width);
Span<float> crRow0 = components.Slice(width * 2, width);
this.ConvertSourceRow(sourceY, packed, yRow0, cbRow0, crRow0);
WriteSamples<TSample, TStorer>(
yRow0,
this.GetPlaneRow(Av1Plane.Y, sourceY),
this.colorConverter.LumaScale,
this.colorConverter.LumaBias,
this.sampleMaximum);
bool hasSecondSourceRow = this.rowShift != 0 && sourceY + 1 < this.image.Height;
Span<float> yRow1 = Span<float>.Empty;
Span<float> cbRow1 = Span<float>.Empty;
Span<float> crRow1 = Span<float>.Empty;
if (hasSecondSourceRow)
{
yRow1 = components.Slice(width * 3, width);
cbRow1 = components.Slice(width * 4, width);
crRow1 = components.Slice(width * 5, width);
this.ConvertSourceRow(sourceY + 1, packed, yRow1, cbRow1, crRow1);
WriteSamples<TSample, TStorer>(
yRow1,
this.GetPlaneRow(Av1Plane.Y, sourceY + 1),
this.colorConverter.LumaScale,
this.colorConverter.LumaBias,
this.sampleMaximum);
}
if (this.isMonochrome)
{
return;
}
Span<TSample> uDestination = this.GetPlaneRow(Av1Plane.U, y);
Span<TSample> vDestination = this.GetPlaneRow(Av1Plane.V, y);
float chromaScale = this.colorConverter.ChromaScale;
float chromaBias = this.colorConverter.ChromaBias;
if (this.subX == 0)
{
WriteSamples<TSample, TStorer>(cbRow0, uDestination, chromaScale, chromaBias, this.sampleMaximum);
WriteSamples<TSample, TStorer>(crRow0, vDestination, chromaScale, chromaBias, this.sampleMaximum);
}
else
{
float secondRowWeight = hasSecondSourceRow ? 0.5F : 0F;
WriteSubsampledSamples<TSample, TStorer>(
cbRow0,
cbRow1,
uDestination,
true,
secondRowWeight,
chromaScale,
chromaBias,
this.sampleMaximum);
WriteSubsampledSamples<TSample, TStorer>(
crRow0,
crRow1,
vDestination,
true,
secondRowWeight,
chromaScale,
chromaBias,
this.sampleMaximum);
}
}
/// <summary>
/// Converts one source row into normalized planar AV1 components.
/// </summary>
/// <param name="y">The source row index.</param>
/// <param name="packed">The high-bit-depth RGB staging row.</param>
/// <param name="luma">The destination luma values.</param>
/// <param name="chromaBlue">The destination blue-difference values.</param>
/// <param name="chromaRed">The destination red-difference values.</param>
private void ConvertSourceRow(int y, Span<Rgb48> packed, Span<float> luma, Span<float> chromaBlue, Span<float> chromaRed)
{
ReadOnlySpan<TPixel> source = this.image.PixelBuffer.DangerousGetRowSpan(y);
if (typeof(TSample) == typeof(byte))
{
// This is the same planar input contract used by JPEG encoding. Pixel types with optimized
// unpackers reach their existing SIMD path before the AV1 operator consumes the planes.
PixelOperations<TPixel>.Instance.UnpackIntoRgbPlanes(luma, chromaBlue, chromaRed, source);
this.colorConverter.ConvertFromRgbInPlace(luma, chromaBlue, chromaRed, ByteMaximum);
}
else
{
// JPEG's planar unpack contract is eight-bit. AV1 10/12-bit encoding stages Rgb48 instead
// so high-precision source pixels are not truncated before the color transform.
PixelOperations<TPixel>.Instance.ToRgb48(this.configuration, source, packed);
DeinterleaveRgb48(packed, luma, chromaBlue, chromaRed);
this.colorConverter.ConvertFromRgbInPlace(luma, chromaBlue, chromaRed, UShortMaximum);
}
}
/// <summary>
/// Gets a writable row from an eight-bit or high-bit-depth AV1 plane.
/// </summary>
/// <param name="plane">The requested plane.</param>
/// <param name="y">The row index in the requested plane.</param>
/// <returns>The writable sample row.</returns>
private Span<TSample> GetPlaneRow(Av1Plane plane, int y)
{
if (typeof(TSample) == typeof(byte))
{
Buffer2DRegion<byte> region = plane switch
{
Av1Plane.Y => this.yPlane,
Av1Plane.U => this.uPlane,
_ => this.vPlane,
};
return MemoryMarshal.Cast<byte, TSample>(region.DangerousGetRowSpan(y));
}
int subX = plane == Av1Plane.Y ? 0 : this.subX;
int subY = plane == Av1Plane.Y ? 0 : this.rowShift;
return MemoryMarshal.Cast<ushort, TSample>(this.frameBuffer.GetHighBitDepthRowSpan(plane, y, subX, subY));
}
}
}

174
src/ImageSharp/Formats/Heif/Av1/Av1YuvConverter.cs

@ -1,174 +0,0 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using System.Buffers;
using SixLabors.ImageSharp.Advanced;
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Formats.Heif.Color;
using SixLabors.ImageSharp.Memory;
using SixLabors.ImageSharp.Metadata.Profiles.Cicp;
using SixLabors.ImageSharp.PixelFormats;
using static SixLabors.ImageSharp.Formats.Heif.Color.HeifColorConverterBase;
namespace SixLabors.ImageSharp.Formats.Heif.Av1;
/// <summary>
/// Converts between reconstructed AV1 YUV planes and packed ImageSharp pixels.
/// </summary>
internal static partial class Av1YuvConverter
{
/// <summary>
/// The largest value represented by an eight-bit packed RGB component.
/// </summary>
private const float ByteMaximum = byte.MaxValue;
/// <summary>
/// The largest value represented by a 16-bit packed RGB component.
/// </summary>
private const float UShortMaximum = ushort.MaxValue;
/// <summary>
/// Converts the reconstructed YUV planes to packed pixels.
/// </summary>
/// <typeparam name="TPixel">The destination pixel type.</typeparam>
/// <param name="configuration">The configuration used for allocation and pixel conversion.</param>
/// <param name="frameBuffer">The reconstructed AV1 frame.</param>
/// <param name="image">The destination image frame.</param>
public static void ConvertToRgb<TPixel>(Configuration configuration, Av1FrameBuffer<byte> frameBuffer, ImageFrame<TPixel> image)
where TPixel : unmanaged, IPixel<TPixel>
{
HeifColorConversionParameters parameters = GetConversionParameters(frameBuffer, out HeifColorConversionMode mode);
HeifColorConverterBase colorConverter = HeifColorConverterBase.Create(mode, in parameters, frameBuffer.ColorFormat == Av1ColorFormat.Yuv400);
if (frameBuffer.BitDepth == Av1BitDepth.EightBit)
{
YuvToRgbRowConverter<TPixel, byte, HeifByteSampleLoader> converter = new(configuration, frameBuffer, image, colorConverter);
using IMemoryOwner<float> scratchOwner = configuration.MemoryAllocator.Allocate<float>(converter.BufferLength);
Span<float> scratch = scratchOwner.GetSpan();
for (int y = 0; y < image.Height; y++)
{
converter.Convert(y, scratch);
}
}
else
{
YuvToRgbRowConverter<TPixel, ushort, HeifUShortSampleLoader> converter = new(configuration, frameBuffer, image, colorConverter);
using IMemoryOwner<float> owner = configuration.MemoryAllocator.Allocate<float>(converter.BufferLength);
Span<float> scratch = owner.GetSpan();
for (int y = 0; y < image.Height; y++)
{
converter.Convert(y, scratch);
}
}
}
/// <summary>
/// Converts packed pixels to the configured monochrome or YUV planes used by the AV1 encoder.
/// </summary>
/// <typeparam name="TPixel">The source pixel type.</typeparam>
/// <param name="configuration">The configuration used for allocation and pixel conversion.</param>
/// <param name="image">The source image frame.</param>
/// <param name="frameBuffer">The destination AV1 frame.</param>
public static void ConvertFromRgb<TPixel>(Configuration configuration, ImageFrame<TPixel> image, Av1FrameBuffer<byte> frameBuffer)
where TPixel : unmanaged, IPixel<TPixel>
{
HeifColorConversionParameters parameters = GetConversionParameters(frameBuffer, out HeifColorConversionMode mode);
float sampleMaximum = parameters.LumaSampleMaximum;
bool isMonochrome = frameBuffer.ColorFormat == Av1ColorFormat.Yuv400;
HeifColorConverterBase colorConverter = HeifColorConverterBase.Create(mode, in parameters, isMonochrome);
int rowShift = !isMonochrome && frameBuffer.ColorConfig.SubSamplingY ? 1 : 0;
int iterationCount = (image.Height + rowShift) >> rowShift;
if (frameBuffer.BitDepth == Av1BitDepth.EightBit)
{
RgbToYuvRowConverter<TPixel, byte, HeifByteSampleStorer> converter = new(configuration, frameBuffer, image, colorConverter, sampleMaximum);
using IMemoryOwner<float> componentOwner = configuration.MemoryAllocator.Allocate<float>(converter.ComponentBufferLength);
Span<float> components = componentOwner.GetSpan();
for (int y = 0; y < iterationCount; y++)
{
converter.Convert(y, Span<Rgb48>.Empty, components);
}
}
else
{
RgbToYuvRowConverter<TPixel, ushort, HeifUShortSampleStorer> converter = new(configuration, frameBuffer, image, colorConverter, sampleMaximum);
using IMemoryOwner<Rgb48> packedOwner = configuration.MemoryAllocator.Allocate<Rgb48>(image.Width);
using IMemoryOwner<float> componentOwner = configuration.MemoryAllocator.Allocate<float>(converter.ComponentBufferLength);
Span<Rgb48> packed = packedOwner.GetSpan()[..image.Width];
Span<float> components = componentOwner.GetSpan();
for (int y = 0; y < iterationCount; y++)
{
converter.Convert(y, packed, components);
}
}
}
/// <summary>
/// Resolves the H.273 conversion mode, matrix coefficients, and sample range for a frame.
/// </summary>
/// <param name="frameBuffer">The AV1 frame containing the signaled color configuration.</param>
/// <param name="mode">The resolved conversion mode.</param>
/// <returns>The resolved conversion parameters.</returns>
private static HeifColorConversionParameters GetConversionParameters(Av1FrameBuffer<byte> frameBuffer, out HeifColorConversionMode mode)
{
if (frameBuffer.ColorConfig.ChromaSamplePosition == ObuChromoSamplePosition.Reserved)
{
throw new InvalidImageContentException("The reserved AV1 chroma sample position is invalid.");
}
bool isMonochrome = frameBuffer.ColorFormat == Av1ColorFormat.Yuv400;
return HeifColorConversionParameters.Create(
(CicpColorPrimaries)(byte)frameBuffer.ColorConfig.ColorPrimaries,
(CicpTransferCharacteristics)(byte)frameBuffer.ColorConfig.TransferCharacteristics,
(CicpMatrixCoefficients)(byte)frameBuffer.ColorConfig.MatrixCoefficients,
frameBuffer.ColorConfig.ColorRange,
frameBuffer.BitDepth.GetBitCount(),
frameBuffer.BitDepth.GetBitCount(),
isMonochrome,
frameBuffer.ColorFormat == Av1ColorFormat.Yuv444,
out mode);
}
/// <summary>
/// Resolves the two chroma samples and quarter-sample weight surrounding a luma coordinate.
/// </summary>
/// <param name="coordinate">The luma coordinate.</param>
/// <param name="subsampling">The chroma subsampling shift.</param>
/// <param name="isCentered">Whether chroma lies between neighboring luma samples.</param>
/// <param name="maximum">The last available chroma coordinate.</param>
/// <param name="lower">The lower chroma coordinate.</param>
/// <param name="upper">The upper chroma coordinate.</param>
/// <param name="upperWeight">The upper-coordinate weight with a denominator of four.</param>
private static void GetChromaCoordinates(
int coordinate,
int subsampling,
bool isCentered,
int maximum,
out int lower,
out int upper,
out int upperWeight)
{
if (subsampling == 0)
{
lower = coordinate;
upper = coordinate;
upperWeight = 0;
return;
}
int sample = coordinate >> 1;
bool isOdd = (coordinate & 1) != 0;
if (isCentered)
{
lower = isOdd ? sample : Math.Max(sample - 1, 0);
upper = isOdd ? Math.Min(sample + 1, maximum) : sample;
upperWeight = isOdd ? 1 : 3;
return;
}
lower = sample;
upper = isOdd ? Math.Min(sample + 1, maximum) : sample;
upperWeight = isOdd ? 2 : 0;
}
}

133
src/ImageSharp/Formats/Heif/Av1/Color/Av1PlanarSampleBuffer.cs

@ -0,0 +1,133 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using System.Runtime.InteropServices;
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Formats.Heif.Color;
using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Color;
/// <summary>
/// Adapts reconstructed AV1 planes to the shared HEIF planar color pipeline.
/// </summary>
/// <typeparam name="TSample">The native unsigned sample storage type.</typeparam>
internal struct Av1PlanarSampleBuffer<TSample> : IHeifPlanarSampleBuffer<TSample>
where TSample : unmanaged
{
/// <summary>
/// The reconstructed AV1 frame containing the component planes.
/// </summary>
private readonly Av1FrameBuffer<byte> frameBuffer;
/// <summary>
/// The visible luma plane in byte-backed storage.
/// </summary>
private readonly Buffer2DRegion<byte> luma;
/// <summary>
/// The visible blue-difference plane in byte-backed storage.
/// </summary>
private readonly Buffer2DRegion<byte> chromaBlue;
/// <summary>
/// The visible red-difference plane in byte-backed storage.
/// </summary>
private readonly Buffer2DRegion<byte> chromaRed;
/// <summary>
/// Initializes a new instance of the <see cref="Av1PlanarSampleBuffer{TSample}"/> struct.
/// </summary>
/// <param name="frameBuffer">The reconstructed AV1 frame.</param>
public Av1PlanarSampleBuffer(Av1FrameBuffer<byte> frameBuffer)
{
this.frameBuffer = frameBuffer;
this.luma = frameBuffer.DeriveBlockPointer(Av1Plane.Y, 0, 0);
this.chromaBlue = this.IsMonochrome
? default
: frameBuffer.DeriveBlockPointer(Av1Plane.U, this.ChromaSubsamplingX, this.ChromaSubsamplingY);
this.chromaRed = this.IsMonochrome
? default
: frameBuffer.DeriveBlockPointer(Av1Plane.V, this.ChromaSubsamplingX, this.ChromaSubsamplingY);
}
/// <inheritdoc/>
public readonly int Width => this.frameBuffer.Width;
/// <inheritdoc/>
public readonly int Height => this.frameBuffer.Height;
/// <inheritdoc/>
public readonly int LumaBitDepth => this.frameBuffer.BitDepth.GetBitCount();
/// <inheritdoc/>
public readonly int ChromaBitDepth => this.frameBuffer.BitDepth.GetBitCount();
/// <inheritdoc/>
public readonly bool IsMonochrome => this.frameBuffer.ColorFormat == Av1ColorFormat.Yuv400;
/// <inheritdoc/>
public readonly int ChromaSubsamplingX => this.frameBuffer.ColorConfig.SubSamplingX ? 1 : 0;
/// <inheritdoc/>
public readonly int ChromaSubsamplingY => this.frameBuffer.ColorConfig.SubSamplingY ? 1 : 0;
/// <inheritdoc/>
public readonly int ChromaPositionX
{
get
{
if (this.ChromaSubsamplingX == 0)
{
return 0;
}
// AV1 4:2:2 chroma is centered horizontally. For 4:2:0, CSP_UNKNOWN is centered while the two
// explicitly positioned layouts are co-sited with the left luma sample.
bool isCentered = this.ChromaSubsamplingY == 0
|| this.frameBuffer.ColorConfig.ChromaSamplePosition == ObuChromoSamplePosition.Unknown;
return isCentered ? 1 : 0;
}
}
/// <inheritdoc/>
public readonly int ChromaPositionY
=> this.ChromaSubsamplingY != 0 && this.frameBuffer.ColorConfig.ChromaSamplePosition != ObuChromoSamplePosition.Colocated ? 1 : 0;
/// <inheritdoc/>
public Span<TSample> GetLumaRowSpan(int row)
{
if (typeof(TSample) == typeof(byte))
{
return MemoryMarshal.Cast<byte, TSample>(this.luma.DangerousGetRowSpan(row));
}
return MemoryMarshal.Cast<ushort, TSample>(this.frameBuffer.GetHighBitDepthRowSpan(Av1Plane.Y, row, 0, 0));
}
/// <inheritdoc/>
public Span<TSample> GetChromaBlueRowSpan(int row)
{
if (typeof(TSample) == typeof(byte))
{
return MemoryMarshal.Cast<byte, TSample>(this.chromaBlue.DangerousGetRowSpan(row));
}
return MemoryMarshal.Cast<ushort, TSample>(
this.frameBuffer.GetHighBitDepthRowSpan(Av1Plane.U, row, this.ChromaSubsamplingX, this.ChromaSubsamplingY));
}
/// <inheritdoc/>
public Span<TSample> GetChromaRedRowSpan(int row)
{
if (typeof(TSample) == typeof(byte))
{
return MemoryMarshal.Cast<byte, TSample>(this.chromaRed.DangerousGetRowSpan(row));
}
return MemoryMarshal.Cast<ushort, TSample>(
this.frameBuffer.GetHighBitDepthRowSpan(Av1Plane.V, row, this.ChromaSubsamplingX, this.ChromaSubsamplingY));
}
}

109
src/ImageSharp/Formats/Heif/Av1/Color/Av1YuvConverter.cs

@ -0,0 +1,109 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Formats.Heif.Color;
using SixLabors.ImageSharp.Metadata.Profiles.Cicp;
using SixLabors.ImageSharp.PixelFormats;
using static SixLabors.ImageSharp.Formats.Heif.Color.HeifColorConverterBase;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Color;
/// <summary>
/// Adapts AV1 color signaling and reconstructed planes to the shared HEIF color pipeline.
/// </summary>
internal static class Av1YuvConverter
{
/// <summary>
/// Converts the reconstructed component planes to packed pixels.
/// </summary>
/// <typeparam name="TPixel">The destination pixel type.</typeparam>
/// <param name="configuration">The configuration used for allocation and pixel conversion.</param>
/// <param name="frameBuffer">The reconstructed AV1 frame.</param>
/// <param name="image">The destination image frame.</param>
public static void ConvertToRgb<TPixel>(Configuration configuration, Av1FrameBuffer<byte> frameBuffer, ImageFrame<TPixel> image)
where TPixel : unmanaged, IPixel<TPixel>
{
HeifColorConversionParameters parameters = GetConversionParameters(frameBuffer, out HeifColorConversionMode mode);
if (frameBuffer.BitDepth == Av1BitDepth.EightBit)
{
Av1PlanarSampleBuffer<byte> buffer = new(frameBuffer);
HeifPlanarColorConverter.ConvertToRgb<TPixel, Av1PlanarSampleBuffer<byte>, byte, HeifByteSampleLoader>(
configuration,
buffer,
image,
in parameters,
mode);
return;
}
Av1PlanarSampleBuffer<ushort> highBitDepthBuffer = new(frameBuffer);
HeifPlanarColorConverter.ConvertToRgb<TPixel, Av1PlanarSampleBuffer<ushort>>(
configuration,
highBitDepthBuffer,
image,
in parameters,
mode);
}
/// <summary>
/// Converts packed pixels to the configured monochrome or component planes used by the AV1 encoder.
/// </summary>
/// <typeparam name="TPixel">The source pixel type.</typeparam>
/// <param name="configuration">The configuration used for allocation and pixel conversion.</param>
/// <param name="image">The source image frame.</param>
/// <param name="frameBuffer">The destination AV1 frame.</param>
public static void ConvertFromRgb<TPixel>(Configuration configuration, ImageFrame<TPixel> image, Av1FrameBuffer<byte> frameBuffer)
where TPixel : unmanaged, IPixel<TPixel>
{
HeifColorConversionParameters parameters = GetConversionParameters(frameBuffer, out HeifColorConversionMode mode);
if (frameBuffer.BitDepth == Av1BitDepth.EightBit)
{
Av1PlanarSampleBuffer<byte> buffer = new(frameBuffer);
HeifPlanarColorConverter.ConvertFromRgb<TPixel, Av1PlanarSampleBuffer<byte>, byte, HeifByteSampleStorer>(
configuration,
image,
buffer,
in parameters,
mode);
return;
}
Av1PlanarSampleBuffer<ushort> highBitDepthBuffer = new(frameBuffer);
HeifPlanarColorConverter.ConvertFromRgb<TPixel, Av1PlanarSampleBuffer<ushort>, ushort, HeifUShortSampleStorer>(
configuration,
image,
highBitDepthBuffer,
in parameters,
mode);
}
/// <summary>
/// Resolves the H.273 conversion mode, matrix coefficients, and sample range for a frame.
/// </summary>
/// <param name="frameBuffer">The AV1 frame containing the signaled color configuration.</param>
/// <param name="mode">The resolved conversion mode.</param>
/// <returns>The resolved conversion parameters.</returns>
private static HeifColorConversionParameters GetConversionParameters(Av1FrameBuffer<byte> frameBuffer, out HeifColorConversionMode mode)
{
if (frameBuffer.ColorConfig.ChromaSamplePosition == ObuChromoSamplePosition.Reserved)
{
throw new InvalidImageContentException("The reserved AV1 chroma sample position is invalid.");
}
bool isMonochrome = frameBuffer.ColorFormat == Av1ColorFormat.Yuv400;
return HeifColorConversionParameters.Create(
(CicpColorPrimaries)(byte)frameBuffer.ColorConfig.ColorPrimaries,
(CicpTransferCharacteristics)(byte)frameBuffer.ColorConfig.TransferCharacteristics,
(CicpMatrixCoefficients)(byte)frameBuffer.ColorConfig.MatrixCoefficients,
frameBuffer.ColorConfig.ColorRange,
frameBuffer.BitDepth.GetBitCount(),
frameBuffer.BitDepth.GetBitCount(),
isMonochrome,
frameBuffer.ColorFormat == Av1ColorFormat.Yuv444,
out mode);
}
}

18
src/ImageSharp/Formats/Heif/Color/HeifColorConversionParameters.cs

@ -16,6 +16,8 @@ internal readonly struct HeifColorConversionParameters
/// <param name="kr">The red luma coefficient.</param> /// <param name="kr">The red luma coefficient.</param>
/// <param name="kg">The green luma coefficient.</param> /// <param name="kg">The green luma coefficient.</param>
/// <param name="kb">The blue luma coefficient.</param> /// <param name="kb">The blue luma coefficient.</param>
/// <param name="matrixCoefficients">The signaled H.273 matrix-coefficient code point.</param>
/// <param name="isFullRange">Whether encoded components use their complete numeric range.</param>
/// <param name="transferCharacteristics">The signaled transfer characteristics.</param> /// <param name="transferCharacteristics">The signaled transfer characteristics.</param>
/// <param name="constantLuminanceScales">The constant-luminance chroma scales.</param> /// <param name="constantLuminanceScales">The constant-luminance chroma scales.</param>
/// <param name="lumaBias">The encoded luma bias.</param> /// <param name="lumaBias">The encoded luma bias.</param>
@ -31,6 +33,8 @@ internal readonly struct HeifColorConversionParameters
float kr, float kr,
float kg, float kg,
float kb, float kb,
CicpMatrixCoefficients matrixCoefficients,
bool isFullRange,
CicpTransferCharacteristics transferCharacteristics, CicpTransferCharacteristics transferCharacteristics,
in HeifConstantLuminanceScales constantLuminanceScales, in HeifConstantLuminanceScales constantLuminanceScales,
float lumaBias, float lumaBias,
@ -50,6 +54,8 @@ internal readonly struct HeifColorConversionParameters
this.BlueChromaScale = 2F * (1F - kb); this.BlueChromaScale = 2F * (1F - kb);
this.GreenRedChromaScale = 2F * kr * (1F - kr) / kg; this.GreenRedChromaScale = 2F * kr * (1F - kr) / kg;
this.GreenBlueChromaScale = 2F * kb * (1F - kb) / kg; this.GreenBlueChromaScale = 2F * kb * (1F - kb) / kg;
this.MatrixCoefficients = matrixCoefficients;
this.IsFullRange = isFullRange;
this.TransferCharacteristics = transferCharacteristics; this.TransferCharacteristics = transferCharacteristics;
this.ConstantLuminanceScales = constantLuminanceScales; this.ConstantLuminanceScales = constantLuminanceScales;
this.LumaBias = lumaBias; this.LumaBias = lumaBias;
@ -98,6 +104,16 @@ internal readonly struct HeifColorConversionParameters
/// </summary> /// </summary>
public float GreenBlueChromaScale { get; } public float GreenBlueChromaScale { get; }
/// <summary>
/// Gets the signaled H.273 matrix-coefficient code point.
/// </summary>
public CicpMatrixCoefficients MatrixCoefficients { get; }
/// <summary>
/// Gets a value indicating whether encoded components use their complete numeric range.
/// </summary>
public bool IsFullRange { get; }
/// <summary> /// <summary>
/// Gets the signaled transfer characteristics. /// Gets the signaled transfer characteristics.
/// </summary> /// </summary>
@ -296,6 +312,8 @@ internal readonly struct HeifColorConversionParameters
kr, kr,
kg, kg,
kb, kb,
matrixCoefficients,
isFullRange,
transferCharacteristics, transferCharacteristics,
in constantLuminanceScales, in constantLuminanceScales,
lumaBias, lumaBias,

738
src/ImageSharp/Formats/Heif/Color/HeifPlanarColorConverter.cs

@ -0,0 +1,738 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using System.Buffers;
using System.Runtime.InteropServices;
using SixLabors.ImageSharp.Advanced;
using SixLabors.ImageSharp.Common.Helpers;
using SixLabors.ImageSharp.Memory;
using SixLabors.ImageSharp.PixelFormats;
using static SixLabors.ImageSharp.Formats.Heif.Color.HeifColorConverterBase;
namespace SixLabors.ImageSharp.Formats.Heif.Color;
/// <summary>
/// Converts between native HEIF component planes and packed ImageSharp pixels.
/// </summary>
internal static class HeifPlanarColorConverter
{
/// <summary>
/// The largest value represented by an eight-bit packed RGB component.
/// </summary>
private const float ByteMaximum = byte.MaxValue;
/// <summary>
/// The largest value represented by a 16-bit packed RGB component.
/// </summary>
private const float UShortMaximum = ushort.MaxValue;
/// <summary>
/// Converts native unsigned 16-bit component storage to packed pixels and selects eligible exact integer kernels.
/// </summary>
/// <typeparam name="TPixel">The destination pixel type.</typeparam>
/// <typeparam name="TBuffer">The codec adapter exposing the native component planes.</typeparam>
/// <param name="configuration">The configuration used for allocation and pixel conversion.</param>
/// <param name="buffer">The native component-plane buffer.</param>
/// <param name="image">The destination image frame.</param>
/// <param name="parameters">The resolved H.273 conversion parameters.</param>
/// <param name="mode">The resolved H.273 conversion mode.</param>
/// <param name="sourceX">The horizontal luma-sample offset of the first converted pixel.</param>
/// <param name="sourceY">The vertical luma-sample offset of the first converted pixel.</param>
public static void ConvertToRgb<TPixel, TBuffer>(
Configuration configuration,
TBuffer buffer,
ImageFrame<TPixel> image,
in HeifColorConversionParameters parameters,
HeifColorConversionMode mode,
int sourceX = 0,
int sourceY = 0)
where TPixel : unmanaged, IPixel<TPixel>
where TBuffer : struct, IHeifPlanarSampleBuffer<ushort>
{
if (HeifYuv420ToRgb8Converter.IsSupported(
buffer.ChromaSubsamplingX,
buffer.ChromaSubsamplingY,
buffer.LumaBitDepth,
buffer.ChromaBitDepth,
parameters.IsFullRange,
parameters.MatrixCoefficients,
mode))
{
// The fixed-point operator preserves the exact code-value rounding used by the verified eight-bit
// presentation path. Selection belongs here so no codec can acquire a private color-conversion route.
HeifYuv420ToRgb8Converter.Convert(configuration, buffer, image, in parameters, sourceX, sourceY);
return;
}
ConvertToRgb<TPixel, TBuffer, ushort, HeifUShortSampleLoader>(
configuration,
buffer,
image,
in parameters,
mode,
sourceX,
sourceY);
}
/// <summary>
/// Converts native component planes to packed pixels.
/// </summary>
/// <typeparam name="TPixel">The destination pixel type.</typeparam>
/// <typeparam name="TBuffer">The codec adapter exposing the native component planes.</typeparam>
/// <typeparam name="TSample">The native unsigned sample storage type.</typeparam>
/// <typeparam name="TLoader">The SIMD widening operations for the sample type.</typeparam>
/// <param name="configuration">The configuration used for allocation and pixel conversion.</param>
/// <param name="buffer">The native component-plane buffer.</param>
/// <param name="image">The destination image frame.</param>
/// <param name="parameters">The resolved H.273 conversion parameters.</param>
/// <param name="mode">The resolved H.273 conversion mode.</param>
/// <param name="sourceX">The horizontal luma-sample offset of the first converted pixel.</param>
/// <param name="sourceY">The vertical luma-sample offset of the first converted pixel.</param>
public static void ConvertToRgb<TPixel, TBuffer, TSample, TLoader>(
Configuration configuration,
TBuffer buffer,
ImageFrame<TPixel> image,
in HeifColorConversionParameters parameters,
HeifColorConversionMode mode,
int sourceX = 0,
int sourceY = 0)
where TPixel : unmanaged, IPixel<TPixel>
where TBuffer : struct, IHeifPlanarSampleBuffer<TSample>
where TSample : unmanaged
where TLoader : struct, IHeifSampleLoader<TSample>
{
HeifColorConverterBase colorConverter = HeifColorConverterBase.Create(mode, in parameters, buffer.IsMonochrome);
YuvToRgbRowConverter<TPixel, TBuffer, TSample, TLoader> converter = new(
configuration,
buffer,
image,
colorConverter,
sourceX,
sourceY);
using IMemoryOwner<float> scratchOwner = configuration.MemoryAllocator.Allocate<float>(converter.BufferLength);
Span<float> scratch = scratchOwner.GetSpan();
for (int y = 0; y < image.Height; y++)
{
converter.Convert(y, scratch);
}
}
/// <summary>
/// Converts packed pixels to native component planes.
/// </summary>
/// <typeparam name="TPixel">The source pixel type.</typeparam>
/// <typeparam name="TBuffer">The codec adapter exposing the native component planes.</typeparam>
/// <typeparam name="TSample">The native unsigned sample storage type.</typeparam>
/// <typeparam name="TStorer">The SIMD narrowing and storage operations for the sample type.</typeparam>
/// <param name="configuration">The configuration used for allocation and pixel conversion.</param>
/// <param name="image">The source image frame.</param>
/// <param name="buffer">The destination component-plane buffer.</param>
/// <param name="parameters">The resolved H.273 conversion parameters.</param>
/// <param name="mode">The resolved H.273 conversion mode.</param>
public static void ConvertFromRgb<TPixel, TBuffer, TSample, TStorer>(
Configuration configuration,
ImageFrame<TPixel> image,
TBuffer buffer,
in HeifColorConversionParameters parameters,
HeifColorConversionMode mode)
where TPixel : unmanaged, IPixel<TPixel>
where TBuffer : struct, IHeifPlanarSampleBuffer<TSample>
where TSample : unmanaged
where TStorer : struct, IHeifSampleStorer<TSample>
{
HeifColorConverterBase colorConverter = HeifColorConverterBase.Create(mode, in parameters, buffer.IsMonochrome);
RgbToYuvRowConverter<TPixel, TBuffer, TSample, TStorer> converter = new(
configuration,
buffer,
image,
colorConverter,
in parameters);
using IMemoryOwner<float> componentOwner = configuration.MemoryAllocator.Allocate<float>(converter.ComponentBufferLength);
Span<float> components = componentOwner.GetSpan();
if (converter.UsesByteInput)
{
converter.Convert(Span<Rgb48>.Empty, components);
return;
}
using IMemoryOwner<Rgb48> packedOwner = configuration.MemoryAllocator.Allocate<Rgb48>(image.Width);
converter.Convert(packedOwner.GetSpan()[..image.Width], components);
}
/// <summary>
/// Resolves the two chroma rows and quarter-sample weight surrounding a luma row.
/// </summary>
/// <param name="coordinate">The luma row coordinate.</param>
/// <param name="subsampling">The vertical chroma subsampling shift.</param>
/// <param name="position">The chroma offset in half-luma-sample units.</param>
/// <param name="maximum">The last available chroma row.</param>
/// <param name="lower">The lower chroma row.</param>
/// <param name="upper">The upper chroma row.</param>
/// <param name="upperWeight">The upper-row weight with a denominator of four.</param>
private static void GetChromaCoordinates(
int coordinate,
int subsampling,
int position,
int maximum,
out int lower,
out int upper,
out int upperWeight)
{
if (subsampling == 0)
{
lower = coordinate;
upper = coordinate;
upperWeight = 0;
return;
}
// Multiplication by two expresses the luma coordinate in half-sample units, while division by four
// addresses the subsampled plane. Floor division is required before the first centered chroma sample.
int quarterCoordinate = (coordinate * 2) - position;
int unclampedLower = quarterCoordinate >= 0 ? quarterCoordinate >> 2 : -((-quarterCoordinate + 3) >> 2);
int fraction = quarterCoordinate - (unclampedLower * 4);
lower = Numerics.Clamp(unclampedLower, 0, maximum);
upper = Numerics.Clamp(unclampedLower + 1, 0, maximum);
upperWeight = lower == upper ? 0 : fraction;
}
/// <summary>
/// Converts one native component row using pooled planar and packed-pixel storage.
/// </summary>
/// <typeparam name="TPixel">The destination pixel type.</typeparam>
/// <typeparam name="TBuffer">The codec adapter exposing the native component planes.</typeparam>
/// <typeparam name="TSample">The native unsigned sample storage type.</typeparam>
/// <typeparam name="TLoader">The SIMD widening operations for the sample type.</typeparam>
private struct YuvToRgbRowConverter<TPixel, TBuffer, TSample, TLoader>
where TPixel : unmanaged, IPixel<TPixel>
where TBuffer : struct, IHeifPlanarSampleBuffer<TSample>
where TSample : unmanaged
where TLoader : struct, IHeifSampleLoader<TSample>
{
/// <summary>
/// The configuration used for packed-pixel conversion.
/// </summary>
private readonly Configuration configuration;
/// <summary>
/// The codec adapter exposing the reconstructed component planes.
/// </summary>
private TBuffer buffer;
/// <summary>
/// The destination image frame.
/// </summary>
private readonly ImageFrame<TPixel> image;
/// <summary>
/// The selected H.273 color converter.
/// </summary>
private readonly HeifColorConverterBase colorConverter;
/// <summary>
/// The horizontal luma-sample offset of the output window.
/// </summary>
private readonly int sourceX;
/// <summary>
/// The vertical luma-sample offset of the output window.
/// </summary>
private readonly int sourceY;
/// <summary>
/// The full coded luma-plane width.
/// </summary>
private readonly int bufferWidth;
/// <summary>
/// The chroma-plane width in samples.
/// </summary>
private readonly int chromaWidth;
/// <summary>
/// The chroma-plane height in samples.
/// </summary>
private readonly int chromaHeight;
/// <summary>
/// The horizontal chroma subsampling shift.
/// </summary>
private readonly int subsamplingX;
/// <summary>
/// The vertical chroma subsampling shift.
/// </summary>
private readonly int subsamplingY;
/// <summary>
/// The horizontal chroma position in half-luma-sample units.
/// </summary>
private readonly int chromaPositionX;
/// <summary>
/// The vertical chroma position in half-luma-sample units.
/// </summary>
private readonly int chromaPositionY;
/// <summary>
/// Whether the buffer contains only the luma plane.
/// </summary>
private readonly bool isMonochrome;
/// <summary>
/// Whether the destination uses JPEG-compatible eight-bit RGB plane packing.
/// </summary>
private readonly bool usesBytePacking;
/// <summary>
/// Whether chroma reconstruction must retain the complete coded row before selecting the output window.
/// </summary>
private readonly bool reconstructCompleteRow;
/// <summary>
/// Initializes a new instance of the <see cref="YuvToRgbRowConverter{TPixel, TBuffer, TSample, TLoader}"/> struct.
/// </summary>
/// <param name="configuration">The configuration used for pixel conversion.</param>
/// <param name="buffer">The codec adapter exposing the reconstructed component planes.</param>
/// <param name="image">The destination image frame.</param>
/// <param name="colorConverter">The selected H.273 color converter.</param>
/// <param name="sourceX">The horizontal luma-sample offset of the output window.</param>
/// <param name="sourceY">The vertical luma-sample offset of the output window.</param>
public YuvToRgbRowConverter(
Configuration configuration,
TBuffer buffer,
ImageFrame<TPixel> image,
HeifColorConverterBase colorConverter,
int sourceX,
int sourceY)
{
this.configuration = configuration;
this.buffer = buffer;
this.image = image;
this.colorConverter = colorConverter;
this.sourceX = sourceX;
this.sourceY = sourceY;
this.bufferWidth = buffer.Width;
this.subsamplingX = buffer.ChromaSubsamplingX;
this.subsamplingY = buffer.ChromaSubsamplingY;
this.chromaPositionX = buffer.ChromaPositionX;
this.chromaPositionY = buffer.ChromaPositionY;
this.isMonochrome = buffer.IsMonochrome;
this.usesBytePacking = buffer.LumaBitDepth == 8 && (buffer.IsMonochrome || buffer.ChromaBitDepth == 8);
this.chromaWidth = (buffer.Width + (1 << this.subsamplingX) - 1) >> this.subsamplingX;
this.chromaHeight = (buffer.Height + (1 << this.subsamplingY) - 1) >> this.subsamplingY;
this.reconstructCompleteRow = sourceX != 0 || image.Width != buffer.Width;
}
/// <summary>
/// Gets a value indicating whether the destination uses JPEG-compatible eight-bit RGB plane packing.
/// </summary>
public readonly bool UsesBytePacking => this.usesBytePacking;
/// <summary>
/// Gets the number of float elements required by the reusable row buffer.
/// </summary>
public readonly int BufferLength
{
get
{
int componentLength = this.image.Width * 3;
if (!this.isMonochrome && this.subsamplingX != 0)
{
// Full-image conversion reconstructs directly into the component rows. Cropped conversion
// retains the complete coded row so interpolation phase is preserved at the window boundary.
componentLength += (this.chromaWidth * 2) + (this.reconstructCompleteRow ? this.bufferWidth : 0);
}
int packedRowCount = this.UsesBytePacking ? 1 : 2;
return componentLength + (this.image.Width * packedRowCount);
}
}
/// <summary>
/// Converts one reconstructed row to packed pixels.
/// </summary>
/// <param name="y">The zero-based output row.</param>
/// <param name="scratch">The reusable pooled row buffer.</param>
public void Convert(int y, Span<float> scratch)
{
int width = this.image.Width;
Span<float> red = scratch[..width];
Span<float> green = scratch.Slice(width, width);
Span<float> blue = scratch.Slice(width * 2, width);
int sourceY = y + this.sourceY;
ReadOnlySpan<TSample> luma = this.buffer.GetLumaRowSpan(sourceY).Slice(this.sourceX, width);
ConvertSamplesToFloat<TSample, TLoader>(luma, red);
int packedOffset = width * 3;
if (!this.isMonochrome)
{
GetChromaCoordinates(
sourceY,
this.subsamplingY,
this.chromaPositionY,
this.chromaHeight - 1,
out int y0,
out int y1,
out int y1Weight);
ReadOnlySpan<TSample> cb0 = this.buffer.GetChromaBlueRowSpan(y0);
ReadOnlySpan<TSample> cb1 = this.buffer.GetChromaBlueRowSpan(y1);
ReadOnlySpan<TSample> cr0 = this.buffer.GetChromaRedRowSpan(y0);
ReadOnlySpan<TSample> cr1 = this.buffer.GetChromaRedRowSpan(y1);
if (this.subsamplingX == 0)
{
ConvertSamplesToFloat<TSample, TLoader>(cb0.Slice(this.sourceX, width), green);
ConvertSamplesToFloat<TSample, TLoader>(cr0.Slice(this.sourceX, width), blue);
}
else
{
Span<float> chroma0 = scratch.Slice(packedOffset, this.chromaWidth);
Span<float> chroma1 = scratch.Slice(packedOffset + this.chromaWidth, this.chromaWidth);
Span<float> reconstructed = this.reconstructCompleteRow
? scratch.Slice(packedOffset + (this.chromaWidth * 2), this.bufferWidth)
: green;
bool isCenteredX = this.chromaPositionX == 1;
ReconstructChromaRow<TSample, TLoader>(
cb0,
cb1,
y1Weight,
this.subsamplingX,
isCenteredX,
reconstructed,
chroma0,
chroma1);
if (this.reconstructCompleteRow)
{
reconstructed.Slice(this.sourceX, width).CopyTo(green);
}
reconstructed = this.reconstructCompleteRow ? reconstructed : blue;
ReconstructChromaRow<TSample, TLoader>(
cr0,
cr1,
y1Weight,
this.subsamplingX,
isCenteredX,
reconstructed,
chroma0,
chroma1);
if (this.reconstructCompleteRow)
{
reconstructed.Slice(this.sourceX, width).CopyTo(blue);
}
packedOffset += (this.chromaWidth * 2) + (this.reconstructCompleteRow ? this.bufferWidth : 0);
}
}
this.colorConverter.ConvertToRgbInPlace(red, green, blue);
Span<TPixel> destination = this.image.PixelBuffer.DangerousGetRowSpan(y);
Span<float> packedStorage = scratch[packedOffset..];
if (this.UsesBytePacking)
{
// This is JPEG's planar packing contract. Existing pixel-specific SIMD packers therefore own the
// final RGB-to-TPixel conversion rather than a HEIF-codec-specific per-pixel implementation.
Span<byte> byteStorage = MemoryMarshal.AsBytes(packedStorage)[..(width * 3)];
Span<byte> redBytes = byteStorage[..width];
Span<byte> greenBytes = byteStorage.Slice(width, width);
Span<byte> blueBytes = byteStorage.Slice(width * 2, width);
SimdUtils.NormalizedFloatToByteSaturate(red, redBytes);
SimdUtils.NormalizedFloatToByteSaturate(green, greenBytes);
SimdUtils.NormalizedFloatToByteSaturate(blue, blueBytes);
PixelOperations<TPixel>.Instance.PackFromRgbPlanes(redBytes, greenBytes, blueBytes, destination);
return;
}
Span<Rgba64> packed = MemoryMarshal.Cast<float, Rgba64>(packedStorage)[..width];
PackRgba64(red, green, blue, packed);
PixelOperations<TPixel>.Instance.FromRgba64(this.configuration, packed, destination);
}
}
/// <summary>
/// Converts packed image rows to native component planes using pooled planar storage.
/// </summary>
/// <typeparam name="TPixel">The source pixel type.</typeparam>
/// <typeparam name="TBuffer">The codec adapter exposing the native component planes.</typeparam>
/// <typeparam name="TSample">The native unsigned sample storage type.</typeparam>
/// <typeparam name="TStorer">The SIMD narrowing and storage operations for the sample type.</typeparam>
private struct RgbToYuvRowConverter<TPixel, TBuffer, TSample, TStorer>
where TPixel : unmanaged, IPixel<TPixel>
where TBuffer : struct, IHeifPlanarSampleBuffer<TSample>
where TSample : unmanaged
where TStorer : struct, IHeifSampleStorer<TSample>
{
/// <summary>
/// The configuration used for packed-pixel conversion.
/// </summary>
private readonly Configuration configuration;
/// <summary>
/// The codec adapter exposing the destination component planes.
/// </summary>
private TBuffer buffer;
/// <summary>
/// The source image frame.
/// </summary>
private readonly ImageFrame<TPixel> image;
/// <summary>
/// The selected H.273 color converter.
/// </summary>
private readonly HeifColorConverterBase colorConverter;
/// <summary>
/// The largest encoded luma sample value.
/// </summary>
private readonly float lumaMaximum;
/// <summary>
/// The largest encoded chroma sample value.
/// </summary>
private readonly float chromaMaximum;
/// <summary>
/// The horizontal chroma subsampling shift.
/// </summary>
private readonly int subsamplingX;
/// <summary>
/// The vertical chroma subsampling shift.
/// </summary>
private readonly int subsamplingY;
/// <summary>
/// The horizontal chroma position in half-luma-sample units.
/// </summary>
private readonly int chromaPositionX;
/// <summary>
/// The vertical chroma position in half-luma-sample units.
/// </summary>
private readonly int chromaPositionY;
/// <summary>
/// Whether the destination buffer contains only the luma plane.
/// </summary>
private readonly bool isMonochrome;
/// <summary>
/// Whether source pixels use JPEG-compatible eight-bit RGB plane unpacking.
/// </summary>
private readonly bool usesByteInput;
/// <summary>
/// Initializes a new instance of the <see cref="RgbToYuvRowConverter{TPixel, TBuffer, TSample, TStorer}"/> struct.
/// </summary>
/// <param name="configuration">The configuration used for pixel conversion.</param>
/// <param name="buffer">The codec adapter exposing the destination component planes.</param>
/// <param name="image">The source image frame.</param>
/// <param name="colorConverter">The selected H.273 color converter.</param>
/// <param name="parameters">The resolved H.273 component ranges.</param>
public RgbToYuvRowConverter(
Configuration configuration,
TBuffer buffer,
ImageFrame<TPixel> image,
HeifColorConverterBase colorConverter,
in HeifColorConversionParameters parameters)
{
this.configuration = configuration;
this.buffer = buffer;
this.image = image;
this.colorConverter = colorConverter;
this.lumaMaximum = parameters.LumaSampleMaximum;
this.chromaMaximum = parameters.ChromaSampleMaximum;
this.subsamplingX = buffer.ChromaSubsamplingX;
this.subsamplingY = buffer.ChromaSubsamplingY;
this.chromaPositionX = buffer.ChromaPositionX;
this.chromaPositionY = buffer.ChromaPositionY;
this.isMonochrome = buffer.IsMonochrome;
this.usesByteInput = buffer.LumaBitDepth == 8 && (buffer.IsMonochrome || buffer.ChromaBitDepth == 8);
}
/// <summary>
/// Gets a value indicating whether source pixels use JPEG-compatible eight-bit RGB plane unpacking.
/// </summary>
public readonly bool UsesByteInput => this.usesByteInput;
/// <summary>
/// Gets the number of float elements required by the reusable component buffer.
/// </summary>
public readonly int ComponentBufferLength => this.image.Width * (this.subsamplingY == 0 || this.isMonochrome ? 3 : 6);
/// <summary>
/// Converts every packed source row to the destination component planes.
/// </summary>
/// <param name="packed">The reusable high-bit-depth RGB staging row.</param>
/// <param name="components">The reusable planar component buffer.</param>
public void Convert(Span<Rgb48> packed, Span<float> components)
{
int width = this.image.Width;
Span<float> luma0 = components[..width];
Span<float> blue0 = components.Slice(width, width);
Span<float> red0 = components.Slice(width * 2, width);
if (this.subsamplingY == 0)
{
for (int y = 0; y < this.image.Height; y++)
{
this.ConvertSourceRow(y, packed, luma0, blue0, red0);
WriteSamples<TSample, TStorer>(
luma0,
this.buffer.GetLumaRowSpan(y),
this.colorConverter.LumaScale,
this.colorConverter.LumaBias,
this.lumaMaximum);
if (!this.isMonochrome)
{
this.WriteChromaRows(y, blue0, red0, Span<float>.Empty, Span<float>.Empty, 0F);
}
}
return;
}
Span<float> luma1 = this.isMonochrome ? luma0 : components.Slice(width * 3, width);
Span<float> blue1 = this.isMonochrome ? blue0 : components.Slice(width * 4, width);
Span<float> red1 = this.isMonochrome ? red0 : components.Slice(width * 5, width);
int chromaHeight = (this.image.Height + 1) >> 1;
for (int destinationY = 0; destinationY < chromaHeight; destinationY++)
{
// A vertically subsampled chroma row is owned by one two-row luma cell. Processing that cell as a
// unit removes per-row state and lets the selected chroma position choose or average the two rows.
int sourceY = destinationY << 1;
this.ConvertSourceRow(sourceY, packed, luma0, blue0, red0);
WriteSamples<TSample, TStorer>(
luma0,
this.buffer.GetLumaRowSpan(sourceY),
this.colorConverter.LumaScale,
this.colorConverter.LumaBias,
this.lumaMaximum);
bool hasSecondRow = sourceY + 1 < this.image.Height;
if (hasSecondRow)
{
this.ConvertSourceRow(sourceY + 1, packed, luma1, blue1, red1);
WriteSamples<TSample, TStorer>(
luma1,
this.buffer.GetLumaRowSpan(sourceY + 1),
this.colorConverter.LumaScale,
this.colorConverter.LumaBias,
this.lumaMaximum);
}
if (this.isMonochrome)
{
continue;
}
if (this.chromaPositionY == 0 || !hasSecondRow)
{
this.WriteChromaRows(destinationY, blue0, red0, Span<float>.Empty, Span<float>.Empty, 0F);
}
else if (this.chromaPositionY == 1)
{
this.WriteChromaRows(destinationY, blue0, red0, blue1, red1, 0.5F);
}
else
{
this.WriteChromaRows(destinationY, blue1, red1, Span<float>.Empty, Span<float>.Empty, 0F);
}
}
}
/// <summary>
/// Converts one packed source row into normalized planar components.
/// </summary>
/// <param name="y">The source row index.</param>
/// <param name="packed">The high-bit-depth RGB staging row.</param>
/// <param name="luma">The destination luma or first component values.</param>
/// <param name="chromaBlue">The destination blue-difference or second component values.</param>
/// <param name="chromaRed">The destination red-difference or third component values.</param>
private void ConvertSourceRow(int y, Span<Rgb48> packed, Span<float> luma, Span<float> chromaBlue, Span<float> chromaRed)
{
ReadOnlySpan<TPixel> source = this.image.PixelBuffer.DangerousGetRowSpan(y);
if (this.UsesByteInput)
{
// JPEG's planar unpack contract reaches the existing pixel-specific SIMD implementation before
// the closed H.273 operator transforms the component rows in place.
PixelOperations<TPixel>.Instance.UnpackIntoRgbPlanes(luma, chromaBlue, chromaRed, source);
this.colorConverter.ConvertFromRgbInPlace(luma, chromaBlue, chromaRed, ByteMaximum);
return;
}
PixelOperations<TPixel>.Instance.ToRgb48(this.configuration, source, packed);
DeinterleaveRgb48(packed, luma, chromaBlue, chromaRed);
this.colorConverter.ConvertFromRgbInPlace(luma, chromaBlue, chromaRed, UShortMaximum);
}
/// <summary>
/// Filters and writes one pair of chroma component rows.
/// </summary>
/// <param name="destinationY">The destination chroma row.</param>
/// <param name="blue0">The first blue-difference source row.</param>
/// <param name="red0">The first red-difference source row.</param>
/// <param name="blue1">The optional second blue-difference source row.</param>
/// <param name="red1">The optional second red-difference source row.</param>
/// <param name="row1Weight">The second-row contribution.</param>
private void WriteChromaRows(
int destinationY,
ReadOnlySpan<float> blue0,
ReadOnlySpan<float> red0,
ReadOnlySpan<float> blue1,
ReadOnlySpan<float> red1,
float row1Weight)
{
Span<TSample> blueDestination = this.buffer.GetChromaBlueRowSpan(destinationY);
Span<TSample> redDestination = this.buffer.GetChromaRedRowSpan(destinationY);
if (this.subsamplingX == 0)
{
WriteSamples<TSample, TStorer>(
blue0,
blueDestination,
this.colorConverter.ChromaScale,
this.colorConverter.ChromaBias,
this.chromaMaximum);
WriteSamples<TSample, TStorer>(
red0,
redDestination,
this.colorConverter.ChromaScale,
this.colorConverter.ChromaBias,
this.chromaMaximum);
return;
}
bool isCenteredX = this.chromaPositionX == 1;
WriteSubsampledSamples<TSample, TStorer>(
blue0,
blue1,
blueDestination,
isCenteredX,
row1Weight,
this.colorConverter.ChromaScale,
this.colorConverter.ChromaBias,
this.chromaMaximum);
WriteSubsampledSamples<TSample, TStorer>(
red0,
red1,
redDestination,
isCenteredX,
row1Weight,
this.colorConverter.ChromaScale,
this.colorConverter.ChromaBias,
this.chromaMaximum);
}
}
}

18
src/ImageSharp/Formats/Heif/Color/HeifYuv420ToRgb8Converter.cs

@ -60,22 +60,22 @@ internal static partial class HeifYuv420ToRgb8Converter
/// Converts supported HEIF component planes to packed pixels using integer SIMD with a scalar tail. /// Converts supported HEIF component planes to packed pixels using integer SIMD with a scalar tail.
/// </summary> /// </summary>
/// <typeparam name="TPixel">The destination pixel type.</typeparam> /// <typeparam name="TPixel">The destination pixel type.</typeparam>
/// <typeparam name="TSource">The codec adapter that exposes reconstructed component rows.</typeparam> /// <typeparam name="TBuffer">The codec adapter that exposes reconstructed component rows.</typeparam>
/// <param name="configuration">The configuration used for allocation and pixel conversion.</param> /// <param name="configuration">The configuration used for allocation and pixel conversion.</param>
/// <param name="source">The reconstructed component-plane source.</param> /// <param name="buffer">The reconstructed component-plane buffer.</param>
/// <param name="image">The destination image frame.</param> /// <param name="image">The destination image frame.</param>
/// <param name="parameters">The resolved H.273 conversion parameters.</param> /// <param name="parameters">The resolved H.273 conversion parameters.</param>
/// <param name="sourceX">The horizontal luma-sample offset of the output window.</param> /// <param name="sourceX">The horizontal luma-sample offset of the output window.</param>
/// <param name="sourceY">The vertical luma-sample offset of the output window.</param> /// <param name="sourceY">The vertical luma-sample offset of the output window.</param>
public static void Convert<TPixel, TSource>( public static void Convert<TPixel, TBuffer>(
Configuration configuration, Configuration configuration,
TSource source, TBuffer buffer,
ImageFrame<TPixel> image, ImageFrame<TPixel> image,
in HeifColorConversionParameters parameters, in HeifColorConversionParameters parameters,
int sourceX, int sourceX,
int sourceY) int sourceY)
where TPixel : unmanaged, IPixel<TPixel> where TPixel : unmanaged, IPixel<TPixel>
where TSource : struct, IHeifYuv420ToRgb8Source where TBuffer : struct, IHeifPlanarSampleBuffer<ushort>
{ {
ConversionParameters conversionParameters = new(in parameters); ConversionParameters conversionParameters = new(in parameters);
using IMemoryOwner<byte> componentOwner = configuration.MemoryAllocator.Allocate<byte>(image.Width * 3); using IMemoryOwner<byte> componentOwner = configuration.MemoryAllocator.Allocate<byte>(image.Width * 3);
@ -84,7 +84,7 @@ internal static partial class HeifYuv420ToRgb8Converter
Span<byte> green = components.Slice(image.Width, image.Width); Span<byte> green = components.Slice(image.Width, image.Width);
Span<byte> blue = components.Slice(image.Width * 2, image.Width); Span<byte> blue = components.Slice(image.Width * 2, image.Width);
// The value-type source closes the row-access contract at the call site. Constrained calls are therefore // The value-type buffer closes the row-access contract at the call site. Constrained calls are therefore
// devirtualized without boxing while keeping codec-specific buffer ownership outside the color pipeline. // devirtualized without boxing while keeping codec-specific buffer ownership outside the color pipeline.
for (int y = 0; y < image.Height; y++) for (int y = 0; y < image.Height; y++)
{ {
@ -92,9 +92,9 @@ internal static partial class HeifYuv420ToRgb8Converter
// The codec boundary validates 4:2:0 crop offsets in complete chroma-sample units. Each native chroma // The codec boundary validates 4:2:0 crop offsets in complete chroma-sample units. Each native chroma
// sample therefore covers one 2x2 luma cell without an alignment branch in the SIMD loop. // sample therefore covers one 2x2 luma cell without an alignment branch in the SIMD loop.
ReadOnlySpan<ushort> luma = source.GetLumaRow(lumaY).Slice(sourceX, image.Width); ReadOnlySpan<ushort> luma = buffer.GetLumaRowSpan(lumaY).Slice(sourceX, image.Width);
ReadOnlySpan<ushort> chromaBlue = source.GetChromaBlueRow(lumaY >> 1).Slice(sourceX >> 1); ReadOnlySpan<ushort> chromaBlue = buffer.GetChromaBlueRowSpan(lumaY >> 1).Slice(sourceX >> 1);
ReadOnlySpan<ushort> chromaRed = source.GetChromaRedRow(lumaY >> 1).Slice(sourceX >> 1); ReadOnlySpan<ushort> chromaRed = buffer.GetChromaRedRowSpan(lumaY >> 1).Slice(sourceX >> 1);
ConvertRow<FixedPointCoefficientOperator>(luma, chromaBlue, chromaRed, red, green, blue, in conversionParameters); ConvertRow<FixedPointCoefficientOperator>(luma, chromaBlue, chromaRed, red, green, blue, in conversionParameters);

78
src/ImageSharp/Formats/Heif/Color/IHeifPlanarSampleBuffer.cs

@ -0,0 +1,78 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
namespace SixLabors.ImageSharp.Formats.Heif.Color;
/// <summary>
/// Exposes the native planar sample layout shared by HEIF image codecs.
/// </summary>
/// <typeparam name="TSample">The native unsigned sample storage type.</typeparam>
internal interface IHeifPlanarSampleBuffer<TSample>
where TSample : unmanaged
{
/// <summary>
/// Gets the full luma-plane width in samples.
/// </summary>
public int Width { get; }
/// <summary>
/// Gets the full luma-plane height in samples.
/// </summary>
public int Height { get; }
/// <summary>
/// Gets the luma sample precision in bits.
/// </summary>
public int LumaBitDepth { get; }
/// <summary>
/// Gets the chroma sample precision in bits.
/// </summary>
public int ChromaBitDepth { get; }
/// <summary>
/// Gets a value indicating whether the buffer contains only the luma plane.
/// </summary>
public bool IsMonochrome { get; }
/// <summary>
/// Gets the horizontal chroma subsampling shift.
/// </summary>
public int ChromaSubsamplingX { get; }
/// <summary>
/// Gets the vertical chroma subsampling shift.
/// </summary>
public int ChromaSubsamplingY { get; }
/// <summary>
/// Gets the horizontal chroma position in half-luma-sample units.
/// </summary>
public int ChromaPositionX { get; }
/// <summary>
/// Gets the vertical chroma position in half-luma-sample units.
/// </summary>
public int ChromaPositionY { get; }
/// <summary>
/// Gets one complete writable row from the luma, green, or intensity plane.
/// </summary>
/// <param name="row">The zero-based row index in plane samples.</param>
/// <returns>The native luma, green, or intensity samples.</returns>
public Span<TSample> GetLumaRowSpan(int row);
/// <summary>
/// Gets one complete writable row from the blue-difference, blue, or first opponent-color plane.
/// </summary>
/// <param name="row">The zero-based row index in plane samples.</param>
/// <returns>The native blue-difference, blue, or first opponent-color samples.</returns>
public Span<TSample> GetChromaBlueRowSpan(int row);
/// <summary>
/// Gets one complete writable row from the red-difference, red, or second opponent-color plane.
/// </summary>
/// <param name="row">The zero-based row index in plane samples.</param>
/// <returns>The native red-difference, red, or second opponent-color samples.</returns>
public Span<TSample> GetChromaRedRowSpan(int row);
}

31
src/ImageSharp/Formats/Heif/Color/IHeifYuv420ToRgb8Source.cs

@ -1,31 +0,0 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
namespace SixLabors.ImageSharp.Formats.Heif.Color;
/// <summary>
/// Exposes reconstructed eight-bit 4:2:0 component rows to the shared fixed-point HEIF converter.
/// </summary>
internal interface IHeifYuv420ToRgb8Source
{
/// <summary>
/// Gets one complete luma row in native sample storage.
/// </summary>
/// <param name="row">The zero-based luma row.</param>
/// <returns>The reconstructed luma samples.</returns>
public ReadOnlySpan<ushort> GetLumaRow(int row);
/// <summary>
/// Gets one complete blue-difference chroma row in native sample storage.
/// </summary>
/// <param name="row">The zero-based chroma row.</param>
/// <returns>The reconstructed blue-difference samples.</returns>
public ReadOnlySpan<ushort> GetChromaBlueRow(int row);
/// <summary>
/// Gets one complete red-difference chroma row in native sample storage.
/// </summary>
/// <param name="row">The zero-based chroma row.</param>
/// <returns>The reconstructed red-difference samples.</returns>
public ReadOnlySpan<ushort> GetChromaRedRow(int row);
}

81
src/ImageSharp/Formats/Heif/Hevc/Color/HevcPlanarSampleBuffer.cs

@ -0,0 +1,81 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using SixLabors.ImageSharp.Formats.Heif.Color;
namespace SixLabors.ImageSharp.Formats.Heif.Hevc.Color;
/// <summary>
/// Adapts reconstructed HEVC planes to the shared HEIF planar color pipeline.
/// </summary>
internal struct HevcPlanarSampleBuffer : IHeifPlanarSampleBuffer<ushort>
{
/// <summary>
/// The reconstructed HEVC picture containing the component planes.
/// </summary>
private readonly HevcPictureBuffer picture;
/// <summary>
/// The progressive-frame 4:2:0 chroma sample location.
/// </summary>
private readonly HevcChromaSampleLocation chromaSampleLocation;
/// <summary>
/// Initializes a new instance of the <see cref="HevcPlanarSampleBuffer"/> struct.
/// </summary>
/// <param name="picture">The reconstructed HEVC picture.</param>
/// <param name="chromaSampleLocation">The progressive-frame 4:2:0 chroma sample location.</param>
public HevcPlanarSampleBuffer(HevcPictureBuffer picture, HevcChromaSampleLocation chromaSampleLocation)
{
this.picture = picture;
this.chromaSampleLocation = chromaSampleLocation;
}
/// <summary>
/// Gets the horizontal offset in half-luma-sample units for each HEVC 4:2:0 chroma-location code.
/// </summary>
private static ReadOnlySpan<byte> ChromaLocationX => [0, 1, 0, 1, 0, 1];
/// <summary>
/// Gets the vertical offset in half-luma-sample units for each HEVC 4:2:0 chroma-location code.
/// </summary>
private static ReadOnlySpan<byte> ChromaLocationY => [1, 1, 0, 0, 2, 2];
/// <inheritdoc/>
public readonly int Width => this.picture.Width;
/// <inheritdoc/>
public readonly int Height => this.picture.Height;
/// <inheritdoc/>
public readonly int LumaBitDepth => this.picture.BitDepthLuma;
/// <inheritdoc/>
public readonly int ChromaBitDepth => this.picture.BitDepthChroma;
/// <inheritdoc/>
public readonly bool IsMonochrome => this.picture.ChromaFormat == 0;
/// <inheritdoc/>
public readonly int ChromaSubsamplingX => this.picture.GetSubsamplingX(HevcPlane.Cb);
/// <inheritdoc/>
public readonly int ChromaSubsamplingY => this.picture.GetSubsamplingY(HevcPlane.Cb);
/// <inheritdoc/>
public readonly int ChromaPositionX
=> this.picture.ChromaFormat == 1 && !this.picture.SeparateColorPlane ? ChromaLocationX[(int)this.chromaSampleLocation] : 0;
/// <inheritdoc/>
public readonly int ChromaPositionY
=> this.picture.ChromaFormat == 1 && !this.picture.SeparateColorPlane ? ChromaLocationY[(int)this.chromaSampleLocation] : 0;
/// <inheritdoc/>
public Span<ushort> GetLumaRowSpan(int row) => this.picture.GetRowSpan(HevcPlane.Y, row);
/// <inheritdoc/>
public Span<ushort> GetChromaBlueRowSpan(int row) => this.picture.GetRowSpan(HevcPlane.Cb, row);
/// <inheritdoc/>
public Span<ushort> GetChromaRedRowSpan(int row) => this.picture.GetRowSpan(HevcPlane.Cr, row);
}

32
src/ImageSharp/Formats/Heif/Hevc/Color/HevcYuv420ToRgb8Source.cs

@ -1,32 +0,0 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using SixLabors.ImageSharp.Formats.Heif.Color;
namespace SixLabors.ImageSharp.Formats.Heif.Hevc.Color;
/// <summary>
/// Adapts reconstructed HEVC planes to the shared fixed-point HEIF color converter.
/// </summary>
internal readonly struct HevcYuv420ToRgb8Source : IHeifYuv420ToRgb8Source
{
/// <summary>
/// The reconstructed HEVC picture containing the component planes.
/// </summary>
private readonly HevcPictureBuffer picture;
/// <summary>
/// Initializes a new instance of the <see cref="HevcYuv420ToRgb8Source"/> struct.
/// </summary>
/// <param name="picture">The reconstructed HEVC picture.</param>
public HevcYuv420ToRgb8Source(HevcPictureBuffer picture) => this.picture = picture;
/// <inheritdoc/>
public ReadOnlySpan<ushort> GetLumaRow(int row) => this.picture.GetRowSpan(HevcPlane.Y, row);
/// <inheritdoc/>
public ReadOnlySpan<ushort> GetChromaBlueRow(int row) => this.picture.GetRowSpan(HevcPlane.Cb, row);
/// <inheritdoc/>
public ReadOnlySpan<ushort> GetChromaRedRow(int row) => this.picture.GetRowSpan(HevcPlane.Cr, row);
}

597
src/ImageSharp/Formats/Heif/Hevc/Color/HevcYuvConverter.cs

@ -1,12 +1,7 @@
// Copyright (c) Six Labors. // Copyright (c) Six Labors.
// Licensed under the Six Labors Split License. // Licensed under the Six Labors Split License.
using System.Buffers;
using System.Runtime.InteropServices;
using SixLabors.ImageSharp.Advanced;
using SixLabors.ImageSharp.Common.Helpers;
using SixLabors.ImageSharp.Formats.Heif.Color; using SixLabors.ImageSharp.Formats.Heif.Color;
using SixLabors.ImageSharp.Memory;
using SixLabors.ImageSharp.Metadata.Profiles.Cicp; using SixLabors.ImageSharp.Metadata.Profiles.Cicp;
using SixLabors.ImageSharp.PixelFormats; using SixLabors.ImageSharp.PixelFormats;
using static SixLabors.ImageSharp.Formats.Heif.Color.HeifColorConverterBase; using static SixLabors.ImageSharp.Formats.Heif.Color.HeifColorConverterBase;
@ -14,30 +9,10 @@ using static SixLabors.ImageSharp.Formats.Heif.Color.HeifColorConverterBase;
namespace SixLabors.ImageSharp.Formats.Heif.Hevc.Color; namespace SixLabors.ImageSharp.Formats.Heif.Hevc.Color;
/// <summary> /// <summary>
/// Converts between reconstructed HEVC component planes and packed ImageSharp pixels. /// Adapts HEVC color signaling and reconstructed planes to the shared HEIF color pipeline.
/// </summary> /// </summary>
internal static partial class HevcYuvConverter internal static class HevcYuvConverter
{ {
/// <summary>
/// The largest value represented by an eight-bit packed RGB component.
/// </summary>
private const float ByteMaximum = byte.MaxValue;
/// <summary>
/// The largest value represented by a 16-bit packed RGB component.
/// </summary>
private const float UShortMaximum = ushort.MaxValue;
/// <summary>
/// Gets the horizontal offset in half-luma-sample units for each HEVC 4:2:0 chroma-location code.
/// </summary>
private static ReadOnlySpan<byte> ChromaLocationX => [0, 1, 0, 1, 0, 1];
/// <summary>
/// Gets the vertical offset in half-luma-sample units for each HEVC 4:2:0 chroma-location code.
/// </summary>
private static ReadOnlySpan<byte> ChromaLocationY => [1, 1, 0, 0, 2, 2];
/// <summary> /// <summary>
/// Converts reconstructed HEVC component planes to packed pixels. /// Converts reconstructed HEVC component planes to packed pixels.
/// </summary> /// </summary>
@ -60,42 +35,15 @@ internal static partial class HevcYuvConverter
where TPixel : unmanaged, IPixel<TPixel> where TPixel : unmanaged, IPixel<TPixel>
{ {
HeifColorConversionParameters parameters = GetConversionParameters(picture, colorProfile, out HeifColorConversionMode mode); HeifColorConversionParameters parameters = GetConversionParameters(picture, colorProfile, out HeifColorConversionMode mode);
HevcPlanarSampleBuffer buffer = new(picture, chromaSampleLocation);
// The common full-range eight-bit path remains in the integer sample domain. The codec adapter exposes HeifPlanarColorConverter.ConvertToRgb<TPixel, HevcPlanarSampleBuffer>(
// only native rows; shared HEIF code owns coefficient arithmetic, SIMD traversal, pooling, and packing. configuration,
if (HeifYuv420ToRgb8Converter.IsSupported( buffer,
picture.GetSubsamplingX(HevcPlane.Cb), image,
picture.GetSubsamplingY(HevcPlane.Cb), in parameters,
picture.BitDepthLuma, mode,
picture.BitDepthChroma, sourceX,
colorProfile.FullRange, sourceY);
colorProfile.MatrixCoefficients,
mode))
{
HevcYuv420ToRgb8Source source = new(picture);
HeifYuv420ToRgb8Converter.Convert(configuration, source, image, in parameters, sourceX, sourceY);
return;
}
HeifColorConverterBase colorConverter = HeifColorConverterBase.Create(mode, in parameters, picture.ChromaFormat == 0);
YuvToRgbRowConverter<TPixel> converter = new(configuration, picture, image, colorConverter, chromaSampleLocation, sourceX, sourceY);
using IMemoryOwner<float> scratchOwner = configuration.MemoryAllocator.Allocate<float>(converter.BufferLength);
Span<float> scratch = scratchOwner.GetSpan();
if (converter.UsesBytePacking)
{
for (int y = 0; y < image.Height; y++)
{
converter.Convert(y, scratch);
}
return;
}
for (int y = 0; y < image.Height; y++)
{
converter.Convert(y, scratch);
}
} }
/// <summary> /// <summary>
@ -116,19 +64,13 @@ internal static partial class HevcYuvConverter
where TPixel : unmanaged, IPixel<TPixel> where TPixel : unmanaged, IPixel<TPixel>
{ {
HeifColorConversionParameters parameters = GetConversionParameters(picture, colorProfile, out HeifColorConversionMode mode); HeifColorConversionParameters parameters = GetConversionParameters(picture, colorProfile, out HeifColorConversionMode mode);
HeifColorConverterBase colorConverter = HeifColorConverterBase.Create(mode, in parameters, picture.ChromaFormat == 0); HevcPlanarSampleBuffer buffer = new(picture, chromaSampleLocation);
RgbToYuvRowConverter<TPixel> converter = new(configuration, picture, image, colorConverter, chromaSampleLocation, in parameters); HeifPlanarColorConverter.ConvertFromRgb<TPixel, HevcPlanarSampleBuffer, ushort, HeifUShortSampleStorer>(
using IMemoryOwner<float> componentOwner = configuration.MemoryAllocator.Allocate<float>(converter.ComponentBufferLength); configuration,
Span<float> components = componentOwner.GetSpan(); image,
buffer,
if (converter.UsesByteInput) in parameters,
{ mode);
converter.Convert(Span<Rgb48>.Empty, components);
return;
}
using IMemoryOwner<Rgb48> packedOwner = configuration.MemoryAllocator.Allocate<Rgb48>(image.Width);
converter.Convert(packedOwner.GetSpan()[..image.Width], components);
} }
/// <summary> /// <summary>
@ -152,507 +94,4 @@ internal static partial class HevcYuvConverter
picture.ChromaFormat == 0, picture.ChromaFormat == 0,
picture.ChromaFormat == 3, picture.ChromaFormat == 3,
out mode); out mode);
/// <summary>
/// Resolves the two chroma rows and quarter-sample weight surrounding a luma row.
/// </summary>
/// <param name="coordinate">The luma row coordinate.</param>
/// <param name="subsampling">The vertical chroma subsampling shift.</param>
/// <param name="position">The chroma offset in half-luma-sample units.</param>
/// <param name="maximum">The last available chroma row.</param>
/// <param name="lower">The lower chroma row.</param>
/// <param name="upper">The upper chroma row.</param>
/// <param name="upperWeight">The upper-row weight with a denominator of four.</param>
private static void GetChromaCoordinates(
int coordinate,
int subsampling,
int position,
int maximum,
out int lower,
out int upper,
out int upperWeight)
{
if (subsampling == 0)
{
lower = coordinate;
upper = coordinate;
upperWeight = 0;
return;
}
// Multiplying the source coordinate by four keeps all six HEVC 4:2:0 locations integral. Floor division
// is required above the top edge because the first centered sample lies after the first luma row.
int quarterCoordinate = (coordinate * 2) - position;
int unclampedLower = quarterCoordinate >= 0 ? quarterCoordinate >> 2 : -((-quarterCoordinate + 3) >> 2);
int fraction = quarterCoordinate - (unclampedLower * 4);
lower = Numerics.Clamp(unclampedLower, 0, maximum);
upper = Numerics.Clamp(unclampedLower + 1, 0, maximum);
upperWeight = lower == upper ? 0 : fraction;
}
/// <summary>
/// Resolves the luma-relative chroma offsets for the selected plane layout.
/// </summary>
/// <param name="picture">The HEVC picture layout.</param>
/// <param name="chromaSampleLocation">The signaled 4:2:0 location.</param>
/// <param name="horizontalPosition">The horizontal offset in half-luma-sample units.</param>
/// <param name="verticalPosition">The vertical offset in half-luma-sample units.</param>
private static void GetChromaPosition(
HevcPictureBuffer picture,
HevcChromaSampleLocation chromaSampleLocation,
out int horizontalPosition,
out int verticalPosition)
{
if (picture.ChromaFormat != 1 || picture.SeparateColorPlane)
{
horizontalPosition = 0;
verticalPosition = 0;
return;
}
int location = (int)chromaSampleLocation;
horizontalPosition = ChromaLocationX[location];
verticalPosition = ChromaLocationY[location];
}
/// <summary>
/// Converts one reconstructed HEVC row using pooled planar and packed-pixel storage.
/// </summary>
/// <typeparam name="TPixel">The destination pixel type.</typeparam>
private readonly struct YuvToRgbRowConverter<TPixel>
where TPixel : unmanaged, IPixel<TPixel>
{
/// <summary>
/// The configuration used for packed-pixel conversion.
/// </summary>
private readonly Configuration configuration;
/// <summary>
/// The reconstructed HEVC picture.
/// </summary>
private readonly HevcPictureBuffer picture;
/// <summary>
/// The destination image frame.
/// </summary>
private readonly ImageFrame<TPixel> image;
/// <summary>
/// The selected H.273 color converter.
/// </summary>
private readonly HeifColorConverterBase colorConverter;
/// <summary>
/// The horizontal chroma offset in half-luma-sample units.
/// </summary>
private readonly int horizontalPosition;
/// <summary>
/// The vertical chroma offset in half-luma-sample units.
/// </summary>
private readonly int verticalPosition;
/// <summary>
/// The horizontal chroma subsampling shift.
/// </summary>
private readonly int subX;
/// <summary>
/// The vertical chroma subsampling shift.
/// </summary>
private readonly int subY;
/// <summary>
/// The horizontal luma-sample offset of the output window.
/// </summary>
private readonly int sourceX;
/// <summary>
/// The vertical luma-sample offset of the output window.
/// </summary>
private readonly int sourceY;
/// <summary>
/// Initializes a new instance of the <see cref="YuvToRgbRowConverter{TPixel}"/> struct.
/// </summary>
/// <param name="configuration">The configuration used for pixel conversion.</param>
/// <param name="picture">The reconstructed HEVC picture.</param>
/// <param name="image">The destination image frame.</param>
/// <param name="colorConverter">The selected H.273 color converter.</param>
/// <param name="chromaSampleLocation">The progressive-frame 4:2:0 chroma sample location.</param>
/// <param name="sourceX">The horizontal luma-sample offset of the output window.</param>
/// <param name="sourceY">The vertical luma-sample offset of the output window.</param>
public YuvToRgbRowConverter(
Configuration configuration,
HevcPictureBuffer picture,
ImageFrame<TPixel> image,
HeifColorConverterBase colorConverter,
HevcChromaSampleLocation chromaSampleLocation,
int sourceX,
int sourceY)
{
this.configuration = configuration;
this.picture = picture;
this.image = image;
this.colorConverter = colorConverter;
this.subX = picture.GetSubsamplingX(HevcPlane.Cb);
this.subY = picture.GetSubsamplingY(HevcPlane.Cb);
this.sourceX = sourceX;
this.sourceY = sourceY;
GetChromaPosition(picture, chromaSampleLocation, out this.horizontalPosition, out this.verticalPosition);
}
/// <summary>
/// Gets a value indicating whether the destination uses JPEG-compatible eight-bit RGB plane packing.
/// </summary>
public bool UsesBytePacking => this.picture.BitDepthLuma == 8 && (this.picture.ChromaFormat == 0 || this.picture.BitDepthChroma == 8);
/// <summary>
/// Gets the number of float elements required by the reusable row buffer.
/// </summary>
public int BufferLength
{
get
{
int componentLength = this.image.Width * 3;
if (this.picture.ChromaFormat != 0 && this.subX != 0)
{
// Cropped output can begin between subsampled chroma positions. Reconstructing one complete
// coded-width row preserves the edge interpolation before selecting the visible window.
componentLength += this.picture.Width + (this.picture.GetWidth(HevcPlane.Cb) * 2);
}
int packedRowCount = this.UsesBytePacking ? 1 : 2;
return componentLength + (this.image.Width * packedRowCount);
}
}
/// <summary>
/// Converts one reconstructed row to packed pixels.
/// </summary>
/// <param name="y">The zero-based luma row.</param>
/// <param name="scratch">The reusable pooled row buffer.</param>
public void Convert(int y, Span<float> scratch)
{
int width = this.image.Width;
Span<float> red = scratch[..width];
Span<float> green = scratch.Slice(width, width);
Span<float> blue = scratch.Slice(width * 2, width);
int sourceY = y + this.sourceY;
ReadOnlySpan<ushort> luma = this.picture.GetRowSpan(HevcPlane.Y, sourceY).Slice(this.sourceX, width);
ConvertSamplesToFloat<ushort, HeifUShortSampleLoader>(luma, red);
int packedOffset = width * 3;
if (this.picture.ChromaFormat != 0)
{
int chromaHeight = this.picture.GetHeight(HevcPlane.Cb);
GetChromaCoordinates(sourceY, this.subY, this.verticalPosition, chromaHeight - 1, out int y0, out int y1, out int y1Weight);
ReadOnlySpan<ushort> cb0 = this.picture.GetRowSpan(HevcPlane.Cb, y0);
ReadOnlySpan<ushort> cb1 = this.picture.GetRowSpan(HevcPlane.Cb, y1);
ReadOnlySpan<ushort> cr0 = this.picture.GetRowSpan(HevcPlane.Cr, y0);
ReadOnlySpan<ushort> cr1 = this.picture.GetRowSpan(HevcPlane.Cr, y1);
if (this.subX == 0)
{
ConvertSamplesToFloat<ushort, HeifUShortSampleLoader>(cb0.Slice(this.sourceX, width), green);
ConvertSamplesToFloat<ushort, HeifUShortSampleLoader>(cr0.Slice(this.sourceX, width), blue);
}
else
{
int chromaWidth = this.picture.GetWidth(HevcPlane.Cb);
Span<float> reconstructed = scratch.Slice(packedOffset, this.picture.Width);
Span<float> chroma0 = scratch.Slice(packedOffset + this.picture.Width, chromaWidth);
Span<float> chroma1 = scratch.Slice(packedOffset + this.picture.Width + chromaWidth, chromaWidth);
bool isCenteredX = this.horizontalPosition == 1;
ReconstructChromaRow<ushort, HeifUShortSampleLoader>(
cb0,
cb1,
y1Weight,
this.subX,
isCenteredX,
reconstructed,
chroma0,
chroma1);
reconstructed.Slice(this.sourceX, width).CopyTo(green);
ReconstructChromaRow<ushort, HeifUShortSampleLoader>(
cr0,
cr1,
y1Weight,
this.subX,
isCenteredX,
reconstructed,
chroma0,
chroma1);
reconstructed.Slice(this.sourceX, width).CopyTo(blue);
packedOffset += this.picture.Width + (chromaWidth * 2);
}
}
this.colorConverter.ConvertToRgbInPlace(red, green, blue);
Span<TPixel> destination = this.image.PixelBuffer.DangerousGetRowSpan(y);
Span<float> packedStorage = scratch[packedOffset..];
if (this.UsesBytePacking)
{
// This is JPEG's planar packing contract. Existing pixel-specific SIMD packers therefore own the
// final RGB-to-TPixel conversion instead of a HEVC-specific per-pixel loop.
Span<byte> byteStorage = MemoryMarshal.AsBytes(packedStorage)[..(width * 3)];
Span<byte> redBytes = byteStorage[..width];
Span<byte> greenBytes = byteStorage.Slice(width, width);
Span<byte> blueBytes = byteStorage.Slice(width * 2, width);
SimdUtils.NormalizedFloatToByteSaturate(red, redBytes);
SimdUtils.NormalizedFloatToByteSaturate(green, greenBytes);
SimdUtils.NormalizedFloatToByteSaturate(blue, blueBytes);
PixelOperations<TPixel>.Instance.PackFromRgbPlanes(redBytes, greenBytes, blueBytes, destination);
return;
}
Span<Rgba64> packed = MemoryMarshal.Cast<float, Rgba64>(packedStorage)[..width];
PackRgba64(red, green, blue, packed);
PixelOperations<TPixel>.Instance.FromRgba64(this.configuration, packed, destination);
}
}
/// <summary>
/// Converts packed image rows to HEVC component planes using pooled planar storage.
/// </summary>
/// <typeparam name="TPixel">The source pixel type.</typeparam>
private readonly struct RgbToYuvRowConverter<TPixel>
where TPixel : unmanaged, IPixel<TPixel>
{
/// <summary>
/// The configuration used for packed-pixel conversion.
/// </summary>
private readonly Configuration configuration;
/// <summary>
/// The destination HEVC picture.
/// </summary>
private readonly HevcPictureBuffer picture;
/// <summary>
/// The source image frame.
/// </summary>
private readonly ImageFrame<TPixel> image;
/// <summary>
/// The selected H.273 color converter.
/// </summary>
private readonly HeifColorConverterBase colorConverter;
/// <summary>
/// The largest encoded luma sample value.
/// </summary>
private readonly float lumaMaximum;
/// <summary>
/// The largest encoded chroma sample value.
/// </summary>
private readonly float chromaMaximum;
/// <summary>
/// The horizontal chroma offset in half-luma-sample units.
/// </summary>
private readonly int horizontalPosition;
/// <summary>
/// The vertical chroma offset in half-luma-sample units.
/// </summary>
private readonly int verticalPosition;
/// <summary>
/// The horizontal chroma subsampling shift.
/// </summary>
private readonly int subX;
/// <summary>
/// The vertical chroma subsampling shift.
/// </summary>
private readonly int subY;
/// <summary>
/// Initializes a new instance of the <see cref="RgbToYuvRowConverter{TPixel}"/> struct.
/// </summary>
/// <param name="configuration">The configuration used for pixel conversion.</param>
/// <param name="picture">The destination HEVC picture.</param>
/// <param name="image">The source image frame.</param>
/// <param name="colorConverter">The selected H.273 color converter.</param>
/// <param name="chromaSampleLocation">The progressive-frame 4:2:0 chroma sample location.</param>
/// <param name="parameters">The resolved H.273 component ranges.</param>
public RgbToYuvRowConverter(
Configuration configuration,
HevcPictureBuffer picture,
ImageFrame<TPixel> image,
HeifColorConverterBase colorConverter,
HevcChromaSampleLocation chromaSampleLocation,
in HeifColorConversionParameters parameters)
{
this.configuration = configuration;
this.picture = picture;
this.image = image;
this.colorConverter = colorConverter;
this.lumaMaximum = parameters.LumaSampleMaximum;
this.chromaMaximum = parameters.ChromaSampleMaximum;
this.subX = picture.GetSubsamplingX(HevcPlane.Cb);
this.subY = picture.GetSubsamplingY(HevcPlane.Cb);
GetChromaPosition(picture, chromaSampleLocation, out this.horizontalPosition, out this.verticalPosition);
}
/// <summary>
/// Gets a value indicating whether source pixels use JPEG-compatible eight-bit RGB plane unpacking.
/// </summary>
public bool UsesByteInput => this.picture.BitDepthLuma == 8 && (this.picture.ChromaFormat == 0 || this.picture.BitDepthChroma == 8);
/// <summary>
/// Gets the number of float elements required by the reusable component buffer.
/// </summary>
public int ComponentBufferLength => this.image.Width * (this.subY == 0 ? 3 : 6);
/// <summary>
/// Converts every packed source row to the destination component planes.
/// </summary>
/// <param name="packed">The reusable high-bit-depth RGB staging row.</param>
/// <param name="components">The reusable planar component buffer.</param>
public void Convert(Span<Rgb48> packed, Span<float> components)
{
int width = this.image.Width;
for (int y = 0; y < this.image.Height; y++)
{
int slot = this.subY == 0 ? 0 : y & 1;
int offset = slot * width * 3;
Span<float> luma = components.Slice(offset, width);
Span<float> chromaBlue = components.Slice(offset + width, width);
Span<float> chromaRed = components.Slice(offset + (width * 2), width);
this.ConvertSourceRow(y, packed, luma, chromaBlue, chromaRed);
WriteSamples<ushort, HeifUShortSampleStorer>(
luma,
this.picture.GetRowSpan(HevcPlane.Y, y),
this.colorConverter.LumaScale,
this.colorConverter.LumaBias,
this.lumaMaximum);
if (this.picture.ChromaFormat == 0)
{
continue;
}
if (this.subY == 0)
{
this.WriteChromaRows(y, chromaBlue, chromaRed, Span<float>.Empty, Span<float>.Empty, 0F);
continue;
}
bool isLastRow = y == this.image.Height - 1;
if (this.verticalPosition == 0 && (y & 1) == 0)
{
this.WriteChromaRows(y >> 1, chromaBlue, chromaRed, Span<float>.Empty, Span<float>.Empty, 0F);
}
else if (this.verticalPosition == 1 && ((y & 1) != 0 || isLastRow))
{
if ((y & 1) != 0)
{
Span<float> previousBlue = components.Slice(width, width);
Span<float> previousRed = components.Slice(width * 2, width);
this.WriteChromaRows(y >> 1, previousBlue, previousRed, chromaBlue, chromaRed, 0.5F);
}
else
{
this.WriteChromaRows(y >> 1, chromaBlue, chromaRed, Span<float>.Empty, Span<float>.Empty, 0F);
}
}
else if (this.verticalPosition == 2 && ((y & 1) != 0 || isLastRow))
{
this.WriteChromaRows(y >> 1, chromaBlue, chromaRed, Span<float>.Empty, Span<float>.Empty, 0F);
}
}
}
/// <summary>
/// Converts one packed source row into normalized planar components.
/// </summary>
/// <param name="y">The source row index.</param>
/// <param name="packed">The high-bit-depth RGB staging row.</param>
/// <param name="luma">The destination luma or first component values.</param>
/// <param name="chromaBlue">The destination blue-difference or second component values.</param>
/// <param name="chromaRed">The destination red-difference or third component values.</param>
private void ConvertSourceRow(int y, Span<Rgb48> packed, Span<float> luma, Span<float> chromaBlue, Span<float> chromaRed)
{
ReadOnlySpan<TPixel> source = this.image.PixelBuffer.DangerousGetRowSpan(y);
if (this.UsesByteInput)
{
// JPEG's planar unpack contract reaches the existing pixel-specific SIMD implementation before
// the closed H.273 operator transforms the rows in place.
PixelOperations<TPixel>.Instance.UnpackIntoRgbPlanes(luma, chromaBlue, chromaRed, source);
this.colorConverter.ConvertFromRgbInPlace(luma, chromaBlue, chromaRed, ByteMaximum);
return;
}
PixelOperations<TPixel>.Instance.ToRgb48(this.configuration, source, packed);
DeinterleaveRgb48(packed, luma, chromaBlue, chromaRed);
this.colorConverter.ConvertFromRgbInPlace(luma, chromaBlue, chromaRed, UShortMaximum);
}
/// <summary>
/// Filters and writes one pair of chroma component rows.
/// </summary>
/// <param name="destinationY">The destination chroma row.</param>
/// <param name="blue0">The first blue-difference source row.</param>
/// <param name="red0">The first red-difference source row.</param>
/// <param name="blue1">The optional second blue-difference source row.</param>
/// <param name="red1">The optional second red-difference source row.</param>
/// <param name="row1Weight">The second-row contribution.</param>
private void WriteChromaRows(
int destinationY,
ReadOnlySpan<float> blue0,
ReadOnlySpan<float> red0,
ReadOnlySpan<float> blue1,
ReadOnlySpan<float> red1,
float row1Weight)
{
Span<ushort> blueDestination = this.picture.GetRowSpan(HevcPlane.Cb, destinationY);
Span<ushort> redDestination = this.picture.GetRowSpan(HevcPlane.Cr, destinationY);
if (this.subX == 0)
{
WriteSamples<ushort, HeifUShortSampleStorer>(
blue0,
blueDestination,
this.colorConverter.ChromaScale,
this.colorConverter.ChromaBias,
this.chromaMaximum);
WriteSamples<ushort, HeifUShortSampleStorer>(
red0,
redDestination,
this.colorConverter.ChromaScale,
this.colorConverter.ChromaBias,
this.chromaMaximum);
return;
}
bool isCenteredX = this.horizontalPosition == 1;
WriteSubsampledSamples<ushort, HeifUShortSampleStorer>(
blue0,
blue1,
blueDestination,
isCenteredX,
row1Weight,
this.colorConverter.ChromaScale,
this.colorConverter.ChromaBias,
this.chromaMaximum);
WriteSubsampledSamples<ushort, HeifUShortSampleStorer>(
red0,
red1,
redDestination,
isCenteredX,
row1Weight,
this.colorConverter.ChromaScale,
this.colorConverter.ChromaBias,
this.chromaMaximum);
}
}
} }

1
tests/ImageSharp.Benchmarks/Codecs/Heif/Av1ColorConversionBenchmarks.cs

@ -3,6 +3,7 @@
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using SixLabors.ImageSharp.Formats.Heif.Av1; using SixLabors.ImageSharp.Formats.Heif.Av1;
using SixLabors.ImageSharp.Formats.Heif.Av1.Color;
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.PixelFormats; using SixLabors.ImageSharp.PixelFormats;

35
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1YuvConverterTests.cs

@ -3,6 +3,7 @@
using System; using System;
using SixLabors.ImageSharp.Formats.Heif.Av1; using SixLabors.ImageSharp.Formats.Heif.Av1;
using SixLabors.ImageSharp.Formats.Heif.Av1.Color;
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit; using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Memory; using SixLabors.ImageSharp.Memory;
using SixLabors.ImageSharp.PixelFormats; using SixLabors.ImageSharp.PixelFormats;
@ -320,6 +321,40 @@ public class Av1YuvConverterTests
Assert.Equal(expectedLeftBlue, image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(1)[0].B); Assert.Equal(expectedLeftBlue, image.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(1)[0].B);
} }
/// <summary>
/// Verifies that encoding selects the horizontal and vertical sample coordinates defined by each AV1 chroma position.
/// </summary>
/// <param name="chromaSamplePosition">The signaled AV1 chroma sample position.</param>
/// <param name="expectedChromaBlue">The expected encoded blue-difference sample.</param>
/// <param name="expectedChromaRed">The expected encoded red-difference sample.</param>
[Theory]
[InlineData(ObuChromoSamplePosition.Unknown, 128, 128)]
[InlineData(ObuChromoSamplePosition.Vertical, 96, 192)]
[InlineData(ObuChromoSamplePosition.Colocated, 128, 128)]
public void RgbToYuv420UsesChromaSamplePosition(int chromaSamplePosition, byte expectedChromaBlue, byte expectedChromaRed)
{
using Image<Rgba32> source = new(2, 2);
// The four distinct YCgCo samples make left, centered, and vertically averaged selection observable as
// exact integer chroma values without introducing transfer-function or coefficient-rounding tolerances.
source[0, 0] = new Rgba32(0, 0, 0);
source[1, 0] = new Rgba32(0, byte.MaxValue, 0);
source[0, 1] = new Rgba32(byte.MaxValue, 0, 0);
source[1, 1] = new Rgba32(0, 0, byte.MaxValue);
ObuSequenceHeader sequenceHeader = CreateSequenceHeader(
2,
2,
matrixCoefficients: ObuMatrixCoefficients.SmpteYCgCo,
colorFormat: Av1ColorFormat.Yuv420,
chromaSamplePosition: (ObuChromoSamplePosition)chromaSamplePosition);
using Av1FrameBuffer<byte> frameBuffer = new(Configuration.Default, sequenceHeader, Av1ColorFormat.Yuv420, false);
Av1YuvConverter.ConvertFromRgb(Configuration.Default, source.Frames.RootFrame, frameBuffer);
Assert.Equal(expectedChromaBlue, frameBuffer.DeriveBlockPointer(Av1Plane.U, 1, 1).DangerousGetRowSpan(0)[0]);
Assert.Equal(expectedChromaRed, frameBuffer.DeriveBlockPointer(Av1Plane.V, 1, 1).DangerousGetRowSpan(0)[0]);
}
/// <summary> /// <summary>
/// Verifies that odd image dimensions retain the final YUV 4:2:0 chroma row and column. /// Verifies that odd image dimensions retain the final YUV 4:2:0 chroma row and column.
/// </summary> /// </summary>

Loading…
Cancel
Save