tsai
/
ImageSharp
mirror of https://github.com/SixLabors/ImageSharp
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
Browse Source

Decode AV1 palette blocks

pull/2633/head
James Jackson-South 2 months ago
parent
77cc468752
commit
7505f77598
9 changed files with 914 additions and 32 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 6
      HEIF_IMPLEMENTATION_PLAN.md
  2. 142
      THIRD-PARTY-NOTICES.TXT
  3. 75
      src/ImageSharp/Formats/Heif/Av1/Entropy/Av1DefaultDistributions.cs
  4. 123
      src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolDecoder.cs
  5. 20
      src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs
  6. 41
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1BlockModeInfo.cs
  7. 10
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ParseAboveNeighbor4x4Context.cs
  8. 9
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ParseLeftNeighbor4x4Context.cs
  9. 520
      src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs

6
HEIF_IMPLEMENTATION_PLAN.md
View File

@ -70,6 +70,7 @@ This snapshot pins or classifies the available references and failures; it does
| `Av1YuvConverter.ConvertToRgb`, `ConvertFromRgb`, scalar row conversion, chroma reconstruction, and chroma downsampling | H.273 formulas 20-31 and the identity, YCgCo, and non-constant-luminance matrix formulas; AV1 section 6.4.2 chroma sample positions | libavif `src/reformat.c` and `src/colr.c` at `092276ce89098ead06db80975173191e5fee1826`; libaom `aom/aom_image.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Scalar behavioral oracle for full/limited-range conversion at 8, 10, and 12 bits. Decode covers monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with AV1 chroma sample positioning. Encode covers the same plane layouts, using libavif's actual-edge-count box average for subsampled chroma. Later SIMD paths must preserve this scalar behavior. |
| `Av1FrameBuffer` high-bit-depth sample layout and `Av1YuvConverter` 10/12-bit packed-pixel conversion | AV1 section 6.4.1 bit depth and H.273 sample-range scaling | libaom `aom_scale/yv12config.h`, `av1/common/idct.c`, and `av1/common/reconintra.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f`; libavif `src/avif.c` and `src/reformat.c` at `092276ce89098ead06db80975173191e5fee1826` | Establish two-byte native sample storage with sample-unit strides for 10/12-bit reconstruction and use ImageSharp's existing `Rgb48` pixel-operation paths in both directions so packed-pixel staging does not reduce high-bit-depth samples to eight bits. |
| `Av1PredictionDecoder`, `Av1HighBitDepthPredictor`, `Av1ChromaFromLumaContext`, `Av1PartitionInfo`, and the scalar DC, directional, Paeth, smooth, filter-intra, and chroma-from-luma predictors | AV1 sections 7.11.2 and 7.11.2.3 intra prediction | libaom `aom_dsp/intrapred.c`, `av1/common/reconintra.c`, `av1/common/av1_common_int.h`, `av1/common/blockd.h`, `av1/common/cfl.c`, and `av1/common/cfl.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Behavioral oracle for luma/chroma mode-neighbor addressing, directional upsampling, Paeth selection, smooth normalization, filter-intra taps, high-bit-depth clipping, chroma-from-luma storage/subsampling, and chroma-from-luma row strides. Existing managed scalar tables and predictors remain the implementation base. The WIP rectangular byte-pipeline smooth digest expectations encode width/height-swapped weights and must be replaced only from an independently generated oracle, not regenerated from this implementation. |
| `Av1TileReader` palette mode/color-map parsing, `Av1SymbolDecoder` palette distributions, `Av1BlockModeInfo` palette state, and `Av1PredictionDecoder` palette reconstruction | AV1 sections 5.11.46, 5.11.49, and 7.11.2 palette prediction | libaom `av1/decoder/decodemv.c`, `av1/decoder/detokenize.c`, `av1/decoder/decoder.h`, `av1/common/pred_common.c`, `av1/common/pred_common.h`, and `av1/common/entropymode.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Reuse the existing tile range decoder and frame-owned mode map while porting the normative palette cache merge, high-bit-depth color deltas, tile-adaptive mode/size/index distributions, diagonal color-map traversal, edge padding, and direct palette-sample reconstruction. This is AV1 still-image compression syntax and does not add retained video reference state or any ISO BMFF surface. |
| `Av1FrameInfo`, `Av1TileReader`, and `Av1BlockDecoder` transform/coefficient storage | AV1 section 5.11.39 coefficient syntax and section 7.11.2 reconstruction | libaom `av1/decoder/decodetxb.c` and `av1/decoder/decoder.h` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Preserve separate luma and chroma transform coefficients at monotonically advancing per-plane offsets within each superblock so reconstruction consumes the same transform-block order produced by tile parsing. |
| `Av1InverseQuantizer` and `Av1InverseQuantizationLookup` | AV1 section 7.12.3 inverse quantization | libaom `aom_dsp/aom_dsp_common.h`, `av1/common/quant_common.c`, and `av1/decoder/decodetxb.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Select the per-segment matrix level, alias 64-pixel transform dimensions to their adjusted matrices, retain a flat level-15 matrix, and apply the five-bit inverse-matrix weight scale. The large managed lookup remains a single process-wide table. |
| `Av1Inverse2dTransformer` and `Av1InverseTransformerFactory` | AV1 section 7.11.2 inverse transform and reconstruction | libaom `av1/common/av1_inv_txfm1d.c`, `av1/common/av1_inv_txfm2d.c`, and `av1/common/idct.c` at `03087864cf4bea6abb0d28f95cf7843511413d8f` | Scalar transform oracle for coefficient-row traversal, intermediate layout, stage ranges, clipping, and high-bit-depth sample addition. The managed 16-bit overload is also used as a parity oracle for the byte overload. |
@ -116,9 +117,10 @@ This assessment is based on the current source after the upstream ImageSharp mer
### AV1 decoder
- The single-still `Av1Decoder` path now parses tile state before allocating and reconstructing one frame, and it disposes the reconstruction planes after pixel conversion. Reference-frame, `show_existing_frame`, and multi-frame ownership remain incomplete.
- The single-still `Av1Decoder` path now parses tile state before allocating and reconstructing one independently decodable frame, and it disposes the reconstruction planes after pixel conversion. It deliberately does not retain animation/video reference frames or implement `show_existing_frame` playback state.
- The reconstruction pipeline disables loop filtering, CDEF, super-resolution, loop restoration, and padding with constant flags. These are normative stages when signaled, not optional quality improvements.
- Loop restoration, palette paths, `show_existing_frame`, reference/CDF state, and other syntax paths contain `NotImplementedException` or equivalent unsupported branches.
- Palette mode now reads the normative luma/chroma mode and size CDFs, neighbor color caches, high-bit-depth color syntax, diagonal color-index maps, clipped-edge padding, and direct sample prediction through the existing reconstruction pipeline. The scalar implementation matches the pinned libaom source, but an independently encoded palette AVIF fixture is still required before this path is independently verified.
- Loop restoration and other normative independently decodable still-image syntax paths still contain `NotImplementedException` or equivalent unsupported branches. Tile-local palette CDF adaptation is now present; the remaining still-image frame-context behavior requires a separate source audit without introducing sequence playback state.
- The frame buffer now establishes two-byte native sample storage, logical plane rows, and sample-unit block strides for 10/12-bit frames. The active intra-prediction, inverse-transform, and block-reconstruction path selects native 16-bit samples for 10/12-bit frames and has focused pipeline wiring coverage. Independently encoded high-bit-depth AVIF conformance files are still required; chroma-from-luma storage and the other normative reconstruction stages listed below remain incomplete.
- `Av1YuvConverter` now consumes the signaled range, supported H.273 matrix coefficients, subsampling, and chroma sample position for 8, 10, and 12-bit output. Its high-bit-depth decode and encode paths use allocator-backed `Rgb48` rows and the existing `PixelOperations<TPixel>` conversions, avoiding the former eight-bit intermediate. Encoder conversion covers monochrome, YUV 4:2:0, 4:2:2, and 4:4:4 with libavif-compatible box averaging. Constant-luminance and chromaticity-derived matrices and ICtCp remain incomplete.
- The inverse-transform path allocates arrays in a per-transform hot path.

142
THIRD-PARTY-NOTICES.TXT
View File

@ -30,3 +30,145 @@ See the LICENSE file in the CoreFX project root for more information.
https://github.com/dotnet/corefx/blob/17300169760c61a90cab8d913636c1058a30a8c1/LICENSE.TXT
https://github.com/dotnet/corefx/blob/17300169760c61a90cab8d913636c1058a30a8c1/src/Common/src/CoreLib/System/IO/Stream.cs#L742
https://github.com/dotnet/corefx/blob/17300169760c61a90cab8d913636c1058a30a8c1/src/Common/src/CoreLib/System/IO/Stream.cs#L775
License notice for the AOM AV1 Codec Library
-----
Parts of the AV1 implementation are adapted from the AOM AV1 Codec Library.
https://aomedia.googlesource.com/aom/
Copyright (c) 2016, Alliance for Open Media. All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in
the documentation and/or other materials provided with the
distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
Alliance for Open Media Patent License 1.0
1. License Terms.
1.1. Patent License. Subject to the terms and conditions of this License, each
Licensor, on behalf of itself and successors in interest and assigns,
grants Licensee a non-sublicensable, perpetual, worldwide, non-exclusive,
no-charge, royalty-free, irrevocable (except as expressly stated in this
License) patent license to its Necessary Claims to make, use, sell, offer
for sale, import or distribute any Implementation.
1.2. Conditions.
1.2.1. Availability. As a condition to the grant of rights to Licensee to make,
sell, offer for sale, import or distribute an Implementation under
Section 1.1, Licensee must make its Necessary Claims available under
this License, and must reproduce this License with any Implementation
as follows:
a. For distribution in source code, by including this License in the
root directory of the source code with its Implementation.
b. For distribution in any other form (including binary, object form,
and/or hardware description code (e.g., HDL, RTL, Gate Level Netlist,
GDSII, etc.)), by including this License in the documentation, legal
notices, and/or other written materials provided with the
Implementation.
1.2.2. Additional Conditions. This license is directly from Licensor to
Licensee. Licensee acknowledges as a condition of benefiting from it
that no rights from Licensor are received from suppliers, distributors,
or otherwise in connection with this License.
1.3. Defensive Termination. If any Licensee, its Affiliates, or its agents
initiates patent litigation or files, maintains, or voluntarily
participates in a lawsuit against another entity or any person asserting
that any Implementation infringes Necessary Claims, any patent licenses
granted under this License directly to the Licensee are immediately
terminated as of the date of the initiation of action unless 1) that suit
was in response to a corresponding suit regarding an Implementation first
brought against an initiating entity, or 2) that suit was brought to
enforce the terms of this License (including intervention in a third-party
action by a Licensee).
1.4. Disclaimers. The Reference Implementation and Specification are provided
"AS IS" and without warranty. The entire risk as to implementing or
otherwise using the Reference Implementation or Specification is assumed
by the implementer and user. Licensor expressly disclaims any warranties
(express, implied, or otherwise), including implied warranties of
merchantability, non-infringement, fitness for a particular purpose, or
title, related to the material. IN NO EVENT WILL LICENSOR BE LIABLE TO
ANY OTHER PARTY FOR LOST PROFITS OR ANY FORM OF INDIRECT, SPECIAL,
INCIDENTAL, OR CONSEQUENTIAL DAMAGES OF ANY CHARACTER FROM ANY CAUSES OF
ACTION OF ANY KIND WITH RESPECT TO THIS LICENSE, WHETHER BASED ON BREACH
OF CONTRACT, TORT (INCLUDING NEGLIGENCE), OR OTHERWISE, AND WHETHER OR
NOT THE OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
2. Definitions.
2.1. Affiliate. "Affiliate" means an entity that directly or indirectly
Controls, is Controlled by, or is under common Control of that party.
2.2. Control. "Control" means direct or indirect control of more than 50% of
the voting power to elect directors of that corporation, or for any other
entity, the power to direct management of such entity.
2.3. Decoder. "Decoder" means any decoder that conforms fully with all
non-optional portions of the Specification.
2.4. Encoder. "Encoder" means any encoder that produces a bitstream that can
be decoded by a Decoder only to the extent it produces such a bitstream.
2.5. Final Deliverable. "Final Deliverable" means the final version of a
deliverable approved by the Alliance for Open Media as a Final
Deliverable.
2.6. Implementation. "Implementation" means any implementation, including the
Reference Implementation, that is an Encoder and/or a Decoder. An
Implementation also includes components of an Implementation only to the
extent they are used as part of an Implementation.
2.7. License. "License" means this license.
2.8. Licensee. "Licensee" means any person or entity who exercises patent
rights granted under this License.
2.9. Licensor. "Licensor" means (i) any Licensee that makes, sells, offers
for sale, imports or distributes any Implementation, or (ii) a person
or entity that has a licensing obligation to the Implementation as a
result of its membership and/or participation in the Alliance for Open
Media working group that developed the Specification.
2.10. Necessary Claims. "Necessary Claims" means all claims of patents or
patent applications, (a) that currently or at any time in the future,
are owned or controlled by the Licensor, and (b) (i) would be an
Essential Claim as defined by the W3C Policy as of February 5, 2004
(https://www.w3.org/Consortium/Patent-Policy-20040205/#def-essential)
as if the Specification was a W3C Recommendation; or (ii) are infringed
by the Reference Implementation.
2.11. Reference Implementation. "Reference Implementation" means an Encoder
and/or Decoder released by the Alliance for Open Media as a Final
Deliverable.
2.12. Specification. "Specification" means the specification designated by
the Alliance for Open Media as a Final Deliverable for which this
License was issued.

75
src/ImageSharp/Formats/Heif/Av1/Entropy/Av1DefaultDistributions.cs
View File

@ -113,6 +113,81 @@ internal static class Av1DefaultDistributions
/// </summary>
public static Av1Distribution IntraBlockCopy => new(30531);
/// <summary>
/// Gets the luma palette-mode distributions indexed by block-size and neighboring-palette contexts.
/// </summary>
public static Av1Distribution[][] PaletteYMode =>
[
[new(31676), new(3419), new(1261)],
[new(31912), new(2859), new(980)],
[new(31823), new(3400), new(781)],
[new(32030), new(3561), new(904)],
[new(32309), new(7337), new(1462)],
[new(32265), new(4015), new(1521)],
[new(32450), new(7946), new(129)]
];
/// <summary>
/// Gets the chroma palette-mode distributions indexed by whether luma uses a palette.
/// </summary>
public static Av1Distribution[] PaletteUvMode => [new(32461), new(21488)];
/// <summary>
/// Gets the luma palette-size distributions indexed by block-size context.
/// </summary>
public static Av1Distribution[] PaletteYSize =>
[
new(7952, 13000, 18149, 21478, 25527, 29241),
new(7139, 11421, 16195, 19544, 23666, 28073),
new(7788, 12741, 17325, 20500, 24315, 28530),
new(8271, 14064, 18246, 21564, 25071, 28533),
new(12725, 19180, 21863, 24839, 27535, 30120),
new(9711, 14888, 16923, 21052, 25661, 27875),
new(14940, 20797, 21678, 24186, 27033, 28999)
];
/// <summary>
/// Gets the chroma palette-size distributions indexed by block-size context.
/// </summary>
public static Av1Distribution[] PaletteUvSize =>
[
new(8713, 19979, 27128, 29609, 31331, 32272),
new(5839, 15573, 23581, 26947, 29848, 31700),
new(4426, 11260, 17999, 21483, 25863, 29430),
new(3228, 9464, 14993, 18089, 22523, 27420),
new(3768, 8886, 13091, 17852, 22495, 27207),
new(2464, 8451, 12861, 21632, 25525, 28555),
new(1269, 5435, 10433, 18963, 21700, 25865)
];
/// <summary>
/// Gets the luma palette color-index distributions indexed by palette size minus two and spatial context.
/// </summary>
public static Av1Distribution[][] PaletteYColorIndex =>
[
[new(28710), new(16384), new(10553), new(27036), new(31603)],
[new(27877, 30490), new(11532, 25697), new(6544, 30234), new(23018, 28072), new(31915, 32385)],
[new(25572, 28046, 30045), new(9478, 21590, 27256), new(7248, 26837, 29824), new(19167, 24486, 28349), new(31400, 31825, 32250)],
[new(24779, 26955, 28576, 30282), new(8669, 20364, 24073, 28093), new(4255, 27565, 29377, 31067), new(19864, 23674, 26716, 29530), new(31646, 31893, 32147, 32426)],
[new(23132, 25407, 26970, 28435, 30073), new(7443, 17242, 20717, 24762, 27982), new(6300, 24862, 26944, 28784, 30671), new(18916, 22895, 25267, 27435, 29652), new(31270, 31550, 31808, 32059, 32353)],
[new(23105, 25199, 26464, 27684, 28931, 30318), new(6950, 15447, 18952, 22681, 25567, 28563), new(7560, 23474, 25490, 27203, 28921, 30708), new(18544, 22373, 24457, 26195, 28119, 30045), new(31198, 31451, 31670, 31882, 32123, 32391)],
[new(21689, 23883, 25163, 26352, 27506, 28827, 30195), new(6892, 15385, 17840, 21606, 24287, 26753, 29204), new(5651, 23182, 25042, 26518, 27982, 29392, 30900), new(19349, 22578, 24418, 25994, 27524, 29031, 30448), new(31028, 31270, 31504, 31705, 31927, 32153, 32392)]
];
/// <summary>
/// Gets the chroma palette color-index distributions indexed by palette size minus two and spatial context.
/// </summary>
public static Av1Distribution[][] PaletteUvColorIndex =>
[
[new(29089), new(16384), new(8713), new(29257), new(31610)],
[new(25257, 29145), new(12287, 27293), new(7033, 27960), new(20145, 25405), new(30608, 31639)],
[new(24210, 27175, 29903), new(9888, 22386, 27214), new(5901, 26053, 29293), new(18318, 22152, 28333), new(30459, 31136, 31926)],
[new(22980, 25479, 27781, 29986), new(8413, 21408, 24859, 28874), new(2257, 29449, 30594, 31598), new(19189, 21202, 25915, 28620), new(31844, 32044, 32281, 32518)],
[new(22217, 24567, 26637, 28683, 30548), new(7307, 16406, 19636, 24632, 28424), new(4441, 25064, 26879, 28942, 30919), new(17210, 20528, 23319, 26750, 29582), new(30674, 30953, 31396, 31735, 32207)],
[new(21239, 23168, 25044, 26962, 28705, 30506), new(6545, 15012, 18004, 21817, 25503, 28701), new(3448, 26295, 27437, 28704, 30126, 31442), new(15889, 18323, 21704, 24698, 26976, 29690), new(30988, 31204, 31479, 31734, 31983, 32325)],
[new(21442, 23288, 24758, 26246, 27649, 28980, 30563), new(5863, 14933, 17552, 20668, 23683, 26411, 29273), new(3415, 25810, 26877, 27990, 29223, 30394, 31618), new(17965, 20084, 22232, 23974, 26274, 28402, 30390), new(31190, 31329, 31516, 31679, 31825, 32026, 32322)]
];
/// <summary>
/// Gets the partition-type distributions indexed by block-size and neighboring split context.
/// </summary>

123
src/ImageSharp/Formats/Heif/Av1/Entropy/Av1SymbolDecoder.cs
View File

@ -23,6 +23,36 @@ internal ref struct Av1SymbolDecoder
/// </summary>
private readonly Av1Distribution tileIntraBlockCopy;
/// <summary>
/// The tile-adaptive luma palette-mode distributions.
/// </summary>
private readonly Av1Distribution[][] paletteYMode;
/// <summary>
/// The tile-adaptive chroma palette-mode distributions.
/// </summary>
private readonly Av1Distribution[] paletteUvMode;
/// <summary>
/// The tile-adaptive luma palette-size distributions.
/// </summary>
private readonly Av1Distribution[] paletteYSize;
/// <summary>
/// The tile-adaptive chroma palette-size distributions.
/// </summary>
private readonly Av1Distribution[] paletteUvSize;
/// <summary>
/// The tile-adaptive luma palette color-index distributions.
/// </summary>
private readonly Av1Distribution[][] paletteYColorIndex;
/// <summary>
/// The tile-adaptive chroma palette color-index distributions.
/// </summary>
private readonly Av1Distribution[][] paletteUvColorIndex;
/// <summary>
/// The tile-adaptive partition-type distributions.
/// </summary>
@ -160,6 +190,12 @@ internal ref struct Av1SymbolDecoder
// Every tile starts from its own frame-context copy. Sharing these objects would let one image's adaptive
// updates change the initial probabilities used to decode the next tile or image.
this.tileIntraBlockCopy = Av1DefaultDistributions.IntraBlockCopy.CreateCopy();
this.paletteYMode = Av1Distribution.CreateCopy(Av1DefaultDistributions.PaletteYMode);
this.paletteUvMode = Av1Distribution.CreateCopy(Av1DefaultDistributions.PaletteUvMode);
this.paletteYSize = Av1Distribution.CreateCopy(Av1DefaultDistributions.PaletteYSize);
this.paletteUvSize = Av1Distribution.CreateCopy(Av1DefaultDistributions.PaletteUvSize);
this.paletteYColorIndex = Av1Distribution.CreateCopy(Av1DefaultDistributions.PaletteYColorIndex);
this.paletteUvColorIndex = Av1Distribution.CreateCopy(Av1DefaultDistributions.PaletteUvColorIndex);
this.tilePartitionTypes = Av1Distribution.CreateCopy(Av1DefaultDistributions.PartitionTypes);
this.keyFrameYMode = Av1Distribution.CreateCopy(Av1DefaultDistributions.KeyFrameYMode);
this.uvMode = Av1Distribution.CreateCopy(Av1DefaultDistributions.UvMode);
@ -198,6 +234,93 @@ internal ref struct Av1SymbolDecoder
return r.ReadLiteral(bitCount);
}
/// <summary>
/// Reads an unsigned fixed-width literal from the tile entropy stream.
/// </summary>
/// <param name="bitCount">The number of literal bits to read.</param>
/// <returns>The decoded literal.</returns>
public int ReadLiteral(int bitCount)
{
ref Av1SymbolReader r = ref this.reader;
return r.ReadLiteral(bitCount);
}
/// <summary>
/// Reads a uniformly coded value from a non-power-of-two alphabet.
/// </summary>
/// <param name="valueCount">The number of possible values.</param>
/// <returns>A value in the range from zero through <paramref name="valueCount"/> minus one.</returns>
public int ReadUniform(int valueCount)
{
ref Av1SymbolReader r = ref this.reader;
int bitCount = Av1Math.Log2(valueCount) + 1;
int threshold = (1 << bitCount) - valueCount;
int value = r.ReadLiteral(bitCount - 1);
if (value < threshold)
{
// The short prefix covers the lower values; only the remaining prefixes consume a final bit.
return value;
}
return (value << 1) - threshold + r.ReadLiteral(1);
}
/// <summary>
/// Reads whether the current luma block uses palette prediction.
/// </summary>
/// <param name="blockSizeContext">The block-area context in the range from zero through six.</param>
/// <param name="neighborContext">The number of available above and left luma neighbors that use palettes.</param>
/// <returns><see langword="true"/> when luma palette prediction is selected; otherwise, <see langword="false"/>.</returns>
public bool ReadPaletteYMode(int blockSizeContext, int neighborContext)
{
ref Av1SymbolReader r = ref this.reader;
return r.ReadSymbol(this.paletteYMode[blockSizeContext][neighborContext]) != 0;
}
/// <summary>
/// Reads whether the current chroma block uses palette prediction.
/// </summary>
/// <param name="hasLumaPalette">A value indicating whether the current block uses a luma palette.</param>
/// <returns><see langword="true"/> when chroma palette prediction is selected; otherwise, <see langword="false"/>.</returns>
public bool ReadPaletteUvMode(bool hasLumaPalette)
{
ref Av1SymbolReader r = ref this.reader;
return r.ReadSymbol(this.paletteUvMode[hasLumaPalette ? 1 : 0]) != 0;
}
/// <summary>
/// Reads a luma or chroma palette size.
/// </summary>
/// <param name="blockSizeContext">The block-area context in the range from zero through six.</param>
/// <param name="planeType">The luma or chroma plane class.</param>
/// <returns>The palette size in the range from two through eight.</returns>
public int ReadPaletteSize(int blockSizeContext, Av1PlaneType planeType)
{
ref Av1SymbolReader r = ref this.reader;
Av1Distribution distribution = planeType == Av1PlaneType.Y
? this.paletteYSize[blockSizeContext]
: this.paletteUvSize[blockSizeContext];
return r.ReadSymbol(distribution) + 2;
}
/// <summary>
/// Reads a palette color-order index from the selected spatial context.
/// </summary>
/// <param name="paletteSize">The number of colors in the palette.</param>
/// <param name="colorContext">The color-index context derived from decoded neighboring indices.</param>
/// <param name="planeType">The luma or chroma plane class.</param>
/// <returns>The decoded index into the context-specific color order.</returns>
public int ReadPaletteColorIndex(int paletteSize, int colorContext, Av1PlaneType planeType)
{
ref Av1SymbolReader r = ref this.reader;
Av1Distribution distribution = planeType == Av1PlaneType.Y
? this.paletteYColorIndex[paletteSize - 2][colorContext]
: this.paletteUvColorIndex[paletteSize - 2][colorContext];
return r.ReadSymbol(distribution);
}
/// <summary>
/// Reads the frame-local intra-block-copy flag.
/// </summary>

20
src/ImageSharp/Formats/Heif/Av1/Prediction/Av1PredictionDecoder.cs
View File

@ -378,8 +378,24 @@ internal class Av1PredictionDecoder
if (usePalette)
{
// Palette blocks use their decoded color-index map rather than neighboring-sample
// prediction. The tile parser currently rejects palette syntax before this point.
ReadOnlySpan<ushort> paletteColors = modeInfo.GetPaletteColors(plane);
ReadOnlySpan<byte> colorIndexMap = modeInfo.GetPaletteColorIndexMap(plane);
int paletteStride = partitionInfo.WidthInPixels[(int)plane];
int mapOffset = ((blockModeInfoRowOffset << Av1Constants.ModeInfoSizeLog2) * paletteStride) +
(blockModeInfoColumnOffset << Av1Constants.ModeInfoSizeLog2);
// Every transform reconstructs its own window of the block-level palette map. Keeping the map padded to
// the coded block dimensions lets edge transforms use the same addressing rule as interior transforms.
for (int row = 0; row < transformHeight; row++)
{
Span<T> destinationRow = pixelBuffer.Slice(row * pixelBufferStride, transformWidth);
ReadOnlySpan<byte> mapRow = colorIndexMap.Slice(mapOffset + (row * paletteStride), transformWidth);
for (int column = 0; column < transformWidth; column++)
{
destinationRow[column] = T.CreateChecked(paletteColors[mapRow[column]]);
}
}
return;
}

41
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1BlockModeInfo.cs
View File

@ -15,6 +15,16 @@ internal class Av1BlockModeInfo
/// </summary>
private int[] paletteSize;
/// <summary>
/// Stores the decoded palette colors for the Y, U, and V planes.
/// </summary>
private readonly ushort[][] paletteColors = [[], [], []];
/// <summary>
/// Stores the luma and shared chroma palette color-index maps.
/// </summary>
private readonly byte[][] paletteColorIndexMaps = [[], []];
/// <summary>
/// Initializes a new instance of the <see cref="Av1BlockModeInfo"/> class.
/// </summary>
@ -127,4 +137,35 @@ internal class Av1BlockModeInfo
/// <param name="ySize">The luma palette size.</param>
/// <param name="uvSize">The palette size shared by the chroma planes.</param>
public void SetPaletteSizes(int ySize, int uvSize) => this.paletteSize = [ySize, uvSize];
/// <summary>
/// Gets the decoded palette colors for a color plane.
/// </summary>
/// <param name="plane">The color plane.</param>
/// <returns>The palette colors in prediction-index order.</returns>
public ReadOnlySpan<ushort> GetPaletteColors(Av1Plane plane) => this.paletteColors[(int)plane];
/// <summary>
/// Stores the decoded palette colors for a color plane.
/// </summary>
/// <param name="plane">The color plane.</param>
/// <param name="colors">The palette colors in prediction-index order.</param>
public void SetPaletteColors(Av1Plane plane, ReadOnlySpan<ushort> colors)
=> this.paletteColors[(int)plane] = colors.ToArray();
/// <summary>
/// Gets the palette color-index map for a color plane.
/// </summary>
/// <param name="plane">The color plane.</param>
/// <returns>The luma map for <see cref="Av1Plane.Y"/> or the shared chroma map for either chroma plane.</returns>
public ReadOnlySpan<byte> GetPaletteColorIndexMap(Av1Plane plane)
=> this.paletteColorIndexMaps[Math.Min(1, (int)plane)];
/// <summary>
/// Stores the palette color-index map for a plane class.
/// </summary>
/// <param name="planeType">The luma or shared chroma plane class.</param>
/// <param name="colorIndexMap">The row-major color-index map including coded-block edge padding.</param>
public void SetPaletteColorIndexMap(Av1PlaneType planeType, byte[] colorIndexMap)
=> this.paletteColorIndexMaps[(int)planeType] = colorIndexMap;
}

10
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ParseAboveNeighbor4x4Context.cs
View File

@ -7,7 +7,7 @@ using SixLabors.ImageSharp.Formats.Heif.Av1.Transform;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
/// <summary>
/// Stores entropy, palette, partition, and transform contexts for 4-by-4 blocks above the current block.
/// Stores entropy, partition, and transform contexts for 4-by-4 blocks above the current block.
/// </summary>
internal class Av1ParseAboveNeighbor4x4Context
{
@ -21,11 +21,6 @@ internal class Av1ParseAboveNeighbor4x4Context
/// </summary>
private readonly int[] aboveSegmentIdPredictionContext;
/// <summary>
/// Stores palette base colors for each plane above the current block.
/// </summary>
private readonly int[][] abovePaletteColors = new int[Av1Constants.MaxPlanes][];
/// <summary>
/// Stores compound-reference group contexts from the preceding 4x4 row.
/// </summary>
@ -38,13 +33,11 @@ internal class Av1ParseAboveNeighbor4x4Context
/// <param name="modeInfoColumnCount">The frame width in 4x4 mode-information columns.</param>
public Av1ParseAboveNeighbor4x4Context(int planesCount, int modeInfoColumnCount)
{
int wide64x64Count = Av1BlockSize.Block64x64.Get4x4WideCount();
this.AboveTransformWidth = new int[modeInfoColumnCount];
this.AbovePartitionWidth = new int[modeInfoColumnCount];
for (int i = 0; i < planesCount; i++)
{
this.aboveContext[i] = new int[modeInfoColumnCount];
this.abovePaletteColors[i] = new int[wide64x64Count * Av1Constants.PaletteMaxSize];
}
this.aboveSegmentIdPredictionContext = new int[modeInfoColumnCount];
@ -83,7 +76,6 @@ internal class Av1ParseAboveNeighbor4x4Context
for (int i = 0; i < planeCount; i++)
{
Array.Fill(this.aboveContext[i], 0, 0, width);
Array.Fill(this.abovePaletteColors[i], 0, 0, width);
}
Array.Fill(this.aboveSegmentIdPredictionContext, 0, 0, width);

9
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1ParseLeftNeighbor4x4Context.cs
View File

@ -7,7 +7,7 @@ using SixLabors.ImageSharp.Formats.Heif.Av1.Transform;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
/// <summary>
/// Stores entropy, palette, partition, and transform contexts for 4-by-4 blocks left of the current block.
/// Stores entropy, partition, and transform contexts for 4-by-4 blocks left of the current block.
/// </summary>
internal class Av1ParseLeftNeighbor4x4Context
{
@ -21,11 +21,6 @@ internal class Av1ParseLeftNeighbor4x4Context
/// </summary>
private readonly int[] leftSegmentIdPredictionContext;
/// <summary>
/// Stores palette base colors for each plane left of the current block.
/// </summary>
private readonly int[][] leftPaletteColors = new int[Av1Constants.MaxPlanes][];
/// <summary>
/// Stores compound-reference group contexts for the current superblock row.
/// </summary>
@ -43,7 +38,6 @@ internal class Av1ParseLeftNeighbor4x4Context
for (int i = 0; i < planesCount; i++)
{
this.leftContext[i] = new int[superblockModeInfoSize];
this.leftPaletteColors[i] = new int[superblockModeInfoSize * Av1Constants.PaletteMaxSize];
}
this.leftSegmentIdPredictionContext = new int[superblockModeInfoSize];
@ -76,7 +70,6 @@ internal class Av1ParseLeftNeighbor4x4Context
for (int i = 0; i < planeCount; i++)
{
Array.Fill(this.leftContext[i], 0, 0, blockCount);
Array.Fill(this.leftPaletteColors[i], 0, 0, blockCount);
}
Array.Fill(this.leftSegmentIdPredictionContext, 0, 0, blockCount);

520
src/ImageSharp/Formats/Heif/Av1/Tiling/Av1TileReader.cs
View File

@ -44,6 +44,11 @@ internal class Av1TileReader : IAv1TileReader
private static readonly int[][] SkipContexts = [
[1, 2, 2, 2, 3], [1, 4, 4, 4, 5], [1, 4, 4, 4, 5], [1, 4, 4, 4, 5], [1, 4, 4, 4, 6]];
/// <summary>
/// Maps the weighted palette-neighbor score hash to its color-index entropy context.
/// </summary>
private static readonly int[] PaletteColorIndexContexts = [-1, -1, 0, -1, -1, 4, 3, 2, 1];
/// <summary>
/// Stores the preceding self-guided restoration coefficients for each color plane.
/// </summary>
@ -55,12 +60,12 @@ internal class Av1TileReader : IAv1TileReader
private int[][][] referenceLrWiener = [];
/// <summary>
/// Tracks entropy, partition, transform, and palette state above the current block.
/// Tracks entropy, partition, and transform state above the current block.
/// </summary>
private readonly Av1ParseAboveNeighbor4x4Context aboveNeighborContext;
/// <summary>
/// Tracks entropy, partition, transform, and palette state left of the current block.
/// Tracks entropy, partition, and transform state left of the current block.
/// </summary>
private readonly Av1ParseLeftNeighbor4x4Context leftNeighborContext;
@ -440,7 +445,7 @@ internal class Av1TileReader : IAv1TileReader
partitionInfo.PopulateModeInfoNeighbors(this.SequenceHeader.ColorConfig);
this.ReadModeInfo(ref reader, partitionInfo);
ReadPaletteTokens(ref reader, partitionInfo);
this.ReadPaletteTokens(ref reader, partitionInfo);
this.ReadBlockTransformSize(ref reader, modeInfoLocation, partitionInfo, superblockInfo, tileInfo);
if (partitionInfo.ModeInfo.Skip)
{
@ -1135,18 +1140,54 @@ internal class Av1TileReader : IAv1TileReader
/// </summary>
/// <param name="reader">The tile symbol decoder.</param>
/// <param name="partitionInfo">The current coding block.</param>
/// <exception cref="NotImplementedException">The block selects a nonempty luma or chroma palette.</exception>
/// <remarks>Implements AV1 section 5.11.49.</remarks>
private static void ReadPaletteTokens(ref Av1SymbolDecoder reader, Av1PartitionInfo partitionInfo)
private void ReadPaletteTokens(ref Av1SymbolDecoder reader, Av1PartitionInfo partitionInfo)
{
if (partitionInfo.ModeInfo.GetPaletteSize(Av1PlaneType.Y) != 0)
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo;
if (modeInfo.GetPaletteSize(Av1PlaneType.Y) != 0)
{
throw new NotImplementedException();
GetPaletteMapDimensions(
partitionInfo,
Av1PlaneType.Y,
this.SequenceHeader.ColorConfig,
out int planeWidth,
out int planeHeight,
out int rows,
out int columns);
byte[] colorIndexMap = DecodePaletteColorMap(
ref reader,
modeInfo.GetPaletteSize(Av1PlaneType.Y),
Av1PlaneType.Y,
planeWidth,
planeHeight,
rows,
columns);
modeInfo.SetPaletteColorIndexMap(Av1PlaneType.Y, colorIndexMap);
}
if (partitionInfo.ModeInfo.GetPaletteSize(Av1PlaneType.Uv) != 0)
if (modeInfo.GetPaletteSize(Av1PlaneType.Uv) != 0)
{
throw new NotImplementedException();
GetPaletteMapDimensions(
partitionInfo,
Av1PlaneType.Uv,
this.SequenceHeader.ColorConfig,
out int planeWidth,
out int planeHeight,
out int rows,
out int columns);
byte[] colorIndexMap = DecodePaletteColorMap(
ref reader,
modeInfo.GetPaletteSize(Av1PlaneType.Uv),
Av1PlaneType.Uv,
planeWidth,
planeHeight,
rows,
columns);
modeInfo.SetPaletteColorIndexMap(Av1PlaneType.Uv, colorIndexMap);
}
}
@ -1293,10 +1334,467 @@ internal class Av1TileReader : IAv1TileReader
/// </summary>
/// <param name="reader">The tile symbol decoder.</param>
/// <param name="partitionInfo">The current coding block.</param>
/// <exception cref="NotImplementedException">Palette-mode syntax is not implemented.</exception>
/// <remarks>Implements AV1 section 5.11.46.</remarks>
private void PaletteModeInfo(ref Av1SymbolDecoder reader, Av1PartitionInfo partitionInfo)
=> throw new NotImplementedException();
{
Av1BlockModeInfo modeInfo = partitionInfo.ModeInfo;
Av1BlockSize blockSize = modeInfo.BlockSize;
// The palette block-size context is the base-two block-area difference from an 8-by-8 block.
int blockSizeContext = Av1Math.Log2(blockSize.GetWidth() * blockSize.GetHeight()) - 6;
int yPaletteSize = 0;
int uvPaletteSize = 0;
int bitDepth = this.SequenceHeader.ColorConfig.BitDepth.GetBitCount();
if (modeInfo.YMode == Av1PredictionMode.DC)
{
int neighborContext = 0;
if (partitionInfo.AboveModeInfo is not null && partitionInfo.AboveModeInfo.GetPaletteSize(Av1PlaneType.Y) != 0)
{
neighborContext++;
}
if (partitionInfo.LeftModeInfo is not null && partitionInfo.LeftModeInfo.GetPaletteSize(Av1PlaneType.Y) != 0)
{
neighborContext++;
}
if (reader.ReadPaletteYMode(blockSizeContext, neighborContext))
{
yPaletteSize = reader.ReadPaletteSize(blockSizeContext, Av1PlaneType.Y);
Span<ushort> yColors = stackalloc ushort[Av1Constants.PaletteMaxSize];
ReadPaletteColorsY(ref reader, partitionInfo, yPaletteSize, bitDepth, yColors);
modeInfo.SetPaletteColors(Av1Plane.Y, yColors[..yPaletteSize]);
}
}
if (this.SequenceHeader.ColorConfig.PlaneCount > 1 &&
modeInfo.UvMode == Av1PredictionMode.DC &&
partitionInfo.IsChroma &&
reader.ReadPaletteUvMode(yPaletteSize != 0))
{
uvPaletteSize = reader.ReadPaletteSize(blockSizeContext, Av1PlaneType.Uv);
Span<ushort> uColors = stackalloc ushort[Av1Constants.PaletteMaxSize];
Span<ushort> vColors = stackalloc ushort[Av1Constants.PaletteMaxSize];
ReadPaletteColorsUv(ref reader, partitionInfo, uvPaletteSize, bitDepth, uColors, vColors);
modeInfo.SetPaletteColors(Av1Plane.U, uColors[..uvPaletteSize]);
modeInfo.SetPaletteColors(Av1Plane.V, vColors[..uvPaletteSize]);
}
modeInfo.SetPaletteSizes(yPaletteSize, uvPaletteSize);
}
/// <summary>
/// Reads the sorted luma palette colors, including selections from neighboring palette caches.
/// </summary>
/// <param name="reader">The tile symbol decoder.</param>
/// <param name="partitionInfo">The current coding block and its palette neighbors.</param>
/// <param name="paletteSize">The number of luma palette colors.</param>
/// <param name="bitDepth">The number of bits in each color sample.</param>
/// <param name="colors">The destination palette-color buffer.</param>
private static void ReadPaletteColorsY(
ref Av1SymbolDecoder reader,
Av1PartitionInfo partitionInfo,
int paletteSize,
int bitDepth,
scoped Span<ushort> colors)
{
Span<ushort> colorCache = stackalloc ushort[Av1Constants.PaletteMaxSize * 2];
Span<ushort> cachedColors = stackalloc ushort[Av1Constants.PaletteMaxSize];
int cacheSize = GetPaletteCache(partitionInfo, Av1Plane.Y, colorCache);
int colorIndex = 0;
for (int i = 0; i < cacheSize && colorIndex < paletteSize; i++)
{
if (reader.ReadLiteral(1) != 0)
{
cachedColors[colorIndex++] = colorCache[i];
}
}
if (colorIndex == paletteSize)
{
cachedColors[..paletteSize].CopyTo(colors);
return;
}
int cachedColorCount = colorIndex;
colors[colorIndex++] = (ushort)reader.ReadLiteral(bitDepth);
if (colorIndex < paletteSize)
{
int bits = bitDepth - 3 + reader.ReadLiteral(2);
int maximumColor = (1 << bitDepth) - 1;
int range = maximumColor - colors[colorIndex - 1];
for (; colorIndex < paletteSize; colorIndex++)
{
int delta = reader.ReadLiteral(bits) + 1;
colors[colorIndex] = (ushort)Av1Math.Clip3(0, maximumColor, colors[colorIndex - 1] + delta);
range -= colors[colorIndex] - colors[colorIndex - 1];
bits = Math.Min(bits, (int)Av1Math.CeilLog2((uint)range));
}
}
MergePaletteColors(colors, cachedColors, paletteSize, cachedColorCount);
}
/// <summary>
/// Reads the U and V palette colors, including neighboring U colors and optional V delta coding.
/// </summary>
/// <param name="reader">The tile symbol decoder.</param>
/// <param name="partitionInfo">The current coding block and its palette neighbors.</param>
/// <param name="paletteSize">The number of chroma palette colors.</param>
/// <param name="bitDepth">The number of bits in each color sample.</param>
/// <param name="uColors">The destination U palette-color buffer.</param>
/// <param name="vColors">The destination V palette-color buffer.</param>
private static void ReadPaletteColorsUv(
ref Av1SymbolDecoder reader,
Av1PartitionInfo partitionInfo,
int paletteSize,
int bitDepth,
scoped Span<ushort> uColors,
scoped Span<ushort> vColors)
{
Span<ushort> colorCache = stackalloc ushort[Av1Constants.PaletteMaxSize * 2];
Span<ushort> cachedColors = stackalloc ushort[Av1Constants.PaletteMaxSize];
int cacheSize = GetPaletteCache(partitionInfo, Av1Plane.U, colorCache);
int colorIndex = 0;
for (int i = 0; i < cacheSize && colorIndex < paletteSize; i++)
{
if (reader.ReadLiteral(1) != 0)
{
cachedColors[colorIndex++] = colorCache[i];
}
}
if (colorIndex < paletteSize)
{
int cachedColorCount = colorIndex;
uColors[colorIndex++] = (ushort)reader.ReadLiteral(bitDepth);
if (colorIndex < paletteSize)
{
int bits = bitDepth - 3 + reader.ReadLiteral(2);
int maximumColor = (1 << bitDepth) - 1;
int range = (1 << bitDepth) - uColors[colorIndex - 1];
for (; colorIndex < paletteSize; colorIndex++)
{
int delta = reader.ReadLiteral(bits);
uColors[colorIndex] = (ushort)Av1Math.Clip3(0, maximumColor, uColors[colorIndex - 1] + delta);
range -= uColors[colorIndex] - uColors[colorIndex - 1];
bits = Math.Min(bits, (int)Av1Math.CeilLog2((uint)range));
}
}
MergePaletteColors(uColors, cachedColors, paletteSize, cachedColorCount);
}
else
{
cachedColors[..paletteSize].CopyTo(uColors);
}
if (reader.ReadLiteral(1) != 0)
{
// V deltas wrap in the unsigned sample domain so complementary chroma colors remain compact.
int bits = bitDepth - 4 + reader.ReadLiteral(2);
int maximumColorPlusOne = 1 << bitDepth;
vColors[0] = (ushort)reader.ReadLiteral(bitDepth);
for (int i = 1; i < paletteSize; i++)
{
int delta = reader.ReadLiteral(bits);
if (delta != 0 && reader.ReadLiteral(1) != 0)
{
delta = -delta;
}
int value = vColors[i - 1] + delta;
if (value < 0)
{
value += maximumColorPlusOne;
}
if (value >= maximumColorPlusOne)
{
value -= maximumColorPlusOne;
}
vColors[i] = (ushort)value;
}
}
else
{
for (int i = 0; i < paletteSize; i++)
{
vColors[i] = (ushort)reader.ReadLiteral(bitDepth);
}
}
}
/// <summary>
/// Builds the sorted unique palette cache from the available above and left block palettes.
/// </summary>
/// <param name="partitionInfo">The current coding block and its decoded neighbors.</param>
/// <param name="plane">The luma or U plane whose sorted base colors form the cache.</param>
/// <param name="cache">The destination cache, which can hold both neighboring palettes.</param>
/// <returns>The number of colors written to <paramref name="cache"/>.</returns>
private static int GetPaletteCache(Av1PartitionInfo partitionInfo, Av1Plane plane, Span<ushort> cache)
{
// AV1 deliberately excludes the block above at a 64-by-64 superblock-row boundary.
int minimumSuperblockHeight = Av1BlockSize.Block64x64.Get4x4HighCount();
Av1BlockModeInfo? aboveModeInfo = partitionInfo.RowIndex % minimumSuperblockHeight == 0
? null
: partitionInfo.AboveModeInfo;
Av1BlockModeInfo? leftModeInfo = partitionInfo.LeftModeInfo;
int abovePaletteSize = aboveModeInfo?.GetPaletteSize(plane) ?? 0;
int leftPaletteSize = leftModeInfo?.GetPaletteSize(plane) ?? 0;
ReadOnlySpan<ushort> aboveColors = aboveModeInfo is null ? [] : aboveModeInfo.GetPaletteColors(plane);
ReadOnlySpan<ushort> leftColors = leftModeInfo is null ? [] : leftModeInfo.GetPaletteColors(plane);
int aboveIndex = 0;
int leftIndex = 0;
int count = 0;
while (aboveIndex < abovePaletteSize && leftIndex < leftPaletteSize)
{
ushort aboveColor = aboveColors[aboveIndex];
ushort leftColor = leftColors[leftIndex];
if (leftColor < aboveColor)
{
AddPaletteCacheColor(cache, ref count, leftColor);
leftIndex++;
}
else
{
AddPaletteCacheColor(cache, ref count, aboveColor);
aboveIndex++;
if (leftColor == aboveColor)
{
leftIndex++;
}
}
}
while (aboveIndex < abovePaletteSize)
{
AddPaletteCacheColor(cache, ref count, aboveColors[aboveIndex++]);
}
while (leftIndex < leftPaletteSize)
{
AddPaletteCacheColor(cache, ref count, leftColors[leftIndex++]);
}
return count;
}
/// <summary>
/// Appends a palette cache color unless it duplicates the preceding sorted value.
/// </summary>
/// <param name="cache">The sorted cache being populated.</param>
/// <param name="count">The number of colors currently stored.</param>
/// <param name="color">The next sorted color.</param>
private static void AddPaletteCacheColor(Span<ushort> cache, ref int count, ushort color)
{
if (count == 0 || cache[count - 1] != color)
{
cache[count++] = color;
}
}
/// <summary>
/// Merges selected cached colors with the sorted transmitted colors in one prediction-order palette.
/// </summary>
/// <param name="colors">The transmitted colors beginning at <paramref name="cachedColorCount"/> and the merged output.</param>
/// <param name="cachedColors">The selected cached colors in ascending order.</param>
/// <param name="paletteSize">The total palette size.</param>
/// <param name="cachedColorCount">The number of selected cached colors.</param>
private static void MergePaletteColors(Span<ushort> colors, ReadOnlySpan<ushort> cachedColors, int paletteSize, int cachedColorCount)
{
if (cachedColorCount == 0)
{
return;
}
int cacheIndex = 0;
int transmittedIndex = cachedColorCount;
for (int i = 0; i < paletteSize; i++)
{
if (cacheIndex < cachedColorCount &&
(transmittedIndex >= paletteSize || cachedColors[cacheIndex] <= colors[transmittedIndex]))
{
colors[i] = cachedColors[cacheIndex++];
}
else
{
colors[i] = colors[transmittedIndex++];
}
}
}
/// <summary>
/// Computes the padded plane dimensions and the portion that lies inside the coded image.
/// </summary>
/// <param name="partitionInfo">The current coding block and frame-edge distances.</param>
/// <param name="planeType">The luma or shared chroma plane class.</param>
/// <param name="colorConfig">The sequence chroma-subsampling configuration.</param>
/// <param name="planeWidth">The padded plane-block width in samples.</param>
/// <param name="planeHeight">The padded plane-block height in samples.</param>
/// <param name="rows">The number of plane-block rows inside the coded image.</param>
/// <param name="columns">The number of plane-block columns inside the coded image.</param>
private static void GetPaletteMapDimensions(
Av1PartitionInfo partitionInfo,
Av1PlaneType planeType,
ObuColorConfig colorConfig,
out int planeWidth,
out int planeHeight,
out int rows,
out int columns)
{
int subX = planeType == Av1PlaneType.Uv && colorConfig.SubSamplingX ? 1 : 0;
int subY = planeType == Av1PlaneType.Uv && colorConfig.SubSamplingY ? 1 : 0;
int blockWidth = partitionInfo.ModeInfo.BlockSize.GetWidth();
int blockHeight = partitionInfo.ModeInfo.BlockSize.GetHeight();
int columnsInsideImage = partitionInfo.ModeBlockToRightEdge >= 0
? blockWidth
: blockWidth + (partitionInfo.ModeBlockToRightEdge >> 3);
int rowsInsideImage = partitionInfo.ModeBlockToBottomEdge >= 0
? blockHeight
: blockHeight + (partitionInfo.ModeBlockToBottomEdge >> 3);
planeWidth = blockWidth >> subX;
planeHeight = blockHeight >> subY;
columns = columnsInsideImage >> subX;
rows = rowsInsideImage >> subY;
}
/// <summary>
/// Decodes a padded palette color-index map in AV1 diagonal wavefront order.
/// </summary>
/// <param name="reader">The tile symbol decoder.</param>
/// <param name="paletteSize">The number of palette colors.</param>
/// <param name="planeType">The luma or shared chroma plane class.</param>
/// <param name="planeWidth">The padded plane-block width.</param>
/// <param name="planeHeight">The padded plane-block height.</param>
/// <param name="rows">The number of rows inside the coded image.</param>
/// <param name="columns">The number of columns inside the coded image.</param>
/// <returns>The decoded row-major color-index map.</returns>
private static byte[] DecodePaletteColorMap(
ref Av1SymbolDecoder reader,
int paletteSize,
Av1PlaneType planeType,
int planeWidth,
int planeHeight,
int rows,
int columns)
{
byte[] colorIndexMap = new byte[planeWidth * planeHeight];
colorIndexMap[0] = (byte)reader.ReadUniform(paletteSize);
Span<byte> colorOrder = stackalloc byte[Av1Constants.PaletteMaxSize];
for (int diagonal = 1; diagonal < rows + columns - 1; diagonal++)
{
int firstColumn = Math.Min(diagonal, columns - 1);
int lastColumn = Math.Max(0, diagonal - rows + 1);
for (int column = firstColumn; column >= lastColumn; column--)
{
int row = diagonal - column;
int colorContext = GetPaletteColorIndexContext(
colorIndexMap,
planeWidth,
row,
column,
paletteSize,
colorOrder);
int colorOrderIndex = reader.ReadPaletteColorIndex(paletteSize, colorContext, planeType);
colorIndexMap[(row * planeWidth) + column] = colorOrder[colorOrderIndex];
}
}
if (columns < planeWidth)
{
// Blocks clipped by the right image edge repeat their final coded column into the padded block area.
for (int row = 0; row < rows; row++)
{
int rowOffset = row * planeWidth;
colorIndexMap.AsSpan(rowOffset + columns, planeWidth - columns)
.Fill(colorIndexMap[rowOffset + columns - 1]);
}
}
// Blocks clipped by the bottom image edge repeat their final coded row for later transform reconstruction.
ReadOnlySpan<byte> finalRow = colorIndexMap.AsSpan((rows - 1) * planeWidth, planeWidth);
for (int row = rows; row < planeHeight; row++)
{
finalRow.CopyTo(colorIndexMap.AsSpan(row * planeWidth, planeWidth));
}
return colorIndexMap;
}
/// <summary>
/// Derives the palette color order and entropy context from the left, upper-left, and above indices.
/// </summary>
/// <param name="colorIndexMap">The partially decoded color-index map.</param>
/// <param name="stride">The map row stride.</param>
/// <param name="row">The current map row.</param>
/// <param name="column">The current map column.</param>
/// <param name="paletteSize">The number of palette colors.</param>
/// <param name="colorOrder">The destination color order for the current context.</param>
/// <returns>The color-index entropy context in the range from zero through four.</returns>
private static int GetPaletteColorIndexContext(
ReadOnlySpan<byte> colorIndexMap,
int stride,
int row,
int column,
int paletteSize,
Span<byte> colorOrder)
{
Span<int> neighborColors = stackalloc int[3];
neighborColors[0] = column > 0 ? colorIndexMap[(row * stride) + column - 1] : -1;
neighborColors[1] = column > 0 && row > 0 ? colorIndexMap[((row - 1) * stride) + column - 1] : -1;
neighborColors[2] = row > 0 ? colorIndexMap[((row - 1) * stride) + column] : -1;
Span<int> scores = stackalloc int[Av1Constants.PaletteMaxSize];
ReadOnlySpan<int> neighborWeights = [2, 1, 2];
for (int i = 0; i < neighborColors.Length; i++)
{
if (neighborColors[i] >= 0)
{
scores[neighborColors[i]] += neighborWeights[i];
}
}
for (int i = 0; i < colorOrder.Length; i++)
{
colorOrder[i] = (byte)i;
}
// Stable descending score order keeps lower palette indices ahead when neighboring scores tie.
for (int i = 0; i < 3; i++)
{
int maximumScore = scores[i];
int maximumIndex = i;
for (int j = i + 1; j < paletteSize; j++)
{
if (scores[j] > maximumScore)
{
maximumScore = scores[j];
maximumIndex = j;
}
}
if (maximumIndex != i)
{
byte maximumColor = colorOrder[maximumIndex];
for (int j = maximumIndex; j > i; j--)
{
scores[j] = scores[j - 1];
colorOrder[j] = colorOrder[j - 1];
}
scores[i] = maximumScore;
colorOrder[i] = maximumColor;
}
}
int contextHash = scores[0] + (2 * scores[1]) + (2 * scores[2]);
return PaletteColorIndexContexts[contextHash];
}
/// <summary>
/// Reads the joint signs and nonzero alpha magnitudes for chroma-from-luma prediction.

Write Preview
Loading…
Cancel
Save