tsai
/
ImageSharp
mirror of https://github.com/SixLabors/ImageSharp
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
Browse Source

Evaluate AV1 inter modes before intra candidates

pull/2633/head
James Jackson-South 4 weeks ago
parent
6716c8e9ff
commit
26c86f7ff3
3 changed files with 200 additions and 207 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 15
      HEIF_IMPLEMENTATION_PLAN.md
  2. 165
      src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ModeDecision.cs
  3. 227
      src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ReferenceModeDecision.cs

15
HEIF_IMPLEMENTATION_PLAN.md
View File

@ -23,6 +23,19 @@ maximum error zero, differing samples zero, and samples exceeding one zero. Temp
uncommitted Hadamard-screening experiment remain excluded. Decoder-wide coverage and performance acceptance
remain open; these results verify the recorded cases and do not establish complete codec conformance.
The inter-mode ordering checkpoint evaluates inter candidates before intra and retains only the winning syntax
and transform choices across those trials. Final inter reconstruction regenerates the selected transforms using
existing scratch storage; empty planes retain prediction without re-quantization. Equal-cost intra candidates
do not replace the preceding inter winner. The source basis is `av1/encoder/rdopt.c:6355-6399,6491-6494`
and `av1/encoder/partition_search.c:474-503`. This corrects ordering and winner publication, but does not implement
the reference's full intra gating, TPL decisions, winner refinement, larger inter partitions, or additional references.
After the final code edit, Release .NET 11 built with zero errors and zero warnings; serialized VSTest passed
379/379 focused encoder, superblock, intra-copy, and public HEIF cases (`inter-first-r2.trx`, 41.5269 seconds).
Fresh optimized-native comparisons matched 86,859 samples across 39 motion streams and 477,243 samples across
three larger row-refresh streams. Encoder reconstruction also matched native decoding over 102,390 samples in
12 deblocking streams. Each comparison reported maximum error zero, differing samples zero, and counts exceeding
one zero. These checks establish the recorded same-bitstream behavior, not separate-encoder parity or performance.
Acceptance criteria are separate for encoding and decoding:
- Encoder parity permits at most one component unit per sample when comparing separately encoded results
@ -331,7 +344,7 @@ Line numbers describe the inspected starting tree, before subsequent corrections
| Missing functionality | `Av1FrameEncoder.cs:372-408`, under `src/ImageSharp/Formats/Heif/Av1/Pipeline` | `av1/encoder/encoder.c:641-646`; `av1/av1_cx_iface.c:287-288,1284-1286,1561-1562` | Sequence setup unconditionally disables CDEF, restoration, and intra-edge filtering. These are not equivalent to the reference's configured tool decisions. |
| Architectural deviation | `Av1FrameEncoder.cs:508-542,1551-1557` | `av1/encoder/encode_strategy.c:168-230,1664-1669` | Every frame is error resilient, refreshes all slots, disables frame-end CDF publication, and resets probabilities. The reference selects retained primary-reference state. |
| Missing functionality | `Av1IntraSuperblockEncoder.ModeDecision.cs:185-245`; `Av1IntraSuperblockEncoder.ReferenceModeDecision.cs:526-566` | `av1/encoder/partition_search.c:3320` onward; `av1/encoder/rdopt.c:6196-6236` | Inter frames retain a fixed 8x8 partition tree and search only LAST. Larger partitions and additional reference roles are not implemented by this path. |
| Architectural deviation | `Av1IntraSuperblockEncoder.ModeDecision.cs:597-609,662-672,824-834` | `av1/encoder/rdopt.c:111-142,6186-6236`; `av1/encoder/intra_mode_search.c:1291-1344` | Managed coding finishes intra search before inter evaluation. Reference inter search has its own ordered candidates, pruning state, bounds, and later intra evaluation. |
| Architectural deviation, partly corrected | `Av1IntraSuperblockEncoder.ModeDecision.cs:606-753` | `av1/encoder/rdopt.c:6355-6399,6491-6494`; `av1/encoder/partition_search.c:474-503` | Inter now precedes intra and the final winner is reconstructed after competition. Full intra gating, pruning state, bounds, and winner refinement remain incomplete. |
| Architectural deviation | `Av1IntraSuperblockEncoder.ReferenceModeDecision.cs:576-617` | `av1/encoder/rdopt.c:111-142` | Managed single-reference mode order is NEAREST, NEAR, GLOBAL, NEW. The reference default order is NEAREST, NEW, NEAR, GLOBAL across eligible references. |
| Architectural deviation | `Av1IntraSuperblockEncoder.ReferenceModeDecision.cs:1278-1414` | `av1/encoder/mcomp.c`; caller policy in `av1/encoder/motion_search_facade.c` | The managed radius is an effort-shifted value capped by its border; each scale visits eight offsets once. This is a simplified search controller whose full reference-policy reconciliation remains open. |
| Missing functionality | `Av1TransformBlockEncoder.cs:1047-1097` | `av1/encoder/encodemb.c:842-885` | Lossy transform coding ends at fast quantization. Reference coding selects quantization with trellis policy and can optimize coefficients before reconstruction. Native primitive arithmetic alone does not establish encoder parity. |

165
src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ModeDecision.cs
View File

@ -3,6 +3,7 @@
using System.Runtime.InteropServices;
using SixLabors.ImageSharp.Formats.Heif.Av1.Entropy;
using SixLabors.ImageSharp.Formats.Heif.Av1.Motion;
using SixLabors.ImageSharp.Formats.Heif.Av1.OpenBitstreamUnit;
using SixLabors.ImageSharp.Formats.Heif.Av1.Prediction;
using SixLabors.ImageSharp.Formats.Heif.Av1.Tiling;
@ -602,6 +603,36 @@ internal static partial class Av1IntraSuperblockEncoder
block.QuantizationIndex = qIndex;
block.SegmentId = 0;
bool isInterFrame = !this.picture.Parent.FrameHeader.IsIntra;
Av1RateDistortionStatistics interStatistics = Av1RateDistortionStatistics.Invalid;
Av1MacroBlockModeInfo interModeInfo = default;
Av1EncoderBlockStruct interBlock = default;
InlineArray3<Av1EncoderTransformBlockState> interStates = default;
Av1MotionVector interVector = default;
if (isInterFrame)
{
Av1MacroBlockModeInfo initialModeInfo = modeInfo;
Av1EncoderBlockStruct initialBlock = block;
interStatistics = this.SelectInterBlock(
writer,
macroBlock,
blockOrigin,
tileIndex,
ref modeInfo,
ref block,
out interVector,
out interStates);
interModeInfo = modeInfo;
interBlock = block;
// Only the winning syntax and transform choices survive across mode families. Intra trials
// reuse prediction and coefficient scratch; the selected inter block is reconstructed afterward.
modeInfo = initialModeInfo;
block = initialBlock;
paletteInfo = default;
}
Span<int> lumaCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.Y);
Span<Av1EncoderTransformBlockState> lumaTransformBlocks =
this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.Y);
@ -628,83 +659,39 @@ internal static partial class Av1IntraSuperblockEncoder
block.FilterIntraMode = filterIntraMode;
modeInfo.Block.TransformSize = lumaTransformSize;
// Ordinary intra keeps the block non-skipped, including when all transforms are empty. Its RD cost
// includes those transform symbols and the non-skip flag; only inter or IBC winners can replace this state.
if (this.source.IsMonochrome)
int chromaArea = 0;
if (block.HasChroma)
{
bool allowIntraBlockCopy = blockSize == Av1BlockSize.Block8x8 &&
this.picture.Parent.FrameHeader.AllowIntraBlockCopy;
Av1RateDistortionStatistics regularStatistics = this.GetRegularBlockCost(
writer,
macroBlock,
lumaStatistics,
allowIntraBlockCopy);
if (!this.picture.Parent.FrameHeader.IsIntra)
{
this.SelectedBlockStatistics = this.SelectInterPrediction(
writer,
macroBlock,
blockOrigin,
tileIndex,
regularStatistics,
ref modeInfo,
ref block,
ref paletteInfo);
}
else
{
this.SelectedBlockStatistics = allowIntraBlockCopy
? this.SelectIntraBlockCopy(
writer,
macroBlock,
blockOrigin,
tileIndex,
regularStatistics,
ref modeInfo,
ref block,
ref paletteInfo)
: regularStatistics;
}
this.codedAreaLuma += blockSize.GetWidth() * blockSize.GetHeight();
return;
}
ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig;
int subsamplingX = colorConfig.SubSamplingX ? 1 : 0;
int subsamplingY = colorConfig.SubSamplingY ? 1 : 0;
Point chromaOrigin = Av1TileWriter.GetChromaBlockOrigin(
blockOrigin,
subsamplingX,
subsamplingY);
ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig;
int subsamplingX = colorConfig.SubSamplingX ? 1 : 0;
int subsamplingY = colorConfig.SubSamplingY ? 1 : 0;
Point chromaOrigin = Av1TileWriter.GetChromaBlockOrigin(
blockOrigin,
subsamplingX,
subsamplingY);
Av1TransformSize chromaTransformSize = this.picture.Parent.FrameHeader.CodedLossless
? Av1TransformSize.Size4x4
: blockSize.GetMaxUvTransformSize(
colorConfig.SubSamplingX,
colorConfig.SubSamplingY);
Av1TransformSize chromaTransformSize = this.picture.Parent.FrameHeader.CodedLossless
? Av1TransformSize.Size4x4
: blockSize.GetMaxUvTransformSize(
Av1BlockSize chromaBlockSize = blockSize.GetSubsampled(
colorConfig.SubSamplingX,
colorConfig.SubSamplingY);
Av1BlockSize chromaBlockSize = blockSize.GetSubsampled(
colorConfig.SubSamplingX,
colorConfig.SubSamplingY);
Span<int> blueCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.U);
Span<int> redCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.V);
Span<Av1EncoderTransformBlockState> blueTransformBlocks =
this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.U);
Span<Av1EncoderTransformBlockState> redTransformBlocks =
this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.V);
Span<int> blueCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.U);
Span<int> redCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.V);
Span<Av1EncoderTransformBlockState> blueTransformBlocks =
this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.U);
Span<Av1EncoderTransformBlockState> redTransformBlocks =
this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.V);
int chromaTransformIndex = this.codedAreaChroma /
Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount;
int chromaTransformIndex = this.codedAreaChroma /
Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount;
Span<Av1EncoderTransformBlockState> retainedBlueStates = blueTransformBlocks[chromaTransformIndex..];
Span<Av1EncoderTransformBlockState> retainedRedStates = redTransformBlocks[chromaTransformIndex..];
Av1RateDistortionStatistics chromaStatistics = default;
if (block.HasChroma)
{
Span<Av1EncoderTransformBlockState> retainedBlueStates = blueTransformBlocks[chromaTransformIndex..];
Span<Av1EncoderTransformBlockState> retainedRedStates = redTransformBlocks[chromaTransformIndex..];
modeInfo.Block.UvMode = this.SelectChromaMode(
writer,
macroBlock,
@ -723,55 +710,51 @@ internal static partial class Av1IntraSuperblockEncoder
out int chromaAngleDelta,
out byte chromaFromLumaIndex,
out sbyte chromaFromLumaSigns,
out chromaStatistics);
out Av1RateDistortionStatistics chromaStatistics);
block.PredictionUnit.AngleDelta[(int)Av1PlaneType.Uv] = (sbyte)chromaAngleDelta;
block.PredictionUnit.ChromaFromLumaIndex = chromaFromLumaIndex;
block.PredictionUnit.ChromaFromLumaSigns = chromaFromLumaSigns;
chromaArea = chromaBlockSize.GetWidth() * chromaBlockSize.GetHeight();
lumaStatistics.Add(this.rateMultiplier, in chromaStatistics);
}
bool allowColorIntraBlockCopy = blockSize == Av1BlockSize.Block8x8 &&
bool allowIntraBlockCopy = blockSize == Av1BlockSize.Block8x8 &&
this.picture.Parent.FrameHeader.AllowIntraBlockCopy;
lumaStatistics.Add(this.rateMultiplier, in chromaStatistics);
Av1RateDistortionStatistics regularColorStatistics = this.GetRegularBlockCost(
Av1RateDistortionStatistics regularStatistics = this.GetRegularBlockCost(
writer,
macroBlock,
lumaStatistics,
allowColorIntraBlockCopy);
allowIntraBlockCopy);
if (!this.picture.Parent.FrameHeader.IsIntra)
if (isInterFrame && interStatistics.Cost <= regularStatistics.Cost)
{
this.SelectedBlockStatistics = this.SelectInterPrediction(
writer,
macroBlock,
blockOrigin,
tileIndex,
regularColorStatistics,
ref modeInfo,
ref block,
ref paletteInfo);
// Inter candidates precede intra candidates, so an equal cost retains the inter winner.
modeInfo = interModeInfo;
block = interBlock;
paletteInfo = default;
this.ReconstructSelectedInterBlock(blockOrigin, modeInfo, block, interVector, interStates);
this.picture.SetDisplacementVector(modeInfoPosition, interVector);
this.SelectedBlockStatistics = interStatistics;
}
else
{
this.SelectedBlockStatistics = allowColorIntraBlockCopy
this.SelectedBlockStatistics = allowIntraBlockCopy
? this.SelectIntraBlockCopy(
writer,
macroBlock,
blockOrigin,
tileIndex,
regularColorStatistics,
regularStatistics,
ref modeInfo,
ref block,
ref paletteInfo)
: regularColorStatistics;
: regularStatistics;
}
this.codedAreaLuma += blockSize.GetWidth() * blockSize.GetHeight();
if (block.HasChroma)
{
this.codedAreaChroma += chromaBlockSize.GetWidth() * chromaBlockSize.GetHeight();
}
this.codedAreaChroma += chromaArea;
}
private Av1RateDistortionStatistics EvaluatePartitionLeaf(

227
src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ReferenceModeDecision.cs
View File

@ -14,7 +14,7 @@ using SixLabors.ImageSharp.Memory;
namespace SixLabors.ImageSharp.Formats.Heif.Av1.Pipeline;
/// <content>
/// Provides full rate-distortion selection for reference-frame and intra-block-copy candidates.
/// Provides reference-frame and intra-block-copy mode decisions.
/// </content>
internal static partial class Av1IntraSuperblockEncoder
{
@ -449,53 +449,17 @@ internal static partial class Av1IntraSuperblockEncoder
}
/// <summary>
/// Compares the retained intra result with an inter candidate without disturbing the intra result on loss.
/// </summary>
private Av1RateDistortionStatistics SelectInterPrediction(
Av1SymbolEncoder writer,
Av1MacroBlockD macroBlock,
Point blockOrigin,
ushort tileIndex,
Av1RateDistortionStatistics regularStatistics,
ref Av1MacroBlockModeInfo modeInfo,
ref Av1EncoderBlockStruct block,
ref Av1EncoderPaletteInfo paletteInfo)
{
Av1MacroBlockModeInfo interModeInfo = modeInfo;
Av1EncoderBlockStruct interBlock = block;
Av1EncoderPaletteInfo interPaletteInfo = default;
Av1RateDistortionStatistics selectedStatistics = this.SelectInterBlock(
writer,
macroBlock,
blockOrigin,
tileIndex,
regularStatistics,
ref interModeInfo,
ref interBlock,
ref interPaletteInfo);
if (selectedStatistics.Cost < regularStatistics.Cost)
{
modeInfo = interModeInfo;
block = interBlock;
paletteInfo = interPaletteInfo;
}
return selectedStatistics;
}
/// <summary>
/// Evaluates the supported LAST_FRAME modes and publishes only a strict improvement over the intra result.
/// Evaluates reference-frame modes and retains the winning syntax and transform choices.
/// </summary>
private Av1RateDistortionStatistics SelectInterBlock(
Av1SymbolEncoder writer,
Av1MacroBlockD macroBlock,
Point blockOrigin,
ushort tileIndex,
Av1RateDistortionStatistics regularStatistics,
ref Av1MacroBlockModeInfo modeInfo,
ref Av1EncoderBlockStruct block,
ref Av1EncoderPaletteInfo paletteInfo)
out Av1MotionVector selectedVector,
out InlineArray3<Av1EncoderTransformBlockState> selectedStates)
{
const Av1BlockSize BlockSize = Av1BlockSize.Block8x8;
const Av1TransformSize LumaTransformSize = Av1TransformSize.Size8x8;
@ -511,7 +475,6 @@ internal static partial class Av1IntraSuperblockEncoder
block.PredictionUnit.AngleDelta[(int)Av1PlaneType.Uv] = 0;
block.PredictionUnit.ChromaFromLumaIndex = 0;
block.PredictionUnit.ChromaFromLumaSigns = 0;
paletteInfo = default;
Av1EncoderInterPredictionWorkspace<TSample> workspace =
this.blockWorkspace.GetInterPredictionWorkspace<TSample>();
@ -601,15 +564,15 @@ internal static partial class Av1IntraSuperblockEncoder
transformPartitionRate = writer.GetTransformPartitionCost(false, transformPartitionContext);
}
Av1RateDistortionStatistics selectedStatistics = regularStatistics;
Av1MotionVector selectedVector = default;
Av1RateDistortionStatistics selectedStatistics = Av1RateDistortionStatistics.Invalid;
selectedVector = default;
selectedStates = default;
Av1PredictionMode selectedMode = default;
int selectedReferenceIndex = 0;
bool selectedSkip = false;
Av1EncoderTransformBlockState selectedLumaState = default;
Av1EncoderTransformBlockState selectedBlueState = default;
Av1EncoderTransformBlockState selectedRedState = default;
bool hasInterWinner = false;
ObuSequenceHeader sequenceHeader = this.picture.Sequence.SequenceHeader;
bool isSwitchable = frameHeader.InterpolationFilter == Av1InterpolationFilter.Switchable;
@ -939,88 +902,122 @@ internal static partial class Av1IntraSuperblockEncoder
selectedLumaState = candidateLumaState;
selectedBlueState = candidateBlueState;
selectedRedState = candidateRedState;
hasInterWinner = true;
}
// Inter trials never overwrite retained picture state. The complete intra result remains authoritative
// when no inter candidate strictly improves its rate-distortion cost.
if (!hasInterWinner)
{
return regularStatistics;
}
Span<int> retainedLumaCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.Y);
Span<Av1EncoderTransformBlockState> retainedLumaTransformBlocks =
this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.Y);
int lumaTransformIndex = this.codedAreaLuma /
Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount;
CopyCandidate(
selectedLumaReconstruction,
selectedLumaCoefficients,
this.reconstruction.GetPlane(Av1Plane.Y),
blockOrigin,
retainedLumaCoefficients[this.codedAreaLuma..],
LumaTransformSize,
selectedLumaState,
ref retainedLumaTransformBlocks[lumaTransformIndex]);
if (block.HasChroma)
{
ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig;
int subsamplingX = colorConfig.SubSamplingX ? 1 : 0;
int subsamplingY = colorConfig.SubSamplingY ? 1 : 0;
Point chromaOrigin = Av1TileWriter.GetChromaBlockOrigin(
blockOrigin,
subsamplingX,
subsamplingY);
Av1TransformSize chromaTransformSize = BlockSize.GetMaxUvTransformSize(
colorConfig.SubSamplingX,
colorConfig.SubSamplingY);
Span<int> retainedBlueCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.U);
Span<int> retainedRedCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, Av1Plane.V);
Span<Av1EncoderTransformBlockState> retainedBlueTransformBlocks =
this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.U);
Span<Av1EncoderTransformBlockState> retainedRedTransformBlocks =
this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, Av1Plane.V);
int chromaTransformIndex = this.codedAreaChroma /
Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount;
CopyCandidate(
selectedBlueReconstruction,
selectedBlueCoefficients,
this.reconstruction.GetPlane(Av1Plane.U),
chromaOrigin,
retainedBlueCoefficients[this.codedAreaChroma..],
chromaTransformSize,
selectedBlueState,
ref retainedBlueTransformBlocks[chromaTransformIndex]);
CopyCandidate(
selectedRedReconstruction,
selectedRedCoefficients,
this.reconstruction.GetPlane(Av1Plane.V),
chromaOrigin,
retainedRedCoefficients[this.codedAreaChroma..],
chromaTransformSize,
selectedRedState,
ref retainedRedTransformBlocks[chromaTransformIndex]);
}
// Candidate pixels and coefficients remain scratch. Preserve the transform decisions so final
// reconstruction can regenerate only the winner after other mode families reuse this storage.
selectedStates[0] = selectedLumaState;
selectedStates[1] = selectedBlueState;
selectedStates[2] = selectedRedState;
modeInfo.Block.Mode = selectedMode;
modeInfo.Block.Skip = selectedSkip;
modeInfo.Block.VerticalInterpolationFilter = selectedVerticalFilter;
modeInfo.Block.HorizontalInterpolationFilter = selectedHorizontalFilter;
block.ReferenceMotionVectorIndex = selectedReferenceIndex;
this.picture.SetDisplacementVector(modeInfoPosition, selectedVector);
return selectedStatistics;
}
/// <summary>
/// Reconstructs the selected inter mode after intra trials have reused its arithmetic storage.
/// </summary>
/// <param name="blockOrigin">The luma block origin.</param>
/// <param name="modeInfo">The selected prediction and interpolation syntax.</param>
/// <param name="block">The selected block parameters.</param>
/// <param name="vector">The selected motion vector in eighth-luma-sample units.</param>
/// <param name="states">The selected transform choices, indexed by plane.</param>
private void ReconstructSelectedInterBlock(
Point blockOrigin,
Av1MacroBlockModeInfo modeInfo,
Av1EncoderBlockStruct block,
Av1MotionVector vector,
ReadOnlySpan<Av1EncoderTransformBlockState> states)
{
Av1EncoderInterPredictionWorkspace<TSample> workspace = this.blockWorkspace.GetInterPredictionWorkspace<TSample>();
ObuColorConfig colorConfig = this.picture.Sequence.SequenceHeader.ColorConfig;
int planeCount = block.HasChroma ? 3 : 1;
for (int planeIndex = 0; planeIndex < planeCount; planeIndex++)
{
Av1Plane plane = (Av1Plane)planeIndex;
int subX = planeIndex == 0 ? 0 : this.source.ChromaSubsamplingX;
int subY = planeIndex == 0 ? 0 : this.source.ChromaSubsamplingY;
Point planeOrigin = new(blockOrigin.X >> subX, blockOrigin.Y >> subY);
Av1TransformSize transformSize = planeIndex == 0
? modeInfo.Block.TransformSize
: modeInfo.Block.BlockSize.GetMaxUvTransformSize(colorConfig.SubSamplingX, colorConfig.SubSamplingY);
int sampleCount = transformSize.GetSize2d();
Span<TSample> reconstruction = workspace.LumaCandidateReconstruction[..sampleCount];
Span<int> coefficients = workspace.LumaCandidateCoefficients[..sampleCount];
Span<TSample> prediction = workspace.LumaPrediction[..sampleCount];
Span<short> residual = workspace.Residual[..sampleCount];
// Convert eighth-luma-sample motion into the plane's sixteenth-sample interpolation
// coordinates. The low four bits carry the phase; the remaining bits locate the reference.
int columnQ4 = (planeOrigin.X << 4) + (vector.Column << (1 - subX));
int rowQ4 = (planeOrigin.Y << 4) + (vector.Row << (1 - subY));
TOperator.PrepareTranslationalInterPrediction(
this.source.GetPlane(plane),
planeOrigin,
this.reference.GetPlane(plane),
new Point(columnQ4 >> 4, rowQ4 >> 4),
modeInfo.Block.HorizontalInterpolationFilter,
modeInfo.Block.VerticalInterpolationFilter,
columnQ4 & 15,
rowQ4 & 15,
prediction,
residual,
workspace.PredictionScratch,
transformSize,
this.bitDepth);
Av1EncoderTransformBlockState state = default;
if (modeInfo.Block.Skip || states[planeIndex].EndOfBlock == 0)
{
// An empty transform retains prediction even when other planes have coded residuals.
// Re-quantizing its inferred DCT could otherwise introduce coefficients absent in the winner.
reconstruction = prediction;
coefficients.Clear();
}
else
{
// Regenerate only the selected transform. Motion, transform choice, coefficient-rate
// measurement, and skip decisions are complete before this final reconstruction.
_ = TOperator.EncodePredictionCandidate(
this.blockWorkspace,
this.source.GetPlane(plane),
planeOrigin,
prediction,
residual,
reconstruction,
transformSize.GetWidth(),
coefficients,
transformSize,
states[planeIndex].TransformType,
plane,
this.quantization.QIndex[0],
this.quantization.DeltaQDc[planeIndex],
this.quantization.DeltaQAc[planeIndex],
this.bitDepth,
ref state);
}
int codedArea = planeIndex == 0 ? this.codedAreaLuma : this.codedAreaChroma;
int transformIndex = codedArea / Av1EncoderCoefficientBuffer.TransformBlockUnitCoefficientCount;
Span<int> retainedCoefficients = this.coefficientBuffer.GetPlaneSpan(this.superblock.Index, plane);
Span<Av1EncoderTransformBlockState> retainedStates = this.coefficientBuffer.GetTransformBlockSpan(this.superblock.Index, plane);
CopyCandidate(
reconstruction,
coefficients,
this.reconstruction.GetPlane(plane),
planeOrigin,
retainedCoefficients[codedArea..],
transformSize,
state,
ref retainedStates[transformIndex]);
}
}
/// <summary>
/// Ranks a filter pair from visible prediction error using the reference curve model, without transforming samples.
/// </summary>

Write Preview
Loading…
Cancel
Save