Browse Source

Fix AV1 live partition traversal and lossless tiled prediction

pull/2633/head
James Jackson-South 4 weeks ago
parent
commit
7cf7fc469c
  1. 8
      HEIF_IMPLEMENTATION_PLAN.md
  2. 11
      src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ChromaModeDecision.cs
  3. 18
      src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ModeDecision.cs
  4. 83
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EncoderFrameTests.cs
  5. 169
      tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraSuperblockEncoderTests.cs

8
HEIF_IMPLEMENTATION_PLAN.md

@ -44,12 +44,18 @@ Reconciled with the worktree on 2026-09-05.
Current interpolation-search checkpoint, implemented on 2026-09-05 with focused verification in progress:
- [x] The requested in-progress tree was committed as `433afd1a9` before further encoder work. That commit is a checkpoint, not a claim of completed interpolation or codec delivery.
- [x] Production interpolation verification now forces Smooth and Sharp at effort eight, both dual-filter axis orders at effort nine, and native 10/12-bit two-axis half-sample motion. The tests assert actual retained filter symbols, vectors, exact reconstruction, and no allocator rent during tile coding. The sequence fixture now uses the retained-reference decode contract; the still-image buffer-transfer API intentionally releases the reference map and cannot decode dependent samples in succession.
- [x] This verification exposed a live partition traversal defect: after search changed an earlier node's child count, a later node could consume an unrelated entry from the initial flat 8x8 skeleton. Unsearched intra partitions now derive their default from the current block size, preserving the geometry-driven traversal used by current libaom. Wide and tall lossless regressions cover clipped parents and superblock boundaries at efforts nine and ten.
- [x] The affected frame encoder, intra-superblock encoder, and public HEIF encoder set passes all 225 cases on the corrected tree with zero failures or skips. Evidence: `artifacts/TestResults/av1-interpolation-partition-broad-20260905/partition-broad-r3.trx`. This is the affected encoder surface, not the full codec release matrix.
- [x] Lossless tiled luma and chroma search no longer evaluate unsignaled angle deltas on 4x8/8x4 coding blocks; tiled chroma also respects the ordinary effort limits. Partition trials publish lossless chroma coefficient contexts at 4x4 transform granularity. The extended native RGB-plane regression exposed the chroma angle defect as a real lossless mismatch reproduced by libaom, and passes after the correction without changing expected samples.
- [x] All 20 focused partition/interpolation/lossless cases pass with hardware intrinsics enabled and all 20 pass with them disabled through serialized net11.0 Release Visual Studio VSTest with stop-on-failure. Current-main libaom `d565eec60f084421fa34fc0534b760c6452b6a6c` decodes all 20 emitted streams (28 frames) with exact native-plane equality against the retained reconstruction or lossless source planes. Evidence: `artifacts/TestResults/av1-interpolation-partition-interpolation-20260905/partition-interpolation-r3.trx`, `partition-interpolation-scalar-r3.trx`, and `artifacts/av1-partition-interpolation-net11-20260905-r3.log`. Roslyn compiler and analyzer passes report no errors or changed-file warnings. Subsampled inter-plane coverage, broader current-tree verification, and end-to-end performance remain open.
- [x] The 8x8 single-candidate SAD, four-candidate SAD, and variance paths now share closed-generic traversal in `Av1ResidualBuilder`. Its existing byte/ushort residual operators own scalar and SIMD arithmetic; the superblock operators no longer duplicate these row loops or hardware dispatch. The four-candidate path retains one source load/conversion per row, exact eight-sample loads preserve final-row bounds, and both variance moments retain native precision until the existing normalization boundary. Six known-result cases cover 8/10/12-bit signed extrema, distinct source/prediction rows and strides, unaligned starts, exact final-row lengths, and four-candidate output order. Together with nine existing intra-block-copy cases and the extended zero-allocation test, all 16 pass with hardware intrinsics enabled and all 16 pass with them disabled. All 58 public HEIF encoder cases also pass. Verification used serialized net11.0 Release Visual Studio VSTest with stop-on-failure; evidence is `artifacts/TestResults/av1-search-metrics-20260905/search-metrics-r2.trx`, `search-metrics-scalar-r2.trx`, and `search-metrics-encoder-r2.trx` in that directory. The exact Release build has zero errors and the existing 1,009 warnings; Roslyn reports no diagnostics in the changed files. This verifies the search-metric refactor, not the remaining interpolation conformance or end-to-end performance work.
- [~] Effort eight searches the common regular, smooth, and sharp interpolation families; efforts nine and ten enable independent vertical/horizontal filter selection. Lower efforts retain the fixed regular-filter path.
- [~] Filter ranking follows the curve-fit prediction-error model in the official `d565eec60f084421fa34fc0534b760c6452b6a6c` source, before full transform search. The existing rate-distortion type owns the static model tables and paired portable SIMD cubic evaluation. Visible-plane SSE uses the existing SIMD residual reduction, including native-bit-depth normalization and cropped edges.
- [~] Candidate and retained prediction views alternate within the existing inter workspace. At most one normalization copy per active plane retains the chosen predictor for transform search; no new pixel owner, coefficient owner, per-block rent, or reconstructed-frame copy is introduced. Zero-phase axes retain only the cheapest signaled filter instead of repeating equivalent prediction trials.
- [~] The selected filters, skip flags, segment, and primary reference fit the original seven-byte block-mode record and eight-byte macroblock record. Filter costing and writing use the live tile CDFs. Encoder and decoder share the context-combination mapping, and encoder search and writing share filter-symbol eligibility.
- [~] All 67 focused net11.0 Release cases pass through serialized Visual Studio VSTest: model curve samples and skip decisions, 8/10/12-bit quantizer normalization, exact entropy bytes and live adaptation, packed-field independence, tile-boundary contexts, syntax eligibility, and sequence-header signaling. All 12 model cases also pass with hardware intrinsics disabled in the test-host environment. The four retained-reference sequences are accepted by current libaom. These flat-image cases do not force non-regular filter selections; those production cases, broader current-tree verification, allocation assertions, and end-to-end timing remain required before this checkpoint can be committed.
- [~] The earlier 67-case net11.0 Release set covered model curve samples and skip decisions, 8/10/12-bit quantizer normalization, exact entropy bytes and live adaptation, packed-field independence, tile-boundary contexts, syntax eligibility, and sequence-header signaling. Its 12 model cases also passed with hardware intrinsics disabled. The four flat retained-reference sequences were accepted by current libaom but did not force non-regular filters. The production, allocation, and exact native-plane evidence above extends that coverage; subsampled inter planes and end-to-end timing remain required before closing this checkpoint.
- [x] Focused runtime verification exposed an invalid low-effort inter-frame header: `force_integer_mv` was set while screen-content tools were disabled, making the writer omit the high-precision flag that a conforming reader expects. The frame encoder now retains the inferred false flag and controls integer-only search through the existing effort boundary. All four retained-reference cases assert the parsed precision, quantizer, and filter fields and decode both frames; current libaom accepts the same saved streams.
- [x] The preceding interpolation checkpoint's affected net11.0 Release run passed 2,494 cases with zero failures or skips through one serialized Visual Studio VSTest process with stop-on-failure enabled. It combined 2,269 entropy cases, 167 frame/superblock/transform/picture-storage cases, and 58 public HEIF encoder cases. That build had zero errors and the existing 1,009 test-project warnings, with none in the changed files. Evidence: `artifacts/TestResults/av1-interpolation-20260905/verified-encoder-entropy-r10.trx` and `artifacts/av1-interpolation-net11-release-20260905-r10.log`. This predates the search-metric refactor above and is not the complete current-tree decoder/encoder release matrix.
- [x] Allocation regressions now reflect the implemented lifetimes instead of the former layout: all seven trailing tile-state integers are proved contiguous within the picture's second allocator owner; coefficient level, context, and output owners are proved allocated at construction, reused across costing, writing, finalization, and frame resets, and returned exactly once. The two-owner picture and three-owner symbol-encoder limits are retained.

11
src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ChromaModeDecision.cs

@ -636,7 +636,16 @@ internal static partial class Av1IntraSuperblockEncoder
(int)Av1ChromaPredictionMode.Vertical +
1;
int candidateCount = baseModeCount + (directionalModeCount * deltaCount);
// Multiple transforms do not expand the block's prediction syntax. Preserve the ordinary
// effort tiers and exclude angle adjustments for 4x8/8x4 blocks, where no delta is signaled.
int candidateCount = this.effort switch
{
0 => 1,
1 => baseModeCount,
_ when blockSize >= Av1BlockSize.Block8x8 => baseModeCount + (directionalModeCount * deltaCount),
_ => baseModeCount
};
long bestCost = long.MaxValue;
Av1ChromaPredictionMode bestMode = Av1ChromaPredictionMode.DC;
selectedAngleDelta = 0;

18
src/ImageSharp/Formats/Heif/Av1/Pipeline/Av1IntraSuperblockEncoder.ModeDecision.cs

@ -197,6 +197,11 @@ internal static partial class Av1IntraSuperblockEncoder
return preparedPartition;
}
// Live decisions change the number of nodes visited before this position. The original flat
// skeleton's index no longer identifies this block, so derive its default from current geometry.
// Otherwise an earlier unsplit 16x16 can make a later 32x32 consume an old 8x8 NONE entry.
preparedPartition = blockSize == Av1BlockSize.Block8x8 ? Av1PartitionType.None : Av1PartitionType.Split;
bool searchPartition = blockSize is Av1BlockSize.Block8x8 or Av1BlockSize.Block16x16 ||
(this.effort == 10 &&
blockSize is Av1BlockSize.Block32x32 or Av1BlockSize.Block64x64 or Av1BlockSize.Block128x128);
@ -983,9 +988,11 @@ internal static partial class Av1IntraSuperblockEncoder
colorConfig.SubSamplingX,
colorConfig.SubSamplingY);
Av1TransformSize chromaTransformSize = blockSize.GetMaxUvTransformSize(
colorConfig.SubSamplingX,
colorConfig.SubSamplingY);
// Lossless residuals retain one state per 4x4 transform, including chroma. Publish those exact
// edges during partition trials so a later sibling sees the contexts that final writing will use.
Av1TransformSize chromaTransformSize = this.picture.Parent.FrameHeader.CodedLossless
? Av1TransformSize.Size4x4
: blockSize.GetMaxUvTransformSize(colorConfig.SubSamplingX, colorConfig.SubSamplingY);
Av1BlockSize maximumChromaUnitBlockSize =
Av1BlockSize.Block64x64.GetSubsampled(colorConfig.SubSamplingX, colorConfig.SubSamplingY);
@ -2071,11 +2078,14 @@ internal static partial class Av1IntraSuperblockEncoder
int directionalModeCount =
(int)Av1PredictionMode.Directional67Degrees - (int)Av1PredictionMode.Vertical + 1;
// A lossless 4x8 or 8x4 block has multiple 4x4 transforms but carries no angle-delta symbol.
// Its predictor must therefore use the unadjusted direction, just as the decoder does.
int candidateCount = this.effort switch
{
0 => 1,
1 => baseModeCount,
_ => baseModeCount + (directionalModeCount * deltaCount)
_ when blockSize >= Av1BlockSize.Block8x8 => baseModeCount + (directionalModeCount * deltaCount),
_ => baseModeCount
};
Buffer2DRegion<TSample> sourcePlane = this.source.GetPlane(Av1Plane.Y);

83
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1EncoderFrameTests.cs

@ -1,6 +1,7 @@
// Copyright (c) Six Labors.
// Licensed under the Six Labors Split License.
using System.Runtime.InteropServices;
using SixLabors.ImageSharp.Formats;
using SixLabors.ImageSharp.Formats.Heif.Av1;
using SixLabors.ImageSharp.Formats.Heif.Av1.Motion;
@ -436,12 +437,14 @@ public class Av1EncoderFrameTests
}
[Theory]
[InlineData(TenBit)]
[InlineData(TwelveBit)]
public void LosslessHighBitDepthEncodingPreservesNativePlanes(int bitDepthValue)
[InlineData(TenBit, 8, 8, 0)]
[InlineData(TwelveBit, 8, 8, 0)]
[InlineData(TenBit, 24, 16, 9)]
[InlineData(TwelveBit, 24, 16, 9)]
[InlineData(TenBit, 16, 24, 10)]
[InlineData(TwelveBit, 16, 24, 10)]
public void LosslessHighBitDepthEncodingPreservesNativePlanes(int bitDepthValue, int width, int height, int effort)
{
const int width = 8;
const int height = 8;
Av1BitDepth bitDepth = (Av1BitDepth)bitDepthValue;
using Image<Rgb48> source = new(width, height);
for (int row = 0; row < height; row++)
@ -450,9 +453,9 @@ public class Av1EncoderFrameTests
for (int column = 0; column < width; column++)
{
pixels[column] = new Rgb48(
(ushort)((column * 7001) + (row * 997)),
(ushort)((row * 6007) + (column * 1231)),
(ushort)((column * 4001) + (row * 3001)));
(ushort)(((column * 7001) + (row * 997)) & ushort.MaxValue),
(ushort)(((row * 6007) + (column * 1231)) & ushort.MaxValue),
(ushort)(((column * 4001) + (row * 3001)) & ushort.MaxValue));
}
}
@ -488,7 +491,7 @@ public class Av1EncoderFrameTests
stream,
colorConfig,
qIndex: 0,
effort: 0);
effort);
byte[] payload = stream.ToArray();
string outputDirectory = Path.Combine(
@ -498,8 +501,9 @@ public class Av1EncoderFrameTests
"Av1");
Directory.CreateDirectory(outputDirectory);
string outputName = $"encoder-frame-{width}x{height}-{bitDepth.GetBitCount()}b-444-lossless-effort{effort}";
File.WriteAllBytes(
Path.Combine(outputDirectory, $"encoder-frame-8x8-{bitDepth.GetBitCount()}b-444-lossless.obu"),
Path.Combine(outputDirectory, outputName + ".obu"),
payload);
using Av1Decoder decoder = new(Configuration.Default);
@ -512,18 +516,71 @@ public class Av1EncoderFrameTests
Assert.True(frameHeader.AllLossless);
Assert.Equal(Av1TransformMode.Only4x4, frameHeader.TransformMode);
// Lossless native planes are the oracle for external decoding, not the packed RGB conversion on return.
// UInt16 raw samples are explicitly little-endian even when these tests run on a different host byte order.
using BinaryWriter rawOutput = new(File.Create(Path.Combine(outputDirectory, outputName + ".source.yuv")));
foreach (Av1Plane plane in new[] { Av1Plane.Y, Av1Plane.U, Av1Plane.V })
{
Buffer2DRegion<ushort> expectedPlane = expected.Frame.View.GetPlane(plane);
for (int row = 0; row < height; row++)
{
Assert.Equal(
expectedPlane.DangerousGetRowSpan(row)[..width].ToArray(),
actual.GetHighBitDepthRowSpan(plane, row, 0, 0).ToArray());
ReadOnlySpan<ushort> expectedRow = expectedPlane.DangerousGetRowSpan(row);
Assert.Equal(expectedRow, actual.GetHighBitDepthRowSpan(plane, row, 0, 0));
foreach (ushort sample in expectedRow)
{
rawOutput.Write(sample);
}
}
}
}
/// <summary>
/// Verifies that live partition search preserves lossless syntax across clipped parent nodes and superblocks.
/// </summary>
[Theory]
[InlineData(48, 24, 9)]
[InlineData(24, 48, 9)]
[InlineData(80, 24, 9)]
[InlineData(24, 80, 9)]
[InlineData(96, 24, 10)]
[InlineData(24, 96, 10)]
public void EncodeLosslessPartitionSearchAcrossClippedSuperblocks(int width, int height, int effort)
{
ReadOnlySpan<int> period = [0, 28, 40, 28, 0, -28, -40, -12];
using Image<L8> source = new(width, height);
for (int y = 0; y < height; y++)
{
Span<L8> row = source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y);
for (int x = 0; x < width; x++)
{
row[x] = new L8((byte)(128 + period[x % period.Length] + period[y % period.Length]));
}
}
// The repeated surface favors larger early leaves. Later clipped parents must still split from their
// own geometry instead of reading a stale position in the original fixed-eight partition preorder.
ObuColorConfig colorConfig = CreateColorConfig(Av1BitDepth.EightBit, Av1ColorFormat.Yuv400);
colorConfig.ColorRange = true;
using MemoryStream stream = new();
Av1FrameEncoder.Encode(Configuration.Default, source.Frames.RootFrame, stream, colorConfig, qIndex: 0, effort);
byte[] payload = stream.ToArray();
string outputDirectory = TestEnvironment.CreateOutputDirectory("Heif", "Av1", nameof(this.EncodeLosslessPartitionSearchAcrossClippedSuperblocks));
string outputName = $"{width}x{height}-effort{effort}";
File.WriteAllBytes(Path.Combine(outputDirectory, outputName + ".obu"), payload);
using Av1Decoder decoder = new(Configuration.Default);
using Av1FrameBuffer<byte> decoded = decoder.DecodeFrameBuffer(payload, null, null, out _);
Assert.Equal(width, decoded.Width);
Assert.Equal(height, decoded.Height);
Buffer2DRegion<byte> actual = decoded.DeriveBlockPointer(Av1Plane.Y, 0, 0);
using FileStream rawOutput = File.Create(Path.Combine(outputDirectory, outputName + ".source.yuv"));
for (int y = 0; y < height; y++)
{
ReadOnlySpan<byte> expectedRow = MemoryMarshal.AsBytes(source.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y));
Assert.Equal(expectedRow, actual.DangerousGetRowSpan(y));
rawOutput.Write(expectedRow);
}
}
[Fact]
public void EncodeEffortNineSelectsSubEightPartition()
{

169
tests/ImageSharp.Tests/Formats/Heif/Av1/Av1IntraSuperblockEncoderTests.cs

@ -159,8 +159,11 @@ public class Av1IntraSuperblockEncoderTests
using ObuWriter obuWriter = new(configuration);
obuWriter.WriteFrame(secondSample, sequenceHeader, frameHeader, tileWriter);
using Av1Decoder decoder = new(configuration);
using Av1FrameBuffer<byte> decodedFirst = decoder.DecodeFrameBuffer(firstSample.ToArray(), null, null, out _);
using Av1FrameBuffer<byte> decodedSecond = decoder.DecodeFrameBuffer(secondSample.ToArray(), null, null, out _);
// Sequence decoding retains the first frame's reference slots. The still-image transfer API deliberately
// releases those slots, so it cannot be used between dependent samples. Full-range monochrome L8 is exact.
using ImageFrame<L8> decodedFirst = decoder.DecodeSequenceFrame<L8>(firstSample.ToArray(), null, null);
using ImageFrame<L8> decodedSecond = decoder.DecodeSequenceFrame<L8>(secondSample.ToArray(), null, null);
// Preserve both the production stream and every managed reconstructed luma sample for exact libaom comparison.
string outputDirectory = TestEnvironment.CreateOutputDirectory("Heif", "Av1", nameof(this.ProductionTileSelectsNonRegularInterpolation));
@ -174,7 +177,7 @@ public class Av1IntraSuperblockEncoderTests
for (int y = 0; y < Height; y++)
{
ReadOnlySpan<byte> expected = reference.Frame.View.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y);
ReadOnlySpan<byte> actual = decodedFirst.DeriveBlockPointer(Av1Plane.Y, new Point(0, y), 0, 0, out _)[..Width];
ReadOnlySpan<byte> actual = MemoryMarshal.AsBytes(decodedFirst.PixelBuffer.DangerousGetRowSpan(y));
Assert.Equal(expected, actual);
rawOutput.Write(actual);
}
@ -182,12 +185,170 @@ public class Av1IntraSuperblockEncoderTests
for (int y = 0; y < Height; y++)
{
ReadOnlySpan<byte> expected = reconstruction.Frame.View.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y);
ReadOnlySpan<byte> actual = decodedSecond.DeriveBlockPointer(Av1Plane.Y, new Point(0, y), 0, 0, out _)[..Width];
ReadOnlySpan<byte> actual = MemoryMarshal.AsBytes(decodedSecond.PixelBuffer.DangerousGetRowSpan(y));
Assert.Equal(expected, actual);
rawOutput.Write(actual);
}
}
/// <summary>
/// Verifies independent half-sample filters on both axes without discarding native sample precision.
/// </summary>
[Theory]
[InlineData(10, false)]
[InlineData(10, true)]
[InlineData(12, false)]
[InlineData(12, true)]
public void ProductionTileSelectsDualAxisInterpolationHighBitDepth(int bitDepth, bool reverseFilters)
{
const int Width = 48;
const int Height = 24;
const int TargetColumn = 16;
const int TargetRow = 8;
const int BlockSize = 8;
const int QIndex = 1;
const int Effort = 9;
const int TileBufferLength = 8192;
const int FilterScale = 128;
int sampleScale = 1 << (bitDepth - 8);
int maximumSample = (1 << bitDepth) - 1;
Av1InterpolationFilter horizontalFilter = reverseFilters ? Av1InterpolationFilter.Sharp : Av1InterpolationFilter.Smooth;
Av1InterpolationFilter verticalFilter = reverseFilters ? Av1InterpolationFilter.Smooth : Av1InterpolationFilter.Sharp;
ReadOnlySpan<int> referencePeriod = [0, 28, 40, 28, 0, -28, -40, -12];
// These Q7 sums are the fixed half-sample responses of the periodic reference to libaom's eight-tap
// kernels. The source is separable: 128 + horizontal period + vertical period. Each horizontal sum
// is divisible by the first-pass rounding unit (including the five-bit shift at 12 bits), so the
// two-axis result is the sum of these responses with one final Q7 rounding, not two rounded pixels.
// The asymmetric final phase separates sharp from regular after eight-bit error normalization. A low
// quantizer makes retaining the exact two-axis predictor preferable to saving a filter symbol.
ReadOnlySpan<int> smoothResponse = [1872, 3952, 3984, 1648, -1680, -3760, -3152, -816];
ReadOnlySpan<int> sharpResponse = [1536, 4896, 4640, 1856, -1728, -5088, -3424, -640];
ReadOnlySpan<int> horizontalResponse = reverseFilters ? sharpResponse : smoothResponse;
ReadOnlySpan<int> verticalResponse = reverseFilters ? smoothResponse : sharpResponse;
TestMemoryAllocator allocator = new();
allocator.EnableNonThreadSafeLogging();
Configuration configuration = Configuration.Default.Clone();
configuration.MemoryAllocator = allocator;
ObuColorConfig colorConfig = new()
{
IsMonochrome = true,
ColorRange = true,
SubSamplingX = true,
SubSamplingY = true,
BitDepth = (Av1BitDepth)((bitDepth - 8) / 2)
};
using Image<L16> referenceImage = new(Width, Height);
using Av1EncoderFrameBuffer<ushort> reference = new(configuration, Width, Height, bitDepth, Av1ColorFormat.Yuv400, 0, 0);
using Av1EncoderFrameBuffer<ushort> source = new(configuration, Width, Height, bitDepth, Av1ColorFormat.Yuv400, 0, 0);
using Av1EncoderFrameBuffer<ushort> reconstruction = new(configuration, Width, Height, bitDepth, Av1ColorFormat.Yuv400, 0, 0);
for (int y = 0; y < Height; y++)
{
Span<L16> pixels = referenceImage.Frames.RootFrame.PixelBuffer.DangerousGetRowSpan(y);
Span<ushort> referenceRow = reference.Frame.CodedView.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y);
Span<ushort> sourceRow = source.Frame.CodedView.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y);
for (int x = 0; x < Width; x++)
{
int sample = (128 + referencePeriod[x % BlockSize] + referencePeriod[y % BlockSize]) * sampleScale;
referenceRow[x] = (ushort)sample;
// Map native samples to L16's complete range. The lossless key-frame comparison below proves
// that the public pixel conversion recovers every original 10/12-bit reference sample.
pixels[x] = new L16((ushort)(((sample * ushort.MaxValue) + (maximumSample / 2)) / maximumSample));
int response = (128 * FilterScale) + horizontalResponse[x % BlockSize] + verticalResponse[y % BlockSize];
sourceRow[x] = (ushort)(((response * sampleScale) + (FilterScale / 2)) / FilterScale);
}
}
reference.Frame.ExtendBorders();
source.Frame.ExtendBorders();
ClearPlane(reconstruction.Luma);
using MemoryStream firstSample = new();
using Av1FrameEncoder.SequenceEncoder keyEncoder = Av1FrameEncoder.CreateColorSequenceEncoder(
configuration, Width, Height, colorConfig, qIndex: 0, Effort);
keyEncoder.EncodeKeyFrame(referenceImage.Frames.RootFrame, firstSample);
ObuSequenceHeader sequenceHeader = keyEncoder.SequenceHeader;
using Av1EncoderModeInfoBuffer modeInfo = new(configuration, Width, Height, disallow4x4AllFrames: true);
Av1PictureControlSet template = CreatePicture(modeInfo, colorConfig, use128x128Superblock: false, QIndex);
ObuFrameHeader frameHeader = template.Parent.FrameHeader;
frameHeader.FrameType = ObuFrameType.InterFrame;
frameHeader.ShowFrame = true;
frameHeader.ErrorResilientMode = true;
frameHeader.RefreshFrameFlags = byte.MaxValue;
frameHeader.DisableFrameEndUpdateCdf = true;
frameHeader.ReferenceMode = ObuReferenceMode.SingleReference;
frameHeader.InterpolationFilter = Av1InterpolationFilter.Switchable;
frameHeader.AllowHighPrecisionMotionVector = true;
frameHeader.TransformMode = Av1TransformMode.Select;
frameHeader.FrameSize.FrameWidth = Width;
frameHeader.FrameSize.FrameHeight = Height;
frameHeader.FrameSize.SuperResolutionUpscaledWidth = Width;
frameHeader.FrameSize.RenderWidth = Width;
frameHeader.FrameSize.RenderHeight = Height;
frameHeader.TilesInfo.HasUniformTileSpacing = true;
Av1QuantizationLookup.UpdateFrameQuantizationState(frameHeader);
using Av1EncoderPictureBuffer picture = new(configuration, sequenceHeader, frameHeader, Width, Height, disallow4x4AllFrames: true);
using Av1EncoderCoefficientBuffer coefficients = new(configuration, sequenceHeader, Width, Height);
using Av1EncoderSuperblockWorkspace superblockWorkspace = new(configuration);
using Av1EncoderBlockWorkspace blockWorkspace = new(configuration);
using Av1SymbolEncoder symbolEncoder = new(configuration, TileBufferLength, QIndex, updateCdf: true);
Av1EncoderTileWorkspace tileWorkspace = new(frameHeader, superblockWorkspace);
int allocationCount = allocator.AllocationLog.Count;
Av1TileEncoder tileWriter = new(
symbolEncoder, source.Frame, reference.Frame, reconstruction.Frame, picture.Picture, coefficients, tileWorkspace, blockWorkspace, Effort);
Assert.Equal(allocationCount, allocator.AllocationLog.Count);
// This block is at least three reference taps from every frame edge. It must retain two genuinely
// fractional axes, not a zero-phase filter alias.
Point targetPosition = new(TargetColumn >> Av1Constants.ModeInfoSizeLog2, TargetRow >> Av1Constants.ModeInfoSizeLog2);
ref Av1MacroBlockModeInfo targetMode = ref picture.Picture.GetMacroBlockModeInfo(targetPosition);
Assert.Equal(Av1ReferenceFrameType.Last, targetMode.Block.ReferenceFrame);
Assert.Equal(horizontalFilter, targetMode.Block.HorizontalInterpolationFilter);
Assert.Equal(verticalFilter, targetMode.Block.VerticalInterpolationFilter);
Assert.Equal(4, picture.Picture.GetDisplacementVector(targetPosition).Column);
Assert.Equal(4, picture.Picture.GetDisplacementVector(targetPosition).Row);
for (int y = TargetRow; y < TargetRow + BlockSize; y++)
{
Assert.Equal(
source.Frame.View.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y).Slice(TargetColumn, BlockSize),
reconstruction.Frame.View.GetPlane(Av1Plane.Y).DangerousGetRowSpan(y).Slice(TargetColumn, BlockSize));
}
using MemoryStream secondSample = new();
using ObuWriter obuWriter = new(configuration);
obuWriter.WriteFrame(secondSample, sequenceHeader, frameHeader, tileWriter);
string outputDirectory = TestEnvironment.CreateOutputDirectory("Heif", "Av1", nameof(this.ProductionTileSelectsDualAxisInterpolationHighBitDepth));
string outputName = $"{bitDepth}-{horizontalFilter}-{verticalFilter}";
using FileStream output = File.Create(Path.Combine(outputDirectory, outputName + ".obu"));
firstSample.Position = 0;
firstSample.CopyTo(output);
secondSample.Position = 0;
secondSample.CopyTo(output);
using BinaryWriter rawOutput = new(File.Create(Path.Combine(outputDirectory, outputName + ".managed.yuv")));
using Av1Decoder decoder = new(configuration);
for (int frameIndex = 0; frameIndex < 2; frameIndex++)
{
// Consume native retained planes before the next sample can replace them. BinaryWriter emits explicit
// little-endian UInt16 samples, matching the raw reference-decoder output independently of host byte order.
decoder.DecodeSequenceReference((frameIndex == 0 ? firstSample : secondSample).ToArray(), null, null);
Av1FrameBuffer<byte> decoded = Assert.IsType<Av1FrameBuffer<byte>>(decoder.FrameBuffer);
Buffer2DRegion<ushort> expected = (frameIndex == 0 ? reference : reconstruction).Frame.View.GetPlane(Av1Plane.Y);
for (int y = 0; y < Height; y++)
{
ReadOnlySpan<ushort> actualRow = decoded.GetHighBitDepthRowSpan(Av1Plane.Y, y, 0, 0);
Assert.Equal(expected.DangerousGetRowSpan(y), actualRow);
foreach (ushort sample in actualRow)
{
rawOutput.Write(sample);
}
}
}
}
/// <summary>
/// Gets the normative eight-sample weights used to build independent smooth-mode fixtures.
/// </summary>

Loading…
Cancel
Save