MotionAnalysisSharp

High-performance, SIMD-accelerated motion compensation/estimation library for .NET

Features

• Motion Compensation
• Motion Estimation (Diamond Search, Hexagon Search, Cross Search)
• Transfer Functions (14 supported algorithms) as outlined in ITU-T H.273
• Discrete Cosine Transform (DCT-II)
• Encoding of Intra-prediction modes on top of decoder functions
• Motion Vector Difference (MVD)
• Non-square block sizes
• Any block size from 4x4 to 64x64

How to use

Start by installing MotionAnalysisSharp through NuGet.

dotnet add package MotionAnalysisSharp

Once done, here are some examples on using features that MotionAnalysisSharp brings to you.

Blocks

You can represent small pixel blocks.

Example for 4x4 block with 8-bit depth:

using MotionAnalysisSharp.Blocks;
using System.Drawing;

Span<byte> image = ...; // full frame data
Size imageSize = ...; // 2D size of your image
Point blockLocation = new Point(10, 8); // 4x4 block must be loaded at 10,8 (x,y)
Block4x4Byte block = Block4x4Byte.Create(image, blockLocation, imageSize);

// example: Sum Absolute Differences

long sad = block.SumAbsoluteDifferences(in anotherBlock);

ref byte first = ref block.GetRef(); // reference

Example for 8x4 block with 16-bit depth:

Span<ushort> image = ...; // full frame data
Size imageSize = ...; // 2D size of your image
Point blockLocation = new Point(10, 8); // 4x4 block must be loaded at 10,8 (x,y)
Block8x4UInt16 block = Block8x4UInt16.Create(image, blockLocation, imageSize);

Motion Compensation

You can copy a block onto another image at location:

using MotionAnalysisSharp.Compensation;

// copy into 30,40
MotionCompensation.Compensate(in block, image, imageSize, new Point(30, 40));

DCT

You can apply Discrete Cosine Transform (DCT-II). Inverse/forward with 1D/2D is supported.

using MotionAnalysisSharp.Dct;

Span<double> input = [5.0, 2.5, 4.1, 2.6, 5.5, 7.9, 4.8, 6.5];
Span<double> output = stackalloc double[input.Length];

Dct.Forward(input, output);

// and back
Dct.Inverse1D(output, input);

2D:

Span<double> input4x4 = [
	1, 2, 3, 4,
	2, 3, 4, 5,
	3, 4, 5, 6,
	4, 5, 6, 7
];
Span<double> output4x4 = stackalloc double[input4x4.Length];
Dct.Forward2D(input4x4, output4x4, 4, 4);

// and back
Dct.Inverse2D(output4x4, input4x4, 4, 4);

Motion estimation

You can pick one of the 3 built-in motion estimation algorithms (Diamond Search, Hexagon Search and Cross Search), and use it to get motion vectors using a current and reference picture. Supports 1 to 4 planes and 1 to 16 bit depths.

Example (three-plane, 8-bit-depth, 640x480 image, 4x4 search, Diamond Search, start at 48,32):

using MotionAnalysisSharp.Estimation;

Span<byte> y = ...; // luma plane (current)
Span<byte> u = ...; // cb plane (current)
Span<byte> v = ...; // cr plane (current)
Span<byte> y2 = ...; // luma plane (reference)
Span<byte> u2 = ...; // cb plane (reference)
Span<byte> v2 = ...; // cr plane (reference)
Size imageSize = new(640, 480);
Point startAt = new(48, 32);

SearchResult result = MotionEstimation.DiamondSearch.Search4x4(
	y, y2,
	u, u2,
	v, v2,
	startAt, imageSize);

Console.WriteLine($"New inter-frame location: {result.NonRelativePoint}");
// subtract NonRelativePoint from startAt to get a motion vector

Intraframe Encoding

Assuming that you have methods to decode Intra-frame blocks, you can build an encoder on top of them.

Example (4x4, up to 8 bit depth):

using MotionAnalysisSharp.Intra;

Dictionary<uint, IntraFunctionByte> functions = new() {
	{ 0, IntraDecoder.DecodeVertical },
	{ 1, IntraDecoder.DecodeHorizontal },
	{ 2, IntraDecoder.DecodeDC },
	{ 3, IntraDecoder.DecodePlane }
};

IntraSamples samples = ...; // Neighboring pixels and their availabilities
Block4x4Byte block = ...; // Actual pixels you want to encode into Intra prediction

uint mode = IntraSearch.GetBestModeByTag4x4(in block, functions, in samples);
// whichever key does 'mode' represent in the 'functions' dictionary, that's the
// encoded Intra prediction mode

MVD

A simple utility for motion vector differences.

It can be handled like this:

MotionVectorDifference mvd = new();

// Frame 1
Point actualMV = mvd.ComputeRelativeDifference(new Point(3, 4));
mvd.UpdateMvd(actualMV);
// Returns 3,4 because no previous frames exist to get differences of

// Frame 2
actualMV = mvd.ComputeRelativeDifference(new Point(-2, -2));
mvd.UpdateMvd(actualMV);
// Returns 5,6 now; adapts automatically

Transfer functions

The library supports encoding/decoding of over 14 transfer functions outlined in ITU-T H.273.

Example:

Span<float> input = ...;		// Must be 0 < x < 1
Span<float> output = ...;

TransferCurve.Bt709_6.DangerousEncodeSignals(input, output);
// See also "EncodeSignals": this one prevents crashes but is slow due to checks

or:

Span<float> input = ...;		// Must be 0 < x < 1
Span<float> output = ...;

TransferCurve.Bt709_6.DangerousDecodeSignals(input, output);

This example uses BT.709-6.

Performance

MotionAnalysisSharp is purely CPU-only. That being said, the following took:

Type	Speed
Encoding BT.709-6 (1920x1080)	1.8ms
Computing Forward Discrete Cosine Transform on 1920x1080 floats	~86fps

Building

Open the terminal, and run these 3 commands. Make sure Git and .NET 10 SDK is installed.

git clone https://github.com/winscripter/MotionAnalysisSharp.git
cd MotionAnalysisSharp
dotnet build

Once done, you can also run the following to run all tests:

dotnet test

License

Licensed under MIT. Free to use. See LICENSE.txt for full licensing information.