[API Proposal]: Buffer.BlockCopy overload with long dstOffset

[pfusik]

Background and motivation

https://learn.microsoft.com/en-us/dotnet/api/system.buffer.blockcopy?view=net-10.0 cannot access dst array above 2 GB because the byte offset is passed as int.
Looking at the current implementation, it already extends the offset to nuint internally, so this should be an easy addition.

API Proposal

public class Buffer
{
    public static void BlockCopy(Array src, int srcOffset, Array dst, long dstOffset, int count);
}

API Usage

	// Read `a` from `BR`
	void ReadWeights(BinaryReader BR, short[] a, int n)
	{
		const int bufSize = 4096;
		byte[] buf = new byte[bufSize * 2];
		for (int i = 0; i < n; i += bufSize) {
			int m = Math.Min(n - i, bufSize);
			if (BR.Read(buf, 0, m * 2) != m * 2)
				throw new IOException("Truncated read");
#if false
			for (int j = 0; j < m; j++)
				a[i + j] = (short) (buf[j * 2] | buf[j * 2 + 1] << 8);
#else
			Buffer.BlockCopy(buf, 0, a, i * 2L, m * 2);
#endif
		}
	}

Alternative Designs

nint or nuint - my experience with these is limited.

Risks

No response

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-buffers
See info in area-owners.md if you want to be subscribed.

[tannergooding]

If an overload was provided, it should be nint and not long, it simplifies things as it can never be more than nint anyways.

That being said, if you are dealing with multiple gigabytes of data you likely want to be using NativeMemory.Copy or explicitly chunking/segmenting your data anyways. An array that big is not likely to get moved by the GC and pinning is cheap.

[huoyaoyuan]

The entire Buffer class is legacy. The modern equivalent of the code is:

buf.AsSpan(0, m).CopyTo(MemoryMarshal.AsBytes(a.AsSpan(i)));

which can handle array(span) with 2G elements, instead of 2G bytes.

Additionally, if the BinaryReader is constructed from a stream, then the entire method can be replaced by one line:

stream.ReadExactly(MemoryMarshal.AsBytes(a.AsSpan(0, n)))

[pfusik]

@tannergooding I don't see how I could use NativeMemory.Copy in my method?

@huoyaoyuan MemoryMarshal.AsBytes only handles 2 GB at a time and throws OverflowException if passed more. Which is confusing. Anyway, I replaced my inner loop, thanks!

[huoyaoyuan]

MemoryMarshal.AsBytes only handles 2 GB at a time and throws OverflowException if passed more.

You can also invert the conversion and convert the byte span to larger type:

MemoryMarshal.Cast<byte, short>(buf.AsSpan(0, m)).CopyTo(a.AsSpan(0, n));

[tannergooding]

I don't see how I could use NativeMemory.Copy in my method?

You pin the two arrays and copy from one to the other.

MemoryMarshal.AsBytes only handles 2 GB at a time and throws OverflowException if passed more

Yes. Working with that much memory in a single allocation is often a sign you're doing something wrong. It limits portability of the code, places enormous pressure on the machine, and requires loading all the data up front. This typically means you're massively pessimizing throughput compared to appropriately chunking, streaming, and buffering the data into manageable segments.

Keep in mind that anything over around 85KB is considered "large", so something over 2GB is even more so and is going to be a bottleneck in any app.

[pfusik]

Thank you for your comment. It's a toy local LLM chat and I only do the allocation and loading of the weights at the start.