Fix deserialization of big files

[shaosss]

This is a fix for deserialization of files with Length more than int.MaxValue.

[AArnott]

Holy cow, that would be a huge stream. Thanks for submitting a fix, but this would still leave MessagePack trying to allocate potentially a 4GB area of contiguous memory, which may be problematic as well. On a 64-bit process this might be feasible, but in a 32-bit process, even finding 1GB of contiguous memory may be unlikely.

I think instead of this particular change, we should simply set a reasonable cap (somewhere between 80KB and 1MB perhaps) on how large the arg is to GetSpan and simply loop to copy the entire content of the stream as necessary.

[AArnott]

It looks like the loop is already there, so if we just use Math.Max(..., 1024*1024) that might be best.

[shaosss]

Did you mean Math.Min? Because right now we are trying to cast long value to int and getting arithmetic exception. That ruins the deserialization completely(you can serialize file of that size, but cannot deserialize).
I will look at you suggestion tomorrow, thanks.
And yes, we use Messagepack to save some captured data,which sometimes is a hell of amount.:)

[shaosss]

I checked the code and profile it. It seems it is already using the chunks of size:
private static readonly int DefaultLengthFromArrayPool = 1 + (4095 / Marshal.SizeOf<T>());
So for the stream scenario it seems line no additional work needed. The memoryStream code branch doesn't need any changes since you can't have that long memory streams right now (since you can't have array of bytes with length more than 0x7FFFFFC7).
Not sure about any other overrides of Deserialize function (since most of them are probably can't have big sizes because of array of bytes limitations), but async version needs the same fix (will push in a few minutes and update PR).
Not sure if any tests are needed, since it is fix for simple arithmetic exception.

[shaosss]

@AArnott updated the PR. Found that between 0x7FFFFFC7 and int.MaxValue things still not work because of the array of bytes length limit, so I added the limit to the HintSize as you suggested (not sure that it's in the right place though).
Really appreciate your feedback.

[AArnott]

As commented, please revert the change to Sequence<T> and apply the fix to MessagePackSerializer itself to avoid asking for egregiously large spans.

[AArnott]

Looks good. Thank you.