Erlang cannot read nor write files with to 2^31 bytes or more on macOS

[josevalim]

Describe the bug

1> file:write_file("ohno", binary:copy(<<$a>>, 2147483648)).
{error,einval}
2> file:write_file("ohno", binary:copy(<<$a>>, 2147483648-1)).
ok

where 2147483648 is 2^31.

I have been told this error does not happen on Windows. It has been reproduced on macOS.

Upon further inspection, it also happens with the file descriptor read/write APIs, even when reading small files. In this case, however, the limit is slightly lower:

2> {ok, Fd0} = file:open("README.md", [read]).
{ok,<0.89.0>}
3> file:read(Fd0, 2147483648).
{error,einval}
5> {ok, Fd1} = file:open("README.md", [read]).
{ok,<0.93.0>}
6> file:read(Fd1, 2147483648-1).
{error,einval}
7> {ok, Fd2} = file:open("README.md", [read]).
{ok,<0.97.0>}
8> file:read(Fd1, 2147479552-1).
{ok,<<...>>}

Note the Linux manual says:

       On Linux, read() (and similar system calls) will transfer at most
       0x7ffff000 (2,147,479,552) bytes, returning the number of bytes
       actually transferred.  (This is true on both 32-bit and 64-bit
       systems.)

Although I could only read 2147479552-1. So perhaps it is generally safer to not try to read more than 2147479552-1 bytes in said systems?

Expected behavior

That it can read files with 2^31 bytes or more. :)

Affected versions

Checked it on 25 and master as of 23b7b7e.

[garazdawi]

How odd. We have test cases that should cover those scenarios.

[mikpe]

I can't reproduce:

Erlang/OTP 25 [erts-13.0.4] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Eshell V13.0.4  (abort with ^G)
1> file:write_file("ohno", binary:copy(<<$a>>, 2147483648)).
ok
2> file:write_file("README.md", <<"foo\n">>).
ok
3> {ok, Fd0} = file:open("README.md", [read]).
{ok,<0.85.0>}
4> file:read(Fd0, 2147483648).
{ok,"foo\n"}

This is on x86_64-pc-linux-gnu, 64-bit everything, glibc, and ext4 file system.
An strace of the failing beam.smp would be useful.

[garazdawi]

On macOS I think it is the EINVAL limit for writev and readv that is hit:

     [EINVAL]           The sum of the iov_len values in the iov array over-
                        flowed a 32-bit integer.

In the Linux manual the same text reads:

    EINVAL The sum of the iov_len values overflows an ssize_t value.

[josevalim]

Thanks @mikpe! Can you also try with 4294967297? In any case, I will get more precise information about the Linux system where this was reproduced. All of my snippets above do happen on macOS though.

[josevalim]

Sorry, I misspoke. The issue happens on macOS only. I have amended the title accordingly.

[mikpe]

I guess it's moot now, but retesting on Linux with s/2147483648/4294967297/g showed no error.

[josevalim]

@mikpe I mean, it was very useful to confirm it does not happen on Linux, because despite my initial claim it was not tested on Linux. :) Thank you!

[garazdawi]

There seems to be an undocumented limitation in write/read for macOS that a single read/write cannot be larger than 1 bsl 31. This limitation is documented for writev, but not write. So my guess is that write is just a wrapper for writev and that the same limitation applies to both even if it is not documented as such.

I've put together a fix in #6248

[josevalim]

@garazdawi thanks! I have also tried it locally and it worked. :)

[dmorneau]

There seems to be an undocumented limitation in write/read for macOS that a single read/write cannot be larger than 1 bsl 31. This limitation is documented for writev, but not write.

The man page changed recently: https://github.com/apple-oss-distributions/xnu/blame/27b03b360a988dfd3dfdf34262bb0042026747cc/bsd/man/man2/write.2#L228