Read wavfiles of size > 4GB

[TimFelixBeyer]

Adds the ability to open a .wav file with the 'RF64' format for files with a file size > 4GB, I'm not sure how to write a unit test for this, since it'd require uploading a huge .wav file.

[larsoner]

Would it be a lot of extra work to add the equivalent writer? If not, then a round-trip test could work.

[pv]

Or, add a small wav file in the large-file format. Presumably, the format can describe also small files.

[TimFelixBeyer]

I've added the corresponding writer, however I'm still not sure how to go about implementing the test, since the format only really makes sense for data > 4GB, so the writer should only use the format if the data is too large for RIFF.

[larsoner]

the format only really makes sense for data > 4GB, so the writer should only use the format if the data is too large for RIFF.

Although this might be true in practice, can you opt in to using the RF64 format even for files < 4 GB (even if just for the purpose of testing round-trip IO)?

[TimFelixBeyer]

According to the official specs, the standard filesize-locations should be used if the file is smaller than 4GB: https://tech.ebu.ch/docs/tech/tech3306-2009.pdf
It‘d be possible to force the RF64 format with a smaller file but then a large part (writing/reading the new 64bit-size-fields) of the changes wouldn’t be tested.
Alternatively one could disregard the specs, and write the filesize into the 64Bit fields even if it’s < 4GB. Let me know which of these options is the best and I’ll try to implement it

[larsoner]

According to the official specs, the standard filesize-locations should be used if the file is smaller than 4GB

This makes sense, and seems like reasonable behavior for SciPy to follow.

Alternatively one could disregard the specs, and write the filesize into the 64Bit fields even if it’s < 4GB.

This is what I propose to do for testing purposes only, yes. Especially if there is some way to do it that does not expose the option to the user (since it's non-standard). Maybe a private function that does the heavy lifting could be used for this purpose.

[777arc]

This would be a valuable addition!!

[lucascolley]

Alternatively one could disregard the specs, and write the filesize into the 64Bit fields even if it’s < 4GB.

This is what I propose to do for testing purposes only, yes. Especially if there is some way to do it that does not expose the option to the user (since it's non-standard). Maybe a private function that does the heavy lifting could be used for this purpose.

@TimFelixBeyer are you interested in returning to do this? Sounds like it's all that is needed to test this properly and get this in. Judging by the recent comment, there is still some demand for this feature.

[TimFelixBeyer]

Sure, I’m currently busy but can take another look in about 3 weeks if that’s ok.

[TimFelixBeyer]

@lucascolley I'd like some input regarding testing.
In addition to testing with small modified RF64 files, I have a working test on my local machine that just performs a round-trip write of a large (>4GB) array, but it obviously takes quite a while and requires a large amount of memory to complete.
However, I think a round-trip test would be valuable in principle.
Are there any guidelines regarding how long test cases are allowed to last?

[lucascolley]

We have a marker for extremely slow tests, which can be applied with pytest.mark.xslow, to avoid running on every CI job. How about you add the test with that marker, then we can see how long it takes?

[TimFelixBeyer]

Opened #20079 to avoid having to rebase etc. @lucascolley feel free to check out the new PR