Add method to read file into user given Uint8Array

[petkaantonov]

Reading file contents into user supplied array avoids unnecessary copies. For example you can just get a pointer to an area in a WASM memory and directly read the file instead of first reading it into an useless ArrayBuffer and then doing a slow copy:

// file is a File or Blob
// wasm is a WebAssembly module instance wrapper with some additional methods like malloc added
const reader = new FileReader();
const ptr = wasm.malloc(file.size);
const array = new Uint8Array(wasm.memory.buffer, ptr, file.size);
try {
    let bytesRead = await reader.readIntoUint8Array(file, array);
    // Done
} catch (e) {
   // Error
}

It also allows reading files while using minimal memory and allocations:

const reader = new FileReader();
const bufferPtr = wasm.malloc(8192);
const buffer = new Uint8Array(wasm.memory.buffer, bufferPtr, 8192);
let totalBytesRead = 0;

while (totalBytesRead < file.size) {
    const start = totalBytesRead;
    const end = Math.min(start + 8192, file.size);
    const bytesRead = await readIntoUint8Array(file.slice(start, end), buffer);
    wasm.process(bufferPtr, bytesRead);
    totalBytesRead += bytesRead;
}

Doing the same with current APIs would allocate a ton of small ArrayBuffers...

[mkruisselbrink]

This sounds very similar to what you would be able to do with a https://streams.spec.whatwg.org/#byob-reader-class right? I'm not necessarily against having similar functionality in FileReader as well, but I think eventually the streams API should be the one way to stream data from whatever source, so I'd be hesitant to add more functionality to FileReader. A somewhat sensible API to get a ReadableStream out of a Blob would seem like a much better solution to improving the ways you can read from blobs/files (note that you can already get a ReadableStream for a blob by wrapping the blob in a Response and calling its body method).

[domenic]

See also #40

[petkaantonov]

Readable stream with byob sounds kinda like what I'm looking for but since it's not a specific API I fear it will still internally perform copy.

[thiccar]

Until BYOB reader is supported by more browsers, this would be nice to have for use with SharedArrayBuffer.

[Fuzzyma]

For me it is not clear how byob-streams will solve this problem.
It is not possible to get a stream from a file isnt it? As far as I understand the streaming api (https://streams.spec.whatwg.org/) there is no plan on creating a stream from a file-input.
The FileReader still is the only option to create a buffer from the file-input. Even if you slice the file into chunks and stream it via a byob-stream, you still need to copy the contents of the FileReader-Buffer over to the buffer from the byob-stream.
It would be far easier, to just pass the byob-buffer to readAsArrayBuffer(file, buf).
The required changes for something like that should be very small!

Please enlighten me on this issue!

[annevk]

@Fuzzyma #117 adds blobOrFile.stream() (among some nice-to-haves).

[saschanaz]

Fixed by #188 I'd say.