[idea]: Web API Response for Store?

[manzt]

Rather than returning Promise<Uint8Array> , we could unify the store interface around Web-standard Response to expose additional features to zarr.Array instances.

Changes:

• Use the Response.ok or response.status to determine if the item is in the store or not (rather than Uint8Array | undefined).
• Exposes conveniences to read response directly via .json(), .text(), .arrayBuffer(), and .blob(), rather than copying array buffers etc.
• Ability process Response.body as a stream. This is the most interesting/exciting to me, but it's not clear if it would be useful. Ideally there is a world were chunks could be partially decoded, etc.

Rather than handling Responses within zarr.Array, we could look into something like a StorageTransformer layer, which by default just reads the response in full.

• Add Sharding v1.0 Spec & Storage Transformers to Zarr v3.0 zarr-developers/zarr-specs#134

[manzt]

hmm doesn't seem like Response is provided as a global in Node.js... would likely need to allow the current interface as well with Uint8Array.

Actually this works in Node v18:

import { open } from 'node:fs/promises';

let file = await open('./package.json');
let response = new Response(file.readableWebStream());
await response.json();
file.read() // promise! oh no, file still open :x 

EDIT: Errr actually, a tiny detail in the node docs:

While the ReadableStream will read the file to completion, it will not close the FileHandle automatically. User code must still call the fileHandle.close() method.

Since we don't want to manage open file handles, it seems like the correct way to do this is to first create a Readable which auto-closes the file (when read) and get a ReadableStream from that. This way the caller is in control over then the file is closed (and doesn't need a reference to the original FileHandle).

import { open } from 'node:fs/promises';
import { Readable } from 'node:stream';

let file = await open('./package.json');
let readable = file.createReadStream({ autoClose: true });
let response = new Response(Readable.toWeb(readable));
await response.json()
file.read() // Error! file closed