New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding generic seek or read/write-at-offset abilities to readable/writable streams #1128
Comments
Here is the start of a proposal for a seek-based API:
In this model, the underlying source/sink is responsible for knowing what seek means, and how it interacts with reads/writes. The expectation is that they implement things so that reads/writes advance the current position, e.g. Here is the start of a proposal for an offset-based API:
On balance the offset-based API seems a bit cleaner. |
In my zip file reader, I used the |
What if an http request supports byte range requests?
or how about http2, would you be able to do some seeking with that? |
FWIW there is a difference between streaming read, where you probably want aggressive readahead etc., and delimited random access read, where you probably don't and might even want discard-type behavior. So I feel like the solution here might need to be at a higher level, where a |
A lot of the streams ecosystem so far has focused on network and device streams, where data is definitely sequential and read in order. However, File System Access and the Storage Foundation API (which will hopefully soon become an extension of File System Access) operate on file system streams, which are slightly different. In particular, a common operation on file system streams is random access, i.e. reading/writing to a specified offset.
One could imagine using a separate API for random access, and leaving streams only for the kind of streaming sequential reads/writes that they're already good at. But this feels like a bad outcome. It would result in two similar APIs side-by-side, e.g. read/cancel on stream readers, and read/cancel/seek on random access readers.
Instead we could imagine augmenting stream readers/writers to support this use case. If the underlying source/sink advertises seeking or random access support, then the corresponding reader could expose that capability. Most streams on the web platform today would not support random access. (E.g., seeking a HTTP response doesn't make much sense. Except maybe seeking forward?) But file system streams, and maybe
blob.stream()
, could support it.There are a few API details that come to mind:
Is
reader.seek(offset)
the right API, or should it be something likereader.read(view, { offset })
orreader.readAt(offset, view)
? (That's the BYOB case; omit theview
for the default reader case.) This seems like a big fork in the road that affects other parts of the API. E.g. if it's a seek-type API, then we need to consider how to queue up the seeks vs. the reads/writes/read requests, or how reads/writes advance the "current position".Should this be done via adding a
seek()
orreadAt()
method toReadableStreamDefaultReader
/ReadableStreamBYOBReader
/WritableStreamDefaultWriter
, which throws if the underlying sink doesn't support it? Or should we create dedicated "seekable reader/writer" classes or subclasses? The former is a good deal simpler on the spec and implementation side, and is perhaps a better precedent for any future such expansions. But then feature detection would need some kind ofcanSeek
orsupportsOffset
getter, which is a bit annoying.What are the "units" for the seeking
offset
? They could be totally opaque: just a value you pass through to the underlying source/sink. (This starts feeling like some of the generic message-passing mechanisms discussed in Idiomatic Way to Explicitly Tell an Underlying Transform Stream to Flush to the Sink #960 and Reset for TransformStream (and WritableStream?) #1026.) Or there could be some minimal validation, e.g. has to be a number (integer?), has to be nonnegative, has to be finite.Relatedly, should there be a convention for whether seeking past the end throws an exception vs. clamps to the end? I don't know if we can enforce this in the streams infrastructure, but if we could that'd be cool.
The text was updated successfully, but these errors were encountered: