Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upIntegrate file-reading with streams #26759
Open
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Similar to #26743 (although perhaps smaller in scope and a bit easier).
When reading a file, we currently just loop and read chunks from the file "as fast as possible" and send each chunk to
scriptover IPC.servo/components/net/filemanager_thread.rs
Line 211 in f240140
Instead, we could use a mechanism as described in the linked-to issue, where the stream in
scriptwould "pull" chunks of the file fromnet, based on it's own(and possibly user provided) queuing and high-watermark strategies.That way, one could imagine a situation where a file is read from disk, chunks are pulled into script by a readable stream, then perhaps put through a transform stream of some kind, and then transmitted over the network as the body of a request(another stream). Then, as bytes of the request body would be transmitted over the network, this would result in more chunks being "pulled" from the file on disk(once the internal buffer of a stream would go below some desired size).
(One can also imagine a fast path where if it's just the native underlying source for the blob stream that is directly feeding data into another native underlying source for the stream of the request body, we wouldn't have to go through SpiderMonkey, and perhaps we wouldn't event have to send file data to script over IPC at all, although it might be hard to know in advance how the stream will end-up being used in script, if at all)
Currently we'd just have one thread n
netreading the file as fast as possible, sending bytes toscriptover IPC where they would accumulate in the buffer of the blob stream.