Fix infinite stream and its missing incumbent script environment when newing a new stream

[CYBAI]

---

• ./mach build -d does not report any errors
• ./mach test-tidy does not report any errors
• These changes fix #26807
• There are tests for these changes OR
• These changes do not require tests because ___

[highfive]

Heads up! This PR modifies the following files:

• @asajeffrey: components/script/dom/readablestream.rs, components/script/body.rs
• @KiChjang: components/script/dom/readablestream.rs, components/script/body.rs

[highfive]

Warning

• These commits modify script code, but no tests are modified. Please consider adding a test!

[CYBAI]

r? @gterzian or @jdm

hmmm.... I tried several ways for an unit test but didn't figure out a workable version :/

a simplest test is like this; I confirmed it will get into the IPC router but we will even get it pass without patches in this PR... 🤔

promise_test((t) => {
  var blob = new Blob(["This is my blob content"], { type: "text/plain" });

  var formData = new FormData();
  formData.append("blob", blob, "blob.txt");

  return fetch("/fetch/api/resources/echo-content.py", {
    method: "POST",
    body: formData,
  });
}, "Fetch with POST blob");

[CYBAI]

@bors-servo try=wpt

• Just curious will this accidentally fix some tests?

[bors-servo]

⌛ Trying commit 4835422 with merge e7ee127...

[gterzian]

Thanks for the fixes! Just a few comments...

I'm not sure about the tests, actually I was surprised this wasn't caught before. Maybe you need a bit of a delay like would happen in a real network, you can try using https://web-platform-tests.org/writing-tests/server-pipes.html#trickle

[bors-servo]

💔 Test failed - status-taskcluster

[CYBAI]

hmm... interesting, even I do ?pipe=trickle(100:d1:r2), I can still pass the test on current master :/

[CYBAI]

ah!! cool! I think 100 bytes is too big, with changing it to 1 byte, I can get a TIMEOUT on current master now! Let me try to verify if it works for this PR!

[CYBAI]

ah!! cool! I think 100 bytes is too big, with changing it to 1 byte, I can get a TIMEOUT on current master now! Let me try to verify if it works for this PR!

hmm, this is not true; 1:d1:r2 will also TIMEOUT in other browsers :/ but if I try 10:d1:r2, Servo on current master won't crash either... 🤔

[CYBAI]

💔 Test failed - status-taskcluster

ok, I can reproduce these TIMEOUTs locally. Will try to investigate it 👀

[gterzian]

I know what the problem is, so let me know if you want to try to find it yourself first 😄

hint: look at what is done to the TransmitBodyConnectHandler when a re-direct happens, something is missing from that given your new changes...

[gterzian]

Actually I'm not so sure I know what the problem is, but yeah the TransmitBodyConnectHandler is definitely manipulated as part of re-direct, so I'd say just add some print statements at the various places and you should be able to figure it out...

You might also want to check the interplay between the below and what's happening in body.rs

servo/components/net_traits/request.rs

Line 173 in 99ab109

BodySource::Object => {

I'm just wondering if it would be possible for the self.chan to drop, and then the other side to remove the route, before BodyChunkRequest::Extract(port) is received by the other side. And then someone what we are now doing with self.stop_reading results in all corresponding senders for the route to drop too quickly, and we're exposing a problem that was hidden previously?

@jdm I couldn't find the details in the semantics of ipc_channel, and I'm wonidering if one does:

1. let _ = self.chan.send(BodyChunkRequest::Extract(port));
2. self.chan = chan.clone();

Can you be certain that the other side will first receive the message, and then notice the channel being closed? Or is it possible for the channel to close first?

That's one thing I can imagine could cause a problem if the route handler is removed too early. Because in theory if we first receive the message, then a new route is setup before the old one being removed, but if the channel closes first, it could break the re-direct.

@CYBAI we could test this by ensuring the BodyChunkRequest::Extract is received, if it's not received and the test times-out, that could be it...

[gterzian]

Yeah reading on it, it seems if you close a file descriptor, you loose all data in transit, so it reads a bit like dropping the chan might close the receiver, and remove the route, even though there is still a message in transit.

@CYBAI you could try to add a old_chans: Vec<IpcSender<BodyChunkRequest>> to RequestBody, and then before doing

servo/components/net_traits/request.rs

Line 176 in 99ab109

self.chan = chan.clone();

do a self.old_chans.push(self.chan.clone()), which should keep the route alive while the request is being re-directed.

I think it's the self.stop_reading that now happens earlier, and it results in removing the route over at

servo/components/net/http_loader.rs

Line 449 in 0645d8c

ROUTER.add_route(

which then drops it's IpcSender<BodyChunkRequest>, and then the RequestBody still has a sender, however it is then dropped on the next take_stream, and then the route in body.rs closes before the new route for the re-direct in setup in response to receiving the BodyChunkRequest::Extract message.

So previously we would not actually call the self.stop_reading, or do it later in the promise callback, so now it happens earlier and then senders drop too soon.

Maybe, it's worth a try...

[CYBAI]

@gterzian thanks for the tip! I will try it later!

[CYBAI]

@gterzian I tried to debug with removing the if self.in_memory_done checking and yes, looks like we drop the IPC router too early.

I haven't tried the old_chans way though, I try to read the spec again.

Then, I just realized, in the step 5.1.3 of request transmit body, it says if read is fulfilled with an object whose done property is true.

The done property means the done property from the readablestream. Looks like it's handled by SM and we can simply just get the value.

servo/components/script/dom/readablestream.rs

Lines 503 to 505 in 0645d8c

	#[allow(unsafe_code)]
	/// Get the `done` property of an object that a read promise resolved to.
	pub fn get_read_promise_done(cx: SafeJSContext, v: &SafeHandleValue) -> Result<bool, Error> {

I'm wondering, if we can know the stream is done or not by that property, maybe we don't need to add a flag like in_memory_done by ourselves?

But, here comes my question:

I found I don't really understand why if we have something in memory, we will just send it and return 🤔

If we can check if done or not by the done property, maybe it's cleaner than adding a old_chans? WDYT?

servo/components/script/body.rs

Lines 165 to 169 in 0645d8c

	// In case of the data being in-memory, send everything in one chunk, by-passing SpiderMonkey.
	if let Some(bytes) = self.in_memory.clone() {
	let _ = bytes_sender.send(bytes);
	return;
	}

[gterzian]

Yes, thanks for looking into these matters.

So in the case where this problem happens, we have all the bytes in memory, and we don't actually go through the stream and SpiderMonkey. So we're not following the spec to the letter in that case, and I think that's ok(there will be many more cases like this, actually we should find a better way to do this, see for example #26709, #26759, and #26686).

So we need the in_memory_done flag, because we don't actually read from the stream and we will not get the done flag from the promise.

yes, looks like we drop the IPC router too early.

Then I think using this old_chans approach to keep the senders alive for the life of the request, is worth a try to see if it fixes it.

[CYBAI]

So we need the in_memory_done flag, because we don't actually read from the stream and we will not get the done flag from the promise.

Indeed! When there's no need to do chunks, it makes sense to just return the in-memory bytes!

Okay! I will give old_chans a try! Will update anything here again!

[CYBAI]

So previously we would not actually call the self.stop_reading, or do it later in the promise callback, so now it happens earlier and then senders drop too soon.

@gterzian I found I don' fully understand what you mean here. Do you mean, we shouldn't drop the TransmitBodyConnectHandler when in_memory is done? If yes, then what I don't understand is how we should drop it 🤔

Does that mean we should drop it in this router?

servo/components/script/body.rs

Line 338 in aaa6cea

ROUTER.add_route(

[gterzian]

It's just what when we do stop_reading, then this will drop the sender corresponding to the route in obtain_response. The changes here call stop_reading earlier than before, so the route in obtain_response will be removed earlier.

So then I think this is causing the current problem, because there are two places that own a IpcSender<BodyChunkRequest>:

1. The route in http_loader,
2. The RequestBody.

The sender in RequestBody drops here

So I think previously, during a re-direct, when RequestBody would drop it's sender, then the route in obtain_response would still have it's sender, because stop_reading had not been called yet(but soon would be).

So now we're calling stop_reading "earlier", which means that when RequestBody drops it's sender, then that's actually the last sender and the route with the TransmitBodyConnectHandler is removed, and then we get the timeout, because the BodyChunkRequest::Extract message isn't handled because the route is already gone(I think).

Or at least I think that is what is happening. And we can solve it by not dropping those sender upon a redirect in RequestBody, instead storing a clone in a Vec<IpcSender<BodyChunkRequest>> on RequestBody. (Note that the "old" chan is swapped for a new one on each re-direct, so instead of dropping the old ones we just store them in the vector to keep the routes alive).

[gterzian]

Ok, thanks for the latest fixes. There is still one point that we should address I think.

[gterzian]

Looks good, there's just one comment that can be made more effective I think.

[gterzian]

Great work!

@bors-servo r+

[gterzian]

@bors-servo r+

[bors-servo]

📌 Commit 63aab1b has been approved by gterzian

[CYBAI]

Ha, this reminds me homu doesn't support commands from review comment yet 😂

[bors-servo]

⌛ Testing commit 63aab1b with merge 1564f53...

[bors-servo]

💔 Test failed - status-taskcluster

[gterzian]

@bors-servo retry #26895

[bors-servo]

⌛ Testing commit 63aab1b with merge 581ade5...

[bors-servo]

☀️ Test successful - status-taskcluster
Approved by: gterzian
Pushing 581ade5 to master...