Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Searching with duckduckgo.com/html freezes Servo #26807

Closed
atouchet opened this issue Jun 5, 2020 · 24 comments · Fixed by #26810
Closed

Searching with duckduckgo.com/html freezes Servo #26807

atouchet opened this issue Jun 5, 2020 · 24 comments · Fixed by #26810
Labels
A-content/script Related to the script thread I-panic Servo encounters a panic.

Comments

@atouchet
Copy link
Contributor

atouchet commented Jun 5, 2020

URL: https://duckduckgo.com/html

Steps to reproduce:

  1. Load https://duckduckgo.com/html in Servo.
  2. Search something in the DuckDuckGo search bar.
  3. Servo will then freeze.

This issue first occurred in the 2020-06-04 nightly build. Tested in Windows 10.

Regression range: ea491b4...8536cee

@atouchet
Copy link
Contributor Author

atouchet commented Jun 5, 2020

Using the https://duckduckgo.com/ URL still appears to work. It's just the https://duckduckgo.com/html version that has the issue.

@atouchet atouchet changed the title Searching with DuckDuckGo freezes Servo Searching with duckduckgo.com/html freezes Servo Jun 5, 2020
@CYBAI
Copy link
Member

CYBAI commented Jun 6, 2020

I tried to follow the steps and can reproduce it with a panic.

→ ./mach run https://duckduckgo.com/html
assertion failed: incumbent_global.is_some() (thread ScriptThread PipelineId { namespace_id: PipelineNamespaceId(1), index: PipelineIndex(1) }, at components/script/script_runtime.rs:175)
   0: backtrace::backtrace::libunwind::trace
             at /Users/cybai/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.48/src/backtrace/libunwind.rs:86
      backtrace::backtrace::trace_unsynchronized
             at /Users/cybai/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.48/src/backtrace/mod.rs:66
   1: <servo::backtrace::Print as core::fmt::Debug>::fmt
             at /Volumes/Transcend/codespace/mozilla/servo/ports/winit/backtrace.rs:49
   2: core::fmt::write
             at /rustc/a74d1862d4d87a56244958416fd05976c58ca1a8/src/libcore/fmt/mod.rs:1069
   3: std::io::Write::write_fmt
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/io/mod.rs:1537
   4: servo::backtrace::print
             at /Volumes/Transcend/codespace/mozilla/servo/ports/winit/backtrace.rs:17
   5: servo::main::{{closure}}
             at /Volumes/Transcend/codespace/mozilla/servo/ports/winit/main2.rs:150
   6: std::panicking::rust_panic_with_hook
             at /rustc/a74d1862d4d87a56244958416fd05976c58ca1a8/src/libstd/panicking.rs:481
   7: std::panicking::begin_panic
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/panicking.rs:404
   8: script::script_runtime::get_incumbent_global::{{closure}}
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/script_runtime.rs:175
   9: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &mut F>::call_once
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libcore/ops/function.rs:285
  10: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/panic.rs:318
  11: std::panicking::try::do_call
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/panicking.rs:297
  12: ___rust_try
  13: std::panicking::try
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/panicking.rs:274
  14: std::panic::catch_unwind
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/panic.rs:394
  15: mozjs::panic::wrap_panic
             at /Users/cybai/.cargo/git/checkouts/rust-mozjs-8611526964119dd6/28248e1/src/panic.rs:22
  16: script::script_runtime::get_incumbent_global
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/script_runtime.rs:172
  17: _ZN12RustJobQueue18getIncumbentGlobalEP9JSContext
             at /Users/cybai/.cargo/git/checkouts/rust-mozjs-8611526964119dd6/28248e1/src/jsglue.cpp:55
  18: _ZN2js28GetObjectFromIncumbentGlobalEP9JSContextN2JS13MutableHandleIP8JSObjectEE
             at /Users/cybai/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/9a6d8fc/mozjs/js/src/vm/JSObject.cpp:2283
  19: _ZL17NewReactionRecordP9JSContextN2JS6HandleI17PromiseCapabilityEENS2_INS1_5ValueEEES6_21IncumbentGlobalObject
             at /Users/cybai/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/9a6d8fc/mozjs/js/src/builtin/Promise.cpp:3992
  20: _ZL18PerformPromiseThenP9JSContextN2JS6HandleIPN2js13PromiseObjectEEENS2_INS1_5ValueEEES8_NS2_I17PromiseCapabilityEE
             at /Users/cybai/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/9a6d8fc/mozjs/js/src/builtin/Promise.cpp:5045
  21: _ZN2js19OriginalPromiseThenEP9JSContextN2JS6HandleIP8JSObjectEENS3_INS2_5ValueEEES8_NS2_13MutableHandleIS5_EENS_22CreateDependentPromiseE
             at /Users/cybai/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/9a6d8fc/mozjs/js/src/builtin/Promise.cpp:4092
  22: _ZL27CallOriginalPromiseThenImplP9JSContextN2JS6HandleIP8JSObjectEES5_S5_NS1_13MutableHandleIS4_EEN2js22CreateDependentPromiseE
             at /Users/cybai/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/9a6d8fc/mozjs/js/src/jsapi.cpp:3957
      _ZN2JS19AddPromiseReactionsEP9JSContextNS_6HandleIP8JSObjectEES5_S5_
             at /Users/cybai/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/9a6d8fc/mozjs/js/src/jsapi.cpp:3978
  23: _ZN2js41SetUpExternalReadableByteStreamControllerEP9JSContextN2JS6HandleIPNS_14ReadableStreamEEEPNS2_30ReadableStreamUnderlyingSourceE
             at /Users/cybai/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/9a6d8fc/mozjs/js/src/builtin/Stream.cpp:338
  24: _ZN2js14ReadableStream26createExternalSourceStreamEP9JSContextPN2JS30ReadableStreamUnderlyingSourceEPvNS3_6HandleIP8JSObjectEE
             at /Users/cybai/.cargo/git/checkouts/mozjs-fa11ffc7d4f1cc2d/9a6d8fc/mozjs/js/src/builtin/streams/ReadableStream.cpp:86
  25: script::dom::readablestream::ReadableStream::new_with_external_underlying_source
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/dom/readablestream.rs:134
  26: script::dom::readablestream::ReadableStream::new_from_bytes
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/dom/readablestream.rs:105
  27: <alloc::vec::Vec<u8> as script::body::Extractable>::extract
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/body.rs:431
  28: script::dom::htmlformelement::HTMLFormElement::submit_entity_body
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/dom/htmlformelement.rs:805
  29: script::dom::htmlformelement::HTMLFormElement::submit
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/dom/htmlformelement.rs:713
  30: <script::dom::htmlinputelement::HTMLInputElement as script::dom::activation::Activatable>::activation_behavior::{{closure}}
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/dom/htmlinputelement.rs:2832
  31: core::option::Option<T>::map
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libcore/option.rs:458
  32: <script::dom::htmlinputelement::HTMLInputElement as script::dom::activation::Activatable>::activation_behavior
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/dom/htmlinputelement.rs:2831
  33: script::dom::event::Event::dispatch
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/dom/event.rs:343
  34: script::dom::eventtarget::EventTarget::dispatch_event
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/dom/eventtarget.rs:393
  35: script::dom::event::Event::fire
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/dom/event.rs:390
  36: script::dom::document::Document::handle_mouse_event
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/dom/document.rs:1178
  37: script::script_thread::ScriptThread::handle_mouse_event
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/script_thread.rs:3519
  38: script::script_thread::ScriptThread::handle_event
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/script_thread.rs:3381
  39: script::script_thread::ScriptThread::handle_msg_from_constellation
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/script_thread.rs:1836
  40: script::script_thread::ScriptThread::handle_msgs::{{closure}}
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/script_thread.rs:1539
  41: script::script_thread::ScriptThread::profile_event
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/script_thread.rs:1784
  42: script::script_thread::ScriptThread::handle_msgs
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/script_thread.rs:1533
  43: script::script_thread::ScriptThread::start
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/script_thread.rs:1356
  44: <script::script_thread::ScriptThread as script_traits::ScriptThreadFactory>::create::{{closure}}::{{closure}}
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/script_thread.rs:796
  45: profile_traits::mem::ProfilerChan::run_with_memory_reporting
             at /Volumes/Transcend/codespace/mozilla/servo/components/profile_traits/mem.rs:88
  46: <script::script_thread::ScriptThread as script_traits::ScriptThreadFactory>::create::{{closure}}
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/script_thread.rs:794
  47: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/sys_common/backtrace.rs:130
  48: std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/thread/mod.rs:475
  49: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/panic.rs:318
  50: std::panicking::try::do_call
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/panicking.rs:297
  51: ___rust_try
  52: std::panicking::try
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/panicking.rs:274
  53: std::panic::catch_unwind
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/panic.rs:394
  54: std::thread::Builder::spawn_unchecked::{{closure}}
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/thread/mod.rs:474
  55: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libcore/ops/function.rs:232
  56: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/a74d1862d4d87a56244958416fd05976c58ca1a8/src/liballoc/boxed.rs:1034
      <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/a74d1862d4d87a56244958416fd05976c58ca1a8/src/liballoc/boxed.rs:1034
      std::sys::unix::thread::Thread::new::thread_start
             at /rustc/a74d1862d4d87a56244958416fd05976c58ca1a8/src/libstd/sys/unix/thread.rs:87
  57: __pthread_body
  58: __pthread_start

@CYBAI
Copy link
Member

CYBAI commented Jun 6, 2020

Because I saw readablestream in the backtrace, @gterzian you might be interested in this?

@CYBAI CYBAI added A-content/script Related to the script thread I-panic Servo encounters a panic. labels Jun 6, 2020
@CYBAI
Copy link
Member

CYBAI commented Jun 6, 2020

Interesting! I think I figured out how to fix the panic but with that fix, we will get a 400 page instead of a ddg search result page.

@CYBAI
Copy link
Member

CYBAI commented Jun 6, 2020

While checking out to the commit before landing readablestream, I'm able to search successfully in https://duckduckgo.com/html.

@jdm
Copy link
Member

jdm commented Jun 6, 2020

I suspect something about the request body that ddg receives from servo is broken, given those pieces of data.

@CYBAI
Copy link
Member

CYBAI commented Jun 6, 2020

This can fix the panic and yes, looks like the request body is incorrect!

diff --git a/components/script/dom/readablestream.rs b/components/script/dom/readablestream.rs
index 1c6ac242e9..90e6c72da7 100644
--- a/components/script/dom/readablestream.rs
+++ b/components/script/dom/readablestream.rs
@@ -6,7 +6,7 @@ use crate::dom::bindings::conversions::{ConversionBehavior, ConversionResult};
 use crate::dom::bindings::error::Error;
 use crate::dom::bindings::reflector::{reflect_dom_object, DomObject, Reflector};
 use crate::dom::bindings::root::DomRoot;
-use crate::dom::bindings::settings_stack::AutoEntryScript;
+use crate::dom::bindings::settings_stack::{AutoEntryScript, AutoIncumbentScript};
 use crate::dom::bindings::utils::get_dictionary_property;
 use crate::dom::globalscope::GlobalScope;
 use crate::dom::promise::Promise;
@@ -118,6 +118,7 @@ impl ReadableStream {
         source: ExternalUnderlyingSource,
     ) -> DomRoot<ReadableStream> {
         let _ar = enter_realm(global);
+        let _ais = AutoIncumbentScript::new(global);
         let cx = global.get_cx();

         let source = Rc::new(ExternalUnderlyingSourceController::new(source));

btw, @jdm @gterzian while trying to spot the bug, I suspect this should be AutoIncumbentScript instead of AutoEntryScript ?

#[allow(unsafe_code)]
pub fn append_native_handler(&self, handler: &PromiseNativeHandler, _comp: InRealm) {
let _ais = AutoEntryScript::new(&*handler.global());

@gterzian
Copy link
Member

gterzian commented Jun 6, 2020

Nice catch.

I'm not entirely sure about whether append_native_handler should use a AutoEntryScript or a AutoIncumbentScript, I think I chose the former because that it what they were using in Gecko, although I can't be 100% sure that the context of that use is the same as in Servo.

In any case it sounds like a good idea to add the call that you added in your fix.

With regards to the request body, how does it look now?

From the backtrace it looks like the body originates from submit_entity_body, which is entirely read into a Vec<u8> and then "streamed" in one chunk, by-passing Spidermonkey, but maybe something goes wrong along the way?

The various places where would check would be:

let bytes = self.clone();

if let Some(bytes) = self.in_memory.clone() {

let bytes: Vec<u8> = message.to().unwrap();

just seeing we're not actually adding a content-type there, whereas in FF I can see the Post has a application/x-www-form-urlencoded content-type.

Maybe try to add this to:

content_type: None,

@CYBAI
Copy link
Member

CYBAI commented Jun 6, 2020

oh, interesting! I didn't notice we miss the content-type here (I did see it's logged from the request though 🤔)

[2020-06-06T14:50:08Z INFO  net::http_loader] POST request for https://duckduckgo.com/html/
[2020-06-06T14:50:08Z INFO  net::http_loader]  - ("content-type", "application/x-www-form-urlencoded")
[2020-06-06T14:50:08Z INFO  net::http_loader]  - ("accept", "text/html, application/xhtml+xml, application/xml; q=0.9, */*; q=0.8")
[2020-06-06T14:50:08Z INFO  net::http_loader]  - ("accept-language", "en-US, en; q=0.5")
[2020-06-06T14:50:08Z INFO  net::http_loader]  - ("content-length", "11")
[2020-06-06T14:50:08Z INFO  net::http_loader]  - ("origin", "https://duckduckgo.com")
[2020-06-06T14:50:08Z INFO  net::http_loader]  - ("user-agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:75.0) Servo/1.0 Firefox/75.0")
[2020-06-06T14:50:08Z INFO  net::http_loader]  - ("accept-encoding", "gzip, deflate, br")

@gterzian
Copy link
Member

gterzian commented Jun 6, 2020

Also another thing: should we not enter into a realm or a script before calling ScriptThread::handle_mouse_event? That happens straight from handling a constellation message. I'm guessing that's why the call to create the stream fails because there is no incumbent global, but should there not be one already before in any case since we're about to fire an event?

@gterzian
Copy link
Member

gterzian commented Jun 6, 2020

Ah ok, so htmlformelement sets headers directly around https://github.com/servo/servo/pull/25873/files#diff-db2031d1f23ecf1a921fc100cfed753dR774

But since the RequestBody still have a None content-type, maybe this fails the request somewhere in the logic in net?

I think it's worth a try to add a Some(DOMString::from( "application/x-www-form-urlencoded;charset=UTF-8", ));
at

content_type: None,

@CYBAI
Copy link
Member

CYBAI commented Jun 6, 2020

Maybe try to add this to:

content_type: None,

I found we've handled the content-type in htmlformelement while submitting the form. (I also tried to add a Some(DOMString::from( "application/x-www-form-urlencoded;charset=UTF-8")); and confirmed it didn't help)

FormEncType::UrlEncoded => {
let charset = encoding.name();
load_data
.headers
.typed_insert(ContentType::from(mime::APPLICATION_WWW_FORM_URLENCODED));

should there not be one already before in any case since we're about to fire an event?

I'm not very sure but if I understand correctly about AutoIncumbentScript, it's required for things like calling JS callbacks or promises (it's kind of JS callback?). We might not always need to get into realm in the constellation? if we just get into the realm for incumbent, would we hold the incumbent for a long time which might be unnecessary? 🤔

@gterzian
Copy link
Member

gterzian commented Jun 6, 2020

Ok so I'd say your fix at #26807 (comment)

then the question is still what is wrong with the request body when it is sent...

@CYBAI
Copy link
Member

CYBAI commented Jun 6, 2020

then the question is still what is wrong with the request body when it is sent...

one thing I also noticed is, if I set a breakpoint at this line 👇

if let Some(bytes) = self.in_memory.clone() {

then I will endlessly get into it and can inspect the same bytes.

Does that mean we get into an infinite loop and result in a 400 request 🤔?

@gterzian
Copy link
Member

gterzian commented Jun 6, 2020

Yes, please add let _ = self.control_sender.send(BodyChunkRequest::Done); 🤣

@CYBAI
Copy link
Member

CYBAI commented Jun 6, 2020

ooooooooooooh!!!!! gooooood catch!!! let me try it 🤣🤣🤣

@CYBAI
Copy link
Member

CYBAI commented Jun 6, 2020

Interesting! With sending the Done message, we do stop the bytes_sender of request body but we then get a panic because it's None.

I suspect the IPC router in obtain_response around line 443 to 467 is also getting into an infinite loop as well 🤔 Will check it tmr!

No bytes sender to transmit chunk. (thread <unnamed>, at components/script/body.rs:160)
   0: backtrace::backtrace::libunwind::trace
             at /Users/cybai/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.48/src/backtrace/libunwind.rs:86
      backtrace::backtrace::trace_unsynchronized
             at /Users/cybai/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.48/src/backtrace/mod.rs:66
   1: <servo::backtrace::Print as core::fmt::Debug>::fmt
             at /Volumes/Transcend/codespace/mozilla/servo/ports/winit/backtrace.rs:49
   2: core::fmt::write
             at /rustc/a74d1862d4d87a56244958416fd05976c58ca1a8/src/libcore/fmt/mod.rs:1069
   3: std::io::Write::write_fmt
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/io/mod.rs:1537
   4: servo::backtrace::print
             at /Volumes/Transcend/codespace/mozilla/servo/ports/winit/backtrace.rs:17
   5: servo::main::{{closure}}
             at /Volumes/Transcend/codespace/mozilla/servo/ports/winit/main2.rs:150
   6: std::panicking::rust_panic_with_hook
             at /rustc/a74d1862d4d87a56244958416fd05976c58ca1a8/src/libstd/panicking.rs:481
   7: rust_begin_unwind
             at /rustc/a74d1862d4d87a56244958416fd05976c58ca1a8/src/libstd/panicking.rs:385
   8: core::panicking::panic_fmt
             at /rustc/a74d1862d4d87a56244958416fd05976c58ca1a8/src/libcore/panicking.rs:89
   9: core::option::expect_failed
             at /rustc/a74d1862d4d87a56244958416fd05976c58ca1a8/src/libcore/option.rs:1264
  10: core::option::Option<T>::expect
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libcore/option.rs:349
  11: script::body::TransmitBodyConnectHandler::transmit_body_chunk
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/body.rs:160
  12: script::body::ExtractedBody::into_net_request_body::{{closure}}
             at /Volumes/Transcend/codespace/mozilla/servo/components/script/body.rs:351
  13: <alloc::boxed::Box<F> as core::ops::function::FnMut<A>>::call_mut
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/liballoc/boxed.rs:1041
  14: ipc_channel::router::Router::run
             at /Users/cybai/.cargo/registry/src/github.com-1ecc6299db9ec823/ipc-channel-0.14.1/src/router.rs:198
  15: ipc_channel::router::RouterProxy::new::{{closure}}
             at /Users/cybai/.cargo/registry/src/github.com-1ecc6299db9ec823/ipc-channel-0.14.1/src/router.rs:50
  16: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/sys_common/backtrace.rs:130
  17: std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/thread/mod.rs:475
  18: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/panic.rs:318
  19: std::panicking::try::do_call
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/panicking.rs:297
  20: ___rust_try
  21: std::panicking::try
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/panicking.rs:274
  22: std::panic::catch_unwind
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/panic.rs:394
  23: std::thread::Builder::spawn_unchecked::{{closure}}
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libstd/thread/mod.rs:474
  24: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /Users/cybai/.rustup/toolchains/nightly-2020-05-15-x86_64-apple-darwin/lib/rustlib/src/rust/src/libcore/ops/function.rs:232
  25: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/a74d1862d4d87a56244958416fd05976c58ca1a8/src/liballoc/boxed.rs:1034
      <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/a74d1862d4d87a56244958416fd05976c58ca1a8/src/liballoc/boxed.rs:1034
      std::sys::unix::thread::Thread::new::thread_start
             at /rustc/a74d1862d4d87a56244958416fd05976c58ca1a8/src/libstd/sys/unix/thread.rs:87
  26: __pthread_body
  27: __pthread_start

@CYBAI
Copy link
Member

CYBAI commented Jun 7, 2020

ok, I just tried a workaround and it fixed the issue!

In this IPC router, we will just send a Chunk request for step 5.1.2.3 of concept-request-transmit-body; however, we might want to take step 5.1.3 which means, when the stream is already done, we won't send chunks anymore.

sender
.send(bytes)
.map(move |_| {
// Step 5.1.2.3
// Request the next chunk.
let _ = chunk_requester.send(BodyChunkRequest::Chunk);
()
})
.map_err(|_| ()),

I say workaround because I wonder doing the checking in above IPC router is more straightforward? but, it seems we don't hold a RequestBody there so we might can only handle it in the following IPC router?

diff --git a/components/script/body.rs b/components/script/body.rs
index 8017519cdd..4ac1b780bd 100644
--- a/components/script/body.rs
+++ b/components/script/body.rs
@@ -165,6 +165,7 @@ impl TransmitBodyConnectHandler {
         // In case of the data being in-memory, send everything in one chunk, by-passing SpiderMonkey.
         if let Some(bytes) = self.in_memory.clone() {
             let _ = bytes_sender.send(bytes);
+            let _ = control_sender.send(BodyChunkRequest::Done);
             return;
         }

@@ -346,7 +347,11 @@ impl ExtractedBody {
                     BodyChunkRequest::Extract(receiver) => {
                         body_handler.re_extract(receiver);
                     },
-                    BodyChunkRequest::Chunk => body_handler.transmit_body_chunk(),
+                    BodyChunkRequest::Chunk => {
+                        if body_handler.bytes_sender.is_some() {
+                            body_handler.transmit_body_chunk();
+                        }
+                    },
                     // Note: this is actually sent from this process
                     // by the TransmitBodyPromiseHandler when reading stops.
                     BodyChunkRequest::Done => body_handler.stop_reading(),

@CYBAI
Copy link
Member

CYBAI commented Jun 7, 2020

Maybe let me send a PR and discuss there!

@CYBAI
Copy link
Member

CYBAI commented Jun 7, 2020

Also, it might be worth adding a new test for this 🤔?

@gterzian
Copy link
Member

gterzian commented Jun 7, 2020

Ok good one.

I think the way to do it so that it matches the flow in the case where we go through the native promise handler, is to add a in_memory_done flag to TransmitBodyConnectHandler, and then set it instead of doing the let _ = control_sender.send(BodyChunkRequest::Done);. (we could still do it there I think, but it's more consistent with the promise-based flow do send the Done message when the next chunk is requested and the flag is set).

Then at the top of transmit_body_chunk, check the flag and if it's set do the let _ = control_sender.send(BodyChunkRequest::Done); and return. Or actually even better, just do self.stop_reading() and return.

So "normally", when we use the native promise handler, then when we send the "last" chunk, the other route in http_loader will then always request another chunk, however then back in script we'll see that the stream is "done"(which we don't know in advance), and then we just drop the sender and do not send another chunk, stopping the flow.

So I think we should replicate this with a flag in the in_memory case(even though there we do know in advance it's done), so that we mark the data as "done", and then when the next chunk is requested we actually stop, as opposed to immediately dropping the sender which is "too early".

Yes I think you can add a test in the mozilla folder, I think you can get the failure by doing a POST using a Blob too(might be easier to write than a form).

@CYBAI
Copy link
Member

CYBAI commented Jun 7, 2020

I see! Thanks for the clear explanation!

as opposed to immediately dropping the sender which is "too early".

Now I can understand why it's better to be handled in transmit_body_chunk 👀!

Yes I think you can add a test in the mozilla folder, I think you can get the failure by doing a POST using a Blob too(might be easier to write than a form).

Thanks! Will try to do that!

CYBAI added a commit to CYBAI/servo that referenced this issue Jun 7, 2020
As discussed in servo#26807 (comment), we'd like to
add a new flag, `in_memory_done`, to `TransmitBodyConnectHandler` so
that we can correctly finish and drop the sender correctly.

When we send the bytes, we will mark the body as done and we can
recognize it's already done in next tick so that we can send a Done
request to finish the sender.
bors-servo added a commit that referenced this issue Jun 7, 2020
Fix infinite stream and its missing incumbent script environment when newing a new stream

---
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix #26807
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___
CYBAI added a commit to CYBAI/servo that referenced this issue Jun 7, 2020
As discussed in servo#26807 (comment), we'd like to
add a new flag, `in_memory_done`, to `TransmitBodyConnectHandler` so
that we can correctly finish and drop the sender correctly.

When we send the bytes, we will mark the body as done and we can
recognize it's already done in next tick so that we can send a Done
request to finish the sender.
CYBAI added a commit to CYBAI/servo that referenced this issue Jun 12, 2020
As discussed in servo#26807 (comment), we'd like to
add a new flag, `in_memory_done`, to `TransmitBodyConnectHandler` so
that we can correctly finish and drop the sender correctly.

When we send the bytes, we will mark the body as done and we can
recognize it's already done in next tick so that we can send a Done
request to finish the sender.

Also, when there comes a redirect request, it will go to `re-extract`
route, we can set the `done` flag to `false` which means we won't
stop the IPC routers yet. Then, if the re-extract sent its bytes, we
will be marked as done again so that we can finish with stopping the IPC
routes when Chunk request comes.
bors-servo added a commit that referenced this issue Jun 12, 2020
Fix infinite stream and its missing incumbent script environment when newing a new stream

---
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix #26807
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___
CYBAI added a commit to CYBAI/servo that referenced this issue Jun 12, 2020
As discussed in servo#26807 (comment), we'd like to
add a new flag, `in_memory_done`, to `TransmitBodyConnectHandler` so
that we can correctly finish and drop the sender correctly.

When we send the bytes, we will mark the body as done and we can
recognize it's already done in next tick so that we can send a Done
request to finish the sender.

Also, when there comes a redirect request, it will go to `re-extract`
route, we can set the `done` flag to `false` which means we won't
stop the IPC routers yet. Then, if the re-extract sent its bytes, we
will be marked as done again so that we can finish with stopping the IPC
routes when Chunk request comes.
CYBAI added a commit to CYBAI/servo that referenced this issue Jun 13, 2020
As discussed in servo#26807 (comment), we'd like to
add a new flag, `in_memory_done`, to `TransmitBodyConnectHandler` so
that we can correctly finish and drop the sender correctly.

When we send the bytes, we will mark the body as done and we can
recognize it's already done in next tick so that we can send a Done
request to finish the sender.

Also, when there comes a redirect request, it will go to `re-extract`
route, we can set the `done` flag to `false` which means we won't
stop the IPC routers yet. Then, if the re-extract sent its bytes, we
will be marked as done again so that we can finish with stopping the IPC
routes when Chunk request comes.
CYBAI added a commit to CYBAI/servo that referenced this issue Jun 13, 2020
As discussed in servo#26807 (comment), we'd like to
add a new flag, `in_memory_done`, to `TransmitBodyConnectHandler` so
that we can correctly finish and drop the sender correctly.

When we send the bytes, we will mark the body as done and we can
recognize it's already done in next tick so that we can send a Done
request to finish the sender.

Also, when there comes a redirect request, it will go to `re-extract`
route, we can set the `done` flag to `false` which means we won't
stop the IPC routers yet. Then, if the re-extract sent its bytes, we
will be marked as done again so that we can finish with stopping the IPC
routes when Chunk request comes.
CYBAI added a commit to CYBAI/servo that referenced this issue Jun 13, 2020
As discussed in servo#26807 (comment), we'd like to
add a new flag, `in_memory_done`, to `TransmitBodyConnectHandler` so
that we can correctly finish and drop the sender correctly.

When we send the bytes, we will mark the body as done and we can
recognize it's already done in next tick so that we can send a Done
request to finish the sender.

Also, when there comes a redirect request, it will go to `re-extract`
route, we can set the `done` flag to `false` which means we won't
stop the IPC routers yet. Then, if the re-extract sent its bytes, we
will be marked as done again so that we can finish with stopping the IPC
routes when Chunk request comes.
bors-servo added a commit that referenced this issue Jun 13, 2020
Fix infinite stream and its missing incumbent script environment when newing a new stream

---
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix #26807
- [ ] There are tests for these changes OR
- [ ] These changes do not require tests because ___
@CYBAI
Copy link
Member

CYBAI commented Jun 13, 2020

Phew! Thanks @gterzian a lot! This is my first time to play with IPC things! and it makes me wonder if I can leverage the power of IPC to make module script work better and more readable 🤔

@gterzian
Copy link
Member

gterzian commented Jun 13, 2020

You're welcome, let me know what you find out!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-content/script Related to the script thread I-panic Servo encounters a panic.
Projects
None yet
4 participants