Replace mpsc with crossbeam-channel #21325

gterzian · 2018-08-02T16:10:32Z

Follow up on #19515

Selecting over multiple channels in std::sync::mpsc is not stable and likely never will be:

It seems the only thing keeping mpsc_select around is Servo.

crossbeam-channel is designed specifically to replace std::sync::mpsc and fix many of its shortcomings:
https://github.com/stjepang/rfcs-crossbeam/blob/channel/text/2017-11-09-channel.md

This is to be landed together with servo/ipc-channel#183.

This change is

highfive · 2018-08-02T16:10:36Z

Heads up! This PR modifies the following files:

@jgraham: components/webdriver_server/lib.rs, components/webdriver_server/Cargo.toml
@emilio: components/style/animation.rs, components/style/Cargo.toml, components/style/lib.rs, components/style/context.rs, components/layout/animation.rs and 2 more
@KiChjang: components/script_traits/lib.rs, components/script/dom/abstractworkerglobalscope.rs, components/script/dom/servoparser/async_html.rs, components/net/http_loader.rs, components/script/dom/workerglobalscope.rs and 42 more
@asajeffrey: components/script/dom/abstractworkerglobalscope.rs, components/script/dom/servoparser/async_html.rs, components/script/dom/workerglobalscope.rs, components/script/task_source/dom_manipulation.rs, components/script/Cargo.toml and 39 more
@cbrewster: components/constellation/timer_scheduler.rs, components/constellation/network_listener.rs, components/constellation/constellation.rs, components/constellation/lib.rs, components/constellation/pipeline.rs and 1 more
@edunham: servo-tidy.toml
@paulrouget: components/constellation/timer_scheduler.rs, components/compositing/compositor_thread.rs, components/compositing/compositor.rs, components/compositing/Cargo.toml, components/constellation/network_listener.rs and 8 more

gterzian · 2018-08-02T17:33:27Z

@bors-servo try

bors-servo · 2018-08-02T17:33:32Z

⌛ Trying commit cb3a52d with merge d6fbaa9...


        Auto merge of #21325 - gterzian:crossbeam_integration, r=<try>

Crossbeam integration  Follow up on #19515 ---  - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #__ (github issue number if applicable).  - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____    --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21325)

bors-servo · 2018-08-02T17:51:01Z

💔 Test failed - linux-dev

stjepang · 2018-08-02T17:57:45Z

@gterzian @SimonSapin

In order to resolve the tidy errors, I'm currently updating dependencies of the crossbeam crates. Just waiting for the Travis tests to pass... (will take an hour or so)

stjepang · 2018-08-02T19:03:15Z

Done! Can we retry the tests?

gterzian · 2018-08-02T21:05:21Z

{"status": "TIMEOUT", "group": "default", "message": null, "stack": null, "subtest": null, "test": "/_mozilla/webgl/conformance-1.0.3/conformance/renderbuffers/framebuffer-object-attachment.html", "line": 103695, "action": "test_result", "expected": "CRASH"}
{"status": "TIMEOUT", "group": "default", "message": null, "stack": null, "subtest": null, "test": "/_mozilla/webgl/conformance-2.0.0/conformance2/renderbuffers/framebuffer-object-attachment.html", "line": 106426, "action": "test_result", "expected": "CRASH"}

{"status": "TIMEOUT", "group": "default", "message": null, "stack": null, "subtest": null, "test": "/_mozilla/webgl/conformance-1.0.3/conformance/canvas/drawingbuffer-test.html", "line": 42832, "action": "test_result", "expected": "CRASH"}
{"status": "TIMEOUT", "group": "default", "message": null, "stack": null, "subtest": null, "test": "/_mozilla/webgl/conformance-1.0.3/conformance/canvas/drawingbuffer-static-canvas-test.html", "line": 50434, "action": "test_result", "expected": "CRASH"}

Actually getting a bunch of positive unexpected test results. Also some websocket timeouts, but I think those are intermittents.

I just removed another similar default in serviceworker...

@stjepang thanks, retrying now

@bors-servo try

bors-servo · 2018-08-02T21:05:27Z

⌛ Trying commit ca86ae9 with merge 0443bcf...


        Auto merge of #21325 - gterzian:crossbeam_integration, r=<try>

Crossbeam integration  Follow up on #19515 ---  - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #__ (github issue number if applicable).  - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____    --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21325)

gterzian · 2018-08-02T21:06:58Z


{"status": "PASS", "group": "default", "message": null, "stack": null, "subtest": "DOMParser", "test": "/html/semantics/embedded-content/the-img-element/non-active-document.html", "line": 132002, "action": "test_result", "expected": "FAIL"}
{"status": "PASS", "group": "default", "message": null, "stack": null, "subtest": "createHTMLDocument", "test": "/html/semantics/embedded-content/the-img-element/non-active-document.html", "line": 132003, "action": "test_result", "expected": "FAIL"}
{"status": "PASS", "group": "default", "message": null, "stack": null, "subtest": "<template>", "test": "/html/semantics/embedded-content/the-img-element/non-active-document.html", "line": 132004, "action": "test_result", "expected": "FAIL"}
{"status": "TIMEOUT", "group": "default", "message": null, "stack": null, "subtest": null, "test": "/_mozilla/webgl/conformance-1.0.3/conformance/renderbuffers/framebuffer-object-attachment.html", "line": 232346, "action": "test_result", "expected": "CRASH"}
{"status": "TIMEOUT", "group": "default", "message": null, "stack": null, "subtest": null, "test": "/_mozilla/webgl/conformance-2.0.0/conformance2/renderbuffers/framebuffer-object-attachment.html", "line": 232868, "action": "test_result", "expected": "CRASH"}

awesome, let's see if those hold up on the next test run... Wasn't there something about the channel to the layout thread crashing sometimes? Could it be that this is fixed which is preventing those crashes of those webgl tests?

The source for this is http://build.servo.org/grid, under the column for merge commit corresponding to what the bot announces...

bors-servo · 2018-08-02T21:22:44Z

💔 Test failed - linux-dev

gterzian · 2018-08-02T21:25:23Z


./Cargo.lock:1: duplicate versions for package `crossbeam-epoch`
	The following packages depend on version 0.3.1 from 'crates.io':
		crossbeam-deque
	The following packages depend on version 0.5.0 from 'crates.io':
		crossbeam-channel

./Cargo.lock:1: duplicate versions for package `crossbeam-utils`
	The following packages depend on version 0.2.2 from 'crates.io':
		crossbeam-deque
		crossbeam-epoch
	The following packages depend on version 0.4.1 from 'crates.io':
		crossbeam-channel
		crossbeam-epoch

./Cargo.lock:1: duplicate versions for package `parking_lot`
	The following packages depend on version 0.5.5 from 'crates.io':
		crossbeam-channel
	The following packages depend on version 0.6.3 from 'crates.io':
		layout
		layout_thread
		script
		style
		style_tests
		winit

Do we need to do something on this side too? (this isn't going to prevent the other tests from running, which will still provide useful info)

stjepang · 2018-08-02T22:03:03Z

Oh, I see what the problem is.

So rayon-core depends on an old version of crossbeam-deque, which depends on old versions of crossbeam-epoch and crossbeam-utils. The reason why Rayon doesn't update to the newest version of crossbeam-deque is because it requires Rust 1.25, and Rayon wants to support older versions of Rust.

I don't see an easy way out of this mess. :(

Can we just allow using multiple versions of the Crossbeam crates and parking_lot in Servo?

jdm · 2018-08-02T22:07:44Z

Yes, our servo-tidy.toml has a list of exclusions for the duplicate crate check.

gterzian · 2018-08-02T22:34:50Z

this looks like progress:

{"status": "PASS", "group": "default", "message": null, "stack": null, "subtest": "DOMParser", "test": "/html/semantics/embedded-content/the-img-element/non-active-document.html", "line": 139976, "action": "test_result", "expected": "FAIL"}
{"status": "PASS", "group": "default", "message": null, "stack": null, "subtest": "createHTMLDocument", "test": "/html/semantics/embedded-content/the-img-element/non-active-document.html", "line": 139977, "action": "test_result", "expected": "FAIL"}
{"status": "PASS", "group": "default", "message": null, "stack": null, "subtest": "<template>", "test": "/html/semantics/embedded-content/the-img-element/non-active-document.html", "line": 139978, "action": "test_result", "expected": "FAIL"}
{"status": "TIMEOUT", "group": "default", "message": null, "stack": null, "subtest": null, "test": "/_mozilla/webgl/conformance-1.0.3/conformance/renderbuffers/framebuffer-object-attachment.html", "line": 226820, "action": "test_result", "expected": "CRASH"}
{"status": "TIMEOUT", "group": "default", "message": null, "stack": null, "subtest": null, "test": "/_mozilla/webgl/conformance-2.0.0/conformance2/renderbuffers/framebuffer-object-attachment.html", "line": 232829, "action": "test_result", "expected": "CRASH"}
{"status": "TIMEOUT", "group": "default", "message": null, "stack": null, "subtest": null, "test": "/_mozilla/webgl/conformance-1.0.3/conformance/canvas/drawingbuffer-static-canvas-test.html", "line": 40960, "action": "test_result", "expected": "CRASH"}
{"status": "PASS", "group": "default", "message": null, "stack": null, "subtest": "DOMParser", "test": "/html/semantics/embedded-content/the-img-element/non-active-document.html", "line": 48774, "action": "test_result", "expected": "FAIL"}
{"status": "PASS", "group": "default", "message": null, "stack": null, "subtest": "createHTMLDocument", "test": "/html/semantics/embedded-content/the-img-element/non-active-document.html", "line": 48775, "action": "test_result", "expected": "FAIL"}
{"status": "PASS", "group": "default", "message": null, "stack": null, "subtest": "<template>", "test": "/html/semantics/embedded-content/the-img-element/non-active-document.html", "line": 48776, "action": "test_result", "expected": "FAIL"}
{"status": "TIMEOUT", "group": "default", "message": null, "stack": null, "subtest": null, "test": "/_mozilla/webgl/conformance-1.0.3/conformance/canvas/drawingbuffer-test.html", "line": 53963, "action": "test_result", "expected": "CRASH"}

{"status": "TIMEOUT", "group": "default", "message": null, "stack": null, "subtest": null, "test": "/_mozilla/webgl/conformance-1.0.3/conformance/renderbuffers/framebuffer-object-attachment.html", "line": 99171, "action": "test_result", "expected": "CRASH"}
{"status": "TIMEOUT", "group": "default", "message": null, "stack": null, "subtest": null, "test": "/_mozilla/webgl/conformance-2.0.0/conformance2/renderbuffers/framebuffer-object-attachment.html", "line": 105733, "action": "test_result", "expected": "CRASH"}

those look intermittent(failing on different boxes on different runs):

{"status": "FAIL", "group": "default", "message": "assert_true: expected true got false", "stack": "@http://web-platform.test:8000/html/semantics/interactive-elements/the-details-element/toggleEvent.html:115:5\nTest.prototype.step_timeout/<@http://web-platform.test:8000/resources/testharness.js:1596:20\nTest.prototype.step@http://web-platform.test:8000/resources/testharness.js:1538:20\nTest.prototype.step_func/<@http://web-platform.test:8000/resources/testharness.js:1562:20\n", "subtest": "Calling open twice on 'details' fires only one toggle event", "test": "/html/semantics/interactive-elements/the-details-element/toggleEvent.html", "line": 155693, "action": "test_result", "expected": "PASS"}
{"status": "FAIL", "group": "default", "message": "assert_true: expected true got false", "stack": "@http://web-platform.test:8000/html/semantics/interactive-elements/the-details-element/toggleEvent.html:144:5\nTest.prototype.step_timeout/<@http://web-platform.test:8000/resources/testharness.js:1596:20\nTest.prototype.step@http://web-platform.test:8000/resources/testharness.js:1538:20\nTest.prototype.step_func/<@http://web-platform.test:8000/resources/testharness.js:1562:20\n", "subtest": "Setting open=true to opened 'details' element should not fire a toggle event at the 'details' element", "test": "/html/semantics/interactive-elements/the-details-element/toggleEvent.html", "line": 155696, "action": "test_result", "expected": "PASS"}

I'll update the test exptectations and the servo-tidy.toml(thanks @jdm) in the morning, and squash all the commits into one I guess.

gterzian · 2018-08-03T07:51:28Z

@bors-servo try


        Auto merge of #21325 - gterzian:crossbeam_integration, r=<try>

Crossbeam integration  Follow up on #19515 ---  - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #__ (github issue number if applicable).  - [ ] There are tests for these changes OR - [ ] These changes do not require tests because _____    --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21325)

bors-servo · 2018-08-03T07:51:34Z

⌛ Trying commit 690e6d3 with merge ff40f1d...

bors-servo · 2018-08-03T08:13:37Z

💔 Test failed - linux-rel-css


        Auto merge of #21325 - gterzian:crossbeam_integration, r=SimonSapin,jdm

Replace mpsc with crossbeam-channel Follow up on #19515 --- Selecting over multiple channels in `std::sync::mpsc` is not stable and likely never will be: rust-lang/rust#27800 (comment) > It seems the only thing keeping `mpsc_select` around is Servo. crossbeam-channel is designed specifically to replace `std::sync::mpsc` and fix many of its shortcomings: https://github.com/stjepang/rfcs-crossbeam/blob/channel/text/2017-11-09-channel.md This is to be landed together with servo/ipc-channel#183.  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21325)

bors-servo · 2018-09-12T15:38:35Z

⌛ Testing commit 2a996fb with merge b3f084e...

bors-servo · 2018-09-12T16:35:40Z

💔 Test failed - mac-rel-wpt4

jdm · 2018-09-12T16:37:16Z

@bors-servo retry

#20734


        Auto merge of #21325 - gterzian:crossbeam_integration, r=SimonSapin,jdm

Replace mpsc with crossbeam-channel Follow up on #19515 --- Selecting over multiple channels in `std::sync::mpsc` is not stable and likely never will be: rust-lang/rust#27800 (comment) > It seems the only thing keeping `mpsc_select` around is Servo. crossbeam-channel is designed specifically to replace `std::sync::mpsc` and fix many of its shortcomings: https://github.com/stjepang/rfcs-crossbeam/blob/channel/text/2017-11-09-channel.md This is to be landed together with servo/ipc-channel#183.  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21325)

bors-servo · 2018-09-12T17:33:48Z

⌛ Testing commit 2a996fb with merge 910cc23...

bors-servo · 2018-09-12T18:38:22Z

💔 Test failed - mac-rel-css1

jdm · 2018-09-12T18:42:19Z

@bors-servo retry

#21697

bors-servo · 2018-09-12T19:33:30Z

⚡ Previous build results for android, android-x86, arm32, arm64, linux-dev, linux-rel-css, linux-rel-wpt, mac-dev-unit, mac-rel-css2, mac-rel-wpt1, mac-rel-wpt2, mac-rel-wpt3, mac-rel-wpt4, windows-msvc-dev are reusable. Rebuilding only mac-rel-css1...

bors-servo · 2018-09-12T20:36:02Z

☀️ Test successful - android, android-x86, arm32, arm64, linux-dev, linux-rel-css, linux-rel-wpt, mac-dev-unit, mac-rel-css1, mac-rel-css2, mac-rel-wpt1, mac-rel-wpt2, mac-rel-wpt3, mac-rel-wpt4, windows-msvc-dev
Approved by: SimonSapin,jdm
Pushing 910cc23 to master...


        Auto merge of #21644 - Eijebong:hyperup, r=<try>

WIP: Update hyper to 0.12 Left to do: - Fix the influent::Client call - servo/webrender#3034 - hyperium/mime#91 - sfackler/typed-headers#3 (still need a lot of work) - https://bugzilla.mozilla.org/show_bug.cgi?id=1489792 - Merge and release the ws stuff (I'll do it once I know everything is ok) - Merge and release the influent stuff (Same here, I'll do it once I know everything is ok) - #21325 - Fix unit tests  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21644)

gterzian · 2018-09-13T02:23:58Z

247 comments on a PR, must be a record of some kind...

Thanks for everyone's help and involvement!

jdm · 2018-09-13T02:54:03Z

Thanks again for tackling this, and @stjepang for so much assistance!


        Bug 1491577 - Replace mpsc with crossbeam/servo channel, update ipc-c…

…hannel. r=emilio This cherry-picks servo/servo#21325. Co-authored-by: Gregory Terzian <gterzian@users.noreply.github.com>


        Bug 1491577 - Replace mpsc with crossbeam/servo channel, update ipc-c…

…hannel. r=emilio This cherry-picks servo/servo#21325. Co-authored-by: Gregory Terzian <gterzian@users.noreply.github.com>


        Bug 1491577 - Replace mpsc with crossbeam/servo channel, update ipc-c…

…hannel. r=emilio This cherry-picks servo/servo#21325. Co-authored-by: Gregory Terzian <gterzianusers.noreply.github.com> UltraBlame original commit: 62fbf37ebfa6929dd6349ccb5557789bbaf30210


        Bug 1491577 - Replace mpsc with crossbeam/servo channel, update ipc-c…

…hannel. r=emilio This cherry-picks servo/servo#21325. Co-authored-by: Gregory Terzian <gterzianusers.noreply.github.com> UltraBlame original commit: 62fbf37ebfa6929dd6349ccb5557789bbaf30210


        Bug 1491577 - Replace mpsc with crossbeam/servo channel, update ipc-c…

…hannel. r=emilio This cherry-picks servo/servo#21325. Co-authored-by: Gregory Terzian <gterzianusers.noreply.github.com> UltraBlame original commit: 62fbf37ebfa6929dd6349ccb5557789bbaf30210

highfive assigned gw3583 Aug 2, 2018

highfive added the S-awaiting-review label Aug 2, 2018

gterzian force-pushed the gterzian:crossbeam_integration branch from 6af0967 to cb3a52d Aug 2, 2018

gterzian mentioned this pull request Aug 2, 2018

Replace mpsc with crossbeam-channel #19515

Closed

highfive added the S-tests-failed label Aug 2, 2018

highfive removed the S-tests-failed label Aug 2, 2018

highfive added the S-tests-failed label Aug 2, 2018

highfive removed the S-tests-failed label Aug 3, 2018

gterzian force-pushed the gterzian:crossbeam_integration branch from fc3c844 to 690e6d3 Aug 3, 2018

highfive added the S-tests-failed label Aug 3, 2018

highfive added S-awaiting-merge and removed S-awaiting-review S-needs-rebase S-tests-failed labels Sep 12, 2018

highfive added S-tests-failed and removed S-awaiting-merge labels Sep 12, 2018

highfive added the S-tests-failed label Sep 12, 2018

bors-servo merged commit 2a996fb into servo:master Sep 12, 2018
1 of 3 checks passed

1 of 3 checks passed

continuous-integration/appveyor/pr AppVeyor build failed
Details

continuous-integration/travis-ci/pr The Travis CI build failed
Details

homu Test successful
Details

gterzian deleted the gterzian:crossbeam_integration branch Sep 13, 2018

SimonSapin mentioned this pull request Sep 14, 2018

Tracking issue for channel selection rust-lang/rust#27800

Closed

stjepang mentioned this pull request Jan 30, 2019

Panic on `send` in channels by default? crossbeam-rs/crossbeam#314

Closed

gterzian mentioned this pull request Feb 8, 2019

update to latest substrate - protocol API update paritytech/polkadot#130

Merged

servo / servo

Replace mpsc with crossbeam-channel #21325

Replace mpsc with crossbeam-channel #21325

gterzian commented Aug 2, 2018 •

edited by SimonSapin

highfive commented Aug 2, 2018

gterzian commented Aug 2, 2018

bors-servo commented Aug 2, 2018

bors-servo commented Aug 2, 2018

stjepang commented Aug 2, 2018

stjepang commented Aug 2, 2018

gterzian commented Aug 2, 2018

bors-servo commented Aug 2, 2018

gterzian commented Aug 2, 2018 •

edited

bors-servo commented Aug 2, 2018

gterzian commented Aug 2, 2018 •

edited

stjepang commented Aug 2, 2018 •

edited

jdm commented Aug 2, 2018

gterzian commented Aug 2, 2018 •

edited

gterzian commented Aug 3, 2018

bors-servo commented Aug 3, 2018

bors-servo commented Aug 3, 2018

bors-servo commented Sep 12, 2018

bors-servo commented Sep 12, 2018

jdm commented Sep 12, 2018

bors-servo commented Sep 12, 2018

bors-servo commented Sep 12, 2018

jdm commented Sep 12, 2018

bors-servo commented Sep 12, 2018

bors-servo commented Sep 12, 2018

gterzian commented Sep 13, 2018 •

edited

jdm commented Sep 13, 2018

servo / servo

Join GitHub today

Replace mpsc with crossbeam-channel #21325

Replace mpsc with crossbeam-channel #21325

Conversation

gterzian commented Aug 2, 2018 • edited by SimonSapin

highfive commented Aug 2, 2018

gterzian commented Aug 2, 2018

bors-servo commented Aug 2, 2018

bors-servo commented Aug 2, 2018

stjepang commented Aug 2, 2018

stjepang commented Aug 2, 2018

gterzian commented Aug 2, 2018

bors-servo commented Aug 2, 2018

gterzian commented Aug 2, 2018 • edited

bors-servo commented Aug 2, 2018

gterzian commented Aug 2, 2018 • edited

stjepang commented Aug 2, 2018 • edited

jdm commented Aug 2, 2018

gterzian commented Aug 2, 2018 • edited

gterzian commented Aug 3, 2018

bors-servo commented Aug 3, 2018

bors-servo commented Aug 3, 2018

bors-servo commented Sep 12, 2018

bors-servo commented Sep 12, 2018

jdm commented Sep 12, 2018

bors-servo commented Sep 12, 2018

bors-servo commented Sep 12, 2018

jdm commented Sep 12, 2018

bors-servo commented Sep 12, 2018

bors-servo commented Sep 12, 2018

gterzian commented Sep 13, 2018 • edited

jdm commented Sep 13, 2018

gterzian commented Aug 2, 2018 •

edited by SimonSapin

gterzian commented Aug 2, 2018 •

edited

gterzian commented Aug 2, 2018 •

edited

stjepang commented Aug 2, 2018 •

edited

gterzian commented Aug 2, 2018 •

edited

gterzian commented Sep 13, 2018 •

edited