Fix brotli decoding #22616

jdm · 2019-01-04T20:44:21Z

This replaces our current decoding setup by https://github.com/seanmonstar/reqwest/blob/master/src/async_impl/decoder.rs, and integrates brotli and deflate decoding to maintain our existing support.

./mach build -d does not report any errors
./mach test-tidy does not report any errors
These changes fix #22228
There are tests for these changes

This change is

highfive · 2019-01-04T20:44:25Z

Heads up! This PR modifies the following files:

@KiChjang: components/net/Cargo.toml, components/net/lib.rs, components/net/http_loader.rs, components/net/decoder.rs, components/net/connector.rs

highfive · 2019-01-04T20:44:27Z

Warning

These commits modify unsafe code. Please review it carefully!

jdm · 2019-01-04T20:44:45Z

Waiting to hear about dropbox/rust-brotli-decompressor#4 to see if we need to maintain the brotli fork long-term.

jdm · 2019-01-04T20:44:56Z

@bors-servo try=wpt

bors-servo · 2019-01-04T20:45:01Z

⌛ Trying commit 4542c80 with merge 7c4f065...


        Auto merge of #22616 - jdm:google-decode, r=<try>

Fix brotli decoding work This replaces our current decoding setup by https://github.com/seanmonstar/reqwest/blob/master/src/async_impl/decoder.rs, and integrates brotli and deflate decoding to maintain our existing support. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #22228 - [x] There are tests for these changes  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/22616)

bors-servo · 2019-01-04T20:45:19Z

💔 Test failed - linux-rel-wpt

Eijebong · 2019-01-04T20:49:38Z

components/net/decoder.rs

+
+impl fmt::Debug for Decoder {
+    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
+        f.debug_struct("Decoder").finish()


Not sure what this is exactly doing but I don't think it gives the type of decode and it might be useful to know it ? i.e plain, gzip...

jdm · 2019-01-04T23:01:11Z

@bors-servo try=wpt

bors-servo · 2019-01-04T23:01:16Z

⌛ Trying commit 76dd959 with merge 909ef89...


        Auto merge of #22616 - jdm:google-decode, r=<try>

Fix brotli decoding This replaces our current decoding setup by https://github.com/seanmonstar/reqwest/blob/master/src/async_impl/decoder.rs, and integrates brotli and deflate decoding to maintain our existing support. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #22228 - [x] There are tests for these changes  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/22616)

bors-servo · 2019-01-04T23:21:47Z

💔 Test failed - linux-rel-wpt

nox

The failures seem relevant to the patch, do you want me to look into them?

nox · 2019-01-08T10:20:13Z

tests/wpt/mozilla/tests/mozilla/response-data-brotli.htm

+          var input = client.responseText;
+          var client2 = new XMLHttpRequest()
+          client2.open("GET", "resources/brotli.py?pipe=trickle(1:d1)", false);
+          client2.onload = test.step_func(function() {


Nit: this can be step_func_done.

nox · 2019-01-08T10:20:13Z

tests/wpt/mozilla/tests/mozilla/response-data-brotli.htm

-                  assert_equals(client.responseText, input);
-                  test.done();
-              }
+      async_test(function(test) {


Nit: you could do var test = async_test(); and not have to indent the entire test.

That's how the test was originally written, but that excludes the code outside of the test from reporting errors. This format provides better coverage.

jdm · 2019-01-09T20:53:20Z

@bors-servo r=nox

bors-servo · 2019-01-09T20:53:22Z

📌 Commit 6e29bdc has been approved by nox

jdm · 2019-01-15T00:42:27Z

@bors-servo r=nox

bors-servo · 2019-01-15T00:42:27Z

📌 Commit 047a6b3 has been approved by nox

bors-servo · 2019-01-15T01:43:08Z

⌛ Testing commit 047a6b3 with merge bc3832f...


        Auto merge of #22616 - jdm:google-decode, r=nox

Fix brotli decoding This replaces our current decoding setup by https://github.com/seanmonstar/reqwest/blob/master/src/async_impl/decoder.rs, and integrates brotli and deflate decoding to maintain our existing support. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #22228 - [x] There are tests for these changes  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/22616)

bors-servo · 2019-01-15T02:33:42Z

💔 Test failed - status-taskcluster

CYBAI · 2019-01-15T02:41:51Z

Both task cluster and Travis failed with following error 👀

+ ./mach test-tidy --no-progress --all
Error running mach:

    ['test-tidy', '--no-progress', '--all']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.

You should consider filing a bug for this issue.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

AttributeError: 'NoneType' object has no attribute 'to_json'

  File "/repo/python/servo/testing_commands.py", line 330, in test_tidy
    manifest_dirty = run_update(self.context.topdir, check_clean=True)
  File "/repo/python/servo/testing_commands.py", line 90, in run_update
    return manifestupdate.update(logger, wpt_dir, check_clean, rebuild)
  File "/repo/tests/wpt/manifestupdate.py", line 54, in update
    return _check_clean(logger, test_paths)
  File "/repo/tests/wpt/manifestupdate.py", line 81, in _check_clean
    old_manifest.to_json())

jdm · 2019-01-15T03:42:09Z

sounds of wailing and gnashing of teeth


        Redesign network response decoding to avoid creating decoders before …

…some content is present.

jdm · 2019-01-15T21:06:31Z

@bors-servo r=nox

bors-servo · 2019-01-15T21:06:31Z

📌 Commit 6404a0e has been approved by nox

bors-servo · 2019-01-15T21:06:36Z

⌛ Testing commit 6404a0e with merge 4f45eea...


        Auto merge of #22616 - jdm:google-decode, r=nox

Fix brotli decoding This replaces our current decoding setup by https://github.com/seanmonstar/reqwest/blob/master/src/async_impl/decoder.rs, and integrates brotli and deflate decoding to maintain our existing support. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #22228 - [x] There are tests for these changes  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/22616)

bors-servo · 2019-01-15T21:28:55Z

💔 Test failed - linux-rel-css

jdm · 2019-01-15T21:34:43Z

@bors-servo retry

#22700

bors-servo · 2019-01-15T22:36:52Z

⚡ Previous build results for android-mac, arm32, arm64, linux-rel-wpt, mac-rel-css1, mac-rel-css2, mac-rel-wpt1, mac-rel-wpt2, mac-rel-wpt3, mac-rel-wpt4, magicleap, status-taskcluster are reusable. Rebuilding only linux-rel-css...

bors-servo · 2019-01-15T22:51:25Z

☀️ Test successful - android-mac, arm32, arm64, linux-rel-css, linux-rel-wpt, mac-rel-css1, mac-rel-css2, mac-rel-wpt1, mac-rel-wpt2, mac-rel-wpt3, mac-rel-wpt4, magicleap, status-taskcluster
Approved by: nox
Pushing 4f45eea to master...

highfive assigned asajeffrey Jan 4, 2019

highfive added the S-awaiting-review label Jan 4, 2019

highfive added the S-tests-failed label Jan 4, 2019

jdm changed the title ~~Fix brotli decoding work~~ Fix brotli decoding Jan 4, 2019

Eijebong reviewed Jan 4, 2019

View changes

jdm force-pushed the jdm:google-decode branch from 4542c80 to 58036d4 Jan 4, 2019

highfive removed the S-tests-failed label Jan 4, 2019

jdm force-pushed the jdm:google-decode branch 3 times, most recently from f7b5b42 to 76dd959 Jan 4, 2019

highfive added the S-tests-failed label Jan 4, 2019

nox reviewed Jan 8, 2019

View changes

nox assigned nox and unassigned asajeffrey Jan 8, 2019

jdm force-pushed the jdm:google-decode branch from 76dd959 to 6e29bdc Jan 9, 2019

highfive removed the S-tests-failed label Jan 9, 2019

highfive removed the S-awaiting-review label Jan 9, 2019

jdm removed the S-awaiting-review label Jan 15, 2019

highfive added the S-awaiting-merge label Jan 15, 2019

highfive added S-tests-failed and removed S-awaiting-merge labels Jan 15, 2019

jdm mentioned this pull request Jan 15, 2019

Can't run tests from tests/wpt/mozilla path #22698

Closed

Redesign network response decoding to avoid creating decoders before …

Loading status checks…

6404a0e

…some content is present.

jdm force-pushed the jdm:google-decode branch from 047a6b3 to 6404a0e Jan 15, 2019

highfive added S-awaiting-review and removed S-tests-failed labels Jan 15, 2019

highfive added S-awaiting-merge and removed S-awaiting-review labels Jan 15, 2019

highfive added S-tests-failed and removed S-awaiting-merge labels Jan 15, 2019

jdm mentioned this pull request Jan 23, 2019

Transparent rects demo doesn't display anything on desktop #22749

Closed

servo / servo

Fix brotli decoding #22616

Fix brotli decoding #22616

jdm commented Jan 4, 2019 •

edited by SimonSapin

highfive commented Jan 4, 2019

highfive commented Jan 4, 2019

jdm commented Jan 4, 2019

jdm commented Jan 4, 2019

bors-servo commented Jan 4, 2019

bors-servo commented Jan 4, 2019

This comment has been minimized.

jdm commented Jan 4, 2019

bors-servo commented Jan 4, 2019

bors-servo commented Jan 4, 2019

nox left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

jdm commented Jan 9, 2019

bors-servo commented Jan 9, 2019

jdm commented Jan 15, 2019

bors-servo commented Jan 15, 2019

bors-servo commented Jan 15, 2019

bors-servo commented Jan 15, 2019

CYBAI commented Jan 15, 2019

jdm commented Jan 15, 2019

jdm commented Jan 15, 2019

bors-servo commented Jan 15, 2019

bors-servo commented Jan 15, 2019

bors-servo commented Jan 15, 2019

jdm commented Jan 15, 2019

bors-servo commented Jan 15, 2019

bors-servo commented Jan 15, 2019

servo / servo

Join GitHub today

Fix brotli decoding #22616

Fix brotli decoding #22616

Conversation

jdm commented Jan 4, 2019 • edited by SimonSapin

highfive commented Jan 4, 2019

highfive commented Jan 4, 2019

jdm commented Jan 4, 2019

jdm commented Jan 4, 2019

bors-servo commented Jan 4, 2019

bors-servo commented Jan 4, 2019

This comment has been minimized.

Eijebong Jan 4, 2019

jdm commented Jan 4, 2019

bors-servo commented Jan 4, 2019

bors-servo commented Jan 4, 2019

nox left a comment

This comment has been minimized.

nox Jan 8, 2019

This comment has been minimized.

nox Jan 8, 2019

This comment has been minimized.

jdm Jan 9, 2019

jdm commented Jan 9, 2019

bors-servo commented Jan 9, 2019

jdm commented Jan 15, 2019

bors-servo commented Jan 15, 2019

bors-servo commented Jan 15, 2019

bors-servo commented Jan 15, 2019

CYBAI commented Jan 15, 2019

jdm commented Jan 15, 2019

jdm commented Jan 15, 2019

bors-servo commented Jan 15, 2019

bors-servo commented Jan 15, 2019

bors-servo commented Jan 15, 2019

jdm commented Jan 15, 2019

bors-servo commented Jan 15, 2019

bors-servo commented Jan 15, 2019

jdm commented Jan 4, 2019 •

edited by SimonSapin