Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Subresource Integrity #14865

Merged
merged 1 commit into from Jan 8, 2017
Merged

Implement Subresource Integrity #14865

merged 1 commit into from Jan 8, 2017

Conversation

@nmvk
Copy link
Contributor

nmvk commented Jan 5, 2017

Implemented response validation part of https://w3c.github.io/webappsec-subresource-integrity/.
Implemented step eighteen of the main fetch. If a request has integrity metadata, then following steps are performed

  1. Wait for response body
  2. If the response does not have a termination reason and response does not match request’s integrity metadata, set response and internalResponse to a network error.

Dependency updated: html5ever-atoms from 0.1.2 to 0.1.3. This will not completely fix #14523, It will implement changes related to response validation. Request validation algorithm implementation needs CSP.

I did not update any WPT-Test. In my local system, I found some assertion issue dependent on the order of execution of test-case. It would be helpful if someone could do "try" build on these changes to get wpt results.

r? @jdm


  • ./mach build -d does not report any errors
  • ./mach test-tidy does not report any errors
  • There are tests for these changes

This change is Reviewable

@nmvk
Copy link
Contributor Author

nmvk commented Jan 5, 2017

r? @jdm

@jdm
Copy link
Member

jdm commented Jan 5, 2017

@bors-servo
Copy link
Contributor

bors-servo commented Jan 5, 2017

Trying commit a64b964 with merge 2457bcf...

bors-servo added a commit that referenced this pull request Jan 5, 2017
Implement Subresource Integrity

Implemented response validation part of https://w3c.github.io/webappsec-subresource-integrity/.
Implemented step eighteen of the main fetch. If a request has integrity metadata, then following steps are performed
1) Wait for response body
2) If the response does not have a termination reason and response does not match request’s integrity metadata, set response and internalResponse to a network error.

Dependency updated: html5ever-atoms from 0.1.2 to 0.1.3. This will not completely fix #14523, It will implement changes related to response validation. Request validation algorithm implementation needs CSP.

I did not update any WPT-Test. In my local system, I found some assertion issue dependent on the order of execution of test-case. It would be helpful if someone could do "try" build on these changes to get wpt results.

r? @jdm
<!-- Please describe your changes on the following line: -->

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors

<!-- Either: -->
- [X] There are tests for these changes

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/14865)
<!-- Reviewable:end -->
@bors-servo
Copy link
Contributor

bors-servo commented Jan 5, 2017

💔 Test failed - mac-rel-wpt2

@jdm
Copy link
Member

jdm commented Jan 5, 2017

  ▶ TIMEOUT [expected OK] /subresource-integrity/subresource-integrity.sub.html
  │ 
  │ VMware, Inc.
  │ Gallium 0.4 on softpipe
  └ 3.3 (Core Profile) Mesa 12.0.1

  ▶ Unexpected subtest result in /subresource-integrity/subresource-integrity.sub.html:
  └ NOTRUN [expected PASS] Style: Same-origin with correct sha256 and sha512 hash, rel='alternate stylesheet' enabled

  ▶ Unexpected subtest result in /subresource-integrity/subresource-integrity.sub.html:
  └ NOTRUN [expected PASS] Style: Same-origin with incorrect sha256 and sha512 hash, rel='alternate stylesheet' enabled

  ▶ Unexpected subtest result in /subresource-integrity/subresource-integrity.sub.html:
  │ FAIL [expected PASS] Style: <crossorigin='anonymous'> with correct hash, ACAO: *
  │   → assert_unreached: Good load fired error handler. Reached unreachable code
  │ FAIL [expected PASS] Style: <crossorigin='use-credentials'> with correct hash, CORS-eligible
  │   → assert_unreached: Good load fired error handler. Reached unreachable code
  │ 
  │ SRIStyleTest.prototype.execute/</<@http://web-platform.test:8000/subresource-integrity/subresource-integrity.sub.html:132:39
  │ Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1406:20
  └ SRIStyleTest.prototype.execute/<@http://web-platform.test:8000/subresource-integrity/subresource-integrity.sub.html:132:17

  ▶ Unexpected subtest result in /subresource-integrity/subresource-integrity.sub.html:
  │ FAIL [expected PASS] Script: <crossorigin='use-credentials'> with correct hash, CORS-eligible
  │   → assert_unreached: Good load fired error handler. Reached unreachable code
  │ 
  │ SRIScriptTest.prototype.execute/</<@http://web-platform.test:8000/subresource-integrity/subresource-integrity.sub.html:59:39
  │ Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1406:20
  └ SRIScriptTest.prototype.execute/<@http://web-platform.test:8000/subresource-integrity/subresource-integrity.sub.html:59:17

  ▶ Unexpected subtest result in /subresource-integrity/subresource-integrity.sub.html:
  │ FAIL [expected PASS] Style: Same-origin with incorrect hash.
  │   → assert_not_equals: got disallowed value "rgb(255, 255, 0)"
  │ FAIL [expected PASS] Style: Same-origin with sha256 match, sha512 mismatch
  │   → assert_not_equals: got disallowed value "rgb(255, 255, 0)"
  │ FAIL [expected PASS] Style: <crossorigin='anonymous'> with incorrect hash, ACAO: *
  │   → assert_not_equals: got disallowed value "rgb(255, 255, 0)"
  │ FAIL [expected PASS] Style: <crossorigin='use-credentials'> with incorrect hash CORS-eligible
  │   → assert_not_equals: got disallowed value "rgb(255, 255, 0)"
  │ FAIL [expected PASS] Style: <crossorigin='anonymous'> with CORS-ineligible resource
  │   → assert_not_equals: got disallowed value "rgb(255, 255, 0)"
  │ FAIL [expected PASS] Style: Cross-origin, not CORS request, with correct hash
  │   → assert_not_equals: got disallowed value "rgb(255, 255, 0)"
  │ FAIL [expected PASS] Style: Cross-origin, not CORS request, with hash mismatch
  │   → assert_not_equals: got disallowed value "rgb(255, 255, 0)"
  │ 
  │ SRIStyleTest.prototype.execute/</<@http://web-platform.test:8000/subresource-integrity/subresource-integrity.sub.html:141:21
  │ Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1406:20
  └ SRIStyleTest.prototype.execute/<@http://web-platform.test:8000/subresource-integrity/subresource-integrity.sub.html:139:17
@jdm
Copy link
Member

jdm commented Jan 5, 2017

Let me know if you want me to start reviewing the changes or whether I should wait until you look into the test failure.

@nmvk
Copy link
Contributor Author

nmvk commented Jan 6, 2017

Thanks Josh, I will work on these test cases today. Will notify you once these gets fixed.

@nmvk nmvk force-pushed the nmvk:sri-fetch branch from a64b964 to 387d5d9 Jan 6, 2017
@nmvk
Copy link
Contributor Author

nmvk commented Jan 6, 2017

None of the test case failures seem to be related to changes.

  1. assertion error in 7 test case is due to stylesheet already been loaded in other success test case. I moved all test case expecting assert_not_equals to execute before any success test case.
  2. crossorigin has not been implemented in htmllinkelement. So i changed assertion of two test case (Style: <crossorigin='anonymous'> with correct hash, ACAO: * and Style: <crossorigin='use-credentials'> with correct hash, CORS-eligible) to false.
  3. Two alternate stylesheet related tests got timed out since we ignore alternate stylesheet. I have filed an issue #14881. Currently, I have commented these tests.
  4. Script: <crossorigin='use-credentials'> with correct hash, CORS-eligible failed due to bug in step six of cors_check of http_loader . This has been fixed.

@jdm If my above analysis is correct, can you please review the changes.

@jdm
Copy link
Member

jdm commented Jan 6, 2017

Rather than modifying the test file, we should be modifying tests/wpt/metadata/subresource-integrity/subresource-integrity.sub.html.ini to include the expected failing test results.

@jdm
Copy link
Member

jdm commented Jan 6, 2017

Can you explain point 1 in more detail? Why does the order make a difference in the test result?

@nmvk
Copy link
Contributor Author

nmvk commented Jan 6, 2017

@jdm I am not sure if point 1 is a bug. Our assertion is to validate that div style background color is not yellow. Moment first success test case executes stylesheet gets loaded and subsequent all assertion fail though we don't load stylesheet again.

I will update tests/wpt/metadata/subresource-integrity/subresource-integrity.sub.html.ini, Thanks

@jdm
Copy link
Member

jdm commented Jan 6, 2017

I know what the problem is - the test relies on removing the stylesheet from any previous test before starting the new test. However, Servo doesn't support removing stylesheets from the document yet (see #976).

@nmvk
Copy link
Contributor Author

nmvk commented Jan 6, 2017

So I will modify subresource-integrity.sub.html.ini to include these tests as well?

@jdm
Copy link
Member

jdm commented Jan 6, 2017

Yes please!

@nmvk nmvk force-pushed the nmvk:sri-fetch branch from 387d5d9 to b44e30e Jan 6, 2017
@nmvk
Copy link
Contributor Author

nmvk commented Jan 6, 2017

@jdm I have done the changes

Copy link
Member

jdm left a comment

This is really great work! I have a number of small changes, but in general I like the way the code is organized and found the changes easy to read.

@@ -1,7 +1,6 @@
/* This Source Code Form is subject to the terms of the Mozilla Public
* License, v. 2.0. If a copy of the MPL was not distributed with this
* file, You can obtain one at http://mozilla.org/MPL/2.0/. */

This comment has been minimized.

Copy link
@jdm

jdm Jan 6, 2017

Member

nit: please add this newline back.

let ref integrity_metadata = *request.integrity_metadata.borrow();
if response.termination_reason.is_none() && !is_response_integrity_valid(integrity_metadata, &response) {
let mut response = Response::network_error(
NetworkError::Internal("Sub-resource integrity validation failed".into()));

This comment has been minimized.

Copy link
@jdm

jdm Jan 6, 2017

Member

nit: Subresource should be one word, not hyphenated.

let mut response = Response::network_error(
NetworkError::Internal("Sub-resource integrity validation failed".into()));
response.internal_response = Some(Box::new(
Response::network_error(response.get_network_error().unwrap().clone())));

This comment has been minimized.

Copy link
@jdm

jdm Jan 6, 2017

Member

I don't believe we need to duplicate the error for the internal value here. It should be enough just to have a default network error response.

Response::network_error(response.get_network_error().unwrap().clone())));
response
} else {
response

This comment has been minimized.

Copy link
@jdm

jdm Jan 6, 2017

Member

nit: indentation

let mut result = vec![];

// Step 3
let tokens: Vec<&str> = integrity_metadata.split(" ").collect();

This comment has been minimized.

Copy link
@jdm

jdm Jan 6, 2017

Member

This doesn't quite match the spec's steps for splitting on spaces - we need to skip characters at the start and repeated characters in between tokens, and the space characters include other whitespace. I recommend using split_html_space_chars instead.

Also, we don't need to call collect here; we can iterate over the result of splitting instead.

}

/// https://w3c.github.io/webappsec-subresource-integrity/#getprioritizedhashfunction
pub fn get_prioritized_hash_function(hash_func_left: &str, hash_func_right: &str) -> String {

This comment has been minimized.

Copy link
@jdm

jdm Jan 6, 2017

Member

Let's return Option<&str> here instead, which will avoid the magic empty string behaviour.

let mut result: Vec<SriEntry> = vec![integrity_metadata_list[0].clone()];
let mut current_algorithm = result[0].alg.clone();

for i in 1..integrity_metadata_list.len() {

This comment has been minimized.

Copy link
@jdm

jdm Jan 6, 2017

Member
for integrity_metadata in &integrity_metadata_list[1..] {
let intgerity_metadata = integrity_metadata_list[i].clone();

let prioritized_hash = get_prioritized_hash_function(&intgerity_metadata.alg,
&*current_algorithm);

This comment has been minimized.

Copy link
@jdm

jdm Jan 6, 2017

Member

nit: indentation.

let response_digest = hash(message_digest, vec);
response_digest.to_base64(STANDARD)
} else {
"".to_owned()

This comment has been minimized.

Copy link
@jdm

jdm Jan 6, 2017

Member

We should probably make this unreachable!("Tried to calculate digest of incomplete response body") instead.

let parsed_metadata_list: Vec<SriEntry> = parsed_metadata(integrity_metadata);

// Step 2 & 4
if parsed_metadata_list.len() == 0 {

This comment has been minimized.

Copy link
@jdm

jdm Jan 6, 2017

Member

if parse_metadata_list.is_empty() {

@nmvk
Copy link
Contributor Author

nmvk commented Jan 7, 2017

I had missed updating expectation of two test case :(. I have corrected this.

@jdm
Copy link
Member

jdm commented Jan 7, 2017

You will also need to adjust the expectations for /html/dom/interfaces.html, /html/dom/reflection-misc.html, and /html/dom/reflection-metadata.html. I recommend running
./mach test-wpt tests/wpt/web-platform-test/path/to/test.html --no-pause-after-test --log-raw /tmp/servo.log && ./mach update-wpt /tmp/servo.log for each of those tests.

@bors-servo
Copy link
Contributor

bors-servo commented Jan 7, 2017

The latest upstream changes (presumably #14867) made this pull request unmergeable. Please resolve the merge conflicts.

Implemented response validation part of
https://w3c.github.io/webappsec-subresource-integrity/.
Implemented step eighteen of the main fetch. If a request has integrity
metadata, then following steps are performed
*Wait for response body
*If the response does not have a termination reason and response does not
match request’s integrity metadata, set response to a
network error.# Please enter the commit message for your changes. Lines starting
@nmvk nmvk force-pushed the nmvk:sri-fetch branch from a153c01 to a302649 Jan 8, 2017
@nmvk
Copy link
Contributor Author

nmvk commented Jan 8, 2017

@jdm Thanks, I have fixed expectations and merge conflicts

@jdm
Copy link
Member

jdm commented Jan 8, 2017

@bors-servo
Copy link
Contributor

bors-servo commented Jan 8, 2017

📌 Commit a302649 has been approved by jdm

@bors-servo
Copy link
Contributor

bors-servo commented Jan 8, 2017

Testing commit a302649 with merge f958daf...

bors-servo added a commit that referenced this pull request Jan 8, 2017
Implement Subresource Integrity

Implemented response validation part of https://w3c.github.io/webappsec-subresource-integrity/.
Implemented step eighteen of the main fetch. If a request has integrity metadata, then following steps are performed
1) Wait for response body
2) If the response does not have a termination reason and response does not match request’s integrity metadata, set response and internalResponse to a network error.

Dependency updated: html5ever-atoms from 0.1.2 to 0.1.3. This will not completely fix #14523, It will implement changes related to response validation. Request validation algorithm implementation needs CSP.

I did not update any WPT-Test. In my local system, I found some assertion issue dependent on the order of execution of test-case. It would be helpful if someone could do "try" build on these changes to get wpt results.

r? @jdm
<!-- Please describe your changes on the following line: -->

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: -->
- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors

<!-- Either: -->
- [X] There are tests for these changes

<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/14865)
<!-- Reviewable:end -->
@bors-servo
Copy link
Contributor

bors-servo commented Jan 8, 2017

💔 Test failed - linux-dev

@nmvk
Copy link
Contributor Author

nmvk commented Jan 8, 2017

Infra issue?

@KiChjang
Copy link
Member

KiChjang commented Jan 8, 2017

@bors-servo retry

@bors-servo
Copy link
Contributor

bors-servo commented Jan 8, 2017

Previous build results for android, arm64, linux-rel-wpt, mac-dev-unit, mac-rel-wpt1, windows-gnu-dev, windows-msvc-dev are reusable. Rebuilding only arm32, linux-dev, linux-rel-css, mac-rel-css, mac-rel-wpt2...

@bors-servo bors-servo merged commit a302649 into servo:master Jan 8, 2017
2 of 3 checks passed
2 of 3 checks passed
continuous-integration/travis-ci/pr The Travis CI build failed
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
homu Test successful
Details
@nmvk nmvk deleted the nmvk:sri-fetch branch Jan 8, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

5 participants
You can’t perform that action at this time.