Implement "update the image data" #16238

gterzian · 2017-04-03T09:18:10Z

Spec compliant implementation of the update the image data algorithm.

Currently still a work in progress, the 'async src complete test` is still passing as it was before, even though I switched to the new code, so I guess that's something.

@jdm I will be picking this up next weekend, I left a bunch of TODO and NOTES in the code, if you or someone else have time this week I would appreciate an initial scan and feedback.

./mach build -d does not report any errors
./mach test-tidy does not report any errors
These changes fix #11517 (github issue number if applicable).

There are tests for these changes OR
These changes do not require tests because _____

This change is

highfive · 2017-04-03T09:18:14Z

Heads up! This PR modifies the following files:

@fitzgen: components/script/dom/htmlimageelement.rs
@KiChjang: components/script/dom/htmlimageelement.rs

highfive · 2017-04-03T09:18:15Z

Warning

These commits modify script code, but no tests are modified. Please consider adding a test!

jdm · 2017-04-03T13:40:24Z

Wonderful! Thanks for tackling this!

jdm · 2017-04-03T13:42:25Z

Note: there's no need to include any implementation related to srcset/sizes/other responsive image things, as that's going to be a student project starting this summer.

gterzian · 2017-04-04T01:29:50Z

@jdm great, glad to hear that! That would include concepts like selecting and image source and updating the source set, correct? Currently I have left those at the bare minimum, essentially just grabbing the src and ignoring the actual spec of those concepts...

jdm · 2017-04-04T03:46:12Z

Yep, that sounds great.

emilio · 2017-04-05T16:26:40Z

r? @jdm

gterzian · 2017-04-09T11:44:05Z

@jdm here is this weekend's harvest:

All img.complete tests are now passing.
I had to edit one test, because from what I could tell the while loop that was in there was preventing the page from reaching ths stable state, while the specific case that is being tested requires the fetch part of the code to have started to run, therefore the test was always bound to fail.
I tried to get the fetch part of the code to be a bit more spec compliant, meaning following the comcept of the image request, as well as adding some things related to step 14 of the spec.
I made Complete more spec compliant.

Re 3, the previous code would always just use self.current_request, however to follow the spec we must first pick the image_request from either the pending or current one, and then associate the fetch code with this image_request. Due to the fact that ImageRequest isn't thread safe in it's current form, I couldn't pass the actual request along the algorithm, so instead I pass around the url and grab the 'right' request based on the url when needed. I "think" that's pretty good for now...

Re 4, I had to decide how to have Complete return true in the case that "The final task that is queued by the networking task source once the resource has been fetched has been queued". I didn't want to add more state to ImageRequest, so instead I made the choice that if current_request has a parsed_url, but isn't yet in either Broken or CompletelyAvailable state, we can assume the final task has been queued but not processed yet... Once again, I "think" that's good enough for now...

If you have time this week, I think the PR is good for a more comprehensive review at this stage...

gterzian · 2017-04-09T12:19:23Z

@jdm I'm reading through the spec again and I see I missed some key concepts regarding current and pending requests in step 12 and 14 of the specs, so that's certainly something I will still need to implement, feedback in the meantime welcome off course...

jdm · 2017-04-09T12:21:35Z

Re: "The final task queued that is queued", see whatwg/html#1055.

gterzian · 2017-04-09T12:27:57Z

@jdm thanks, so that means we can keep that test as it was before, and with a FAIL? Glad to read that because it wasn't entirely clear to me.

Re pending/current request, here is the work I intend to still do, next weekend:

make pending_requent an Option<DOMRefCell<ImageRequest>>, and initially set it to None.
Make the fetch and 'choose a request' code comply with step 12 and 14 of the spec, for example the whole abort current and upgrade pending to current. Also currently I just return in some cases, while in fact the spec speaks of aborting these steps, which I guess means just aborting step 12, not returning form the whole algorithm(This relates to picking the right request to fetch the image).

I guess some of the stuff I currently do with url might become obsolete if pending/current is better implemented...

So it's mainly looking at step 12 and 14 in more details....

I'll also undo the changes I made to the tests, and keep that "final task" one on FAIL...

jdm

Great work so far! I found these changes to be quite readable in general, although it might be nice to follow the spec text about aborting a particular request or creating a new request, rather than what is (presumably) equivalent.

jdm · 2017-04-11T04:08:49Z

components/script/dom/htmlimageelement.rs

@@ -113,7 +116,8 @@ impl Runnable for ImageResponseHandlerRunnable {
        let element = self.element.root();
        // Ignore any image response for a previous request that has been discarded.
        if element.generation.get() == self.generation {
-            element.process_image_response(self.image);
+            let url = self.url.clone();
+            element.process_image_response(self.image, &url);


Wouldn't &self.url avoid the need to clone?

yes indeed, and I also had to use &self.image to prevent use of moved value: self

jdm · 2017-04-11T04:08:49Z

components/script/dom/htmlimageelement.rs

                let _ = task_source.queue_with_wrapper(box runnable, &wrapper);
            });

            image_cache.add_listener(id, ImageResponder::new(responder_sender, id));
        }
-


Let's restore this newline.

jdm · 2017-04-11T04:08:49Z

components/script/dom/htmlimageelement.rs

        if trigger_image_load {
            self.upcast::<EventTarget>().fire_event(atom!("load"));
+            self.upcast::<EventTarget>().fire_event(atom!("loadend"));
        }

        // Fire image.onerror
        if trigger_image_error {
            self.upcast::<EventTarget>().fire_event(atom!("error"));


We should be firing loadend after error as well according to the steps in https://html.spec.whatwg.org/multipage/embedded-content.html#updating-the-image-data .

jdm · 2017-04-11T04:08:49Z

components/script/dom/htmlimageelement.rs

+    fn has_parent_image(&self) -> bool {
+        let node = self.upcast::<Node>();
+        if let Some(ref parent) = node.GetParentNode() {
+            if parent.is::<HTMLImageElement>() {


This looks like it's implementing https://html.spec.whatwg.org/multipage/embedded-content.html#use-srcset-or-picture incorrectly .

jdm · 2017-04-11T04:08:49Z

components/script/dom/htmlimageelement.rs

+                    current_request.state = State::CompletelyAvailable;
+                    current_request.parsed_url = Some(img_url);
+                    current_request.source_url = Some(src);
+                    self.upcast::<EventTarget>().fire_event(atom!("load"));


5.3.7 should happen as part of a queued task.

jdm · 2017-04-11T04:08:49Z

components/script/dom/htmlimageelement.rs

+                let mut image_request = self.request_for_url(url);
+                image_request.state = State::Broken;
+                image_request.image = None;
+                image_request.metadata = None;


This looks like it's missing some steps from the relevant "Otherwise" branches.

jdm · 2017-04-11T04:08:49Z

components/script/dom/htmlimageelement.rs

        };
-        self.current_request.borrow_mut().image = image;
-        self.current_request.borrow_mut().metadata = metadata;

        // Mark the node dirty
        self.upcast::<Node>().dirty(NodeDamage::OtherNodeDamage);


We should only do this if the current request is the one being serviced.

jdm · 2017-04-11T04:08:49Z

tests/wpt/web-platform-tests/html/semantics/embedded-content/the-img-element/img.complete.html

-    document.getElementById("imgTestTag3").src = '3.jpg?nocache=' + Math.random();
+    document.getElementById("imgTestTag3").onload = t.step_func(function(){
+      assert_true(document.getElementById("imgTestTag3").complete);
+      assert_equals(document.getElementById("imgTestTag3").src.split("/").pop().split('?')[0], "3.jpg");


What if we use currentSrc here instead of src?

jdm · 2017-04-11T04:08:49Z

tests/wpt/web-platform-tests/html/semantics/embedded-content/the-img-element/img.complete.html

-      if (imageInstance.complete === true) break;
-    }
-    t.done();
+    setTimeout(function(){


t.step_timeout(function {

jdm · 2017-04-11T04:08:49Z

components/script/dom/htmlimageelement.rs

+                // NOTE: this is an attempt to comply with:
+                // "(return true if)The final task that is queued by the networking task source,
+                //  once the resource has been fetched, has been queued"
+                return request.parsed_url.is_some()


Let's forget about this.

gterzian · 2017-04-11T05:19:41Z

@jdm thanks for the review, I'll look at it in more details over the weekend...

gterzian · 2017-04-16T08:13:28Z

@jdm OK I have addressed most of your comments, I just had a question re "We should store the selected source and pass it as an argument here instead" and "Don't we need to initialize the image request's state here?", and I have also made further attempt to implement the concepts of pending and current requests in a more spec compliant way. Please take a look when you have a chance...

gterzian · 2017-05-25T00:11:59Z

@jdm OK, fixed the double borrowing crash through scoping...

gterzian · 2017-05-28T04:14:50Z

@jdm This one might be ready for another test run. The previous run I think confirmed that the timeouts where simply due to my machine being too slow, and the only code change that would require additional review is where the current request is set to Unavailable at the beginning of the algorithm, this allows an additional test to pass. The crashes from the previous run where due to the fact that I hadn't scoped that part, so the current_request was doubly borrowed(this wasn't apparent from running the img specific tests, since the images are not cached, and the double borrowing occured in the "we found a cached image" part).

jdm · 2017-05-29T21:52:12Z

@bors-servo: try

bors-servo · 2017-05-29T21:52:16Z

⌛ Trying commit 71b0c10 with merge f4333fc...


        Auto merge of #16238 - gterzian:implement_update_the_image_data, r=<try>

@jdm

Implement "update the image data"  Spec compliant implementation of the [update the image data algorithm](https://html.spec.whatwg.org/multipage/embedded-content.html#update-the-image-data). Currently still a work in progress, the ['async src complete test`](https://github.com/servo/servo/blob/master/tests/wpt/web-platform-tests/html/semantics/embedded-content/the-img-element/img.complete.html#L33) is still passing as it was before, even though I switched to the new code, so I guess that's something. @jdm I will be picking this up next weekend, I left a bunch of TODO and NOTES in the code, if you or someone else have time this week I would appreciate an initial scan and feedback. ---  - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #11517 (github issue number if applicable).  - [x] There are tests for these changes OR - [ ] These changes do not require tests because _____    --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/16238)

bors-servo · 2017-05-29T23:01:23Z

☀️ Test successful - android, arm32, arm64, linux-dev, linux-rel-css, linux-rel-wpt, mac-dev-unit, mac-rel-css1, mac-rel-css2, mac-rel-wpt1, mac-rel-wpt2, mac-rel-wpt3, mac-rel-wpt4, windows-msvc-dev
State: approved= try=True

jdm · 2017-05-30T02:03:15Z

@bors-servo: r+

bors-servo · 2017-05-30T02:03:16Z

📌 Commit 71b0c10 has been approved by jdm

bors-servo · 2017-05-30T02:30:52Z

⌛ Testing commit 71b0c10 with merge 7f5bc73...


        Auto merge of #16238 - gterzian:implement_update_the_image_data, r=jdm

@jdm

Implement "update the image data"  Spec compliant implementation of the [update the image data algorithm](https://html.spec.whatwg.org/multipage/embedded-content.html#update-the-image-data). Currently still a work in progress, the ['async src complete test`](https://github.com/servo/servo/blob/master/tests/wpt/web-platform-tests/html/semantics/embedded-content/the-img-element/img.complete.html#L33) is still passing as it was before, even though I switched to the new code, so I guess that's something. @jdm I will be picking this up next weekend, I left a bunch of TODO and NOTES in the code, if you or someone else have time this week I would appreciate an initial scan and feedback. ---  - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #11517 (github issue number if applicable).  - [x] There are tests for these changes OR - [ ] These changes do not require tests because _____    --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/16238)

bors-servo · 2017-05-30T03:17:47Z

💔 Test failed - mac-rel-wpt2

jdm · 2017-05-30T14:46:41Z

@bors-servo: retry

#17096

bors-servo · 2017-05-30T15:27:10Z

⌛ Testing commit 71b0c10 with merge fe7d039...


        Auto merge of #16238 - gterzian:implement_update_the_image_data, r=jdm

@jdm

Implement "update the image data"  Spec compliant implementation of the [update the image data algorithm](https://html.spec.whatwg.org/multipage/embedded-content.html#update-the-image-data). Currently still a work in progress, the ['async src complete test`](https://github.com/servo/servo/blob/master/tests/wpt/web-platform-tests/html/semantics/embedded-content/the-img-element/img.complete.html#L33) is still passing as it was before, even though I switched to the new code, so I guess that's something. @jdm I will be picking this up next weekend, I left a bunch of TODO and NOTES in the code, if you or someone else have time this week I would appreciate an initial scan and feedback. ---  - [ ] `./mach build -d` does not report any errors - [ ] `./mach test-tidy` does not report any errors - [ ] These changes fix #11517 (github issue number if applicable).  - [x] There are tests for these changes OR - [ ] These changes do not require tests because _____    --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/16238)

bors-servo · 2017-05-30T16:28:22Z

☀️ Test successful - android, arm32, arm64, linux-dev, linux-rel-css, linux-rel-wpt, mac-dev-unit, mac-rel-css1, mac-rel-css2, mac-rel-wpt1, mac-rel-wpt2, mac-rel-wpt3, mac-rel-wpt4, windows-msvc-dev
Approved by: jdm
Pushing fe7d039 to master...

gterzian · 2017-05-31T09:27:41Z

@jdm that was a big one, thanks for the efforts you put into reviewing this!


        improve spec compliance of update the image data

Upstreamed from servo/servo#16238 [ci skip]

highfive assigned emilio Apr 3, 2017

highfive added the S-awaiting-review label Apr 3, 2017

nox added the S-needs-squash label Apr 3, 2017

highfive assigned jdm and unassigned emilio Apr 5, 2017

gterzian force-pushed the gterzian:implement_update_the_image_data branch 3 times, most recently from a71c9bc to f25ce3d Apr 8, 2017

gterzian force-pushed the gterzian:implement_update_the_image_data branch 2 times, most recently from 80d3bd9 to f3ff232 Apr 9, 2017

jdm requested changes Apr 11, 2017

View changes

jdm added S-needs-code-changes and removed S-awaiting-review S-needs-squash labels Apr 11, 2017

highfive added S-awaiting-review and removed S-needs-code-changes labels Apr 15, 2017

gterzian force-pushed the gterzian:implement_update_the_image_data branch from f5f94b6 to 0f630f5 Apr 20, 2017

jdm changed the title ~~[WIP] Implement "update the image data"~~ Implement "update the image data" May 29, 2017

jdm approved these changes May 29, 2017

View changes

highfive added S-awaiting-merge and removed S-awaiting-review labels May 30, 2017

highfive added S-tests-failed and removed S-awaiting-merge labels May 30, 2017

highfive added S-awaiting-merge and removed S-tests-failed labels May 30, 2017

bors-servo merged commit 71b0c10 into servo:master May 30, 2017
3 checks passed

3 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details

continuous-integration/travis-ci/pr The Travis CI build passed
Details

homu Test successful
Details

highfive removed the S-awaiting-merge label May 30, 2017

gterzian deleted the gterzian:implement_update_the_image_data branch May 31, 2017

jdm mentioned this pull request May 31, 2017

Intermittent crash in /css21_dev/html4/white-space-pre-002.htm and /css-images-3_dev/html/object-fit-cover-svg-005i.htm #17109

Closed

jdm added a commit to web-platform-tests/wpt that referenced this pull request Jun 19, 2017

improve spec compliance of update the image data

771c7fc

Upstreamed from servo/servo#16238 [ci skip]

jdm mentioned this pull request Jun 19, 2017

improve spec compliance of update the image data web-platform-tests/wpt#6281

Merged

servo / servo

Join GitHub today

Implement "update the image data" #16238

Implement "update the image data" #16238

Conversation

gterzian commented Apr 3, 2017 • edited

highfive commented Apr 3, 2017

highfive commented Apr 3, 2017

jdm commented Apr 3, 2017

jdm commented Apr 3, 2017

gterzian commented Apr 4, 2017 • edited

jdm commented Apr 4, 2017

emilio commented Apr 5, 2017

gterzian commented Apr 9, 2017 • edited

gterzian commented Apr 9, 2017

jdm commented Apr 9, 2017

gterzian commented Apr 9, 2017 • edited

jdm left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

gterzian commented Apr 11, 2017

gterzian commented Apr 16, 2017 • edited

gterzian commented May 25, 2017

gterzian commented May 28, 2017

jdm commented May 29, 2017

bors-servo commented May 29, 2017

bors-servo commented May 29, 2017

jdm commented May 30, 2017

bors-servo commented May 30, 2017

bors-servo commented May 30, 2017

bors-servo commented May 30, 2017

jdm commented May 30, 2017

bors-servo commented May 30, 2017

bors-servo commented May 30, 2017

gterzian commented May 31, 2017

gterzian commented Apr 3, 2017 •

edited

gterzian commented Apr 4, 2017 •

edited

gterzian commented Apr 9, 2017 •

edited

gterzian commented Apr 9, 2017 •

edited

gterzian commented Apr 16, 2017 •

edited