Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Design data structures to support streaming content in display list #448

Closed
brson opened this issue May 14, 2013 · 3 comments
Closed

Design data structures to support streaming content in display list #448

brson opened this issue May 14, 2013 · 3 comments
Labels
A-gfx/displaylist

Comments

@brson
Copy link
Contributor

brson commented May 14, 2013

At the moment images in the display list are represented as something like Option<ARC<~Image>>. This is an immutable type that implies that an image must be completely loaded by the time layout generates the display list. This has some big limitations:

  • There is no opportunity to decode images in parallel during the time between layout and rendering.
  • There is no way to support progressive loading of images.
  • This sort of solution can't work for streaming media like video

The way media is represented in the display list needs to be redesigned.
ChrisParis pushed a commit to ChrisParis/servo that referenced this issue Sep 7, 2014
@jgraham
Merge pull request servo#448 from w3c/jgraham/manifest_review
fb690bc 
Add tool for generating test list in JSON format
glennw pushed a commit to glennw/servo that referenced this issue Jan 16, 2017
Auto merge of servo#448 - glennw:prim-vt3, r=pcwalton
12af87a 
Switch primitives to be stored in SoA style in vertex textures.

The primary goals of this patch are:

(1) Store primitives in SoA style. This makes some passes, such as
    culling primitives much faster due to better cache coherency.
(2) Store primitives in flat arrays, for direct upload to GPU. This
    reduces the amount of redundant copying on the CPU during
    frame construction.
(3) Decouple primitive information from geometry instances. This allows
    us to improve occlusion culling, by submitting segments of primitives
    without any extra CPU overhead of copying primitive data.
(4) Allow incremental updates of the GPU SoA arrays during scrolling and
    zoom. This isn't implemented yet, but the framework is in place and
    will allow significant backend/compositor thread improvements during
    scrolling and zoom frames.

There's also a number of other changes that have been rolled in to this
patch since it made sense to fix them at the same time. In particular:

* Blend/composite batches are stored as ints, avoiding CPU float packing overhead.
* Border segment rectangles are calculated in the vertex shader rather than CPU.
* Gradient colors use vertex shader interpolators for axis aligned gradients, reducing FS overhead.
* Removed separate Text/TextRun shader types, resulting in better batching and fewer draw calls.
* Angle gradients support arbitrary stop count in primitive data.
* Removed some unused interpolators from the border shader.
* Axis aligned gradient segments are calculated in vertex shader, reducing CPU overhead.
* Reduced size of packed image structure - UV type is stored in sign of UV field.
* Reduced size of packed glyph structure - color stored per run, rect calculated from UVs.
* Moved some utility types from tiling.rs to util.rs.
* Remove clip cases from batch enum, store needs_clipping flag in batch key.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/webrender/448)
<!-- Reviewable:end -->
@nox
Copy link
Contributor

nox commented Sep 29, 2017

@nical Ain't that not true anymore?

@nical
Copy link
Contributor

nical commented Sep 29, 2017

These days the image display items refer to an ImageKey which is just an Id. There needs to be at least a placeholder image available to webrender by the time the display list is submitted, but the image can then be updated with new content (and new size if need be). I think that servo now does decode images in parallel but I could be wrong. For the media use case there is another system called "external image" that lets the renderer borrow access to image data that isn't owned by webrender (and isn't necessarily mapped on CPU memory) to avoid copies and select the video frame with the best timestamp at the last moment.
I think this issue can be closed now.

@nox
Copy link
Contributor

nox commented Sep 29, 2017

Closing is good. Thanks for the quick reply @nical.

@nox nox closed this as completed Sep 29, 2017
@jdm jdm mentioned this issue Nov 5, 2019
Closed
jdm pushed a commit to jdm/servo that referenced this issue Mar 6, 2021
Auto merge of servo#448 - paulrouget:0.9.5, r=jdm
f84b87a 
Bump version to 0.9.5

<!-- Reviewable:start -->
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/rust-mozjs/448)
<!-- Reviewable:end -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-gfx/displaylist
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@nox @brson @nical @frewsxcv