Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SVG parsing and rendering runtime benchmark #1656

Merged
merged 3 commits into from
Jul 26, 2023

Conversation

Kobzol
Copy link
Contributor

@Kobzol Kobzol commented Jul 14, 2023

Runtime benchmark that tests parsing and rendering of a big-ass ~30 MiB SVG file from Wikipedia.

So far I have been mostly including these test files directly into the binary (and sometimes inflating/repeating them). I wonder if there is any disadvantage to this, as opposed to loading them at runtime (at the beginning of the program). I like that the embedding makes the runtime benchmarks more self-contained and I don't have to think about in which working directory they are being executed.

The stddev of instructions in rendering is pretty low, which is sweet. For parsing it's much higher on the other hand, but still relatively low percentually wise (0.01%).

Finished svg/svg-parse-1 (1/2)
    [Instructions]: min:   5,457,509,897    mean:    5,458,332,985    stddev:     712,250
          [Cycles]: min:   1,506,487,031    mean:    1,523,466,816    stddev:  13,656,702
  [Wall time [µs]]: min:         355,597    mean:          363,953    stddev:       8,486
   [Branch misses]: min:       3,009,549    mean:        3,033,586    stddev:      23,441
    [Cache misses]: min:       2,316,998    mean:        2,375,970    stddev:      38,273

Finished svg/svg-render-1 (2/2)
    [Instructions]: min:   1,382,696,182    mean:    1,382,696,213    stddev:          18
          [Cycles]: min:     565,619,974    mean:      575,163,087    stddev:  11,329,955
  [Wall time [µs]]: min:         136,351    mean:          138,703    stddev:       1,496
   [Branch misses]: min:       3,213,874    mean:        3,224,041    stddev:      11,126
    [Cache misses]: min:         934,657    mean:          957,595    stddev:      33,268

@Kobzol Kobzol requested a review from nnethercote July 14, 2023 14:50
@nnethercote
Copy link
Contributor

In #1618 you were worried about me adding an 800KiB benchmark, and this is 30MiB. Does it need to be that big?

@Kobzol
Copy link
Contributor Author

Kobzol commented Jul 17, 2023

Yeah I also remembered my comment :) I guess that the difference for me was that the additional files for the compile benchmark were useless to us, while this file serves an actual purpose. But you are right that 30 MiB is a lot.

Repeating/inflating SVG isn't that simple as e.g. for CSS, but I'll try to think of something to reduce the file size, while keeping the benchmark reasonably long. I guess that we can always just do N iterations of parsing.

@Mark-Simulacrum
Copy link
Member

One approach if we don't mind network access is to put the files outside the repo (e.g., S3) - which at least keeps the repo itself small, even though in practice I guess they're still needed to run things.

@Kobzol
Copy link
Contributor Author

Kobzol commented Jul 18, 2023

I would like to avoid introducing such remote dependency into the benchmark, and thus making it less self-contained. It's true that we also have to download all crate dependencies from the internet, but that is cached in CARGO_HOME basically forever. Here, if we would download the file in main (or in a build script), it would download ~30 MiB off the internet everytime the benchmark is built/executed (since the builds are isolated by default and recreated every time), and if the download wouldn't succeed, it would basically fail the benchmark.

After gzipping the file, it is ~4.5 MiB. Would such size be acceptable? It's not that easy to simply repeat SVG files, as it is e.g. with CSS. If it's still too big, I'll try to find a SVG that compresses better or somehow generate it dynamically by repeating some of its inner contents.

@nnethercote
Copy link
Contributor

I can live with the 4.5MB gzipped file.

@Kobzol
Copy link
Contributor Author

Kobzol commented Jul 18, 2023

Ok. I added documentation about adding runtime benchmark input files and how to (de)compress them. I force-pushed the compressed SVG file, so that the larger original file is not in git history. I also compressed the 11 MiB text file that was already present in the repo (this won't help with it being included in git history, but at least it will be smaller on disk and in the binary).

@Kobzol
Copy link
Contributor Author

Kobzol commented Jul 25, 2023

@nnethercote Let me know if you have any further comments :)

Copy link
Contributor

@nnethercote nnethercote left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I overlooked the re-review.

@Kobzol Kobzol merged commit 53e9d0e into rust-lang:master Jul 26, 2023
@Kobzol Kobzol deleted the runtime-svg branch July 26, 2023 06:36
@Kobzol Kobzol restored the runtime-svg branch July 26, 2023 06:36
@Kobzol Kobzol deleted the runtime-svg branch July 26, 2023 06:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants