New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up the macro parser #50855

Merged
merged 3 commits into from May 20, 2018

Conversation

Projects
None yet
6 participants
@nnethercote
Contributor

nnethercote commented May 18, 2018

These three commits reduce the number of allocations done by the macro parser, in some cases dramatically. For example, for a clean check builds of html5ever, the number of allocations is reduced by 40%.

Here are the rustc-benchmarks that are sped up by at least 1%.

html5ever-check
        avg: -6.6%      min: -10.3%     max: -4.1%
html5ever
        avg: -5.2%      min: -9.5%      max: -2.8%
html5ever-opt
        avg: -4.3%      min: -9.3%      max: -1.6%
crates.io-check
        avg: -1.8%      min: -2.9%      max: -0.6%
crates.io-opt
        avg: -1.0%      min: -2.2%      max: -0.1%
crates.io
        avg: -1.1%      min: -2.2%      max: -0.2%
Change `TokenTreeOrTokenTreeVec` to `TokenTreeOrTokenTreeSlice`.
This avoids a `to_owned` call that can be hot, speeding up the various
runs of html5ever by 1--5%, and some runs of crates.io by 2--3%.
@rust-highfive

This comment has been minimized.

Collaborator

rust-highfive commented May 18, 2018

r? @nikomatsakis

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive

This comment has been minimized.

Collaborator

rust-highfive commented May 18, 2018

The job x86_64-gnu-llvm-3.9 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:04:56] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:57] tidy error: /checkout/src/libsyntax/ext/tt/macro_parser.rs:237: tab character
[00:04:57] tidy error: /checkout/src/libsyntax/ext/tt/macro_parser.rs:246: tab character
[00:04:58] some tidy checks failed
[00:04:58] 
[00:04:58] 
[00:04:58] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:58] 
[00:04:58] 
[00:04:58] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:58] Build completed unsuccessfully in 0:02:02
[00:04:58] Build completed unsuccessfully in 0:02:02
[00:04:58] Makefile:79: recipe for target 'tidy' failed
[00:04:58] make: *** [tidy] Error 1

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:1e2f5cfb
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@petrochenkov

This comment has been minimized.

Contributor

petrochenkov commented May 18, 2018

@bors r=me after tidy is fixed

@petrochenkov

This comment has been minimized.

Contributor

petrochenkov commented May 18, 2018

@bors delegate+

@bors

This comment has been minimized.

Contributor

bors commented May 18, 2018

✌️ @nnethercote can now approve this pull request

nnethercote added some commits May 18, 2018

Introduce `MatcherPosHandle`.
This lets us store most `MatcherPos` instances on the stack. This speeds
up various runs of html5ever, the best by 3%.
Make `Directory::path` a `Cow`.
Because we create a lot of these in the macro parser, but only very
rarely modify them.

This speeds up some html5ever runs by 2--3%.
@nnethercote

This comment has been minimized.

Contributor

nnethercote commented May 18, 2018

Thank you for the fast review.

@bors: r=petrochenkov

@bors

This comment has been minimized.

Contributor

bors commented May 18, 2018

📌 Commit ad47145 has been approved by petrochenkov

@bors

This comment has been minimized.

Contributor

bors commented May 20, 2018

⌛️ Testing commit ad47145 with merge 4c26e2e...

bors added a commit that referenced this pull request May 20, 2018

Auto merge of #50855 - nnethercote:fewer-macro_parser-allocs, r=petro…
…chenkov

Speed up the macro parser

These three commits reduce the number of allocations done by the macro parser, in some cases dramatically. For example, for a clean check builds of html5ever, the number of allocations is reduced by 40%.

Here are the rustc-benchmarks that are sped up by at least 1%.
```
html5ever-check
        avg: -6.6%      min: -10.3%     max: -4.1%
html5ever
        avg: -5.2%      min: -9.5%      max: -2.8%
html5ever-opt
        avg: -4.3%      min: -9.3%      max: -1.6%
crates.io-check
        avg: -1.8%      min: -2.9%      max: -0.6%
crates.io-opt
        avg: -1.0%      min: -2.2%      max: -0.1%
crates.io
        avg: -1.1%      min: -2.2%      max: -0.2%
```
@bors

This comment has been minimized.

Contributor

bors commented May 20, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: petrochenkov
Pushing 4c26e2e to master...

@bors bors merged commit ad47145 into rust-lang:master May 20, 2018

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
homu Test successful
Details

@nnethercote nnethercote deleted the nnethercote:fewer-macro_parser-allocs branch May 21, 2018

@mark-i-m

This comment has been minimized.

Contributor

mark-i-m commented Jun 23, 2018

This code has gotten a lot of love this year :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment