Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove `LazyTokenStream`. #58476

Merged
merged 5 commits into from Feb 23, 2019

Conversation

Projects
None yet
7 participants
@nnethercote
Copy link
Contributor

nnethercote commented Feb 15, 2019

LazyTokenStream was added in #40939. Perhaps it was an effective optimization then, but no longer. This PR removes it, making the code both simpler and faster.

r? @alexcrichton

@nnethercote nnethercote force-pushed the nnethercote:rm-LazyTokenStream branch from 6b6d80c to 082b649 Feb 15, 2019

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Feb 15, 2019

@bors try

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Feb 15, 2019

⌛️ Trying commit 082b649 with merge 25f58f3...

bors added a commit that referenced this pull request Feb 15, 2019

Auto merge of #58476 - nnethercote:rm-LazyTokenStream, r=<try>
Remove `LazyTokenStream`.

`LazyTokenStream` was added in #40939. Perhaps it was an effective optimization then, but no longer. This PR removes it, making the code both simpler and faster.

r? @alexcrichton
Show resolved Hide resolved src/librustc/hir/lowering.rs Outdated
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Feb 15, 2019

☀️ Test successful - checks-travis
State: approved= try=True

@petrochenkov petrochenkov self-assigned this Feb 15, 2019

@petrochenkov

This comment has been minimized.

Copy link
Contributor

petrochenkov commented Feb 15, 2019

I'm sorry for the amount of hacks you had to witness while doing this.

@petrochenkov

This comment has been minimized.

Copy link
Contributor

petrochenkov commented Feb 15, 2019

Waiting on perf, otherwise r=me with nits addressed.

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Feb 15, 2019

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Feb 15, 2019

Success: Queued 25f58f3 with parent f47ec2a, comparison URL.

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Feb 15, 2019

I'm sorry for the amount of hacks you had to witness while doing this.

It reminds me of #57486 in which I removed ThinTokenStream -- another token stream "optimization" that actually wasn't...

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Feb 15, 2019

Finished benchmarking try commit 25f58f3

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Feb 15, 2019

The comparison URL shows a clear instructions and max-rss win for deep-vector, a smaller instructions win for keccak, and nothing gets worse.

@alexcrichton

This comment has been minimized.

Copy link
Member

alexcrichton commented Feb 15, 2019

nnethercote added some commits Feb 14, 2019

Change `Token::interpolated_to_tokenstream()`.
It is currently a method of `Token`, but it only is valid to call if
`self` is a `Token::Interpolated`. This commit eliminates the
possibility of misuse by changing it to an associated function that
takes a `Nonterminal`, which also simplifies the call sites.

This requires splitting out a new function, `nonterminal_to_string`.
Remove `LazyTokenStream`.
It's present within `Token::Interpolated` as an optimization, so that if
a nonterminal is converted to a `TokenStream` multiple times, the
first-computed value is saved and reused.

But in practice it's not needed. `interpolated_to_tokenstream()` is a
cold function: it's only called a few dozen times while compiling rustc
itself, and a few hundred times across the entire `rustc-perf` suite.
Furthermore, when it is called, it is almost always the first
conversion, so no benefit is gained from it.

So this commit removes `LazyTokenStream`, along with the now-unnecessary
`Token::interpolated()`.

As well as a significant simplification, the removal speeds things up
slightly, mostly due to not having to `drop` the `LazyTokenStream`
instances.
Avoid a `clone()` in `transcribe()`.
The current code (expensively) clones the value within an `Rc`. This
commit changes things so that the `Rc` itself is (cheaply) cloned
instead, avoid some allocations.

This requires converting a few `Rc` instances to `Lrc`.
Remove some unnecessary `into()` calls.
These are probably leftovers from recent `TokenStream` simplifications.

@nnethercote nnethercote force-pushed the nnethercote:rm-LazyTokenStream branch from 082b649 to 895a794 Feb 17, 2019

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Feb 17, 2019

@petrochenkov: I've pushed two new commits, one to address your review comment, and another very simple clean-up.

@petrochenkov

This comment has been minimized.

Copy link
Contributor

petrochenkov commented Feb 18, 2019

@bors r+

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Feb 18, 2019

📌 Commit 895a794 has been approved by petrochenkov

Centril added a commit to Centril/rust that referenced this pull request Feb 20, 2019

Rollup merge of rust-lang#58476 - nnethercote:rm-LazyTokenStream, r=p…
…etrochenkov

Remove `LazyTokenStream`.

`LazyTokenStream` was added in rust-lang#40939. Perhaps it was an effective optimization then, but no longer. This PR removes it, making the code both simpler and faster.

r? @alexcrichton

bors added a commit that referenced this pull request Feb 20, 2019

Auto merge of #58594 - Centril:rollup, r=Centril
Rollup of 12 pull requests

Successful merges:

 - #55632 (Deny the `overflowing_literals` lint for all editions)
 - #57656 (Deprecate the unstable Vec::resize_default)
 - #57997 (Wrap write_bytes in a function. Move docs)
 - #58059 (deprecate before_exec in favor of unsafe pre_exec)
 - #58122 (RangeInclusive internal iteration performance improvement.)
 - #58198 (Suggest removing parentheses surrounding lifetimes)
 - #58353 (Check the Self-type of inherent associated constants)
 - #58476 (Remove `LazyTokenStream`.)
 - #58511 (Const to op simplification)
 - #58555 (Add a note about 2018e if someone uses `try {` in 2015e)
 - #58588 (remove a bit of dead code)
 - #58589 (cleanup macro after 2018 transition)

Failed merges:

r? @ghost

Centril added a commit to Centril/rust that referenced this pull request Feb 23, 2019

Rollup merge of rust-lang#58476 - nnethercote:rm-LazyTokenStream, r=p…
…etrochenkov

Remove `LazyTokenStream`.

`LazyTokenStream` was added in rust-lang#40939. Perhaps it was an effective optimization then, but no longer. This PR removes it, making the code both simpler and faster.

r? @alexcrichton

Centril added a commit to Centril/rust that referenced this pull request Feb 23, 2019

Rollup merge of rust-lang#58476 - nnethercote:rm-LazyTokenStream, r=p…
…etrochenkov

Remove `LazyTokenStream`.

`LazyTokenStream` was added in rust-lang#40939. Perhaps it was an effective optimization then, but no longer. This PR removes it, making the code both simpler and faster.

r? @alexcrichton

bors added a commit that referenced this pull request Feb 23, 2019

Auto merge of #58665 - Centril:rollup, r=Centril
Rollup of 16 pull requests

Successful merges:

 - #58122 (RangeInclusive internal iteration performance improvement.)
 - #58199 (Add better error message for partial move)
 - #58216 ( Set secure flags when opening a named pipe on Windows )
 - #58227 (Updated RELEASES.md for 1.33.0)
 - #58353 (Check the Self-type of inherent associated constants)
 - #58453 (SGX target: fix panic = abort)
 - #58454 (Refactor Windows stdio and remove stdin double buffering )
 - #58476 (Remove `LazyTokenStream`.)
 - #58526 (Special suggestion for illegal unicode curly quote pairs)
 - #58595 (Turn duration consts into associated consts)
 - #58609 (Allow Self::Module to be mutated.)
 - #58628 (Optimise vec![false; N] to zero-alloc)
 - #58642 (rustdoc: support methods on primitives in intra-doc links)
 - #58643 (Don't generate minification variables if minification disabled)
 - #58648 (Update tests to account for cross-platform testing and miri.)
 - #58654 (Do not underflow after resetting unmatched braces count)

Failed merges:

r? @ghost

Centril added a commit to Centril/rust that referenced this pull request Feb 23, 2019

Rollup merge of rust-lang#58476 - nnethercote:rm-LazyTokenStream, r=p…
…etrochenkov

Remove `LazyTokenStream`.

`LazyTokenStream` was added in rust-lang#40939. Perhaps it was an effective optimization then, but no longer. This PR removes it, making the code both simpler and faster.

r? @alexcrichton

bors added a commit that referenced this pull request Feb 23, 2019

Auto merge of #58666 - Centril:rollup, r=Centril
Rollup of 15 pull requests

Successful merges:

 - #58122 (RangeInclusive internal iteration performance improvement.)
 - #58199 (Add better error message for partial move)
 - #58216 ( Set secure flags when opening a named pipe on Windows )
 - #58227 (Updated RELEASES.md for 1.33.0)
 - #58353 (Check the Self-type of inherent associated constants)
 - #58453 (SGX target: fix panic = abort)
 - #58476 (Remove `LazyTokenStream`.)
 - #58526 (Special suggestion for illegal unicode curly quote pairs)
 - #58595 (Turn duration consts into associated consts)
 - #58609 (Allow Self::Module to be mutated.)
 - #58628 (Optimise vec![false; N] to zero-alloc)
 - #58642 (rustdoc: support methods on primitives in intra-doc links)
 - #58643 (Don't generate minification variables if minification disabled)
 - #58648 (Update tests to account for cross-platform testing and miri.)
 - #58654 (Do not underflow after resetting unmatched braces count)

Failed merges:

r? @ghost

Centril added a commit to Centril/rust that referenced this pull request Feb 23, 2019

Rollup merge of rust-lang#58476 - nnethercote:rm-LazyTokenStream, r=p…
…etrochenkov

Remove `LazyTokenStream`.

`LazyTokenStream` was added in rust-lang#40939. Perhaps it was an effective optimization then, but no longer. This PR removes it, making the code both simpler and faster.

r? @alexcrichton

Centril added a commit to Centril/rust that referenced this pull request Feb 23, 2019

Rollup merge of rust-lang#58476 - nnethercote:rm-LazyTokenStream, r=p…
…etrochenkov

Remove `LazyTokenStream`.

`LazyTokenStream` was added in rust-lang#40939. Perhaps it was an effective optimization then, but no longer. This PR removes it, making the code both simpler and faster.

r? @alexcrichton

bors added a commit that referenced this pull request Feb 23, 2019

Auto merge of #58669 - Centril:rollup, r=Centril
Rollup of 16 pull requests

Successful merges:

 - #58100 (Transition librustdoc to Rust 2018)
 - #58122 (RangeInclusive internal iteration performance improvement.)
 - #58199 (Add better error message for partial move)
 - #58227 (Updated RELEASES.md for 1.33.0)
 - #58353 (Check the Self-type of inherent associated constants)
 - #58453 (SGX target: fix panic = abort)
 - #58476 (Remove `LazyTokenStream`.)
 - #58526 (Special suggestion for illegal unicode curly quote pairs)
 - #58595 (Turn duration consts into associated consts)
 - #58609 (Allow Self::Module to be mutated.)
 - #58628 (Optimise vec![false; N] to zero-alloc)
 - #58643 (Don't generate minification variables if minification disabled)
 - #58648 (Update tests to account for cross-platform testing and miri.)
 - #58654 (Do not underflow after resetting unmatched braces count)
 - #58658 (Add expected/provided byte alignments to validation error message)
 - #58667 (Reduce Miri-related Code Repetition `like (n << amt) >> amt`)

Failed merges:

r? @ghost

bors added a commit that referenced this pull request Feb 23, 2019

Auto merge of #58669 - Centril:rollup, r=Centril
Rollup of 16 pull requests

Successful merges:

 - #58100 (Transition librustdoc to Rust 2018)
 - #58122 (RangeInclusive internal iteration performance improvement.)
 - #58199 (Add better error message for partial move)
 - #58227 (Updated RELEASES.md for 1.33.0)
 - #58353 (Check the Self-type of inherent associated constants)
 - #58453 (SGX target: fix panic = abort)
 - #58476 (Remove `LazyTokenStream`.)
 - #58526 (Special suggestion for illegal unicode curly quote pairs)
 - #58595 (Turn duration consts into associated consts)
 - #58609 (Allow Self::Module to be mutated.)
 - #58628 (Optimise vec![false; N] to zero-alloc)
 - #58643 (Don't generate minification variables if minification disabled)
 - #58648 (Update tests to account for cross-platform testing and miri.)
 - #58654 (Do not underflow after resetting unmatched braces count)
 - #58658 (Add expected/provided byte alignments to validation error message)
 - #58667 (Reduce Miri-related Code Repetition `like (n << amt) >> amt`)

Failed merges:

r? @ghost

@bors bors merged commit 895a794 into rust-lang:master Feb 23, 2019

@nnethercote nnethercote deleted the nnethercote:rm-LazyTokenStream branch Feb 26, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.