Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid unnecessary `TokenTree` to `TokenStream` conversions #65455

Merged
Changes from 1 commit
Commits
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.

Always

Just for now

Next

Make `TokenStream::from_iter` less general and more efficient.

The current code has this impl:
```
impl<T: Into<TokenStream>> iter::FromIterator<T> for TokenStream
```
If given an `IntoIterator<Item = TokenTree>`, it will convert each individual
`TokenTree` to a `TokenStream` (at the cost of two allocations: a `Vec`
and an `Lrc`). It will then merge those `TokenStream`s into a single
`TokenStream`. This is inefficient.

This commit changes the impl to this less general one:
```
impl iter::FromIterator<TokenTree> for TokenStream
```
It collects the `TokenTree`s into a single `Vec` first and then converts that
to a `TokenStream` by wrapping it in a single `Lrc`. The previous generality
was unnecessary; no other code needs changing.

This change speeds up several benchmarks by up to 4%.
  • Loading branch information
nnethercote committed Oct 13, 2019
commit a6eef299d3b0ca24f8ffc0c3dc03283c09ec7945
@@ -202,9 +202,9 @@ impl From<TokenTree> for TreeAndJoint {
}
}

impl<T: Into<TokenStream>> iter::FromIterator<T> for TokenStream {
fn from_iter<I: IntoIterator<Item = T>>(iter: I) -> Self {
TokenStream::from_streams(iter.into_iter().map(Into::into).collect::<SmallVec<_>>())
impl iter::FromIterator<TokenTree> for TokenStream {
fn from_iter<I: IntoIterator<Item = TokenTree>>(iter: I) -> Self {
TokenStream::new(iter.into_iter().map(Into::into).collect::<Vec<TreeAndJoint>>())
}
}

ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.