Skip to content

Commit

Permalink
Make TokenStream::from_iter less general and more efficient.
Browse files Browse the repository at this point in the history
The current code has this impl:
```
impl<T: Into<TokenStream>> iter::FromIterator<T> for TokenStream
```
If given an `IntoIterator<Item = TokenTree>`, it will convert each individual
`TokenTree` to a `TokenStream` (at the cost of two allocations: a `Vec`
and an `Lrc`). It will then merge those `TokenStream`s into a single
`TokenStream`. This is inefficient.

This commit changes the impl to this less general one:
```
impl iter::FromIterator<TokenTree> for TokenStream
```
It collects the `TokenTree`s into a single `Vec` first and then converts that
to a `TokenStream` by wrapping it in a single `Lrc`. The previous generality
was unnecessary; no other code needs changing.

This change speeds up several benchmarks by up to 4%.
  • Loading branch information
nnethercote committed Oct 18, 2019
1 parent d0eaf60 commit a6eef29
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions src/libsyntax/tokenstream.rs
Expand Up @@ -202,9 +202,9 @@ impl From<TokenTree> for TreeAndJoint {
}
}

impl<T: Into<TokenStream>> iter::FromIterator<T> for TokenStream {
fn from_iter<I: IntoIterator<Item = T>>(iter: I) -> Self {
TokenStream::from_streams(iter.into_iter().map(Into::into).collect::<SmallVec<_>>())
impl iter::FromIterator<TokenTree> for TokenStream {
fn from_iter<I: IntoIterator<Item = TokenTree>>(iter: I) -> Self {
TokenStream::new(iter.into_iter().map(Into::into).collect::<Vec<TreeAndJoint>>())
}
}

Expand Down

0 comments on commit a6eef29

Please sign in to comment.