Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Derive `Rustc{En,De}codable` for `TokenStream`. #65641

Merged

Conversation

@nnethercote
Copy link
Contributor

nnethercote commented Oct 20, 2019

TokenStream used to be a complex type, but it is now just a newtype
around a Lrc<Vec<TreeAndJoint>>. Currently it uses custom encoding
that discards the IsJoint and custom decoding that adds NonJoint
back in for every token tree. This requires building intermediate
Vec<TokenTree>s.

This commit makes TokenStream derive Rustc{En,De}codable. This
simplifies the code, and avoids the creation of the intermediate
vectors, saving up to 3% on various benchmarks. It also changes the AST
JSON output in one test.

r? @petrochenkov

`TokenStream` used to be a complex type, but it is now just a newtype
around a `Lrc<Vec<TreeAndJoint>>`. Currently it uses custom encoding
that discards the `IsJoint` and custom decoding that adds `NonJoint`
back in for every token tree. This requires building intermediate
`Vec<TokenTree>`s.

This commit makes `TokenStream` derive `Rustc{En,De}codable`. This
simplifies the code, and avoids the creation of the intermediate
vectors, saving up to 3% on various benchmarks. It also changes the AST
JSON output in one test.
@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Oct 20, 2019

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Oct 20, 2019

Awaiting bors try build completion

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Oct 20, 2019

⌛️ Trying commit c290293 with merge 1b0e1f5...

bors added a commit that referenced this pull request Oct 20, 2019
…able, r=<try>

Derive `Rustc{En,De}codable` for `TokenStream`.

`TokenStream` used to be a complex type, but it is now just a newtype
around a `Lrc<Vec<TreeAndJoint>>`. Currently it uses custom encoding
that discards the `IsJoint` and custom decoding that adds `NonJoint`
back in for every token tree. This requires building intermediate
`Vec<TokenTree>`s.

This commit makes `TokenStream` derive `Rustc{En,De}codable`. This
simplifies the code, and avoids the creation of the intermediate
vectors, saving up to 3% on various benchmarks. It also changes the AST
JSON output in one test.

r? @petrochenkov
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Oct 21, 2019

☀️ Try build successful - checks-azure
Build commit: 1b0e1f5 (1b0e1f524644251dc9f642f247055f5af49ddddd)

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Oct 21, 2019

Queued 1b0e1f5 with parent 89e645a, future comparison URL.

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Oct 21, 2019

Finished benchmarking try commit 1b0e1f5, comparison URL.

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

nnethercote commented Oct 21, 2019

The perf results show lots of instruction count wins, up to 1.5%. The wall-time results are much noisier, of course, but also show a clear improvement.

@petrochenkov

This comment has been minimized.

Copy link
Contributor

petrochenkov commented Oct 21, 2019

@bors r+

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Oct 21, 2019

📌 Commit c290293 has been approved by petrochenkov

JohnTitor added a commit to JohnTitor/rust that referenced this pull request Oct 22, 2019
…dable-Decodable, r=petrochenkov

Derive `Rustc{En,De}codable` for `TokenStream`.

`TokenStream` used to be a complex type, but it is now just a newtype
around a `Lrc<Vec<TreeAndJoint>>`. Currently it uses custom encoding
that discards the `IsJoint` and custom decoding that adds `NonJoint`
back in for every token tree. This requires building intermediate
`Vec<TokenTree>`s.

This commit makes `TokenStream` derive `Rustc{En,De}codable`. This
simplifies the code, and avoids the creation of the intermediate
vectors, saving up to 3% on various benchmarks. It also changes the AST
JSON output in one test.

r? @petrochenkov
bors added a commit that referenced this pull request Oct 22, 2019
Rollup of 10 pull requests

Successful merges:

 - #63810 (Make <*const/mut T>::offset_from `const fn`)
 - #65583 (rustc_metadata: use a table for super_predicates, fn_sig, impl_trait_ref.)
 - #65617 (Fix WASI sleep impl)
 - #65641 (Derive `Rustc{En,De}codable` for `TokenStream`.)
 - #65656 (Add option to disable keyboard shortcuts in docs)
 - #65681 (Code cleanups following up on #65576.)
 - #65686 (refactor and move `maybe_append` )
 - #65688 (Add some tests for fixed ICEs)
 - #65689 (bring back some Debug instances for Miri)
 - #65695 (self-profiling: Remove module names from some event-ids in codegen backend.)

Failed merges:

r? @ghost
Centril added a commit to Centril/rust that referenced this pull request Oct 23, 2019
…dable-Decodable, r=petrochenkov

Derive `Rustc{En,De}codable` for `TokenStream`.

`TokenStream` used to be a complex type, but it is now just a newtype
around a `Lrc<Vec<TreeAndJoint>>`. Currently it uses custom encoding
that discards the `IsJoint` and custom decoding that adds `NonJoint`
back in for every token tree. This requires building intermediate
`Vec<TokenTree>`s.

This commit makes `TokenStream` derive `Rustc{En,De}codable`. This
simplifies the code, and avoids the creation of the intermediate
vectors, saving up to 3% on various benchmarks. It also changes the AST
JSON output in one test.

r? @petrochenkov
bors added a commit that referenced this pull request Oct 23, 2019
Rollup of 9 pull requests

Successful merges:

 - #65144 (Add Cow::is_borrowed and Cow::is_owned)
 - #65193 (Lockless LintStore)
 - #65583 (rustc_metadata: use a table for super_predicates, fn_sig, impl_trait_ref.)
 - #65641 (Derive `Rustc{En,De}codable` for `TokenStream`.)
 - #65648 (Eliminate `intersect_opt`.)
 - #65657 (Remove `InternedString`)
 - #65666 (Deprecated proc_macro doesn't trigger warning on build library)
 - #65691 (Update E0659 error code long explanation to 2018 edition)
 - #65704 (relax ExactSizeIterator bound on write_bytes)

Failed merges:

 - #65625 (Turn crate store into a resolver output)

r? @ghost
Centril added a commit to Centril/rust that referenced this pull request Oct 23, 2019
…dable-Decodable, r=petrochenkov

Derive `Rustc{En,De}codable` for `TokenStream`.

`TokenStream` used to be a complex type, but it is now just a newtype
around a `Lrc<Vec<TreeAndJoint>>`. Currently it uses custom encoding
that discards the `IsJoint` and custom decoding that adds `NonJoint`
back in for every token tree. This requires building intermediate
`Vec<TokenTree>`s.

This commit makes `TokenStream` derive `Rustc{En,De}codable`. This
simplifies the code, and avoids the creation of the intermediate
vectors, saving up to 3% on various benchmarks. It also changes the AST
JSON output in one test.

r? @petrochenkov
bors added a commit that referenced this pull request Oct 23, 2019
Rollup of 12 pull requests

Successful merges:

 - #64178 (More Clippy fixes for alloc, core and std)
 - #65144 (Add Cow::is_borrowed and Cow::is_owned)
 - #65193 (Lockless LintStore)
 - #65479 (Add the `matches!( $expr, $pat ) -> bool` macro)
 - #65518 (Avoid ICE when checking `Destination` of `break` inside a closure)
 - #65583 (rustc_metadata: use a table for super_predicates, fn_sig, impl_trait_ref.)
 - #65641 (Derive `Rustc{En,De}codable` for `TokenStream`.)
 - #65648 (Eliminate `intersect_opt`.)
 - #65657 (Remove `InternedString`)
 - #65691 (Update E0659 error code long explanation to 2018 edition)
 - #65696 (Fix an issue with const inference variables sticking around under Chalk + NLL)
 - #65704 (relax ExactSizeIterator bound on write_bytes)

Failed merges:

r? @ghost
bors added a commit that referenced this pull request Oct 24, 2019
Rollup of 12 pull requests

Successful merges:

 - #64178 (More Clippy fixes for alloc, core and std)
 - #65144 (Add Cow::is_borrowed and Cow::is_owned)
 - #65193 (Lockless LintStore)
 - #65479 (Add the `matches!( $expr, $pat ) -> bool` macro)
 - #65518 (Avoid ICE when checking `Destination` of `break` inside a closure)
 - #65583 (rustc_metadata: use a table for super_predicates, fn_sig, impl_trait_ref.)
 - #65641 (Derive `Rustc{En,De}codable` for `TokenStream`.)
 - #65648 (Eliminate `intersect_opt`.)
 - #65657 (Remove `InternedString`)
 - #65691 (Update E0659 error code long explanation to 2018 edition)
 - #65696 (Fix an issue with const inference variables sticking around under Chalk + NLL)
 - #65704 (relax ExactSizeIterator bound on write_bytes)

Failed merges:

r? @ghost
@bors bors merged commit c290293 into rust-lang:master Oct 24, 2019
5 checks passed
5 checks passed
homu Test successful
Details
pr Build #20191020.44 succeeded
Details
pr (Linux mingw-check) Linux mingw-check succeeded
Details
pr (Linux x86_64-gnu-llvm-6.0) Linux x86_64-gnu-llvm-6.0 succeeded
Details
pr (LinuxTools) LinuxTools succeeded
Details
@nnethercote nnethercote deleted the nnethercote:derive-TokenStream-Encodable-Decodable branch Oct 25, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.