Commits on Jan 8, 2019

1. Make TokenStream less recursive. …

   `TokenStream` is currently recursive in *two* ways:
   
   - the `TokenTree` variant contains a `ThinTokenStream`, which can
     contain a `TokenStream`;
   
   - the `TokenStream` variant contains a `Vec<TokenStream>`.
   
   The latter is not necessary and causes significant complexity. This
   commit replaces it with the simpler `Vec<(TokenTree, IsJoint)>`.
   
   This reduces complexity significantly. In particular, `StreamCursor` is
   eliminated, and `Cursor` becomes much simpler, consisting now of just a
   `TokenStream` and an index.
   
   The commit also removes the `Extend` impl for `TokenStream`, because it
   is only used in tests. (The commit also removes those tests.)
   
   Overall, the commit reduces the number of lines of code by almost 200.

   

   nnethercote committed Jan 8, 2019
   Configuration menu

   • View commit details
   • Copy full SHA for e80a930
   • Browse repository at this point

   Copy the full SHA

   e80a930 View commit details

   Browse the repository at this point in the history