Support proc-macro TokenStream

[khionu]

I'm currently writing a proc macro that uses SQL to generate code that works with the storage pattern in specs. As is, I have to call display on the TokenStream, which converts it to a raw string. This will mean I'm undoing the tokenization for this parser to re-tokenize, while it should be possible for this parser to use the TokenStream directly.

This is low priority for me, as this issue only causes excess compilation time, but after creating this proc macro, I can add it to my own to-do.

[nickolay]

I'm not sure what "the storage pattern in specs" refers to, but if you manage to convert a TokenTree to a Vec<sqltokenizer::Token>, you can hand it to the parser easily.

This could be a fun excercise, but I expect optimizing the sqlparser's side of things to be more fruitful as it currently does quite a few things in a slow way.

I do wonder how much SQL you have, as parsing 450 KB of SQL takes about 4 seconds for me, whereas compiling sqlparser (about 250 KB of rust) takes 5 seconds.