Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Ignore comments in tokenizer #68
I was just about to open an issue to suggest this. My current use case is reading in files produced by the Stan package for MCMC (example output; note the comment lines at the beginning, middle, and end of the file).
Currently, I have to make two passes through the file and do some extra fiddling, and it would be nice if all of that could be automated.
If it doesn't add too much complication, it might be nice if the comments (or at least their positions) could be included as an attribute as well, similar to how "problems" are handled.
Thanks for making yet another great package!
My bad for just adding an issue to request this -- sorry for the cruft. Seems to me that it does no harm when not used and can be make it or break it when the file you need need to read has comments in it.
Frankly, I'd like to see more people put meta data in commented portions of delimited files because otherwise it tends to go missing.