Allow non-string user tokens

[zepalmer]

I would like to use the AST generated by BNFC throughout my language project. I see from the example of Java 1.1 that tokens which are not built into BNFC are always assumed to be strings. I would like to be able to parse a string like "123L" as a value 123 but I would like to be responsible for generating my own data value. I understand that this would make my tool backend-specific (although it's not hard to imagine writing one piece of code for each backend I want), but carrying an unparsed String value around everywhere that I'm using this AST type is far from ideal. Any thoughts?

Thanks! This tool is great; I hope I'll be able to use it.

[gdetrez]

This is old, I know.

If you really need to have customized data in the tree, one possible workaround is to change the generated code and to "save" your changes in a .diff patch that you could re-apply if you need to regenerate the code with bnfc.

I don't see a way to implement that in bnfc that without introducing language specific code in the grammar, which is a bit against the idea behind bnfc.