Add a possibility to parse the text and view how it was parsed

[iddm]

When I was in the university there was a program for learning grammatics in the programming languages theory course. It was very useful for learners to see how the code was parsed in the grammar tree. Can we do the same thing? It must not be so hard, basically, we already have it while we output Grammar as a tree in the README file, the same thing is needed but with real data which could come not from the random generator but from the user input - this is how learners learn how to write grammar correctly.

[shnewto]

Ah! Thanks for this suggestion! If I'm understanding correctly, this might be a way to analyse input that conforms to the parsed grammar?

Maybe you could help clarify the output. As a very simple example, if we used the dna grammar from the REAMDE:

<dna> ::= <base> | <base> <dna>
<base> ::= "A" | "C" | "G" | "T";

Then we parsed an input string ATCGT, what would you expect to get back?

Something like this?

<dna> --> 
<base> <dna> --> 
<base> <base> <dna> -->
<base> <base> <base> <dna> -->
<base> <base> <base> <base> <dna> -->
<base> <base> <base> <base> <base>

[iddm]

If I'm understanding correctly, this might be a way to analyse input that conforms to the parsed grammar?

Yes.

Then we parsed an input string ATCGT, what would you expect to get back?

Yes.
I am not sure (it was many years ago), however, I think it was something like that:

<dna> --> "ATCGT"
   |
   |--------- <base> -------> "A"
   |--------- <dna> -------> "TCGT"
                     |--------- <base> -------> "T"
                     |--------- <dna> -------> "CGT"
                                       |--------- <base> -------> "C"
                                       |--------- <dna> -------> "GT"
                                                         |--------- <base> -------> "G"
                                                         |--------- <dna> -------> "T"
                                                                           |--------- <base> -------> "T"

So, the idea is to see how the data entered by the user fits the grammar, what rules and how are satisfied.

I would also suggest to make output be supported in different formats - in a human-readable one and in, for example, json, because if anyone wanted to create a gui for it, he would need it.

[shnewto]

Thanks @vityafx this does seem like it'd be a really nice addition. I'll see what I'm able to figure out and keep you posted.

[iddm]

Just noticing, it would also be nice to have a possibility to navigate through them, not only see the output. For example, I am remembering the university course for implementing lisp in rust, so I wanna create a translator scheme -> rust to be used by rustc directly. I know the grammar but if I already have a parser which can be done by this issue, it will also be nice to navigate trough the rules and be able to build the ast and then use it (I will probably just convert it to rust ast and feed the rustc with it).

[shnewto]

@vityafx I think exposing the AST would be a very valuable part a feature like this. I plan on incorporating that too, thanks for bringing it up! As far as timeline for progress on this work / series of features, it's likely that it'll be pushed back until January.

[shnewto]

Time flies like an arrow! I kicked around some work on building an out Earley parser on top of bnf, it seems like a promising solution but I don't know if I'm going to be able to incorporate anything anytime soon. Would welcome anyone who wants to take a stab.

[shnewto]

@vityafx The implementation for this has made it to the main branch, a release is being drafted now and will be published later today.

Wowie, it was a long time coming!

[iddm]

Indeed! I'll check out, thanks!