Add locations to the AST #21

Open
Chris00 opened this Issue Sep 4, 2013 · 6 comments

Comments

Projects
None yet
3 participants
@Chris00
Member

Chris00 commented Sep 4, 2013

Adding locations to the AST would be very useful to report errors. say for example you want to run some code but it fails. Without locations, it is difficult to report a nice error message to the user.

@pw374

This comment has been minimized.

Show comment
Hide comment
@pw374

pw374 Sep 4, 2013

Contributor

Can you be more specific? I mean, there is no notion of error in Markdown...

Contributor

pw374 commented Sep 4, 2013

Can you be more specific? I mean, there is no notion of error in Markdown...

@Chris00

This comment has been minimized.

Show comment
Hide comment
@Chris00

Chris00 Sep 4, 2013

Member

It is not for Markdown processing in itself. It is for what you do once the markdown has been processed (e.g. check that the code is correct).

Member

Chris00 commented Sep 4, 2013

It is not for Markdown processing in itself. It is for what you do once the markdown has been processed (e.g. check that the code is correct).

@pw374

This comment has been minimized.

Show comment
Hide comment
@pw374

pw374 Sep 4, 2013

Contributor

Ok, I understand that basically, for a node in the AST, you want to be able to know which Markdown expression that led to it. I agree that it would be nice to have this feature.

I'm currently thinking that this might need a lot of code refactoring... I have to think more.
(The first problem that I see is that we'd need good locations for the token list manipulated by the parser. When it does out of the lexer, the list is certainly correct, but after a while, it's likely that the parser would have messed with the list and made weird. So the locations would have to be robust to the parser's manipulations. What I mean is that I think we cannot easily retrieve locations just by looking at the current tokens list, whereas if the parser was nicer with this list, we'd just have to read the list...)

If you have suggestions on how to do it, don't hesitate ;)

Contributor

pw374 commented Sep 4, 2013

Ok, I understand that basically, for a node in the AST, you want to be able to know which Markdown expression that led to it. I agree that it would be nice to have this feature.

I'm currently thinking that this might need a lot of code refactoring... I have to think more.
(The first problem that I see is that we'd need good locations for the token list manipulated by the parser. When it does out of the lexer, the list is certainly correct, but after a while, it's likely that the parser would have messed with the list and made weird. So the locations would have to be robust to the parser's manipulations. What I mean is that I think we cannot easily retrieve locations just by looking at the current tokens list, whereas if the parser was nicer with this list, we'd just have to read the list...)

If you have suggestions on how to do it, don't hesitate ;)

@Chris00

This comment has been minimized.

Show comment
Hide comment
@Chris00

Chris00 Sep 4, 2013

Member

There is more. Since we intend to pre-process some files, an annotation like the # in OCaml must be supported to be able to add HTML code without loosing the locations (think about the 99problems page on the web site, the fact that the answers are hidden requires to wrap them with HTML code but, if the solution contains an error — say a syntax problem — then we want to tell the author using the locations in the original file).

Member

Chris00 commented Sep 4, 2013

There is more. Since we intend to pre-process some files, an annotation like the # in OCaml must be supported to be able to add HTML code without loosing the locations (think about the 99problems page on the web site, the fact that the answers are hidden requires to wrap them with HTML code but, if the solution contains an error — say a syntax problem — then we want to tell the author using the locations in the original file).

@pw374

This comment has been minimized.

Show comment
Hide comment
@pw374

pw374 Sep 4, 2013

Contributor

Yes of course, if we know where we come from (at the AST level), then it's easy to produce the locations, in say HTML comments or something more hack-ish like empty span tags. This could be for each block (e.g., paragraph, blockquote, ...), for instance...

Contributor

pw374 commented Sep 4, 2013

Yes of course, if we know where we come from (at the AST level), then it's easy to produce the locations, in say HTML comments or something more hack-ish like empty span tags. This could be for each block (e.g., paragraph, blockquote, ...), for instance...

@darioteixeira

This comment has been minimized.

Show comment
Hide comment
@darioteixeira

darioteixeira Aug 8, 2014

I've added Markdown support to Lambdoc via OMD (see here for the code). And indeed, the major problem is still the lack of location information. This is necessary in Lambdoc for two main reasons: first, because Lambdoc allows customisable feature sets (you may want to forbid your users from formatting text as bold, for a silly example); second, because not all OMD features are present in Lambdoc (nesting beyond H3, for instance). In both cases I need to know the line number where the offending input occurred, so I can present a user-friendly error message.

I've added Markdown support to Lambdoc via OMD (see here for the code). And indeed, the major problem is still the lack of location information. This is necessary in Lambdoc for two main reasons: first, because Lambdoc allows customisable feature sets (you may want to forbid your users from formatting text as bold, for a silly example); second, because not all OMD features are present in Lambdoc (nesting beyond H3, for instance). In both cases I need to know the line number where the offending input occurred, so I can present a user-friendly error message.

@darioteixeira darioteixeira referenced this issue Aug 8, 2014

Open

TyXML #82

@pw374 pw374 added this to the 1.1.0 milestone Aug 10, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment