Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C++ runtime: For UNWANTED_TOKEN_EXCEPTION, expected token is printed instead of seen token. #219

Open
nyh opened this issue Apr 30, 2023 · 1 comment

Comments

@nyh
Copy link

nyh commented Apr 30, 2023

The ScyllaDB project uses Antlr3 with the C++ runtime.
In scylladb/scylladb#1703 it was noted that ugly error messages are reported. One of the strange error messages is:

line 1:71 extraneous input 'STORAGE' expecting <invalid>

This is ScyllaDB's own (broken in its own way) error message printer, but when I try to use displayRecognitionError() from /usr/include/antlr3exception.inl, it's even worse - only this wrong "Expected" is printed. A snippted from that function:

        case UNWANTED_TOKEN_EXCEPTION:
                ...
                                str_stream << " : Extraneous input - expected "
                                                   << tokenNames[ BaseType::m_expecting] << " ...\n";

So in UNWANTED_TOKEN_EXCEPTION this function prints, instead of the seen (and unwanted) token, the "expected" token, which isn't available in the case of this exception, which is why it's printed as "<invalid">. What needs to be printed in this case is the seen token, not the expected token.

@ibre5041
Copy link
Contributor

ibre5041 commented May 1, 2023

Try to use runtime from this branch.
https://github.com/ibre5041/antlr3/commits/master
So time ago I fixed a some bugs in C++ runtime.

Also try to use this fixed version of antlr tool. It generates much faster C++ code(faster than Java)
https://github.com/tora-tool/tora/tree/master/extlibs/parsing.cpp

Long story short, Java string templates generate conditions like:
if (a == 'c' && x.predicate())

While C/C++ parser code has condition order reversed:
if (x.predicate() && a=='c')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants