Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

upgrade to jflex 1.9.1 and remove the workaround for lex scanners limits #4170

Closed
tarzanek opened this issue Feb 7, 2023 · 3 comments · Fixed by #4443
Closed

upgrade to jflex 1.9.1 and remove the workaround for lex scanners limits #4170

tarzanek opened this issue Feb 7, 2023 · 3 comments · Fixed by #4443

Comments

@tarzanek
Copy link
Contributor

tarzanek commented Feb 7, 2023

jflex 1.9.0 is out and
jflex-de/jflex#203
got fixed and jflex now introduces:
(jflex-de/jflex#197 jflex-de/jflex#1045 )

%token_size_limit (<identifier>|<number>)

which makes the workaround of manually patching generated lexical scanner obsolete,
though I see @vladak removed it ?!?!?
d051e17
(I don't see 1.8 fixes it ... )
so it needs to be set where applicable so OpenGrok is in sync with lucene limits and their tokenizers

also jflex-de/jflex#542 is fixed so https://github.com/oracle/opengrok/blob/master/pom.xml#L348
can be cleaned up

@sreenath-tm
Copy link

Hi I would like to work on this issue can you assign this issue to me

@vladak
Copy link
Member

vladak commented Feb 13, 2023

Hi I would like to work on this issue can you assign this issue to me

just submit a PR - the chances that someone else will also work on this are infinitesimaly small.

@vladak vladak added the indexer label Mar 13, 2023
@tarzanek
Copy link
Contributor Author

jflex 1.9.1
now
ok, I will try to set some time aside for this, it should help with performance

@tarzanek tarzanek changed the title upgrade to jflex 1.9.0 and remove the workaround for lex scanners limits upgrade to jflex 1.9.1 and remove the workaround for lex scanners limits May 30, 2023
@ginoaugustine ginoaugustine linked a pull request Oct 18, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants