New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] yypushStream/yypopStream-Bug [sf#49] #51

Closed
lsf37 opened this Issue Feb 15, 2015 · 4 comments

Comments

Projects
None yet
1 participant
@lsf37
Member

lsf37 commented Feb 15, 2015

*Reported by anonymous on 2001-12-20 14:21 UTC

Diagnosis:

yypushStream()/yypopStream() do not save/restore the
whole scanner state context. At least the "cached
variables" xxx_1 are not saved/restored (what are they
good for?). So, after yypushStream, the scanner partly
remains in the state before pushing, which leads to
invalid results or an exception. After the
<<EOF>>-yypopStream, the scanner terminated, forgetting
the input from the restored stream.

Workaround:

in my example, push/pop was always at token boundaries,
so I could afford to make a recursive call to yynext()
immediately after push/pop.

Thanks a lot for maintaining this great tool!

Best regards,

Heiner.

@lsf37 lsf37 changed the title from yypushStream/yypopStream-Bug to [Bug] yypushStream/yypopStream-Bug [sf#49] Feb 15, 2015

@lsf37 lsf37 added this to the jflex bug milestone Feb 15, 2015

@lsf37 lsf37 closed this Feb 15, 2015

@lsf37

This comment has been minimized.

Show comment
Hide comment
@lsf37

lsf37 Feb 15, 2015

Member

Commented by lsf37 on 2002-01-19 15:51 UTC
fixed skeleton.nested file

Member

lsf37 commented Feb 15, 2015

Commented by lsf37 on 2002-01-19 15:51 UTC
fixed skeleton.nested file

@lsf37

This comment has been minimized.

Show comment
Hide comment
@lsf37

lsf37 Feb 15, 2015

Member

Commented by lsf37 on 2002-01-19 15:51 UTC
Logged In: YES
user_id=93534

I've reproduced the bug, and it is as you write a problem
with the cached variables. yypushStream/yypopStream do save
and restore the whole scanner context, but the local chaches
are not updated correctly after an yypopStream. The fix is
quite simple (just move the local cache init into the loop)
and only affects the skeleton file (skeleton.nested). I'll
attach the fixed version to this page, and of course
distribute it with the next version of JFlex.

ps: the cached variables are for speed. access to a local
variable is an order of magnitude faster that access to a
member variable. The technique (although it makes ugly code,
I admit) is also used in some parts of the Java class
library.

Member

lsf37 commented Feb 15, 2015

Commented by lsf37 on 2002-01-19 15:51 UTC
Logged In: YES
user_id=93534

I've reproduced the bug, and it is as you write a problem
with the cached variables. yypushStream/yypopStream do save
and restore the whole scanner context, but the local chaches
are not updated correctly after an yypopStream. The fix is
quite simple (just move the local cache init into the loop)
and only affects the skeleton file (skeleton.nested). I'll
attach the fixed version to this page, and of course
distribute it with the next version of JFlex.

ps: the cached variables are for speed. access to a local
variable is an order of magnitude faster that access to a
member variable. The technique (although it makes ugly code,
I admit) is also used in some parts of the Java class
library.

@lsf37

This comment has been minimized.

Show comment
Hide comment
@lsf37

lsf37 Feb 15, 2015

Member

Updated by lsf37 on 2002-01-19 15:51 UTC

  • status: open --> open-accepted
Member

lsf37 commented Feb 15, 2015

Updated by lsf37 on 2002-01-19 15:51 UTC

  • status: open --> open-accepted
@lsf37

This comment has been minimized.

Show comment
Hide comment
@lsf37

lsf37 Feb 15, 2015

Member

Updated by lsf37 on 2004-04-12 12:26 UTC

  • status: open-accepted --> closed
Member

lsf37 commented Feb 15, 2015

Updated by lsf37 on 2004-04-12 12:26 UTC

  • status: open-accepted --> closed

@lsf37 lsf37 added the bug label Feb 17, 2015

@lsf37 lsf37 modified the milestone: jflex bug Feb 17, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment