Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lexing of `protected/`endprotected sections can never complete #899

Closed
veripoolbot opened this issue Mar 17, 2015 · 3 comments
Closed

Lexing of `protected/`endprotected sections can never complete #899

veripoolbot opened this issue Mar 17, 2015 · 3 comments
Assignees

Comments

@veripoolbot
Copy link
Collaborator

@veripoolbot veripoolbot commented Mar 17, 2015


Author Name: Corey Teffetalor
Original Redmine Issue: 899 from https://www.veripool.org
Original Date: 2015-03-17
Original Assignee: Wilson Snyder (@wsnyder)


protected endprotected sections are supported by a text callback in the Parser, however these may be VERY large in practice.

The lexer (in both preprocessing and parsing) uses yymore() to concatenate the data inside a protected block to a single token. However once you have ~3Mb+ of encrypted data between protected and `endprotected statements, I believe the repeated concatenations to the single token begin to degrade performance, it's not impossible to have enough encrypted data (10's-100's of Mb) that the preprocessing will never complete.

I think the solution would be to not concatenate the entire section into a VP_TEXT token and instead create many VP_TEXT tokens and ignore them incrementally if they come in. But I think this has consequences for the rest of the parser that become quite ugly (and would ultimately change how Verilog::Parser callbacks operate).

As a workaround I just remove these sections prior to handing text to the Preprocessor/Parser/etc.. but for other use-cases discarding this data would be incorrect.

@veripoolbot

This comment has been minimized.

Copy link
Collaborator Author

@veripoolbot veripoolbot commented Mar 17, 2015


Original Redmine Comment
Author Name: Corey Teffetalor
Original Date: 2015-03-17T19:44:05Z


Apologies, this was intended to be filed against Verilog::Perl, not Verilator. I can refile there if necessary.

@veripoolbot

This comment has been minimized.

Copy link
Collaborator Author

@veripoolbot veripoolbot commented Mar 18, 2015


Original Redmine Comment
Author Name: Wilson Snyder (@wsnyder)
Original Date: 2015-03-18T00:32:03Z


Fixed in git towards 3.413.

The preprocessor had a similar issue, for that returning it line by line is fine.

For the parser I added a use_protected argument, which is 1 for backward compatibility, but vhier and the SigParser don't need protected regions so set use_protected = 0.

@veripoolbot

This comment has been minimized.

Copy link
Collaborator Author

@veripoolbot veripoolbot commented Jun 26, 2015


Original Redmine Comment
Author Name: Wilson Snyder (@wsnyder)
Original Date: 2015-06-26T10:30:21Z


In 3.414.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.