Skip trying to substitute macros into lines that do not contain them #296
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While investigating #295 I used
cProfile
on the code and noticed that most of the parsing time is spend inregex.subn
in fortls/parse_fortran.py#L2241.In that code part, we try to substitute all macros into all lines by calling
subn
on the compiled regex, which results in an unmodified line if the line did not contain the macro at all. This call tosubn
still seems to be quite expensive (runtime-wise).To spare the cost, I added a check whether the line actually contains the macro text and skip the substitution check otherwise. Please note that a line containing the macro text does not necessarily mean that a substitution will take place, since the simple containment test is less strict than the regex (which checks that the macro is an actual token separated from the surrounding context, as in
FOO
being inFOO_BAR
but it should not be substituted).With these changes I was able to reduce the parsing time of the example described in #295 from ~9.8 seconds to ~1.4 seconds, which comes far closer to the time it takes without specifying any pre-processor definitions (~1 second).
In a real world example I could reduce the time to parse a single file from ~107 seconds to ~7seconds.
The parsing time of our whole code-base was reduced from more than 1.5 hours to less than 7 minutes.
NOTE: If I exchange the simple containment test via
by checking the actual regex via
a few lines further down, this still reduces the time to parse the real world file from ~107 seconds to ~26 seconds.