Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests: Exhaustive pattern tests #2688

Merged

Conversation

RunDevelopment
Copy link
Member

This improves the static pattern test to check every regex.

Method

This is achieved by running all test files for a given language and overwriting the Prism.tokenize function itself. The tokenize function will be replaced with a function that runs all checks on the grammar given to the function and then uses the original tokenize function. Assuming that our test files cover (nearly) all aspects of our languages, this approach will analyze all regexes used in grammar.

This runtime approach is necessary because some languages (e.g. PHP and Latte) use markup templating (MT) which changes the grammar with one that might not be reachable from Prism.languages.

However, even this isn't enough. MT also requires a regex to detect the embedded language (example). This pattern isn't passed to Prism.tokenize.
To analyze these regexes for super-linear runtime, the prototype of the RegExp type itself is changed to replace the exec and test methods with methods that run the analysis for every regex that is used throughout the tests.

Problem

The only problem with this approach is that it is slow.

To statically analyze all regexes, I need about 2s, and to run all test files, I need about another 2s on my machine. However, the exhaustive pattern test takes 9s on my machine.

This is because of the RegExp prototype magic that forces us to take the slow path. Without the prototype magic, the whole thing only takes 5s on my machine.

@github-actions
Copy link

github-actions bot commented Jan 2, 2021

JS File Size Changes (gzipped)

A total of 1 files have changed, with a combined diff of -1 B (-0.4%).

file master pull size diff % diff
components/prism-erb.min.js 267 B 266 B -1 B -0.4%

Generated by 🚫 dangerJS against 9f02440

@RunDevelopment
Copy link
Member Author

I will merge this. It really doesn't matter that the tests take longer now.

@RunDevelopment RunDevelopment merged commit 5315140 into PrismJS:master May 1, 2021
9 checks passed
@RunDevelopment RunDevelopment deleted the exhaustive-pattern-tests branch May 1, 2021 16:01
This was referenced Jun 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant