-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Snapshot testing #152
Comments
Hi @czosel I like the idea of make tests based on snapshots, I miss them here over the AST and it could help to increase easily my code coverage. By the way, I've actually done some work in the same direction based on tokens :
Sadly I have not enough time to work on the project but I would be very glad if you could help me with that. |
Hi @ichiriac, thanks for your quick response - I didn't know about the two testing repos yet. I guess that integrating snapshot testing in this repo and basically porting the existing test cases would already be a great first step. I hope I'll find the time soon to give this a shot! |
Hi @czosel, I've tried a simplified version that scans the fixtures folder and builds tests from it. Maybe we should have a separate folder between AST and tokens. Related to #49, we could start by migrate actual tests to fixtures, it's a lot of work to be done. /cc @b4dnewz, @DaGhostman if you have time and want to help, it would be highly apreciated |
Would love to be of help, I'll try my best to find some spare time and help with whatever I can :) |
hi everybody, I looked to this snapshot testing and I think is a wonderful idea! otherwise my proposal is to write the .json output only if the parsing succeed so you don't have to manually remove it after you fix the php file template issue. try {
// parse the file
ast = parser.parseCode(buffer, filename, options);
ast = JSON.stringify(ast, null, 2);
try {
original = fs.readFileSync(filename + ".json").toString();
} catch (e) {
console.log("Generating snapshot: " + filename);
// GENERATE THE OUTPUT (IF FILE DOES NOT EXISTS)
original = ast;
fs.writeFileSync(filename + ".json", original);
}
original.should.be.exactly(ast, "Fail " + filename);
} catch (e) {
console.log("Failed file parsing for:", filename);
console.error(e);
} otherwise if is needed for testing errors it's good as it is. also I see is a very huge work to do and I would like some tips from @ichiriac about where to start and how to deal these tests when there are more than one ast parsing for describe method like this one without loosing the readability of the tests. maybe a recursive fixture.js function that look for subfolders and creates describe blocks. |
ah, ok! I agree |
Hi @b4dnewz, glad to see you 😄 I've wrote this quick (& dirty) without handling errors types in order to draft something. I did not know about Jest but it's a more clean way to generate snapshot. We need to handle parsing errors also, maybe with the I can try to bootstrap something with Jest, and @czosel you could check if it's the right way to use it. If we are good to go, @b4dnewz you could start to migrate the code, it should be easy as the only thing to do is to remove all asserts and replace them with a single match assert. The next hard step where everybody can help is to check the results on the code coverage, find uncovered use cases, and trigger them code. This part is hard as sometimes it's tricky to find a way to create the specific code or reproduce edge cases. |
Hi, I've started a new branch here, and switched all tests over jest : https://github.com/glayzzle/php-parser/tree/v3.1 You can make your PR there :) |
This fix is now relased under 3.0.0-alpha.3 |
Currently, a typical test looks roughly like this:
After working with snapshot-based testing for a while in prettier/plugin-php, I've really come to appreciate not having to write my own assertions anymore.
If we were to introduce snapshot-based testing here, the test case above would essentialy boil down to just one line, which could even be stored in a
.php
file:The testing framework would then generate the snapshot and save it in a file:
Like this, we'd see the exact AST changes that every PR introduces to any of the existing test cases.
It seems that Esprima is using a similar approach.
If you're interested, I could try preparing a little proof of concept - let me know what you think! 😄
The text was updated successfully, but these errors were encountered: