test(integration): differentiate between integration and end-to-end tests #3351
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
"integration" tests can be ambiguous to define, so this PR is a step toward drawing better boundaries between our suite of isolated unit/micro tests and larger scoped tests that integrate more parts of the system.
the tests against
index.js
are larger scoped than the rest of the test files that target specific units of the project. they are more targeted at testing through the public interface and cover more of the full lifecycle, but stop short of doing so in the context of simulating a real git repository host, api, and npm registry. this makes them distinct from both the other unit tests and what i've renamed here to e2e (short for end-to-end) tests.while "integration" is still potentially a bit ambiguous, it at least aligns with what i've found to be a valuable scope in other projects. i like "integration" tests to refer to tests that interact through the public interface and exercise the whole project up to the point where it reaches out to external systems. open to feedback if others have additional thoughts
Note
i'm not confident that it is valuable to have all three of these categories in this project long term.
i could see converting some of the now-integration tests to the e2e category over time. alternatively, maybe more of the scenarios covered in e2e make sense with a smaller scope within the "integration" category and e2e becomes more specifically targeted at protecting specifically against external integration regressions rather than verifying behaviors. this likely also becomes influenced by efforts to eventually extract some of the core behavior further away from the composition with a default plugins config
for now, this is just to better recognize the categories that already exist in the project
Caution
i think it is worth integrating this with a normal merge rather than a squash because it will help git better keep track of some of the file renames that happened as part of this change