Allow multiple cases per fixture file#5149
Open
Pedro-Muller29 wants to merge 8 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4506.
Description
In #4506, MeGaGiGaGon flagged that the longest fixture files in
tests/data/cases/are painful to work with.preview_long_strings.pyandpreview_long_strings__regression.pyare both 1000+ lines: input and output for any given case sit hundreds of lines apart, and when a test fails the assertion points at the end-of-file mismatch with nothing tying it back to the original case.MeGaGiGaGon's first proposal was to split big files into folders of numbered files. JelleZijlstra pushed back on numbering ("renumbering files would be needlessly annoying") and suggested an alternative shape: sections within larger files, like mypy's test data format.
This PR takes the mypy-style path. Fixture files can group multiple cases under
[case <name>]headers, and the two longest files are migrated as the first consumers.The format is opt-in. Files without a
[case ]header at the top run through the legacy single-case loader unchanged. Cell names appear in pytest IDs (test_simple_format[preview_long_strings::assign_x_1]), and on failure the assertion message attaches the file path, the cell header line, and the# outputmarker line, so the traceback points at the case under test rather than the file boundary.A
# flags:line above the first[case ]applies to every cell, and cell flags take precedence on overlap. That makes preview-to-stable migrations more granular: cells can graduate one at a time, while neighbors in the same file keep their preview flags.Documentation in
docs/contributing/the_basics.mdcovers the format and the merge semantics.I picked
preview_long_strings*for the first migrations because they're the worst offenders on file length.Checklist - did you ...
--previewstyle, following the stability policy?CHANGES.mdif necessary?