Can't update functional test refs #9688
Description
Bug description
The message counter is checked first before the output text, so we never have a chance to write the updated output file.
Need to flip these two statements:
pylint/pylint/testutils/lint_module_test.py
Lines 256 to 261 in 86e1943
so this has a chance to run:
pylint/pylint/testutils/functional/lint_module_output_update.py
Lines 27 to 32 in 86e1943
Configuration
No response
Command used
python tests/test_functional.py -k abstract --update-functional-outputPylint output
Failure, depending on if you change the refs
Expected behavior
You can update the refs
Pylint version
pylint 3.3.0-dev0
astroid 3.2.2
Python 3.11.4 (main, Jul 20 2023, 22:46:38) [GCC 9.4.0]OS / Environment
Ubuntu 20.04
Additional dependencies
No response
Metadata
Metadata
Assignees
Labels
Type
Projects
Milestone
Relationships
Participants
Activity
Pierre-Sassoulas commentedon Jun 4, 2024
It's "by design", writing a new output file is useless (or misleading you into thinking it worked) if the functional test does not output the expected value. That said the functional test is still going to fail the line after, and you're only going to have a temporarily better output file while you fix the issue with the functional test. Feel free to open a MR to change the order :)
sshane commentedon Jun 4, 2024
Ah, maybe I'm confused about the relationship of the functional test and the output ref. I assumed that the output text file was a reference we compared the pylint output to, without any ignore tags. And the functional test was to assert certain warnings on certain lines.Nvm that's correct. pylint already outputs "here's what you need to copy/paste" in response to a failed test, so I don't believe fixing this would mislead you any more that it already does.
Pierre-Sassoulas commentedon Jun 4, 2024
Yes, but I'm not sure if I understand the "without any ignore tags" part.
Yes, or the absence of them, if you have unexpected warning the functional test file fail. (And if the functional test fail file updating the output is going to create a wrong output).
sshane commentedon Jun 4, 2024
I just meant running pylint on the whole file, ignoring the warning annotations. Disregard.
Pierre-Sassoulas commentedon Jun 4, 2024
It seems you're already deep in the truth/code so I'm not sure if this part of the doc might help (maybe to get a general idea): https://pylint.readthedocs.io/en/stable/development_guide/contributor_guide/tests/writing_test.html#functional-tests