New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Future Design of Python Testing Output #21861
Comments
tagging team for their input: @brettcannon @luabud @cwebster-99 @karthiknadig @karrtikr @paulacamargo25 @anthonykim1 @connor4312 |
One item @karrtikr brought up was if the Test Result panel is always around or if it goes away. Users should be able to reference these logs not just right after run so the result panel will need to have the same lifespan as the output channel. |
I say just merge it all into the Test Result panel. It's the integrated solution in VS Code for this sort of information. |
another related issue: #17371 |
If we got rid of the python test output then this issue will be different as all commands to open the output would then route to the test panel: #21694 |
I would like the Test Results view to be the 'one stop shop' for the test experience in VS Code. If there are things that make this prohibitive or undesirable, let me know and we can solve them.
I'm not a pythonista, but one thing I do find useful is test extensions that print the command they're running (or the equivalent command a user could run) for the tests. Sometimes, for whatever reason, I want to run them manually in a terminal.
From my point of view when writing test extensions, my goal for the output is to write human readable data and also not lose information. If there is VS Code API for providing information, such as showing large failure diffs in the TestMessage, then that counts as "not losing information" and I may write a more concise version of that failure into the output. It seems like "python test output" is a mix of the test output and diagnostic data. I think it's good to have a way to get that diagnostic data, but that could be a "log" type output channel and not something we necessarily direct users to unless they run into problems.
Today the "show output" button always opens the output in the Test Results view. Are you thinking that would be different? |
Based on discussions with team members, the plan is to move fully to the Test Results panel (as we have also confirmed this is feasible) and possible remove the This being said there is one question I have. Discovery (for example pytest discovery which is the equivalent of the command @brettcannon, @luabud, @karthiknadig, @cwebster-99 thoughts?
|
We could stick it in the Python channel of the Output panel since, as you pointed out, the Test Results panel doesn't really have a concept of discovery. |
Yea, I'm not a fan of the idea of putting it in the Test Results panel. You could have an output channel, or even a simple memory buffer that's opened in an editor on some command. Up to you. |
@connor4312 and @brettcannon, thanks for the input! Will move forward with keeping discovery in the python output panel thanks! |
Colors are great, they make output much more readable 👍 |
I say we give it a shot and see what users think! |
closes #21861 and related issues --------- Co-authored-by: Courtney Webster <60238438+cwebster-99@users.noreply.github.com>
Ah, amazing, with The “Peek View” still doesn’t have color, but that can be deactivated. Thank you! |
The following issue describes the current behavior and outlines a few questions that need to be figured out to write a proposal.
Current Behavior (pre-rewrite)
Currently, there are three places where users can view different output types from their test run. First is the "python test log" under the output panel, second is the "Test Result" panel, and finally the in-line pop-up for failed tests. The "Test Result" panel and the in-line pop-up are identical in output (when the in-line is displayed since it isn't displayed on pass). Therefore for this discussion, we will just consider the "python test log" and the "Test Result" panel but all things printed to the "Test Result" panel will show up on the in-line pop-up if changes are made.
Due to prior constraints, the extension used to parse output to determine the result of test run and update the UI. This is the primary reason for the two different output views but due to the testing rewrite this is no longer a constraint. This means a decision needs to be made moving forward regarding these two channels and their information.
The current difference between the two are noted below:
comparison on the differences just as text for a test that fails and has a print statement. (right: python test output, left: Test Result)
comparison on the difference just as text for a test that fails with logging enabled. (right: python test output, left: Test Result)
Moving Forward
The following questions arise after the above analysis. Please add your feedback in comments below.
Pros and Cons
The following issues are related to this spike and those involved in these issues are encouraged to give their opinion. Thanks!
Pytest Test Log
with a single click on any test #17371The text was updated successfully, but these errors were encountered: