Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Line number qualifier to test.suite.path #2202

Open
humblerookie opened this issue Apr 1, 2024 · 6 comments
Open

Add Line number qualifier to test.suite.path #2202

humblerookie opened this issue Apr 1, 2024 · 6 comments

Comments

@humblerookie
Copy link

humblerookie commented Apr 1, 2024

Currently the reporter (expanded or github) spits up only the path of the file under test rather than the full qualifier i.e.

<Path-to-file>:<line>:<column>

This would enable editors like intellij to autolink it to the exact line that failed instead of having to manually scroll through the file.

Current Behaviour

00:09 +117: /Users/abc/demo/test/blocs/auth_onboarding_bloc_test.dart: verify when notification permission is permanently denied it displays toast           
00:11 +136: Users/abc/demo/test/blocs/tickets_bloc_test.dart: verify on successful logIn  we send analytics            

Expected Behaviour


00:09 +117: /Users/abc/demo/test/blocs/auth_onboarding_bloc_test.dart:130:5 verify when notification permission is permanently denied it displays toast           
00:11 +136: Users/abc/demo/test/blocs/tickets_bloc_test.dart:200:5 verify on successful logIn  we send analytics           
@jakemac53
Copy link
Contributor

We should be able to do that, given we support running tests by line/col.

Note that the logic isn't always perfect for which line/col we would report, in particular if you are wrapping your calls to test in another function. I think the current heuristic is to report the first line in the stack trace that appears in the main test file, which is usually pretty good.

@jakemac53
Copy link
Contributor

cc @natebosch thoughts on how we would roll this out?

@natebosch
Copy link
Member

I think we also don't collect the file location unless we're using the JSON reporter or some other feature which uses it. Adding to the expanded or github reporters will add some amount of overhead to collect the file locations when we otherwise would have skipped it. I don't think that should be much of a problem.

I think rolling it out to the GitHub reporter is unlikely to break anyone, and should be safe to do.

There is more risk in changing the expanded reporter format, but given that we've had a machine readable format already available I think I'd be OK risking breaking projects that are parsing the expanded reporter output.

This would enable editors like intellij to autolink it to the exact line that failed instead of having to manually scroll through the file.

Slight nitpick - this will let us find the start of the test that failed, not the specific line within the test that caused the test to fail.

I think overall this would be a fine change to make. I think we've also had a request to add more fine-grained timing information to the expanded reporter.

@jakemac53
Copy link
Contributor

Yeah, the expanded/compact reporter formats while used by some tools are not supported for that use, so I think we should feel free to change it.

Slight nitpick - this will let us find the start of the test that failed, not the specific line within the test that caused the test to fail.

We could possibly show the exact line for any failed expectation too?

@natebosch
Copy link
Member

We could possibly show the exact line for any failed expectation too?

I think we make an attempt at that in the stack trace.

I think it's currently hardcoded to support expect and matchers in stack traces, but we can add support for plugging in to stack trace hiding when we get around to checks again. I think the stack trace is a good place to have the line number for the specific failure.

@jakemac53
Copy link
Contributor

I think the stack trace is a good place to have the line number for the specific failure.

Yeah true, so what value is this providing then really? It does let you jump right to any test (failed or not), but non-failures aren't as interesting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants