-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Spec reporting shows different line numbers in results #5633
Comments
This is a feature: describe "Parser" do
it_parses "nil", nil
it_parses "true", true.bool
it_parses "false", false.bool
end Imagine one of those fail. The important part where I want to look is the line that says This is the main reason |
Basically:
By the way, this is how it works in RSpec, minus the location forwarding, which is quite annoying in RSpec (you have to define helpers in a specific file and then mute those methods with a regex or something like that). |
@asterite That's what this is about: I would like to see the line number where Assuming your is somewhere in the middle of a larger spec file and
The location passed as
When I'm debugging these failures I don't really need to know the line of the failing expectation, I need to know where In general, I find it very un-intuitive that the result shows different line numbers in detail and summary. It's not self-explanatory what they mean. |
Well, I (and probably others) need to know the line of the failing expectation: require "spec"
private def it_does_something(x, y)
it "does something with #{x} and #{y}" do
(x * y).should eq(12)
(x + y).should eq(8)
end
end
describe "foo" do
it_does_something 2, 6
it_does_something 3, 4
end Now you get:
Super clear:
You got all the info you need. Repeating the number 4 instead of saying 6 gives you less information to debug. You really want to know that the assertion that failed was the |
The information is clear once you know how to interpret it but I agree that it's not self-explanitory UX. I'd suggest something like
That's not nearly perfect but you get the idea. I've frequently wanted the |
@RX14 What you said last is a really good point. I also wanted this but didn't realize it, or just accepted things as they are. Feel free to tweak the output format and send a PR :-) |
Assuming you're planning on submitting a PR, can I suggest:
or
It gets the same amount of information across with more clarity, in a smaller space. |
Looks like you're trying to be more expressive... but I don't think that helps to improve readability. I'd keep changes from the current output format at a minimum and go for something like @RX14 suggested. By the way, I've been trying to setup some specs for the spec output. They're missing entirely. |
@RX14, @straight-shoota What's your feeling on:
Changes are fairly minimal but more explicit, and it means you get both line numbers in both outputs? Also, I think I deleted my other comment as you posted, so I'm not sure which bit of it you're referring to. FYI, I'm happy to make it look like whatever :) |
Taking this code as a reference. In the above examples, @asterite and @RX14 referred to the same code, I've just added the require "spec"
private def it_does_something(x, y, file = __FILE__, line = __LINE__)
it "does something with #{x} and #{y}", file, line do
(x * y).should eq(12)
(x + y).should eq(8)
end
end
describe "foo" do
it_does_something 2, 6
it_does_something 3, 4
end I would just add the following minimal changes: .F
Failures:
1) foo does something with 3 and 4
+ at spec/foo_spec.cr:12
+ Failure/Error: (x + y).should eq(8)
Expected: 8
got: 7
# spec/foo_spec.cr:6
Finished in 83 microseconds
2 examples, 1 failures, 0 errors, 0 pending
Failed examples:
crystal spec spec/foo_spec.cr:12 # foo does something with 3 and 4 |
And an example which directly defines an expectation: require "spec"
describe "foo" do
it "does something with 3 and 4" do
(3 + 4).should eq(8)
end
end F
Failures:
1) foo does something with 3 and 4
Expected: 8
got: 7
# spec/foo_spec.cr:5
Finished in 88 microseconds
1 examples, 1 failures, 0 errors, 0 pending
Failed examples:
-crystal spec spec/foo_spec.cr:4 # foo does something with 3 and 4
+crystal spec spec/foo_spec.cr:5 # foo does something with 3 and 4 Maybe we could add |
Personally, I would like I'll see if I can put something together today. |
The result from this spec looks like this:
The line number is
5
in the detailed result and9
in the summary. This is because the detailed view shows the location where theAssertionFailed
was raised while the summary shows the location given toit
usingfile
andline
variables.I'd suggest that both should show the same line number and it should be the location provided to
it
(my_it
=>9
). If you're using such spec helpers (and explicitly set the location of the caller), you're usually interested in the helper call that caused it, not the location in the helper method definition.We should however make sure that it doesn't affect other cases: When an
it
block directly contains several individual expectations, the line number should certainly point the location where the expectation was raised.Take this example spec:
It currently prints this result:
Line
5
is actually the important line. I'd suggest that the summary should also point to this line. I guess pointing to the beginning of the example is actually just a leftover from when spec could only target the beginning of the block for selected execution.In essence, if
it
is invoked with explicitfile
andline
arguments, those should be used as location of the failing spec in both detailed result and summary. Otherwise it should implicitly use the location of the failing expectation (or exception).The text was updated successfully, but these errors were encountered: