Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nice looking detailed reporter #529

Closed
gaborcsardi opened this Issue Sep 28, 2016 · 5 comments

Comments

Projects
None yet
2 participants
@gaborcsardi
Copy link
Member

gaborcsardi commented Sep 28, 2016

AFAICT there is no such reporter currently. And I agree that you don't want to see this much output, usually, but e.g. personally I would be happy with more details on the CI. I will only look at the output when there is an issue, anyway, and then it would be great to see the problem immediately.

Something like this:

# mypackage

## context1 ────────────────────────────────────────────────

✔ [4/4] first_test
✔ [5/5] second_test
✖ [1/2] third_test
Error: "foo" not equal to 1.
Types not compatible: character vs double

## context2

...

## Summary

── 34 passed ✔ | 1 failed ✖ | 1 error ✖ | 1 warning ✖ ─────
@gaborcsardi

This comment has been minimized.

Copy link
Member Author

gaborcsardi commented Oct 20, 2016

Here is an even better looking example for motivation:

@hadley

This comment has been minimized.

Copy link
Member

hadley commented Dec 15, 2016

Looks great. I'd love to review a PR :)

@gaborcsardi

This comment has been minimized.

Copy link
Member Author

gaborcsardi commented Dec 15, 2016

Haha. I actually already started writing it last week, in another package. It is unfortunately impossible to implement reporters in other packages, because I need to call testthat internals. What do you think about that? Should testthat provide a better API for reporter writers, or you would like to keep reporters in testthat?

As for this particular reporter, I think it can be in testthat, or separate, I really don't mind. It'll have some additional dependencies, but all lightweight, e.g. my prettycode package to syntax highlight the code, and/or tracer for nicer stack traces.

I am also looking at https://github.com/brodieG/diffobj#readme to provide better looking reports about object differences.

Anyway, I don't think this will be ready for the next testthat release...

@hadley

This comment has been minimized.

Copy link
Member

hadley commented Dec 15, 2016

Let me know what you need exported and I can do it. I'd rather have new reporters in other packages so I'm not the bottleneck.

@hadley

This comment has been minimized.

Copy link
Member

hadley commented Oct 3, 2017

Concrete idea after iterating in slack:

   OK | F | E | W | S | Context 
✅ 20 |   |   |   |   | compare.character  [10 s]
❌ 12 | 1 |   |   |   | compare.numeric [33 s]
-----------------------------------------------------------
Failure @test-compare-numeric.R#6: numeric types are compatible
compare(1, 2L)$equal isn't true.
-----------------------------------------------------------
/  5 |   |   |   |   | compare.time [15 s]
  • Status/spinner on left
  • Just show counts (updating as the expectations complete)
  • Show errors after each context
  • Time each context
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.