Add testdox format #359

bitfield · 2023-08-13T14:34:08Z

This format shows test names as full sentences, which is useful both for autogenerating documentation ("dox"), and for encouraging programmers to write their test names as sentences describing the behaviour of the unit under test.

The specific method of formatting test names as sentences is that used by https://github.com/bitfield/gotestdox.

dnephin

This is awesome, thank you for building gotestdox and adding this format!

I pushed a commit to use the same icons we use in other formats. There's a --format-hivis option to change the icons to more colorful ones. In a future version we might look at making those the default.

Left a couple suggestions/comments for small changes, but overall this looks great and I would like to add this format.

cmd/main.go

testjson/format.go

dnephin · 2023-08-20T19:06:31Z

testjson/format.go

+			}
+			fmt.Fprintln(buf)
+			return buf.Flush()
+		case event.Action.IsTerminal():


I changed this to IsTerminal so that skipped tests show up in the output as well. Does that sound ok to you?

Again, it makes sense to show skipped tests in a test runner, but a documentation generator takes the opposite view. Since these tests are skipped, they don't by definition assert anything about the behaviour of the code under test.

dnephin · 2023-08-20T19:09:48Z

testjson/format.go

+			if !event.Action.IsTerminal() {
+				return nil
+			}
+			if opts.HideEmptyPackages && len(results[event.Package]) == 0 {


I added opts.HideEmptyPackages here to match the behaviour of other formats. Print the empty package by default, but --format-hide-empty-pkg can be set to hide these (for projects that have lots of empty packages).

bitfield · 2023-08-21T10:10:31Z

I pushed a commit to use the same icons we use in other formats. There's a --format-hivis option to change the icons to more colorful ones. In a future version we might look at making those the default.

I leave that up to you, of course. But you might be interested to know why gotestdox uses the emojis it does. One of my first users was a blind developer who uses a screen reader, and he reported that the ✘ emoji I was using is pronounced by the screen reader as something like "heavy ballot x", which is weird (gotestsum's is pronounced "heavy multiplication x", which is even weirder). Changing it to simply x made the output a lot more accessible for him.

dnephin · 2023-08-26T04:17:58Z

Ah, I had not considered screen readers but that's a great point! I'd love to provide some better support. Maybe --format-audible or something like that could change the x to "failed test", etc? The pattern here for switching between the original icons and the new hivis icons seems like it would make it easy to switch to other options as well.

Thank you for explaining the design decisions in gotestdox!

This format shows test names as full sentences, which is useful both for autogenerating documentation ("dox"), and for encouraging programmers to write their test names as sentences describing the behaviour of the unit under test. The specific method of formatting test names as sentences is that used by https://github.com/bitfield/gotestdox.

And only hide Fuzz test cases, always report the top level test function.

afbjorklund · 2023-08-30T20:30:46Z

I added --format-icons text, which outputs "PASS" for ✓, "SKIP" for ∅ and "FAIL" for ✖

#366 (comment)

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [github.com/cerbos/cerbos/api/genpb](https://togithub.com/cerbos/cerbos) | require | digest | `0d63f1e` -> `341baa4` | | [github.com/grpc-ecosystem/go-grpc-middleware/v2](https://togithub.com/grpc-ecosystem/go-grpc-middleware) | require | patch | `v2.0.0` -> `v2.0.1` | | [github.com/lestrrat-go/jwx/v2](https://togithub.com/lestrrat-go/jwx) | require | patch | `v2.0.12` -> `v2.0.13` | | [google.golang.org/grpc](https://togithub.com/grpc/grpc-go) | require | patch | `v1.58.0` -> `v1.58.2` | | [gotest.tools/gotestsum](https://togithub.com/gotestyourself/gotestsum) | require | minor | `v1.10.1` -> `v1.11.0` | --- ### Release Notes <details> <summary>grpc-ecosystem/go-grpc-middleware (github.com/grpc-ecosystem/go-grpc-middleware/v2)</summary> ### [`v2.0.1`](https://togithub.com/grpc-ecosystem/go-grpc-middleware/releases/tag/v2.0.1) [Compare Source](https://togithub.com/grpc-ecosystem/go-grpc-middleware/compare/v2.0.0...v2.0.1) #### What's Changed - Fix outdated 'make proto' command by [@takp](https://togithub.com/takp) in [grpc-ecosystem/go-grpc-middleware#623 - Fix linting errors by [@takp](https://togithub.com/takp) in [grpc-ecosystem/go-grpc-middleware#624 - Logging: Add missing variadic operator for fields by [@olivierlemasle](https://togithub.com/olivierlemasle) in [grpc-ecosystem/go-grpc-middleware#629 - feat: Support extracting fields from CallMeta by [@fsaintjacques](https://togithub.com/fsaintjacques) in [grpc-ecosystem/go-grpc-middleware#628 - Fix "make test" and "make lint" by [@olivierlemasle](https://togithub.com/olivierlemasle) in [grpc-ecosystem/go-grpc-middleware#627 - Do not set timeout for stream initialization by [@DavyJohnes](https://togithub.com/DavyJohnes) in [grpc-ecosystem/go-grpc-middleware#645 - Add logging option to disable fields in log entry by [@coleenquadros](https://togithub.com/coleenquadros) in [grpc-ecosystem/go-grpc-middleware#631 - Update logging adapter docs by [@aboryslawski](https://togithub.com/aboryslawski) in [grpc-ecosystem/go-grpc-middleware#647 #### New Contributors - [@takp](https://togithub.com/takp) made their first contribution in [grpc-ecosystem/go-grpc-middleware#623 - [@olivierlemasle](https://togithub.com/olivierlemasle) made their first contribution in [grpc-ecosystem/go-grpc-middleware#629 - [@fsaintjacques](https://togithub.com/fsaintjacques) made their first contribution in [grpc-ecosystem/go-grpc-middleware#628 - [@DavyJohnes](https://togithub.com/DavyJohnes) made their first contribution in [grpc-ecosystem/go-grpc-middleware#645 - [@coleenquadros](https://togithub.com/coleenquadros) made their first contribution in [grpc-ecosystem/go-grpc-middleware#631 - [@aboryslawski](https://togithub.com/aboryslawski) made their first contribution in [grpc-ecosystem/go-grpc-middleware#647 **Full Changelog**: grpc-ecosystem/go-grpc-middleware@v2.0.0...v2.0.1 </details> <details> <summary>lestrrat-go/jwx (github.com/lestrrat-go/jwx/v2)</summary> ### [`v2.0.13`](https://togithub.com/lestrrat-go/jwx/releases/tag/v2.0.13) [Compare Source](https://togithub.com/lestrrat-go/jwx/compare/v2.0.12...v2.0.13) v2.0.13 26 Sep 2023 [New Features] * [jwk] jwk.Equal has been added. Please note that this is equivalent to comparing the keys' thumbprints, therefore it does NOT take in consideration non-essential fields. [Miscellaneous] * Various documentation fixes and additions. </details> <details> <summary>grpc/grpc-go (google.golang.org/grpc)</summary> ### [`v1.58.2`](https://togithub.com/grpc/grpc-go/releases/tag/v1.58.2): Release 1.58.2 [Compare Source](https://togithub.com/grpc/grpc-go/compare/v1.58.1...v1.58.2) ### Bug Fixes - balancer/weighted_round_robin: fix ticker leak on update A new ticker is created every time there is an update of addresses or configuration, but was not properly stopped. This change stops the ticker when it is no longer needed. ### [`v1.58.1`](https://togithub.com/grpc/grpc-go/releases/tag/v1.58.1): Release 1.58.1 [Compare Source](https://togithub.com/grpc/grpc-go/compare/v1.58.0...v1.58.1) ##### Bug Fixes - grpc: fix a bug that was decrementing active RPC count too early for streaming RPCs; leading to channel moving to IDLE even though it had open streams - grpc: fix a bug where transports were not being closed upon channel entering IDLE </details> <details> <summary>gotestyourself/gotestsum (gotest.tools/gotestsum)</summary> ### [`v1.11.0`](https://togithub.com/gotestyourself/gotestsum/releases/tag/v1.11.0) [Compare Source](https://togithub.com/gotestyourself/gotestsum/compare/v1.10.1...v1.11.0) #### What's Changed - handle multiple levels in subtest regex generation by [@brycekahle](https://togithub.com/brycekahle) in [gotestyourself/gotestsum#342 - rerun-fails: prevent extra runs when a sub-test fails by [@brycekahle](https://togithub.com/brycekahle) in [gotestyourself/gotestsum#343 - Add note about binary test result caching by [@nfi-hashicorp](https://togithub.com/nfi-hashicorp) in [gotestyourself/gotestsum#350 - dots-v2: hide cursor during output by [@howardjohn](https://togithub.com/howardjohn) in [gotestyourself/gotestsum#354 - Export elapsed time to the post-run environment by [@afbjorklund](https://togithub.com/afbjorklund) in [gotestyourself/gotestsum#360 - Include test.timeout stack trace in the junit.xml by [@dnephin](https://togithub.com/dnephin) in [gotestyourself/gotestsum#357 - Fix dots formats with stderr by [@dnephin](https://togithub.com/dnephin) in [gotestyourself/gotestsum#358 - Rename TESTS_ELAPSED environment variable by [@afbjorklund](https://togithub.com/afbjorklund) in [gotestyourself/gotestsum#361 - GitHub actions format by [@dnephin](https://togithub.com/dnephin) in [gotestyourself/gotestsum#315 - Add the time unit to elapsed time duration by [@afbjorklund](https://togithub.com/afbjorklund) in [gotestyourself/gotestsum#363 - Add testdox format by [@bitfield](https://togithub.com/bitfield) in [gotestyourself/gotestsum#359 - Add post-run-command notify implementation for linux by [@afbjorklund](https://togithub.com/afbjorklund) in [gotestyourself/gotestsum#362 - dotsv2: remove jitters entirely by [@howardjohn](https://togithub.com/howardjohn) in [gotestyourself/gotestsum#368 #### New Contributors - [@brycekahle](https://togithub.com/brycekahle) made their first contribution in [gotestyourself/gotestsum#342 - [@nfi-hashicorp](https://togithub.com/nfi-hashicorp) made their first contribution in [gotestyourself/gotestsum#350 - [@howardjohn](https://togithub.com/howardjohn) made their first contribution in [gotestyourself/gotestsum#354 - [@afbjorklund](https://togithub.com/afbjorklund) made their first contribution in [gotestyourself/gotestsum#360 - [@bitfield](https://togithub.com/bitfield) made their first contribution in [gotestyourself/gotestsum#359 **Full Changelog**: gotestyourself/gotestsum@v1.10.1...v1.11.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "before 4am on Monday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/cerbos/cerbos-sdk-go).  Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Signed-off-by: Oğuzhan Durgun <oguzhandurgun95@gmail.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

bitfield force-pushed the testdox branch from 33a6fa8 to 9073966 Compare August 13, 2023 14:40

dnephin reviewed Aug 20, 2023

View reviewed changes

cmd/main.go Outdated Show resolved Hide resolved

dnephin reviewed Aug 20, 2023

View reviewed changes

bitfield and others added 2 commits August 27, 2023 17:32

Use the same icons as other gotestsum formats

61b66b7

dnephin force-pushed the testdox branch from 73dfa81 to c768413 Compare August 27, 2023 21:44

Keep empty packages by default in testdox format

18c1f57

And only hide Fuzz test cases, always report the top level test function.

dnephin force-pushed the testdox branch 2 times, most recently from 0466efc to 18c1f57 Compare August 27, 2023 21:50

dnephin approved these changes Aug 27, 2023

View reviewed changes

dnephin merged commit 2098fe3 into gotestyourself:main Aug 27, 2023
8 checks passed

dnephin mentioned this pull request Aug 27, 2023

Add format-icons flag for nerdfonts instead of unicode #366

Merged

Jumziey mentioned this pull request Sep 2, 2023

Added a verbose flag which makes gotestdox print the output of tests. bitfield/gotestdox#13

Closed

chenrui333 mentioned this pull request Sep 17, 2023

gotestsum 1.11.0 Homebrew/homebrew-core#143293

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add testdox format #359

Add testdox format #359

bitfield commented Aug 13, 2023

dnephin left a comment •

edited

dnephin Aug 20, 2023

bitfield Aug 21, 2023

dnephin Aug 20, 2023

bitfield commented Aug 21, 2023

dnephin commented Aug 26, 2023

afbjorklund commented Aug 30, 2023 •

edited

Add testdox format #359

Add testdox format #359

Conversation

bitfield commented Aug 13, 2023

dnephin left a comment • edited

Choose a reason for hiding this comment

dnephin Aug 20, 2023

Choose a reason for hiding this comment

bitfield Aug 21, 2023

Choose a reason for hiding this comment

dnephin Aug 20, 2023

Choose a reason for hiding this comment

bitfield commented Aug 21, 2023

dnephin commented Aug 26, 2023

afbjorklund commented Aug 30, 2023 • edited

dnephin left a comment •

edited

afbjorklund commented Aug 30, 2023 •

edited