Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Add guidelines for e2e tests #1466

Merged
merged 26 commits into from
Dec 12, 2023
Merged

docs: Add guidelines for e2e tests #1466

merged 26 commits into from
Dec 12, 2023

Conversation

p-offtermatt
Copy link
Contributor

@p-offtermatt p-offtermatt commented Dec 4, 2023

Description

Closes: #1334

This adds guidelines on how to write e2e tests to tests/e2e/README.md

TODO: add this README to the .md link checker, potentially?


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct docs: prefix in the PR title
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • reviewed "Files changed" and left comments if necessary
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • Confirmed the correct docs: prefix in the PR title
  • Confirmed all author checklist items have been addressed
  • Confirmed that this PR only changes documentation
  • Reviewed content for consistency
  • Reviewed content for spelling and grammar
  • Tested instructions (if applicable)
  • Checked that the documentation website can be built and deployed successfully (run make build-docs)

@github-actions github-actions bot added C:Testing Assigned automatically by the PR labeler C:Build Assigned automatically by the PR labeler C:CI Assigned automatically by the PR labeler labels Dec 4, 2023
@github-actions github-actions bot removed C:Build Assigned automatically by the PR labeler C:CI Assigned automatically by the PR labeler labels Dec 4, 2023
@p-offtermatt p-offtermatt changed the title Add guidelines for e2e tests docs: Add guidelines for e2e tests Dec 4, 2023
@p-offtermatt p-offtermatt marked this pull request as ready for review December 4, 2023 11:55
@p-offtermatt p-offtermatt requested a review from a team as a code owner December 4, 2023 11:55
@p-offtermatt p-offtermatt linked an issue Dec 4, 2023 that may be closed by this pull request
Copy link
Contributor

@mpoke mpoke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @p-offtermatt. See my comments below.

tests/e2e/README.md Outdated Show resolved Hide resolved
tests/e2e/README.md Outdated Show resolved Hide resolved
tests/e2e/README.md Outdated Show resolved Hide resolved
tests/e2e/README.md Outdated Show resolved Hide resolved
tests/e2e/README.md Outdated Show resolved Hide resolved
tests/e2e/README.md Show resolved Hide resolved
tests/e2e/README.md Outdated Show resolved Hide resolved
tests/e2e/README.md Outdated Show resolved Hide resolved
tests/e2e/README.md Outdated Show resolved Hide resolved
tests/e2e/README.md Show resolved Hide resolved
p-offtermatt and others added 15 commits December 6, 2023 12:57
Co-authored-by: Marius Poke <marius.poke@posteo.de>
Co-authored-by: Marius Poke <marius.poke@posteo.de>
Co-authored-by: Marius Poke <marius.poke@posteo.de>
Co-authored-by: Marius Poke <marius.poke@posteo.de>
Co-authored-by: Marius Poke <marius.poke@posteo.de>
Co-authored-by: Marius Poke <marius.poke@posteo.de>
Co-authored-by: Marius Poke <marius.poke@posteo.de>
Co-authored-by: Marius Poke <marius.poke@posteo.de>
@@ -1,4 +1,188 @@
## Updating the trace format and tests when adjusting the framework
# End-to-end testing

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be nice to have a tl;dr or cheat sheet here, about running different kinds of tests & traces with different config.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think go run ./tests/e2e/... --help explains it properly?
I added a note to run that. If you think it's not helpful and there should be more here,
please lmk, otherwise just resolve this :)

Copy link
Contributor

@mmulji-ic mmulji-ic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bar the tl;dr additional, good overall.

@github-actions github-actions bot added the C:CI Assigned automatically by the PR labeler label Dec 8, 2023
tests/e2e/README.md Outdated Show resolved Hide resolved
tests/e2e/README.md Outdated Show resolved Hide resolved
tests/e2e/README.md Outdated Show resolved Hide resolved
// potentially check something in the output, or log something, ...
```

Don't forget to wire your action into [main.go](main.go):runStep, where
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Why not point direclty to the line inside main.go that containsrunStep?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dislike it because it will not point to the right place anymore extremely quickly, i.e. as soon as the file is touched and this is not adjusted.
I'd be happy about suggestions on how to overcome this! I know anchors exist, but I think they make the source file a bit verbose.

tests/e2e/README.md Outdated Show resolved Hide resolved
Copy link
Contributor

@insumity insumity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

tests/e2e/README.md Outdated Show resolved Hide resolved
@sainoe sainoe self-requested a review December 12, 2023 07:46
Copy link
Contributor

@sainoe sainoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Great doc!

Co-authored-by: Simon Noetzlin <simon.ntz@gmail.com>
@p-offtermatt
Copy link
Contributor Author

Thanks everyone for the reviews! 🙌 I think everyone had a chance to review, so I will merge this :)

@p-offtermatt p-offtermatt added this pull request to the merge queue Dec 12, 2023
Merged via the queue into main with commit d8c2cb3 Dec 12, 2023
13 checks passed
@p-offtermatt p-offtermatt deleted the ph/e2e-tutorial branch December 12, 2023 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C:CI Assigned automatically by the PR labeler C:Testing Assigned automatically by the PR labeler
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create guidelines on how to use the e2e testing framework feature: add a way to start a local testnet
8 participants