chore(ci): Use TRACE logging for tests run by nextest #1902
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The default log-level being set to INFO makes it really difficult to debug flaky tests on CI. This is a regression from earlier where we had TRACE-level logging for tests using iroh_test::logging::setup(), which was only possible for single-threaded executor tests. Now that we are running with cargo-nextest per-test output capturing also works seamlessly for multi-threaded runners because of the cargo-nextest execution model. This means we can set a detailed log level again by default, thus giving us detailed logs on any failure without having to reproduce it.
Arqu
reviewed
Dec 19, 2023
Arqu
approved these changes
Dec 19, 2023
fubuloubu
pushed a commit
to ApeWorX/iroh
that referenced
this pull request
Feb 21, 2024
## Description The default log-level being set to INFO makes it really difficult to debug flaky tests on CI. This is a regression from earlier where we had TRACE-level logging for tests using iroh_test::logging::setup(), which was only possible for single-threaded executor tests. Now that we are running with cargo-nextest per-test output capturing also works seamlessly for multi-threaded runners because of the cargo-nextest execution model. This means we can set a detailed log level again by default, thus giving us detailed logs on any failure without having to reproduce it. ## Notes & open questions As can be seen in the doctests changes, where we can not use nextest, we could also do this in some other ways. Given we now depend on cargo-nextest I guess we can as well go for always setting TRACE level as the CI configuration stays much simpler. We could also take this much further and simplify the machinenry of iroh-test::logging to not jump through all those hoops. Though I guess people might still run the test suite with `cargo-test` which would be nicer if we leave it in. ## Change checklist - [x] Self-review. - ~~Documentation updates if relevant.~~ - ~~Tests if relevant.~~
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
The default log-level being set to INFO makes it really difficult
to debug flaky tests on CI. This is a regression from earlier where
we had TRACE-level logging for tests using
iroh_test::logging::setup(), which was only possible for
single-threaded executor tests.
Now that we are running with cargo-nextest per-test output capturing
also works seamlessly for multi-threaded runners because of the
cargo-nextest execution model. This means we can set a detailed log
level again by default, thus giving us detailed logs on any failure
without having to reproduce it.
Notes & open questions
As can be seen in the doctests changes, where we can not use nextest,
we could also do this in some other ways. Given we now depend on
cargo-nextest I guess we can as well go for always setting TRACE level
as the CI configuration stays much simpler.
We could also take this much further and simplify the machinenry of
iroh-test::logging to not jump through all those hoops. Though I guess
people might still run the test suite with
cargo-testwhich wouldbe nicer if we leave it in.
Change checklist
Documentation updates if relevant.Tests if relevant.