2.11.15
Features
Displaying task attempt logs
When running a task with the @retry decorator, previously we were able to only view the logs of the latest attempt of the task.
With this release it is now possible to target a specific attempt with the --attempt option for the logs command
python example.py logs 123/retry_step/456 --attempt 2
Scrubbing log contents
This release introduces a new command for scrubbing log contents of tasks in case they contain sensitive information that needs to be redacted.
Simplest use case is scrubbing the latest task logs. By default both stdout and stderr are scrubbed
python example.py logs scrub 123/example/456
There are also options to target only a specific log stream
python example.py logs scrub 123/example/456 --stderr
python example.py logs scrub 123/example/456 --stdout
when using the@retry decorator, tasks can have multiple attempts with separate logs that require scrubbing. By default only the latest attempt is scrubbed. There are options to make scrubbing multiple attempts easier
# scrub specific attempt
python example.py logs scrub 123/retry_step/456 --attempt 1
# scrub all attempts
python example.py logs scrub 123/retry_step/456 --all
# scrub specified attempt and all prior to it (this would scrub attempts 0,1,2,3)
python example.py logs scrub 123/retry_step/456 --all --attempt 3
The command also accepts only specifying a step for scrubbing. This is useful for steps with multiple tasks, like a foreach split.
python example.py logs scrub 123/foreach_step
all the above options also apply when targeting a step for scrubbing.
Note: Log scrubbing for running tasks is not recommended, and is actively protected against. There can be occasions where a task has failed in such a way that it still counts as not completed. For such a case you can supply the --include-not-done option to try and scrub it as well.
What's Changed
- feature: scrub logs by @saikonen in #1802
- Fix issue #1805 by @narayanacharya6 in #1807
- bug fix: unused arg for batch step --help by @mae5357 in #1817
- bump version to 2.11.15 by @saikonen in #1829
New Contributors
- @narayanacharya6 made their first contribution in #1807
- @mae5357 made their first contribution in #1817
Full Changelog: 2.11.14...2.11.15
