Add script to print GitHub workflow run errors #1761
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This script allows users to specify a workflow name and branch to find the latest run. It then identifies any failed jobs within that run and prints the last N lines of logs for each failed step. Key features: - Fetches the most recent workflow run for a given workflow and branch. - Identifies jobs within the run that have a 'failure' conclusion. - For each failed job, attempts to identify failed steps and extracts their logs. - Prints the last 500 lines (configurable) of the log for each failed step. - Handles various scenarios including successful runs, running workflows, and missing data. - Supports GitHub token via CLI arg, env var, or ~/.github_token. - Auto-detects repository from git remote, or accepts via CLI args.
This commit updates the `print_workflow_run_errors.py` script:
- Workflow name and branch are now optional arguments:
- `--workflow` (or `--workflow-name`) defaults to "integration_test.yml".
- `--branch` defaults to the current Git branch.
- Changed default log lines printed from 500 to 100 (`--log-lines`).
- Added `--all-failed-steps` flag:
- If false (default), only logs for the first failed step in a job are printed.
- If true, logs for all failed steps in a job are printed.
These changes provide more flexibility and sensible defaults for common use cases.
This commit introduces a grep-like feature to the `print_workflow_run_errors.py` script. New features: - Added `--grep-pattern` (`-g`) argument to specify an Extended Regular Expression (ERE) for searching within fetched logs. - Added `--grep-context` (`-C`) argument to specify the number of lines of context to show around matches (default is 5). Behavior: - If a grep pattern is provided, the script will use the system `grep` command to filter the logs of failed steps (or the full job log if a specific step's log cannot be isolated). - Output clearly indicates when grep results are shown, the pattern used, and the context lines. - Handles cases where `grep` finds no matches or if the `grep` command itself fails (e.g., not found, bad pattern). - If no grep pattern is provided, the script defaults to its previous behavior of printing the last N lines of the log.
This commit refines the grep functionality in the `print_workflow_run_errors.py` script by updating default values: - The default `--grep-pattern` is now `"[Ee]rror[: ]"`. Logs will be automatically filtered for this pattern if no other pattern is specified. - The default `--grep-context` is now 10 lines. If an empty string is explicitly passed to `--grep-pattern` (e.g., `--grep-pattern ""`), grep functionality will be disabled, and the script will fall back to printing the last N lines of the log.
This commit introduces a new feature to filter jobs by name using a regular expression in the `print_workflow_run_errors.py` script. New features: - Added `--job-pattern` argument, which accepts a regex string. Only jobs whose names match this pattern will be processed for failures. - The default value for `--job-pattern` is `'^build.*'`, meaning by default, the script will only look at jobs starting with 'build'. Behavior: - Job name filtering is applied before checking for job failures. - If an invalid regex is provided for `--job-pattern`, the script prints an error and exits gracefully. - This new filter works in conjunction with existing log processing options (like `--grep-pattern` and `--all-failed-steps`), which are applied to the jobs that pass the name pattern filter and are failed.
This commit updates the default regular expression for the `--grep-pattern` argument in the `print_workflow_run_errors.py` script. The default pattern is changed from `"[Ee]rror[: ]"` to `"[Ee][Rr][Rr][Oo][Rr][: ]"` for more specific matching of "Error" (case-insensitive) followed by a colon or space.
This commit updates the `print_workflow_run_errors.py` script to format its standard output using Markdown for improved readability when viewed in Markdown-aware environments. Changes include: - Job and step headings are now formatted as Markdown headers (H1-H4). - Workflow run and job URLs are presented as clickable Markdown links. - Actual log content (both grep results and last N lines) is enclosed in ```log ... ``` fenced code blocks. - Horizontal rules (`---`) are used to better separate sections. - Minor textual adjustments for clarity within the Markdown structure. The stderr output remains plain text for informational messages.
This commit adds two main enhancements to the
`print_workflow_run_errors.py` script:
1. **Log Download Status (stderr)**:
- Prints messages to stderr indicating the progress of log downloads
(e.g., "INFO: Downloading log X/Y for job 'job_name'...").
- Prints a summary to stderr after all jobs are processed, showing how
many logs were successfully fetched and processed (e.g.,
"INFO: Processed logs for S/T targeted failed jobs.").
- Includes a warning on stderr if a specific job's log fails to download.
2. **Timestamp Stripping (stdout)**:
- Implemented a function `strip_initial_timestamp` that uses a regex
to remove ISO 8601-like timestamps from the beginning of log lines.
- This stripping is applied to all log lines (from specific step
segments or full job log fallbacks) before they are further
processed by `grep` or printed as the 'last N lines'.
This makes the logs cleaner and potentially easier for other tools
or users to parse.
This commit introduces a prioritized, cascading job pattern checking
feature to the `print_workflow_run_errors.py` script.
Key changes:
- The `--job-pattern` argument now uses `action='append'`, allowing users
to specify multiple patterns. These are checked in the order provided.
- If no `--job-pattern` is specified by the user, a default prioritized
sequence is used: `['^build.*', '^test.*', '.*']`.
- The script iterates through the determined list of patterns:
- For the first pattern that matches jobs AND has failures among those
matched jobs, the script processes and displays logs for those
failures.
- It then stops checking subsequent patterns.
- If a pattern results in no matching jobs, or if all matching jobs
succeeded, the script moves to the next pattern in the sequence.
- Informative messages are printed to stderr indicating which pattern is
being checked, the outcome for that pattern, and if subsequent
patterns are skipped.
- The main log processing loop has been refactored into a helper function
`_process_and_display_logs_for_failed_jobs` for better organization.
- Handles invalid regular expressions within the pattern list by warning
the user and skipping the invalid pattern.
This commit changes the default value for the `--grep-context` argument in the `print_workflow_run_errors.py` script back to 5 (from 10).
This commit enhances `print_workflow_run_errors.py` by adding a `--run-id` command-line option. This allows users to specify a particular workflow run ID for processing, bypassing the default behavior of searching for the latest run by workflow name and branch. Key changes: - Added `--run-id <ID>` optional argument. - If `--run-id` is provided, the script fetches details for that specific run using a new helper function `get_workflow_run_details_by_id`. The `--workflow` and `--branch` arguments are ignored in this mode. - If `--run-id` is not provided, the script retains its existing behavior of using `--workflow` and `--branch` to find the latest run. - The new helper function includes error handling for invalid or non-existent run IDs. - Standard error messages have been updated to reflect whether the script is processing a run by specified ID or by search criteria.
Corrects a NameError in the `main` function when calling
`_process_and_display_logs_for_failed_jobs`. The script was
attempting to use `run.get('html_url')` where it should have been
`run_details.get('html_url')` after a previous refactor.
This fix ensures the correct variable is used, allowing the script
to properly pass the workflow run's HTML URL to the log processing
function.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This script allows users to specify a workflow name and branch to find the latest run. It then identifies any failed jobs within that run and prints the last N lines of logs for each failed step.
Key features:
Description
Testing
Type of Change
Place an
xthe applicable box:Notes
Release Notessection ofrelease_build_files/readme.md.