Improve robustness on terraform log regex matching#1098
Merged
Conversation
liyu-ma
requested review from
alyssa1303,
anson627,
sumanthreddy29,
vittoriasalim and
xinWeiWei24
as code owners
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves Terraform log parsing robustness in extract_terraform_operation_metadata.py, expanding elapsed-time regex matching (notably for terraform destroy) and emitting a default 0s duration for failures where no elapsed time was captured.
Changes:
- Relax
ELAPSED_PATTERNto match elapsed durations even when anid=...prefix is present in “Still creating/destroying” lines. - Emit failure records with
time_taken_seconds=0when a run ends (or is superseded) without any captured elapsed duration. - Extend unit tests to cover destroy logs with
id=prefixes and failures/timeouts without elapsed lines.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| modules/python/terraform/extract_terraform_operation_metadata.py | Updates elapsed regex and changes incomplete-run flushing behavior to default missing durations to 0s. |
| modules/python/tests/test_extract_terraform_operation_metadata.py | Fixes malformed test log lines and adds new tests for destroy id= prefixes and no-elapsed failure/timeout scenarios. |
liyu-ma
force-pushed
the
maliyu/terraform-metatada-fix
branch
from
22c9f2e to
895a980
Compare
vittoriasalim
approved these changes
Mar 17, 2026
xinWeiWei24
approved these changes
Mar 17, 2026
wonderyl
reviewed
Mar 18, 2026
|
|
||
| echo "Delete resource group $RUN_ID" | ||
| az group delete --name $RUN_ID --yes | ||
| az group delete --name $RUN_ID --yes --no-wait |
Collaborator
There was a problem hiding this comment.
would this make our pipeline more flaky?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
ELAPSED_PATTERNused in terraform log extraction does not accommodate the log line pattern where resource id was included. Fixed this soterraform destroycommand failure can be collected successfully.This fixes the case where the terraform command failed before it logs any time elaspsed
It is observed that resource group deletion could encounter a timeout of 10 mins from ARM side (not adjustable from client side) which can fail the pipeline run, despite the RG could be deleted successfully later. Change the deletion call to be async with
--no-waitflag, and poll the resource group with specified wait time (20 mins). If RG was not deleted after 20 mins, fail the pipeline run.