-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
compatible(integration_test_charm.yaml): Collect logs from all models #195
compatible(integration_test_charm.yaml): Collect logs from all models #195
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this looks great, thank you!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should probably remove this convert tool in favor of model-logs-size
, which works on VM & k8s
I was hesitant to do this earlier since I was worried about causing out of disk space failures since we've already seen some out of disk space issues on integration tests. This could potentially be mitigated by configuring juju to use /mnt on the GitHub-hosted runners
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure. I created #197 to track that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, should work... some simplification suggestions.
model = json.loads( | ||
model = next(iter(json.loads( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it might be better to pass the model name here
but imo we should really get rid of this workaround—let me open a PR for this
@@ -418,21 +418,39 @@ jobs: | |||
- name: juju status | |||
timeout-minutes: 1 | |||
if: ${{ success() || (failure() && steps.tests.outcome == 'failure') }} | |||
run: juju status --color --relations | tee ~/logs/juju-status.txt | |||
run: | | |||
mapfile -t MODELS < <(juju models --format json | jq -r '.models[] | select(."short-name" != "controller") | ."short-name"') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if jq is installed by default on the IS-hosted runners, Azure-hosted runners, or GitHub-hosted arm64 runners
consider installing jq or switching to Python here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if you install jq, please do it before the integration test step—since this step needs to run quickly on spot instance eviction
4d65ede
to
ef23e64
Compare
7dee1e9
to
25c5e67
Compare
ef23e64
to
242803e
Compare
242803e
to
ae312dc
Compare
tested on canonical/postgresql-k8s-operator#577 |
@marceloneppel apologizes for the long delay I wanted to improve the UX so that log groups are not used for a single model (so that user doesn't need to expand logs twice)—and at that point, it made sense to go from bash to Python. I wanted to create a python api for interacting with GitHub Actions (#210, #211) and didn't have a chance to implement that until now Please review the PR & share any feedback |
25c5e67
to
6a559e8
Compare
9e97f6d
to
0ff3a6a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Beauty!!!
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [allure-pytest-collection-report](https://togithub.com/canonical/data-platform-workflows) | integration | major | `v16.3.3` -> `v18.0.0` | | [canonical/data-platform-workflows](https://togithub.com/canonical/data-platform-workflows) | action | major | `v16.3.3` -> `v18.0.0` | | [pytest-operator-cache](https://togithub.com/canonical/data-platform-workflows) | integration | major | `v16.3.3` -> `v18.0.0` | | [pytest-operator-groups](https://togithub.com/canonical/data-platform-workflows) | integration | major | `v16.3.3` -> `v18.0.0` | --- ### Release Notes <details> <summary>canonical/data-platform-workflows (allure-pytest-collection-report)</summary> ### [`v18.0.0`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v18.0.0) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v17.0.1...v18.0.0) ##### What's Changed - breaking: Change default charmcraft version to 2.x by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#219 **Full Changelog**: canonical/data-platform-workflows@v17...v18.0.0 ### [`v17.0.1`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v17.0.1) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v17.0.0...v17.0.1) ##### What's Changed - patch(integration_test_charm.yaml): Collect microk8s logs if set up fails by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#218 **Full Changelog**: canonical/data-platform-workflows@v17.0.0...v17.0.1 ### [`v17.0.0`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v17.0.0) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.7.0...v17.0.0) ##### What's Changed - breaking(release_rock.yaml): Create git tag & GitHub release by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#216 **Full Changelog**: canonical/data-platform-workflows@v16...v17.0.0 ##### Migration instructions Replace ```yaml permissions: packages: write # Needed to publish to GitHub Container Registry ``` with ```yaml permissions: packages: write # Needed to publish to GitHub Container Registry contents: write # Needed to create GitHub release ``` when calling release_rock.yaml Example: [canonical/charmed-postgresql-rock#45 ### [`v16.7.0`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.7.0) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.6.0...v16.7.0) #### What's Changed - compatible(release_\*.yaml): Create git tags for all snap & charm revisions by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#213 **Full Changelog**: canonical/data-platform-workflows@v16.6.0...v16.7.0 ### [`v16.6.0`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.6.0) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.5.2...v16.6.0) #### What's Changed - compatible(integration_test_charm.yaml): Add `jhack tail` to logs by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#212 **Full Changelog**: canonical/data-platform-workflows@v16.5.2...v16.6.0 ### [`v16.5.2`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.5.2) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.5.1...v16.5.2) #### What's Changed - patch(integration_test_charm.yaml): Remove `#` from Allure commit message by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#214 **Full Changelog**: canonical/data-platform-workflows@v16.5.1...v16.5.2 ### [`v16.5.1`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.5.1) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.5.0...v16.5.1) #### What's Changed - patch(build_snap.yaml): Increase timeout to 30 minutes by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#215 **Full Changelog**: canonical/data-platform-workflows@v16.5.0...v16.5.1 ### [`v16.5.0`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.5.0) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.4.3...v16.5.0) #### What's Changed - compatible(integration_test_charm.yaml): Collect logs from all models by [@​marceloneppel](https://togithub.com/marceloneppel) in [canonical/data-platform-workflows#195 #### New Contributors - [@​marceloneppel](https://togithub.com/marceloneppel) made their first contribution in [canonical/data-platform-workflows#195 **Full Changelog**: canonical/data-platform-workflows@v16.4.3...v16.5.0 ### [`v16.4.3`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.4.3) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.4.2...v16.4.3) #### What's Changed - patch: Add log groups to GH Actions API by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#211 **Full Changelog**: canonical/data-platform-workflows@v16.4.2...v16.4.3 ### [`v16.4.2`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.4.2) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.4.1...v16.4.2) #### What's Changed - patch: Add Python API for GitHub Actions by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#210 **Full Changelog**: canonical/data-platform-workflows@v16.4.1...v16.4.2 ### [`v16.4.1`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.4.1) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.4.0...v16.4.1) #### What's Changed - patch(integration_test_charm.yaml): Use `model-logs-size=10G` instead of converting controller logsink by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#209 **Full Changelog**: canonical/data-platform-workflows@v16.4.0...v16.4.1 ### [`v16.4.0`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.4.0) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.3.3...v16.4.0) #### What's Changed - compatible(integration_test_charm.yaml): Pre-download lxd image by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#208 **Full Changelog**: canonical/data-platform-workflows@v16.3.3...v16.4.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 1am and before 3am every weekday" in timezone Etc/UTC, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View the [repository job log](https://developer.mend.io/github/canonical/mysql-router-k8s-operator). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MzguMCIsInVwZGF0ZWRJblZlciI6IjM3LjQ0MC43IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [canonical/data-platform-workflows](https://togithub.com/canonical/data-platform-workflows) | action | major | `v16.3.0` -> `v18.0.0` | | [pytest-github-secrets](https://togithub.com/canonical/data-platform-workflows) | integration | major | `v16.3.0` -> `v18.0.0` | | [pytest-operator-cache](https://togithub.com/canonical/data-platform-workflows) | integration | major | `v16.3.0` -> `v18.0.0` | | [pytest-operator-groups](https://togithub.com/canonical/data-platform-workflows) | integration | major | `v16.3.0` -> `v18.0.0` | --- ### Release Notes <details> <summary>canonical/data-platform-workflows (canonical/data-platform-workflows)</summary> ### [`v18.0.0`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v18.0.0) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v17.0.1...v18.0.0) ##### What's Changed - breaking: Change default charmcraft version to 2.x by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#219 **Full Changelog**: canonical/data-platform-workflows@v17...v18.0.0 ### [`v17.0.1`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v17.0.1) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v17.0.0...v17.0.1) ##### What's Changed - patch(integration_test_charm.yaml): Collect microk8s logs if set up fails by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#218 **Full Changelog**: canonical/data-platform-workflows@v17.0.0...v17.0.1 ### [`v17.0.0`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v17.0.0) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.7.0...v17.0.0) ##### What's Changed - breaking(release_rock.yaml): Create git tag & GitHub release by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#216 **Full Changelog**: canonical/data-platform-workflows@v16...v17.0.0 ##### Migration instructions Replace ```yaml permissions: packages: write # Needed to publish to GitHub Container Registry ``` with ```yaml permissions: packages: write # Needed to publish to GitHub Container Registry contents: write # Needed to create GitHub release ``` when calling release_rock.yaml Example: [canonical/charmed-postgresql-rock#45 ### [`v16.7.0`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.7.0) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.6.0...v16.7.0) #### What's Changed - compatible(release_\*.yaml): Create git tags for all snap & charm revisions by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#213 **Full Changelog**: canonical/data-platform-workflows@v16.6.0...v16.7.0 ### [`v16.6.0`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.6.0) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.5.2...v16.6.0) #### What's Changed - compatible(integration_test_charm.yaml): Add `jhack tail` to logs by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#212 **Full Changelog**: canonical/data-platform-workflows@v16.5.2...v16.6.0 ### [`v16.5.2`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.5.2) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.5.1...v16.5.2) #### What's Changed - patch(integration_test_charm.yaml): Remove `#` from Allure commit message by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#214 **Full Changelog**: canonical/data-platform-workflows@v16.5.1...v16.5.2 ### [`v16.5.1`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.5.1) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.5.0...v16.5.1) #### What's Changed - patch(build_snap.yaml): Increase timeout to 30 minutes by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#215 **Full Changelog**: canonical/data-platform-workflows@v16.5.0...v16.5.1 ### [`v16.5.0`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.5.0) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.4.3...v16.5.0) #### What's Changed - compatible(integration_test_charm.yaml): Collect logs from all models by [@​marceloneppel](https://togithub.com/marceloneppel) in [canonical/data-platform-workflows#195 #### New Contributors - [@​marceloneppel](https://togithub.com/marceloneppel) made their first contribution in [canonical/data-platform-workflows#195 **Full Changelog**: canonical/data-platform-workflows@v16.4.3...v16.5.0 ### [`v16.4.3`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.4.3) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.4.2...v16.4.3) #### What's Changed - patch: Add log groups to GH Actions API by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#211 **Full Changelog**: canonical/data-platform-workflows@v16.4.2...v16.4.3 ### [`v16.4.2`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.4.2) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.4.1...v16.4.2) #### What's Changed - patch: Add Python API for GitHub Actions by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#210 **Full Changelog**: canonical/data-platform-workflows@v16.4.1...v16.4.2 ### [`v16.4.1`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.4.1) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.4.0...v16.4.1) #### What's Changed - patch(integration_test_charm.yaml): Use `model-logs-size=10G` instead of converting controller logsink by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#209 **Full Changelog**: canonical/data-platform-workflows@v16.4.0...v16.4.1 ### [`v16.4.0`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.4.0) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.3.3...v16.4.0) #### What's Changed - compatible(integration_test_charm.yaml): Pre-download lxd image by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#208 **Full Changelog**: canonical/data-platform-workflows@v16.3.3...v16.4.0 ### [`v16.3.3`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.3.3) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.3.2...v16.3.3) ##### What's Changed - patch(integration_test_charm.yaml): Fix step name by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#205 **Full Changelog**: canonical/data-platform-workflows@v16.3.2...v16.3.3 ### [`v16.3.2`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.3.2) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.3.1...v16.3.2) ##### What's Changed - patch(integration_test_charm.yaml): Shorten recommended job name by [@​carlcsaposs-canonical](https://togithub.com/carlcsaposs-canonical) in [canonical/data-platform-workflows#206 **Full Changelog**: canonical/data-platform-workflows@v16.3.1...v16.3.2 ##### Migration instructions Replace ```yaml name: Integration test charm ``` with ```yaml name: Integration ``` when calling integration_test_charm.yaml Example: [canonical/mysql-router-k8s-operator#285 ### [`v16.3.1`](https://togithub.com/canonical/data-platform-workflows/releases/tag/v16.3.1) [Compare Source](https://togithub.com/canonical/data-platform-workflows/compare/v16.3.0...v16.3.1) ##### What's Changed - patch(update_bundle.yaml): Assume amd64 architecture by [@​lucasgameiroborges](https://togithub.com/lucasgameiroborges) in [canonical/data-platform-workflows#207 ##### New Contributors - [@​lucasgameiroborges](https://togithub.com/lucasgameiroborges) made their first contribution in [canonical/data-platform-workflows#207 **Full Changelog**: canonical/data-platform-workflows@v16.3.0...v16.3.1 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 1am and before 3am every weekday" in timezone Etc/UTC, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View the [repository job log](https://developer.mend.io/github/canonical/mysql-router-operator). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy40MzEuNCIsInVwZGF0ZWRJblZlciI6IjM4LjE4LjE3IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
For integration tests like the one for async replication, it would be helpful to have status and logs from multiple models. (Fixes #193)