Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add ability to record and replay module for testing purpose #998

Conversation

goneri
Copy link
Member

@goneri goneri commented Sep 7, 2022

Depends-On: ansible/ansible-zuul-jobs#1619

Use lib placebo (https://pypi.org/project/placebo/) to record the API
interaction. This allow us to replay later a previous interaction offline
and cover some cases that are hard to reproduce in a CI.

Use the ``placebo`` (https://pypi.org/project/placebo/) to record the API
interaction. This allow us to replay later a previous interaction offline
and cover some cases that are hard to reproduce in a CI.
Copy link
Contributor

@tremble tremble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for splitting this off, generally LGTM but a couple of minor comments

@goneri goneri force-pushed the add-ability-to-record-and-replay-module-for-testing-purpose_2281 branch from 2375cda to 73b3b61 Compare September 7, 2022 15:20
@goneri goneri requested a review from tremble September 7, 2022 15:25
@softwarefactory-project-zuul
Copy link
Contributor

Build failed.

✔️ ansible-galaxy-importer SUCCESS in 4m 42s
✔️ build-ansible-collection SUCCESS in 6m 02s
✔️ ansible-test-sanity-aws-ansible-python38 SUCCESS in 9m 13s
✔️ ansible-test-sanity-aws-ansible-2.12-python38 SUCCESS in 9m 39s
✔️ ansible-test-sanity-aws-ansible-2.13-python38 SUCCESS in 8m 23s
✔️ ansible-test-units-amazon-aws-python38 SUCCESS in 6m 45s
✔️ ansible-test-units-amazon-aws-python39 SUCCESS in 5m 19s
✔️ ansible-test-splitter SUCCESS in 2m 46s
integration-amazon.aws-1 FAILURE in 9m 39s
⚠️ integration-amazon.aws-2 SKIPPED
⚠️ integration-amazon.aws-3 SKIPPED
⚠️ integration-amazon.aws-4 SKIPPED
⚠️ integration-amazon.aws-5 SKIPPED
⚠️ integration-amazon.aws-6 SKIPPED
⚠️ integration-amazon.aws-7 SKIPPED
⚠️ integration-amazon.aws-8 SKIPPED
⚠️ integration-amazon.aws-9 SKIPPED
⚠️ integration-amazon.aws-10 SKIPPED
⚠️ integration-amazon.aws-11 SKIPPED
⚠️ integration-amazon.aws-12 SKIPPED
⚠️ integration-amazon.aws-13 SKIPPED
✔️ integration-community.aws-1 SUCCESS in 5m 39s
⚠️ integration-community.aws-2 SKIPPED
⚠️ integration-community.aws-3 SKIPPED
⚠️ integration-community.aws-4 SKIPPED
⚠️ integration-community.aws-5 SKIPPED
⚠️ integration-community.aws-6 SKIPPED
⚠️ integration-community.aws-7 SKIPPED
⚠️ integration-community.aws-8 SKIPPED
⚠️ integration-community.aws-9 SKIPPED
⚠️ integration-community.aws-10 SKIPPED
⚠️ integration-community.aws-11 SKIPPED
⚠️ integration-community.aws-12 SKIPPED
⚠️ integration-community.aws-13 SKIPPED
✔️ ansible-test-changelog SUCCESS in 2m 20s

@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded.

✔️ ansible-galaxy-importer SUCCESS in 5m 13s
✔️ build-ansible-collection SUCCESS in 5m 31s
✔️ ansible-test-sanity-aws-ansible-python38 SUCCESS in 10m 00s
✔️ ansible-test-sanity-aws-ansible-2.12-python38 SUCCESS in 9m 19s
✔️ ansible-test-sanity-aws-ansible-2.13-python38 SUCCESS in 9m 18s
✔️ ansible-test-units-amazon-aws-python38 SUCCESS in 6m 57s
✔️ ansible-test-units-amazon-aws-python39 SUCCESS in 6m 34s
✔️ ansible-test-splitter SUCCESS in 2m 43s
✔️ integration-amazon.aws-1 SUCCESS in 7m 36s
⚠️ integration-amazon.aws-2 SKIPPED
⚠️ integration-amazon.aws-3 SKIPPED
⚠️ integration-amazon.aws-4 SKIPPED
⚠️ integration-amazon.aws-5 SKIPPED
⚠️ integration-amazon.aws-6 SKIPPED
⚠️ integration-amazon.aws-7 SKIPPED
⚠️ integration-amazon.aws-8 SKIPPED
⚠️ integration-amazon.aws-9 SKIPPED
⚠️ integration-amazon.aws-10 SKIPPED
⚠️ integration-amazon.aws-11 SKIPPED
⚠️ integration-amazon.aws-12 SKIPPED
⚠️ integration-amazon.aws-13 SKIPPED
✔️ integration-community.aws-1 SUCCESS in 5m 30s
⚠️ integration-community.aws-2 SKIPPED
⚠️ integration-community.aws-3 SKIPPED
⚠️ integration-community.aws-4 SKIPPED
⚠️ integration-community.aws-5 SKIPPED
⚠️ integration-community.aws-6 SKIPPED
⚠️ integration-community.aws-7 SKIPPED
⚠️ integration-community.aws-8 SKIPPED
⚠️ integration-community.aws-9 SKIPPED
⚠️ integration-community.aws-10 SKIPPED
⚠️ integration-community.aws-11 SKIPPED
⚠️ integration-community.aws-12 SKIPPED
⚠️ integration-community.aws-13 SKIPPED
✔️ ansible-test-changelog SUCCESS in 2m 44s

Copy link
Contributor

@tremble tremble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd love to see a follow up which includes some documentation around how to re-record.

We should also keep an eye on the size of any recordings. Depending on how frequently the recordings are updated, this could explode the size of the repo without a clean way to shrink it back down.

@tremble tremble added the mergeit Merge the PR (SoftwareFactory) label Sep 8, 2022
@softwarefactory-project-zuul
Copy link
Contributor

Build succeeded (gate pipeline).

✔️ ansible-galaxy-importer SUCCESS in 4m 23s
✔️ build-ansible-collection SUCCESS in 5m 05s
✔️ ansible-test-sanity-aws-ansible-python38 SUCCESS in 9m 15s
✔️ ansible-test-sanity-aws-ansible-2.12-python38 SUCCESS in 9m 14s
✔️ ansible-test-sanity-aws-ansible-2.13-python38 SUCCESS in 8m 54s
✔️ ansible-test-units-amazon-aws-python38 SUCCESS in 6m 22s
✔️ ansible-test-units-amazon-aws-python39 SUCCESS in 6m 09s
✔️ ansible-test-splitter SUCCESS in 3m 07s
✔️ integration-amazon.aws-1 SUCCESS in 6m 46s
⚠️ integration-amazon.aws-2 SKIPPED
⚠️ integration-amazon.aws-3 SKIPPED
⚠️ integration-amazon.aws-4 SKIPPED
⚠️ integration-amazon.aws-5 SKIPPED
⚠️ integration-amazon.aws-6 SKIPPED
⚠️ integration-amazon.aws-7 SKIPPED
⚠️ integration-amazon.aws-8 SKIPPED
⚠️ integration-amazon.aws-9 SKIPPED
⚠️ integration-amazon.aws-10 SKIPPED
⚠️ integration-amazon.aws-11 SKIPPED
⚠️ integration-amazon.aws-12 SKIPPED
⚠️ integration-amazon.aws-13 SKIPPED
✔️ integration-community.aws-1 SUCCESS in 12m 42s
⚠️ integration-community.aws-2 SKIPPED
⚠️ integration-community.aws-3 SKIPPED
⚠️ integration-community.aws-4 SKIPPED
⚠️ integration-community.aws-5 SKIPPED
⚠️ integration-community.aws-6 SKIPPED
⚠️ integration-community.aws-7 SKIPPED
⚠️ integration-community.aws-8 SKIPPED
⚠️ integration-community.aws-9 SKIPPED
⚠️ integration-community.aws-10 SKIPPED
⚠️ integration-community.aws-11 SKIPPED
⚠️ integration-community.aws-12 SKIPPED
⚠️ integration-community.aws-13 SKIPPED
✔️ ansible-test-changelog SUCCESS in 2m 20s

@softwarefactory-project-zuul softwarefactory-project-zuul bot merged commit 4b5902b into ansible-collections:main Sep 8, 2022
@tremble
Copy link
Contributor

tremble commented Sep 8, 2022

@goneri I've not backported this to stable-4 since I don't think we'll have any specific needs for it, but I wouldn't block a backport if someone requested one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community_review integration tests/integration mergeit Merge the PR (SoftwareFactory) module_utils module_utils needs_triage plugins plugin (any type) tests tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants