Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support BEAKER_FACTER_* env vars in beaker testing #108

Merged
merged 4 commits into from
Oct 11, 2023

Conversation

ekohl
Copy link
Member

@ekohl ekohl commented Oct 10, 2023

This PR changes things so you can run a test matrix by providing an additional fact.

The use case is that you have a test suite for your software and want to support multiple versions of that software. For example, in puppet-pulpcore there are versions 3.21, 3.22 and 3.28 supported at the same time. Rather than designing the test suite to first test x, then y and then z while cleaning up in between, this allows providing the version as a fact (so setup_acceptance_node.rb can set up repos). You then expand the test matrix. It also includes the version number so it's easy to see which version of the software is being tested. That it can use parallelism for this is just a side effect, but a nice one.

To make it more flexible the config for GHA is changed by providing puppet_beaker_test_matrix. This only provides a name and a set of environment variables. This means other CI systems could also use this.

Only a single fact can be provided. Generating the whole matrix was too complex and not needed for my use case. Because it's only providing a name and a set of env vars, it could be extended in the future.

Right now it doesn't have documentation, which should be added before it's merged but this at least allows for review.

@ekohl
Copy link
Member Author

ekohl commented Oct 10, 2023

Something to think about: puppet-zabbix could use this, but there we may need to skip combos: EL7 & Zabbix 6 for example.

@evgeni
Copy link
Member

evgeni commented Oct 10, 2023

Something to think about: puppet-zabbix could use this, but there we may need to skip combos: EL7 & Zabbix 6 for example.

Answering here, just for context: no, this needs no skip as the zabbix6 agent is well supported on EL7, so you only need to skip the server bits, and that the tests already do.

This only presents a name and environment variables. It is intended to
replace github_action_test_matrix, which is now deprecated.
This changes things so you can run a test matrix by providing an
additional fact.

The use case is that you have a test suite for your software and want to
support multiple versions of that software. For example, in
puppet-pulpcore there are versions 3.21, 3.22 and 3.28 supported at the
same time. Rather than designing the test suite to first test x, then y
and then z while cleaning up in between, this allows providing the
version as a fact (so setup_acceptance_node.rb can set up repos). You
then expand the test matrix. It also includes the version number so it's
easy to see which version of the software is being tested. That it can
use parallelism for this is just a side effect, but a nice one.

To make it more flexible the config for GHA is changed by providing
puppet_beaker_test_matrix. This only provides a name and a set of
environment variables. This means other CI systems could also use this.

Only a single fact can be provided. Generating the whole matrix was too
complex and not needed for my use case. Because it's only providing a
name and a set of env vars, it could be extended in the future.
@ekohl
Copy link
Member Author

ekohl commented Oct 11, 2023

Now updated with tests and I think it's ready.

@ekohl ekohl merged commit d262ff4 into voxpupuli:master Oct 11, 2023
6 checks passed
@ekohl ekohl deleted the beaker-facts branch October 11, 2023 11:25
@ekohl ekohl added the enhancement New feature or request label Oct 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants