Break down imgtestlib module and use log sections#2383
Merged
Conversation
- E252 missing whitespace around parameter equals - E713 test for membership should be 'not in' - E302 expected 2 blank lines, found 1
achilleas-k
force-pushed
the
breakdown/imgtestlib
branch
3 times, most recently
from
8e1aa38 to
0e4f74c
Compare
Member
Author
|
The cross-arch test keeps build x86_64 images. Hang on... |
Make the image building scaffolding reusable.
Add functions for printing a log section start and end line.. If running in GitLab CI, use their custom collapsible sections escape sequences [1] to create collapsible build logs. [1] https://docs.gitlab.com/ci/jobs/job_logs/#custom-collapsible-sections
Move imgtestlib.py to an importable module directory by the same (logical) name.
The module got too big for one file. This commit only moves code around and adjusts imports. Functions are grouped into submodules by logical functionality. Groups (submodule boundaries) were chosen to avoid circular imports. The .core module could still use some splitting or tidying up, but the current state is already an improvement. Importing everything into __init__.py might be unnecessary—we only really need to import the functions that are used externally—but it's not very important to reduce the API surface of this internal testing library.
Make them easier to see in the log.
The boot-image script is now a thin wrapper around the new imgtestlib.boot_image() function, so the general functionality is reusable. The boot-image script now simply handles argument parsing and calls into the function. The can_boot_test() function has been moved to imgtestlib/core to avoid circular imports. We might need to reorganise the module at a later date.
The upload-results script is now a thin wrapper around the new imgtestlib.upload_results() function, so the general functionality is reusable. The upload-results script now simply handles argument parsing and calls into the function.
Let's test everything!
boto3 is only needed when interfacing with aws. vmtest is imported by imgtestlib, which we often use for a lot of other smaller tasks, like setting up the osbuild repo. We could install boto3 whenever we need it, but it's simpler to allow importing without it.
Since we moved core parts of the test scripts into the imgtestlib module, the boot_image function now needs to be importable by all the distros we support and test on, including EL9 which only has Python 3.9. This wasn't an issue before because boot-image was only ever run on the CI runners, which are currently Fedora 42. Now that the core functionality is part of the importable module though, we need to rewrite it to run on older Python versions.
Support decorating functions with a log section context. This creates a log section at the start of the function that exits automatically at the end of the function.
Use the log_section decorator to print the section start and end lines when entering and exiting certain functions. This makes the names of log sections static, where previously we could print specific values like the name or configuration of the image being built. While it would be nice to have this kind of information in the names of sections, I think the decorator solution is cleaner in code and the downside of having static log section names overall isn't too bad.
The script reads environment variables to determine if it's running in Gitlab, so we need to preserve the environment when running with sudo.
achilleas-k
force-pushed
the
breakdown/imgtestlib
branch
from
0e4f74c to
970c4b4
Compare
lzap
previously approved these changes
May 29, 2026
Contributor
lzap
left a comment
lzap
left a comment
There was a problem hiding this comment.
Very nice, the context is quite important because unclosed sections mess up the output quite a bit. I would also recommend to make a unique ID because if someone logs Test section twice it is very messed up. I think you could use some internal Python thing maybe stackframe or something to calculate some hash or something.
Member
Author
It creates a UUID for each section / context. |
Contributor
|
Ah now I see it, my implementation was bash based, this has a context! Clever. Nice. |
The beta repos are showing the same behaviour that was discovered before with 10.1 and 10.2. osbuild#2044 (comment)
Member
Author
|
RHEL 10.3 beta repos are experiencing the same issue we say when we first added the I'll also open a follow-up PR to enable testing on 10.1 and 10.2 since those should be working normally now. |
lzap
approved these changes
May 29, 2026
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR splits out the commits from #2359 that do not change the test pipelines.
Changes
imgtestlibinto multiple files. The single file was getting a bit too big and harder to navigate.build-imageandboot-imageare now thin wrappers around functions inimgtestlibthat handle command line arguments.