CI: Let external workflows call into the Build workflow#5
Open
riley-dixon wants to merge 11 commits intohipFilefrom
Open
CI: Let external workflows call into the Build workflow#5riley-dixon wants to merge 11 commits intohipFilefrom
riley-dixon wants to merge 11 commits intohipFilefrom
Conversation
riley-dixon
force-pushed
the
rildixon/ci-hook-for-hipfile
branch
13 times, most recently
from
dadc7c2 to
8e98ac2
Compare
If this workflow is called from a hipFile workflow,
the defaults ${{ github.repo }} and ${{ github.ref }} are
used, which means the hipFile repo gets cloned instead.
This is dual purposed: 1) Save time on setting up the CI image for building FIO 2) In future commits we will be consolidating the build workflows into one. This lets us move those setup steps out of the workflow.
riley-dixon
force-pushed
the
rildixon/ci-hook-for-hipfile
branch
3 times, most recently
from
5d08627 to
c9a6c96
Compare
riley-dixon
changed the title
Collaborator
Author
|
Reviewers: Please note that the commit titled |
riley-dixon
force-pushed
the
rildixon/ci-hook-for-hipfile
branch
from
05cdd18 to
de793bc
Compare
There was a problem hiding this comment.
Pull request overview
This PR refactors the FIO CI/CD infrastructure to consolidate multiple distro-specific build workflows into a single reusable workflow, similar to the approach used in the hipFile repository. The changes enable external repositories to cleanly call into the FIO build process while maintaining a clean GitHub Actions UI.
Changes:
- Consolidated three separate distro-specific workflows (build-debian.yml, build-rpm.yml, build-rpm-suse.yml) into a single unified build-fio.yml workflow that accepts a platform parameter (rocky/suse/ubuntu)
- Renamed hipFile engine from 'hipfile' to 'libhipfile' across Makefile, RPM spec files, and CI workflows for consistency
- Added Docker build environment configurations for Ubuntu, Rocky, and SUSE distributions to standardize CI environments
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/build-fio.yml | New unified workflow that builds FIO packages across all platforms using Docker containers and composite actions |
| .github/workflows/fio-hipfile-ci.yml | New CI workflow that tests builds across all supported platforms using a matrix strategy |
| .github/workflows/hipfile-release.yml | Updated to use the new unified workflow and updated output names with platform prefixes |
| .github/workflows/build-debian.yml | Deleted - functionality moved to build-fio.yml |
| .github/workflows/build-rpm.yml | Deleted - functionality moved to build-fio.yml |
| .github/workflows/build-rpm-suse.yml | Deleted - functionality moved to build-fio.yml |
| .github/actions/build-fio/action.yml | New composite action that encapsulates platform-specific build logic to hide skipped steps in CI UI |
| .github/actions/upload-fio-artifacts/action.yml | New composite action that handles platform-specific artifact uploads |
| docker/DOCKERFILE.ubuntu | New Dockerfile for Ubuntu build environment with ROCm and FIO dependencies |
| docker/DOCKERFILE.rocky | New Dockerfile for Rocky Linux build environment |
| docker/DOCKERFILE.suse | New Dockerfile for openSUSE Leap build environment |
| Makefile | Renamed hipfile engine variables to libhipfile for consistency |
| rpm/fedora/fio.spec | Renamed package from fio-engine-hipfile to fio-engine-libhipfile |
| debian/source/options | Added docker/ to diff ignore list |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
riley-dixon
force-pushed
the
rildixon/ci-hook-for-hipfile
branch
from
de793bc to
09c373d
Compare
The purpose of this move is to align with our existing hipFile CI approach where differences in platform steps are hidden within the workflow. By doing this, it reduces the amount of skipped jobs that display in a matrix context (e.g. workflow is running for Ubuntu and skips non-Ubuntu jobs). There is no way to suppress or hide these skipped jobs.
Update release workflow to use new multi-platform build workflow
The benefit of this move is to further hide steps that get skipped. Functionally, there is no difference. These steps are performed in the same exact container of the build workflow. Simply, this allows for better management of the display of workflow results the GitHub Web UI as the skipped steps for other platforms are just not displayed.
For similar reasons to creating a build action, this lets us better control how the workflow results appear on the Web UI.
FIO's engine loading mechanism expects the .so filename to match the name of the ioengine defined in code. If there is a mismatch, FIO will compile, however it will fail to find the IO engine at runtime. This commit fixes our edits to the Fedora RPM spec and FIO Makefile to align with this requirement. (The RPM feature option 'hipFile' does not need to be modified.)
riley-dixon
force-pushed
the
rildixon/ci-hook-for-hipfile
branch
from
09c373d to
a2b5919
Compare
riley-dixon
requested review from
derobins,
gaoikawa,
jordan-turbofish,
kurtmcmillan and
zbyrne
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refactor the existing FIO distro specific workflows to match what we do over in hipFile - a single workflow that works across multiple distros, hiding distro specific logic within the workflow. Otherwise, we clutter up the CI job summary with a bunch of skipped workflows as the hipFile system test workflow would need to call all of the FIO distro-specific workflows. (This is due to the GitHub Actions Framework and how we cannot choose to hide skipped jobs. For example, hipFile's Ubuntu workflow would call into all 3 of FIO's distro specific workflows, but the non-Ubuntu ones would be skipped.)
Additionally, we also hide the distro-specific CI environment setup steps within a Docker environment. We further refactor the distro-specific build and upload steps into local composite actions to further improve how the workflow result appears on the GitHub Web UI. (By moving from 3 workflows to 1, we move the skipped jobs from separate checks to skipped steps within the single workflow. By further moving these steps into a separate action we hide the skipped steps entirely.)
All-in-all, this PR gives a clean way for external repositories to build an FIO with hipFile package and consume it, while also maintaining a clean status summary.
Note: The docker images are static and not regularly updated. I did not think it was worth the trouble to implement a workflow similar to hipFile that maintains the Docker CI images automatically.
AIHIPFILE-121