Presently, local development and GitHub Actions use different build mechanisms. These mechanisms also provide different capabilities and levels of test coverage.
GitHub Actions is the only standardized entry point for several classes of validation, particularly release-canary installation tests, release-artifact smoke matrices, and multi-platform release build and packaging workflows. Other tests can technically run locally, but their CI configurations may lack equivalent, documented mise entry points. The reverse gap also exists: several tests included in mise run pre-commit do not appear to run in GitHub Actions, including the explicit Rust workspace compile check, SBOM tooling tests, focused install.sh tests, static packaging-asset tests, and documentation synchronization tests.
Release builds have a similar reproducibility gap. GitHub Actions provides the toolchains and target-specific environments used to build release binaries, and developers cannot reliably produce every target artifact through a common local entry point. Because packaging and installation tests depend on those binaries, this limitation propagates downstream. For example, a developer may be unable to build an RPM, install it in a Fedora VM, and reproduce the corresponding release test locally. The shared build mechanism should make cross-compilation, packaging, and artifact testing locally executable wherever practical.
CI and developer workstations should invoke the same shared task runner(s) so that neither environment silently provides unique coverage. Intentional differences should be limited to tests requiring specialized hardware or operating systems. For example, testing macOS artifacts or creating VMs with Hypervisor.framework requires a real Mac, while GPU end-to-end tests require an NVIDIA GPU. Even when execution environments differ, entry points and assertions should remain consistent, with GitHub Actions primarily orchestrating the same tasks developers can run locally.
This issue is intended to supersede the following PRs, which add desired coverage to the current system in different ways and must be considered when developing the shared task runner:
- PR #2093, which expands GitHub Actions coverage across additional targets.
- PR #1465, which adds a method for running some tests locally.
- PR #1981, which expands the release canary GitHub Action to cover upgrades.
Presently, local development and GitHub Actions use different build mechanisms. These mechanisms also provide different capabilities and levels of test coverage.
GitHub Actions is the only standardized entry point for several classes of validation, particularly release-canary installation tests, release-artifact smoke matrices, and multi-platform release build and packaging workflows. Other tests can technically run locally, but their CI configurations may lack equivalent, documented mise entry points. The reverse gap also exists: several tests included in mise run pre-commit do not appear to run in GitHub Actions, including the explicit Rust workspace compile check, SBOM tooling tests, focused install.sh tests, static packaging-asset tests, and documentation synchronization tests.
Release builds have a similar reproducibility gap. GitHub Actions provides the toolchains and target-specific environments used to build release binaries, and developers cannot reliably produce every target artifact through a common local entry point. Because packaging and installation tests depend on those binaries, this limitation propagates downstream. For example, a developer may be unable to build an RPM, install it in a Fedora VM, and reproduce the corresponding release test locally. The shared build mechanism should make cross-compilation, packaging, and artifact testing locally executable wherever practical.
CI and developer workstations should invoke the same shared task runner(s) so that neither environment silently provides unique coverage. Intentional differences should be limited to tests requiring specialized hardware or operating systems. For example, testing macOS artifacts or creating VMs with Hypervisor.framework requires a real Mac, while GPU end-to-end tests require an NVIDIA GPU. Even when execution environments differ, entry points and assertions should remain consistent, with GitHub Actions primarily orchestrating the same tasks developers can run locally.
This issue is intended to supersede the following PRs, which add desired coverage to the current system in different ways and must be considered when developing the shared task runner:
- PR #2093, which expands GitHub Actions coverage across additional targets.
- PR #1465, which adds a method for running some tests locally.
- PR #1981, which expands the release canary GitHub Action to cover upgrades.