Measure a GitHub Actions job on Solari microVMs and pick its CPU size from the curve. Reports per-step speed, cost, run history, and static findings.
Showcase: https://moazessam376-dev.github.io/solari-showcase/
- Inspects workflow shape, matrix cells, runner labels, history, and static findings.
- Runs one selected Linux job natively in isolated Solari microVMs at several CPU sizes.
- Renders a speed-versus-cost curve as terminal output, Markdown, JSON, and an optional PNG.
GitHub's Actions platform fee is $0.002/minute effective 2026-03-01, and measuring the job first makes the trade-off between wall-clock time and compute cost visible before changing a workflow. Measuring before tuning is inspired by Blacksmith's [code]smith CI Tuning.
uv pip install -e .
cp .env.example .envSet SOLARI_API_KEY in .env and log in with gh auth login. GITHUB_TOKEN (or the output of
gh auth token) lets solci clone private repositories inside the sandbox; the token is injected into the
clone URL inside the microVM and masked in logs. For example, export GITHUB_TOKEN=$(gh auth token) also
helps the gh CLI avoid GitHub API rate limits; never paste it into reports.
Check the local setup:
solci doctor▮ SOLCI /// DOCTOR check this Solari setup
CHECK STATUS DETAIL
SOLARI_API_KEY PASS slr_live_...xxxx
gh auth PASS authenticated
Solari sandbox PASS create -> nproc=1 -> delete in 2.66s
Inspect a remote repository:
solci inspect moazessam376-dev/crosstalk◎ moazessam376-dev/crosstalk
WORKFLOWS / JOBS
WORKFLOW JOB ID RUNS-ON STEPS MATRIX SERVICES
ci.yml test ${{ matrix.os }} 6 yes -
FINDINGS
low NO_TIMEOUT job Job `test` has no job-level timeout.
HISTORY BASELINE
runs 18 median 122.0 s p90 160.0 s failure rate 17% est. runs/month 600.0
GitHub $/run $0.0000 GitHub $/month $0.00
public repo: GitHub-hosted minutes are free; private-rate reference $0.0300/run
solci inspect owner/repo --job check on a Blacksmith-hosted job (moazessam376-dev/t3code) surfaces the
BIG_RUNNER finding for blacksmith-8vcpu-ubuntu-2404, mentioning the vendor by name.
Run the selected job at several sizes and write artifacts:
solci run moazessam376-dev/crosstalk --job test --cpu 1,2,4,8 \
--json docs/crosstalk.json --md docs/crosstalk.md --chart docs/crosstalk.png▤ RESULTS moazessam376-dev/crosstalk / test
CPU MEM MB BOOT ONLINE TOTAL SOLARI/RUN SOLARI/MONTH SPEEDUP VS 1
1 2,048 1.9 s 0.0s 191.5 s $0.0030 $1.8188 1.00x
2 2,048 1.9 s 2.6s 197.5 s $0.0050 $3.0289 0.97x
4 4,096 0.2 s 3.1s 228.3 s $0.0117 $7.0006 -
8 8,192 0.3 s 2.6s 277.7 s $0.0284 $17.0323 -
GitHub baseline: median 122.0 s, p90 160.0 s, 18 runs, $0.0000/run, 600.0 runs/month.
RECOMMENDATION
Use 1 vCPU: 191 s for $0.0030 per run, within 10% of the 1 vCPU time (191 s)
at 100% of its cost. GitHub ubuntu-latest median is 122 s (free on public repos; private-rate reference $0.030/run).
The job is not CPU-bound: 191 s at 1 vCPU versus 197 s at 2 vCPU, so 1 vCPU is the size to use at
$0.003/run. The 4 and 8 vCPU runs hit the repository's own flaky
PTY test (tests/harness/submit-turn.test.ts) at the same point, so their totals are excluded from the
recommendation. See the per-step table in docs/crosstalk.md for the full log tail.
A second headline run against moazessam376-dev/Gym-App (typecheck, Node 22 shim, private repo) shows
the opposite, CPU-bound shape: a clean 67 s -> 48 s -> 45 s curve as vCPU increases, recommendation 2 vCPU
(48 s, $0.0012/run, 53% of the 4 vCPU cost, within 10% of its time); see docs/gym-app.md.
Jobs with services: or a container: refuse to run and exit 3:
$ solci run moazessam376-dev/Gym-App --job rls --cpu 2
error solci cannot run this job because service containers/Docker are not
available; solci runs steps natively in a microVM
$ echo $?
3Pass --cloud-browser to solci run or solci agent when a job uses Playwright (including
@playwright/test or pytest-playwright), Puppeteer, or browser-use. solci creates a Solari cloud Chrome
session for each browser test step, injects the CDP endpoint through environment variables and preloads, and
releases the session after the step. The checked out repository is never edited. npx playwright install
and playwright install steps are reported as skipped because Chromium is already provided by Solari. Cypress
is detected but continues to use its local browser, because Cypress cannot attach to this remote CDP session.
Chromium projects only. Firefox and WebKit projects will fail because the local browser install is skipped.
For a Playwright webServer config with a literal port or localhost url, solci maps
http://localhost:N and http://127.0.0.1:N to the sandbox preview URL. Use --expose-port N for a
server that is not declared in the Playwright config. This localhost mapping is best effort, and the cloud
browser cannot reach the sandbox's loopback address directly without it.
A cloud-browser session can drop after roughly 10 minutes. solci keeps the scope to one session per browser
test step and releases it in cleanup. Browser usage is billed separately at $0.10/hour on the Starter
plan; the terminal and Markdown reports show the session count, seconds, and cost for each CPU size.
A full report from a real cloud-browser run is in docs/cloud-browser-demo.md.
solci agent gathers the inspect evidence, measures the selected job across the requested CPU sizes,
and asks a brain to propose the smallest evidence-based edit to the selected workflow YAML.
The codex brain uses the local codex CLI with gpt-5.6-luna. The gemini brain uses the Gemini REST
API with GOOGLE_API_KEY and gemini-2.5-pro. Use --brain codex or --brain gemini to choose one.
Without --brain, solci prefers a codex binary on PATH, then Gemini when GOOGLE_API_KEY is set.
solci agent moazessam376-dev/crosstalk --job test --cpu 1,2,4 --brain codex --effort medium --dry-runAdd --pr to create a branch, commit the workflow edit and curve chart, push the branch, and open a
GitHub pull request. --dry-run still clones the base branch so the brain sees the real workflow, but
skips branch, commit, push, and PR creation. The brain may return no change when the evidence does not
justify an edit.
--pr commits with your local git identity and pushes to origin under your gh login, so it needs push
access to the repository; there is no fork fallback.
For safety, the agent only ever edits the single selected file under .github/workflows. A human always
reviews and merges the pull request; solci never merges anything itself.
Real example: Gym-App PR #111 was opened by solci agent moazessam376-dev/Gym-App --job typecheck --cpu 1,2,4 --pr. From the 67/48/45 s curve and a 35% historical failure rate, the model added timeout-minutes: 15 and a concurrency group with cancel-in-progress, and nothing else. Gym-App is a private repository, so that link resolves only for its owner.
Public example: crosstalk PR #43 was opened by solci agent moazessam376-dev/crosstalk --job test --cpu 1,2 --pr. That run measured 196.2 s at 1 vCPU and 192.1 s at 2. An earlier run of the same job measured 191 s and 197 s: the two sizes swap places across runs and stay within 3% of each other, which is what a job that is not CPU-bound looks like. The model proposed timeout-minutes: 15 and nothing else. Full dry-run output is in docs/agent-demo.md.
If every sandbox size in the sweep fails (for example a transient 429), the agent refuses to propose a
change from history alone and exits with code 4. Pass --allow-history-only to proceed anyway.
solci run uses a mini-runner with one Solari microVM per requested size. The repository is cloned into
/work/repo; workflow run steps execute natively in the Linux VM. The first matrix value is substituted
when it is a literal list value (expression-valued axes, e.g. os: ${{ fromJSON(...) }}, are skipped), and
only one matrix cell is measured.
The action shims are:
actions/setup-pythonastral-sh/setup-uvactions/setup-nodepnpm/action-setupoven-sh/setup-bun
actions/checkout is handled by the runner's clone step. actions/cache, artifact upload/download,
codecov/*, and unsupported actions are skipped or no-op with a note. Each exec call is capped at 24
seconds; long steps use nohup, a log file, and polling. For sizes above one vCPU, the runner polls
nproc for CPU hot-plug completion with a bounded wait (typically 1-15 s, bounded at 40 s). Sandbox
cleanup is guaranteed unless --keep is explicitly supplied; cleanup failures are recorded in the result.
Per-step scripts export the step's own environment first and then source the shim environment file last,
so that tool paths installed by a shim (e.g. actions/setup-node's /opt/node/bin) correctly take
priority over the runner's baseline PATH instead of being clobbered by it.
- No Docker, job containers, or service containers; such jobs are reported as
SERVICES_UNSUPPORTEDandsolci runexits 3. - Linux microVMs only.
- One matrix cell only: the first literal value for each list-valued axis.
- Actions are shimmed/skipped as listed above; arbitrary third-party actions are not executed.
- The Starter plan runs at most 2 concurrent VMs by default.
- Live GitHub and Solari access is required for
inspect owner/repoandrun.
| Code | Severity | Meaning |
|---|---|---|
NO_CACHE_SETUP |
medium | A supported setup action does not enable dependency caching. |
UNPINNED_ACTION |
low | An action uses a mutable or missing ref. |
NO_TIMEOUT |
low | The job has no job-level timeout. |
NO_CONCURRENCY |
low | A pull-request workflow lacks a cancellation group. |
FULL_CLONE |
low | Checkout requests the complete repository history. |
SLOW_INSTALL_HINT |
info | pip install or non-lockfile npm install may be slower. |
HIGH_FAILURE_RATE |
high | The history baseline has a failure rate above 20%. |
BIG_RUNNER |
medium | The job requests a large -cores/vcpu runner (GitHub or Blacksmith). |
SERVICES_UNSUPPORTED |
info | The job uses Docker or service containers unavailable in Solari. |
MATRIX_NOTE |
info | Only the first matrix cell was measured. |
BROWSER_JOB |
info | Browser tests install a local browser; use --cloud-browser for Solari cloud Chrome. |
Facts learned running this tool against real jobs: the Solari exec API call itself takes up to roughly 28
seconds wall-clock, so solci clamps its own request timeout to 24 seconds and polls detached scripts via a
log/exit-file pattern instead of blocking on exec; vCPU hot-plug after boot is typically observed within
1-3 seconds and is bounded by a longer wait, up to 40 seconds, before the runner gives up waiting; a fresh
sandbox's HOME is unset by default (the runner and shims set it explicitly to /root where needed);
default memory is 2,048 MB or 1,024 MB per requested vCPU, whichever is larger; a base sandbox has roughly
2.2 GB of disk available before cloning a repository; and there is no Docker or container runtime available
inside a sandbox, which is why jobs with services:/container: are refused rather than attempted.
MIT. See LICENSE.
Author: Moaz Essam