Skip to content

Save warp cache for CI - #6707

Open
StafaH wants to merge 5 commits into
isaac-sim:developfrom
StafaH:mh/ci-warp-cache
Open

Save warp cache for CI#6707
StafaH wants to merge 5 commits into
isaac-sim:developfrom
StafaH:mh/ci-warp-cache

Conversation

@StafaH

@StafaH StafaH commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Description

Save warp cache post-merge so future CI can run warm.

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@greptile-apps

greptile-apps Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR wires up a Warp kernel cache across the CI pipeline so that the expensive per-job kernel compilation happens once post-merge and is reused by subsequent PR runs. It introduces two new composite-action inputs (warp-cache-family and warp-cache-write), an nvidia-smi–driven GPU identity step, GitHub Actions cache restore/save bookends, and a Python fail-fast check that confirms the container actually honours WARP_CACHE_PATH before tests start.

  • run-package-tests: resolves GPU compute capability with sort -u | paste -sd- (multi-GPU safe), restores the Warp cache before the test container starts, saves it on success() only when warp-cache-write == 'true' and there was no exact cache hit.
  • run-tests: accepts warp-cache-host-dir as the 22nd positional argument, bind-mounts it into the container as /tmp/isaaclab-warp-cache:rw, sets WARP_CACHE_PATH, and validates the path via wp.config.kernel_cache_dir — surfacing any env-var naming mismatch as an immediate job failure.
  • build.yaml: removes the github.event_name != 'push' guard from the tasks, core, and newton jobs (enabling post-merge runs), assigns a unique warp-cache-family per shard, and gates warp-cache-write on push or workflow_dispatch events only.

Confidence Score: 5/5

Safe to merge — all three files are CI-infrastructure only, the cache is gated behind success() so broken test runs never pollute the cache, and the Python sanity check turns any WARP_CACHE_PATH misconfiguration into an immediate, visible job failure.

The logic is tight: the GPU identity step covers multi-GPU runners via sort-unique join; the save condition guards on warp-cache-family, warp-cache-write, success(), and cache-hit; and the in-container path validation catches any env-var naming drift at job startup rather than silently writing to the wrong location. The only observable change to production behavior is that the tasks/core/newton jobs now also run on push events — intentional for cache warming.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename Overview
.github/actions/run-package-tests/action.yml Adds warp-cache-family/warp-cache-write inputs, GPU-capability identity step (sort -u
.github/actions/run-tests/action.yml Accepts warp-cache-host-dir as positional arg 22, mounts the dir into the container as /tmp/isaaclab-warp-cache:rw, sets WARP_CACHE_PATH, and adds a Python fail-fast sanity check via wp.config.kernel_cache_dir
.github/workflows/build.yaml Removes github.event_name != 'push' guards from tasks/core/newton jobs, assigns per-shard warp-cache-family names, and enables warp-cache-write on push and workflow_dispatch events only

Sequence Diagram

sequenceDiagram
    participant WF as build.yaml
    participant PKG as run-package-tests
    participant GHC as actions/cache
    participant RT as run-tests
    participant CTR as Docker Container

    WF->>PKG: "warp-cache-family=isaaclab-tasks-0<br/>warp-cache-write=true (on push)"
    PKG->>PKG: "Resolve Warp cache identity<br/>(nvidia-smi → sort -u | paste -sd-)"
    PKG->>GHC: "cache/restore@v4<br/>key: warp-v1-{family}-{os}-{arch}-{gpu}-{isaacsim}-{hash}-{epoch}"
    GHC-->>PKG: "restored (cache-hit=true/false)"
    PKG->>PKG: "mkdir -p cache_dir && test -w"
    PKG->>RT: "warp-cache-host-dir=/tmp/runner/isaaclab-warp-cache/isaaclab-tasks-0"
    RT->>CTR: "docker run -v host_cache_dir:/tmp/isaaclab-warp-cache:rw<br/>-e WARP_CACHE_PATH=/tmp/isaaclab-warp-cache"
    CTR->>CTR: "wp.init() + verify wp.config.kernel_cache_dir == WARP_CACHE_PATH"
    CTR->>CTR: run pytest (compiles Warp kernels into /tmp/isaaclab-warp-cache)
    CTR-->>RT: exit code
    RT-->>PKG: result
    PKG->>PKG: Check Test Results (if always())
    alt "success() && warp-cache-write=='true' && cache-hit!='true'"
        PKG->>GHC: "cache/save@v4 (key = cache-primary-key)"
    end
Loading

Reviews (2): Last reviewed commit: "Merge branch 'develop' into mh/ci-warp-c..." | Re-trigger Greptile

Comment thread .github/actions/run-package-tests/action.yml
Comment thread .github/actions/run-package-tests/action.yml Outdated
Comment thread .github/actions/run-tests/action.yml
Comment thread .github/workflows/build.yaml
@StafaH

StafaH commented Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

@greptileai

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants