Skip to content

Conversation

@mmcky
Copy link
Contributor

@mmcky mmcky commented Nov 11, 2025

This PR removes PyTorch and Pyro-PPL dependencies from all workflows and updates JAX to use CUDA 13 with the new AMI.

Changes Made:

Dependency Removal

  • Removed PyTorch (torch, torchvision, torchaudio) from all workflows
  • Removed Pyro-PPL from all workflows
  • Updated step names from "Install JAX, Numpyro, PyTorch" to "Install JAX and Numpyro"

JAX Configuration

  • Updated JAX installation to use jax[cuda13] for CUDA 13.0 support
  • Kept NumPyro for probabilistic programming needs
  • Added python scripts/test-jax-install.py to verify JAX installation

Infrastructure Updates

  • Updated .github/runs-on.yml with new AMI: ami-0edec81935264b6d3
  • Added region specification: us-west-2
  • New AMI includes CUDA 13.0 with Driver 580.105.08
  • Moved nvidia-smi check before JAX installation for better debugging

Workflow Improvements

  • Re-enabled build cache for faster CI runs
  • Applied changes consistently across ci.yml, cache.yml, and publish.yml

Technical Details:

  • Previous AMI: ami-09baf66e396fa7cfd (CUDA 12.9, Driver 575.51.03)
  • New AMI: ami-0edec81935264b6d3 (CUDA 13.0, Driver 580.105.08)
  • JAX Version: Latest with CUDA 13 support
  • Runner: RunsOn g4dn.2xlarge (Tesla T4 GPU)

Dependencies:

This PR requires PR #693 to be merged first, as it updates the runs-on.yml configuration on main (RunsOn reads this from the main branch).

- Remove torch, torchvision, torchaudio installation from all workflows
- Remove pyro-ppl installation from all workflows
- Update step names from 'Install JAX, Numpyro, PyTorch' to 'Install JAX and Numpyro'
- Disable build cache in ci.yml for full test run
- Keep JAX and NumPyro installations
@mmcky
Copy link
Contributor Author

mmcky commented Nov 11, 2025

  • pyro-ppl was used in bayes_nonconj but was replaced by jax.
They were used in Bayesian lectures like Non-conjugate prior:
https://python.quantecon.org/bayes_nonconj.html

thanks @HumphreyYang

@github-actions
Copy link

📖 Netlify Preview Ready!

Preview URL: https://pr-689--sunny-cactus-210e3e.netlify.app (8c0d609)

@github-actions
Copy link

📖 Netlify Preview Ready!

Preview URL: https://pr-689--sunny-cactus-210e3e.netlify.app (90f6a6f)

Changes jax[cuda12-local] to jax[cuda12_pip] to avoid cuDNN version
compatibility issues. The cuda12_pip variant includes compatible
CUDA and cuDNN libraries bundled with JAX, preventing runtime errors
from mismatched local CUDA installations.

Fixes cuDNN 9.10.0 backward-compatibility error.
@github-actions
Copy link

📖 Netlify Preview Ready!

Preview URL: https://pr-689--sunny-cactus-210e3e.netlify.app (f652c69)

@mmcky
Copy link
Contributor Author

mmcky commented Nov 11, 2025

@jstac I am going to rebuild the ubuntu image tomorrow morning

I am currently getting

image

so still issues with stability in CUDANN driver versions.

Would you like the lectures live for tomorrow? Let me know and I will get up early.

mmcky and others added 9 commits November 12, 2025 11:12
Updates the quantecon_ubuntu2404 AMI from ami-09baf66e396fa7cfd
to ami-0edec81935264b6d3 which includes the latest CUDA and cuDNN
libraries for improved compatibility with JAX.
Changes all workflows to use jax[cuda12-local] to leverage the CUDA
and cuDNN libraries pre-installed in the new AMI (ami-0edec81935264b6d3).
This is faster than cuda12_pip and uses the system libraries.

Also removes version pin (==0.6.2) from cache.yml and publish.yml
to make all workflows consistent.
Changes all workflows from cuda12-local to cuda13-local to match
the CUDA 13 installation in the new AMI (ami-0edec81935264b6d3).
Changes from 'jax[cuda13-local]' to 'jax[cuda13]' following the
official JAX documentation for CUDA 13 support. Also uses -U flag
instead of --upgrade for consistency with JAX docs.
@github-actions
Copy link

📖 Netlify Preview Ready!

Preview URL: https://pr-689--sunny-cactus-210e3e.netlify.app (4d9f83b)

@jstac
Copy link
Contributor

jstac commented Nov 12, 2025

🥳

@mmcky
Copy link
Contributor Author

mmcky commented Nov 12, 2025

@jstac merging this and will rebase your PRs.

@mmcky mmcky merged commit 065fb58 into main Nov 12, 2025
1 check passed
@mmcky mmcky deleted the remove-pytorch-pyro branch November 12, 2025 06:44
@github-actions
Copy link

📖 Netlify Preview Ready!

Preview URL: https://pr-689--sunny-cactus-210e3e.netlify.app (69ba280)

@github-actions
Copy link

📖 Netlify Preview Ready!

Preview URL: https://pr-689--sunny-cactus-210e3e.netlify.app (77f0c62)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants