Skip to content

ML CI: update image, build TF/JAX with Clang#39

Merged
adamjstewart merged 22 commits intospack:developfrom
adamjstewart:ci/ml
Oct 8, 2025
Merged

ML CI: update image, build TF/JAX with Clang#39
adamjstewart merged 22 commits intospack:developfrom
adamjstewart:ci/ml

Conversation

@adamjstewart
Copy link
Member

@adamjstewart adamjstewart commented Jun 6, 2025

Updates to new image with LLVM installed.

Builds Google stuff (TF/JAX) with Clang and Meta stuff (PyTorch) with GCC.

Except TF + aarch64, which seems to build better with GCC for some reason 🤷

@adamjstewart

This comment was marked as outdated.

Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
@adamjstewart adamjstewart changed the title [WIP] ML CI: update image ML CI: update image Jul 25, 2025
Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
@github-actions github-actions bot added the ci Issues related to Continuous Integration label Sep 15, 2025
@adamjstewart adamjstewart changed the title ML CI: update image ML CI: update image, build TF/JAX with LLVM Sep 15, 2025
@adamjstewart
Copy link
Member Author

@spackbot run pipeline

@spackbot-app
Copy link

spackbot-app bot commented Sep 16, 2025

I've started that pipeline for you!

Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
@adamjstewart adamjstewart changed the title ML CI: update image, build TF/JAX with LLVM ML CI: update image, build TF/JAX with Clang Sep 17, 2025
@adamjstewart
Copy link
Member Author

For the sake of debugging, the current specs that fail to concretize within the time limit:

  • ml-linux-aarch64-cuda-generate: py-jaxlib, py-keras backend=jax
  • ml-linux-x86_64-cpu-generate: py-jaxlib
  • ml-linux-x86_64-cuda-generate: py-jax, py-jaxlib
  • ml-linux-x86_64-rocm-generate: py-jax

So it's always jax, but not always the same jax. Note that py-keras backend=jax depends on py-jax, and py-jax depends on py-jaxlib, so there's no reason it should be able to concretize one and not the other.

@adamjstewart
Copy link
Member Author

adamjstewart commented Sep 29, 2025

Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
Signed-off-by: Adam J. Stewart <ajstewart426@gmail.com>
@adamjstewart adamjstewart merged commit c414362 into spack:develop Oct 8, 2025
17 checks passed
@adamjstewart adamjstewart deleted the ci/ml branch October 8, 2025 14:45
mkrack pushed a commit to mkrack/spack-packages that referenced this pull request Dec 23, 2025
climbfuji added a commit to climbfuji/spack-packages that referenced this pull request Feb 17, 2026
1. Add crtm@3.1.3
2. Add logic to run a basic test as part of spack install --test=root crtm; running other tests requires fixing crtm-fix: added logic to move all fixed files into flat directory structure incompatible with crtm package spack-stack#1910 (after spack-stack-2.1.0)

I plan to keep these changes in our fork/branch of spack-packages until JCSDA/spack-stack#1910 is resolved and crtm@3.2.0 is released and added. Version 3.1.3 is a patch release specifically created for us for testing with ifx.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Issues related to Continuous Integration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants