Repository navigation
Highlights
- NERSC Perlmutter A100 training support — VAE, LDM, and broadband training all work on Perlmutter
- Perlmutter benchmark results added to
docs/BENCHMARKS.md - New
notebooks/directory with three analysis/presentation notebooks - Broadband
--data-path+--wandbcompatibility fixes
Added
- NERSC Perlmutter A100 training support — enables VAE, LDM, and broadband training on Perlmutter
scripts/nersc_train_vae.py— standalone VAE training script withjax.lax.dynamic_sliceworkaround for a CUDA graph capture bug on Perlmutter A100sscripts/prepare_training_data.py— reproducible data prep for training (quality filters + metadata enrichment:ECLIPSE_FRAC,SOLFLUX, galactic/ecliptic coords, broadband aliases)jobs/— SLURM job scripts for VAE, LDM (dark/moon/twilight), and broadband training on NERSCdocs/NERSC_SETUP.md— training section documenting required XLA flags (--xla_gpu_autotune_level=0) and when the NERSC VAE script is needed
- Perlmutter benchmark results in
docs/BENCHMARKS.md— CPU and A100 GPU timings at default and 10K sample sizes - Analysis & research notebooks in a new
notebooks/directory (separate fromexamples/, which hosts user-facing tutorials):notebooks/investigate_outlier_spectra.ipynb— investigation of anomalous LDM-generated spectranotebooks/specsim_polar_comparison.ipynb— desisky vs specsim polar comparisonnotebooks/desi_data_pres.ipynb— DESI data systems presentation companion
Fixed
- Broadband
--data-path+--wandbcompatibility —gather_full_datapreviously assumedSkyBrightnessDatasetinternals; now works withTensorDatasetloaded from user data paths - Broadband
_save_checkpoint— added fallback for datasets without.metadata(e.g.TensorDatasetfrom--data-path) - Duplicate quality filter messages when loading the DESI wavelength grid
Install
bash pip install --upgrade desisky
Full changelog
See CHANGELOG.md for the complete entry.