'gcc' module required by 'slurm-drmaa' results in theano BLAS warning

[jhrcook]

The slurm-drmaa module is required for Snakemake to use O2. The slurm-drmaa module requires the gcc module. Loading the gcc module results in the theano BLAS warning.

[jhrcook]

Without gcc:

Python 3.9.9 | packaged by conda-forge | (main, Dec 20 2021, 02:41:03)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import theano
>>> theano.config.cxx
'/home/jc604/.conda/envs/speclet/bin/g++'
>>> theano.config.blas__ldflags
'-L/home/jc604/.conda/envs/speclet/lib -lcblas -lblas -lcblas -lblas'
>>> theano.config.gcc__cxxflags
''
>>>

With gcc:

Python 3.9.9 | packaged by conda-forge | (main, Dec 20 2021, 02:41:03)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import thaeano
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'thaeano'
>>> import theano
WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
>>> theano.config.cxx
'/n/app/gcc/6.2.0/bin/g++'
>>> theano.config.blas__ldflags
''
>>> theano.config.gcc__cxxflags
''
>>>

[jhrcook]

For reference, pymc-theano (Aesara) configuration: https://aesara.readthedocs.io/en/latest/library/config.html

Setting the flags manually removes the BLAS warning even if O2's gcc module is loaded. (I tested against the control without setting the flags and warning reappears.)

module load gcc
MY_THEANO_FLAGS="cxx='/home/jc604/.conda/envs/speclet/bin/g++',blas__ldflags='-L/home/jc604/.conda/envs/speclet/lib -lcblas -lblas -lcblas -lblas'"
THEANO_FLAGS=$MY_THEANO_FLAGS speclet/command_line_interfaces/fit_bayesian_model_cli.py  \
  "hierarchical-nb" \
  models/model-configs.yaml \
  PYMC3_MCMC  \
  temp/fitting-mcmc \
  --mcmc-chains 1  \
  --mcmc-cores 1  \
  --cache-name='test-run'

(Edit: misspelled "FLAGS" in MY_THEANO_FLAGS.)

[jhrcook]

Solution: Add this flag to the current speclet.pipelines.theano_flags.get_theano_flags() function.

Steps:

• Add this flag to the current speclet.pipelines.theano_flags.get_theano_flags() function.
• Add a project configuration parameter for the contents of this additional flag.
• Document this configuration parameter in the README.

[jhrcook]

I made a mistake: The new flags need to use the correct conda environment. Snakemake makes a new environment for the pipeline, so I cannot set it as the prefix for speclet. I need to use the global environment variable CONDA_PREFIX.

❯ conda activate speclet
❯ echo $CONDA_PREFIX
/home/jc604/.conda/envs/speclet