Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there a way to deal with initializing conda correctly across multiple shells? #434

Closed
rdauria opened this issue Jan 29, 2022 · 8 comments
Milestone

Comments

@rdauria
Copy link

rdauria commented Jan 29, 2022

As this is perhaps a feature request I will chance posting it here and not on https://sourceforge.net/projects/modules/lists/modules-interest.

When creating a module file for anaconda invariably we need to request users to also source a conda shell script that varies according to the user shell, i.e.:

# SH bases shells:
module load anconda
source $CONDA_DIR/etc/profile.d/conda.sh

or:

# CSH based shells:
module load anaconda
source $CONDA_DIR/etc/profile.d/conda.csh

since the file to be sourced are different enough using either sh-to-mod or source-sh would require that the modulefile first check on the user SHELL and branch out accordingly. Is this possible?

Thanks!

@xdelaruelle
Copy link
Member

module-info shelltype helps to know the current shell type that module command has been initialized to.

So you may write your modulefile the following way:

switch -- [module-info shelltype] {
  sh {
    source-sh sh $CONDA_DIR/etc/profile.d/conda.sh
  }
  csh {
    source-sh csh $CONDA_DIR/etc/profile.d/conda.csh
  }
  default {
    error {Shell not supported}
  }
}

@rdauria
Copy link
Author

rdauria commented Jan 31, 2022

Thanks so much for the suggestion. I wanted to point out a strange behavior, when I implemented the module with:

csh {
   source-sh csh $CONDA_DIR/etc/profile.d/conda.csh
}

I would invariably get:

$ conda env list
Badly placed ()'s.

the infuriating thing is that I could see that the conda alias was set correctly by the modulefile, namely:

conda	(source /u/local/apps/anaconda3/2020.11/etc/profile.d/conda.csh)

which is exactly what sourcing the file:

/u/local/apps/anaconda3/2020.11/etc/profile.d/conda.csh

sets it to (but sourcing the file outside of a modulefile does not give the Badly placed ()'s. error).

In the end the only way I could find to make the modulefile work (and the issue of the Badly placed ()'s. go away) was to update the portion of the csh modulefile as follows:

csh {
    #source-sh csh $base_dir/etc/profile.d/conda.csh
    set-alias       conda {source /u/local/apps/anaconda3/2020.11/etc/profile.d/conda.csh}
  }

Any idea why this may be so? In any event I decided to post the trouble and its solution should other people experience the same issue.

@rdauria
Copy link
Author

rdauria commented Feb 1, 2022

One more question, would you know how can I go about debugging this error which appears upon logging in from a zsh shell?

_module_raw:122: parse error near `()'

The error appears only at log-in and it prevents certain default modules from being loaded. Once the login is finished, however, the user can load the modules with no further error.

We are running env mod version 4.7.0 and gcc 4.8.5 on a CentOS 7 cluster.

Also do you know if there is a way to have a zsh not to give:

zsh compinit: insecure directories and files, run compaudit for list.
Ignore insecure directories and files and continue [y] or abort compinit [n]?

compaudit gives:

There are insecure directories and files:
/u/local/Modules/4.7.0/gcc-4.8.5/init/zsh-functions
/u/local/Modules/4.7.0/gcc-4.8.5/init
/u/local/Modules/4.7.0/gcc-4.8.5/init/zsh-functions/_module

but on a close inspection I am not sure what is it that zsh complaints of being insecure as:

% ls -tlr /u/local/Modules/4.7.0/gcc-4.8.5/init
total 108
-rw-r--r--. 1 instuser instgroup  2734 Mar 19  2021 sh
-rw-r--r--. 1 instuser instgroup  2513 Mar 19  2021 ksh
-rw-r--r--. 1 instuser instgroup  2884 Mar 19  2021 bash
-rw-r--r--. 1 instuser instgroup  2754 Mar 19  2021 zsh
-rw-r--r--. 1 instuser instgroup   424 Mar 19  2021 profile.sh
-rw-r--r--. 1 instuser instgroup 10135 Mar 19  2021 bash_completion
-rw-r--r--. 1 instuser instgroup  1418 Mar 19  2021 csh
-rw-r--r--. 1 instuser instgroup  1419 Mar 19  2021 tcsh
-rw-r--r--. 1 instuser instgroup  5981 Mar 19  2021 tcsh_completion
-rw-r--r--. 1 instuser instgroup   133 Mar 19  2021 profile.csh
-rw-r--r--. 1 instuser instgroup   725 Mar 19  2021 perl.pm
-rw-r--r--. 1 instuser instgroup   981 Mar 19  2021 python.py
-rw-r--r--. 1 instuser instgroup   836 Mar 19  2021 ruby.rb
-rw-r--r--. 1 instuser instgroup  3258 Mar 19  2021 lisp
-rw-r--r--. 1 instuser instgroup  1070 Mar 19  2021 tcl
-rw-r--r--. 1 instuser instgroup  1771 Mar 19  2021 fish
-rw-r--r--. 1 instuser instgroup  1193 Mar 19  2021 cmake
-rw-r--r--. 1 instuser instgroup   960 Mar 19  2021 r.R
-rw-r--r--. 1 instuser instgroup 10217 Mar 19  2021 fish_completion
drwxr-xr-x. 2 instuser instgroup  4096 Mar 19  2021 zsh-functions
drwxr-xr-x. 2 instuser instgroup  4096 Mar 19  2021 ksh-functions
-rw-r--r--. 1 instuser instgroup   409 Mar 19  2021 modulerc
lrwxrwxrwx. 1 instuser instgroup    10 Mar 23  2021 modules.sh -> profile.sh
lrwxrwxrwx. 1 instuser instgroup    11 Mar 23  2021 modules.csh -> profile.csh

and:

% ls -tlr /u/local/Modules/4.7.0/gcc-4.8.5/init/zsh-functions 
total 16
-rw-r--r--. 1 instuser instgroup 13010 Mar 19  2021 _module

Do you happen to have any idea?

Thanks :)

@xdelaruelle
Copy link
Member

One more question, would you know how can I go about debugging this error which appears upon logging in from a zsh shell?

_module_raw:122: parse error near `()'

The error appears only at log-in and it prevents certain default modules from being loaded. Once the login is finished, however, the user can load the modules with no further error.

I suggest you add a set +x or set +xv command in the zsh initialization file of Modules to get debugging information.

@xdelaruelle
Copy link
Member

Also do you know if there is a way to have a zsh not to give:

zsh compinit: insecure directories and files, run compaudit for list.
Ignore insecure directories and files and continue [y] or abort compinit [n]?

It seems like compinit expect files to be owned by root: https://stackoverflow.com/questions/13762280/zsh-compinit-insecure-directories

@rdauria
Copy link
Author

rdauria commented Feb 2, 2022

Thanks a million for the suggestions!!! Indeed the zsh shell initialization gets stuck as it tries to load a modulefile for the intel compiler which is basically doing:

source-sh bash $base_dir/compilers_and_libraries_2020/linux/bin/compilervars.sh intel64

if I write the modulefile without sourcing the intel initialization script (but setting all the variables "by hand") the problem goes away. I think the issue is with some of the bash functions that the intel script sets up (I could send you more info in case you are interested).

The interesting fact is that the error only occurs when /bin/zsh is run (we place a file in /etc/profile.d that will load a set of default modulefiles for all of our users), if a user with a zsh shell loads the intel module after logging in then there is no error. At this point I think I will just rewrite the intel modulefile without using the source-sh functionality.

@xdelaruelle
Copy link
Member

Thanks a million for the suggestions!!! Indeed the zsh shell initialization gets stuck as it tries to load a modulefile for the intel compiler which is basically doing:

source-sh bash $base_dir/compilers_and_libraries_2020/linux/bin/compilervars.sh intel64

I suggest you adapt the code to use the current running shell rather bash in order to use zsh to source-sh the intel script:

source-sh zsh $base_dir/compilers_and_libraries_2020/linux/bin/compilervars.sh intel64

@xdelaruelle
Copy link
Member

Issue found on csh shell with Conda should be fixed. Other fixes have been added to improve support of shell function on sh-kind shells.

It would be great if you could test this new Modules version (current master git branch) and do not hesitate to open new issues if you still encounter errors.

@xdelaruelle xdelaruelle added this to the 5.1 milestone Feb 26, 2022
@xdelaruelle xdelaruelle added the bug label Mar 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants