enhance `Conda` easyblock: add support for specifying list of conda packages and Python version to install + using mambaBetter conda support #2992

xerbalind · 2023-09-01T15:31:19Z

No description provided.

hound · 2023-09-01T15:31:22Z

easybuild/easyblocks/generic/conda.py

@@ -88,12 +127,24 @@ def install_step(self):
            cmd = "%s conda create --force -y -p %s %s" % (self.cfg['preinstallopts'], self.installdir, install_args)
            run_cmd(cmd, log_all=True, simple=True)

+
+    def det_python_version(self):


too many blank lines (2)

hound · 2023-09-01T15:31:22Z

easybuild/easyblocks/generic/conda.py

+                self.log.info("Installed conda requirements")
+
+            run_cmd("%s install -p %s --no-deps %s" % (conda_cmd,self.installdir,install_args) ,log_all=True,simple=True)
+            run_cmd("ln -sf $(which python) %s" % (os.path.join(self.installdir,"bin","python")))


missing whitespace after ','

hound · 2023-09-01T15:31:22Z

easybuild/easyblocks/generic/conda.py

+
+                self.log.info("Installed conda requirements")
+
+            run_cmd("%s install -p %s --no-deps %s" % (conda_cmd,self.installdir,install_args) ,log_all=True,simple=True)


line too long (121 > 120 characters)
missing whitespace after ','
whitespace before ','

hound · 2023-09-01T15:31:22Z

easybuild/easyblocks/generic/conda.py

+            run_cmd(cmd, log_all=True, simple=True)
+
+            install_args = ""
+            if isinstance(self.src,list):


missing whitespace after ','

hound · 2023-09-01T15:31:22Z

easybuild/easyblocks/generic/conda.py

+
+            python_version = self.cfg['python_version'] or f"python{self.det_python_version()}"
+
+            cmd = "%s %s create --force -y -p %s python=%s --no-default-packages --no-deps" % (self.cfg['preinstallopts'], conda_cmd, self.installdir,python_version)


line too long (165 > 120 characters)
missing whitespace after ','

hound · 2023-09-01T15:31:23Z

easybuild/easyblocks/generic/conda.py

+                    'source_urls': [base_url + 'noarch',base_url + 'linux-64']
+                })
+
+            self.cfg.update('sources',sources)


missing whitespace after ','

hound · 2023-09-01T15:31:23Z

easybuild/easyblocks/generic/conda.py

+                base_url =  f"https://conda.anaconda.org/{package['channel']}/"
+                sources.append({
+                    'filename': package['fn'],
+                    'source_urls': [base_url + 'noarch',base_url + 'linux-64']


missing whitespace after ','

hound · 2023-09-01T15:31:23Z

easybuild/easyblocks/generic/conda.py

+
+            sources = []
+            for package in self.cfg['conda_packages']:
+                base_url =  f"https://conda.anaconda.org/{package['channel']}/"


multiple spaces after operator

hound · 2023-09-01T15:31:23Z

easybuild/easyblocks/generic/conda.py

@@ -39,6 +39,22 @@
 class Conda(Binary):
    """Support for installing software using 'conda'."""

+    def __init__(self,*args,**kwargs):
+        super(Conda,self).__init__(*args,**kwargs)


missing whitespace after ','

hound · 2023-09-01T15:31:23Z

easybuild/easyblocks/generic/conda.py

@@ -39,6 +39,22 @@
 class Conda(Binary):
    """Support for installing software using 'conda'."""

+    def __init__(self,*args,**kwargs):


missing whitespace after ','

ocaisa · 2023-09-06T10:13:34Z

I think we may need some additional changes in the final module file for a conda package. We should consider:

Should each conda package should have a family attached so that you cannot have multiple loaded at once? In the current approach if you have two loaded, do they interfere with each other (since the second module will overwrite CONDA_ENV/CONDA_PREFIX/CONDA_DEFAULT_ENV , see here)?
Should we conflict with GCCcore? This approach was suggested by @branfosj so that a conda installation is never used as a dependency (since it is extremely likely to interfere with the build)
Currently the final module sets PKG_CONFIG_PATH and I think this might be a mistake as packages may pick up on whatever this contains.

boegel · 2023-09-06T19:14:40Z

I'll look into cleaning this up, and making sure the changes are backwards compatible.

@xerbalind was working in the HPC-UGent team as summer intern in Aug'23, and opened this WIP PR on my request to ensure his efforts were not lost.

ocaisa · 2023-09-08T10:24:43Z

@boegel There are some straightforward improvements to the conda easyblock in #2996 (which also covers the mamba use case in a more generic way) that I think I will merge once I've added some test reports. Since you'll take over here I think it'll be ok to do that and then you can integrate that into this branch.

Update conda block

4b7aefb

hound bot reviewed Sep 1, 2023

View reviewed changes

boegel added the enhancement label Sep 6, 2023

boegel added this to the 4.x milestone Sep 6, 2023

boegel self-assigned this Sep 6, 2023

boegel changed the title ~~Better conda support~~ enhance Conda easyblock: add support for specifying list of conda packages and Python version to install + using mambaBetter conda support Sep 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

enhance `Conda` easyblock: add support for specifying list of conda packages and Python version to install + using mambaBetter conda support #2992

enhance `Conda` easyblock: add support for specifying list of conda packages and Python version to install + using mambaBetter conda support #2992

xerbalind commented Sep 1, 2023

hound bot Sep 1, 2023

hound bot Sep 1, 2023

hound bot Sep 1, 2023

hound bot Sep 1, 2023

hound bot Sep 1, 2023

hound bot Sep 1, 2023

hound bot Sep 1, 2023

hound bot Sep 1, 2023

hound bot Sep 1, 2023

hound bot Sep 1, 2023

ocaisa commented Sep 6, 2023

boegel commented Sep 6, 2023 •

edited

ocaisa commented Sep 8, 2023


		self.log.info("Installed conda requirements")

		run_cmd("%s install -p %s --no-deps %s" % (conda_cmd,self.installdir,install_args) ,log_all=True,simple=True)


		python_version = self.cfg['python_version'] or f"python{self.det_python_version()}"

		cmd = "%s %s create --force -y -p %s python=%s --no-default-packages --no-deps" % (self.cfg['preinstallopts'], conda_cmd, self.installdir,python_version)

enhance Conda easyblock: add support for specifying list of conda packages and Python version to install + using mambaBetter conda support #2992

Are you sure you want to change the base?

enhance Conda easyblock: add support for specifying list of conda packages and Python version to install + using mambaBetter conda support #2992

Conversation

xerbalind commented Sep 1, 2023

hound bot Sep 1, 2023

Choose a reason for hiding this comment

hound bot Sep 1, 2023

Choose a reason for hiding this comment

hound bot Sep 1, 2023

Choose a reason for hiding this comment

hound bot Sep 1, 2023

Choose a reason for hiding this comment

hound bot Sep 1, 2023

Choose a reason for hiding this comment

hound bot Sep 1, 2023

Choose a reason for hiding this comment

hound bot Sep 1, 2023

Choose a reason for hiding this comment

hound bot Sep 1, 2023

Choose a reason for hiding this comment

hound bot Sep 1, 2023

Choose a reason for hiding this comment

hound bot Sep 1, 2023

Choose a reason for hiding this comment

ocaisa commented Sep 6, 2023

boegel commented Sep 6, 2023 • edited

ocaisa commented Sep 8, 2023

enhance `Conda` easyblock: add support for specifying list of conda packages and Python version to install + using mambaBetter conda support #2992

enhance `Conda` easyblock: add support for specifying list of conda packages and Python version to install + using mambaBetter conda support #2992

boegel commented Sep 6, 2023 •

edited