Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker container experimental - run a full env solve with mamba instead of a conda update #2306

Merged
merged 2 commits into from
Sep 23, 2021

Conversation

valeriupredoi
Copy link
Contributor

@valeriupredoi valeriupredoi commented Sep 13, 2021

Description

mamba env update is still risky (have seen issues as of lately) and it chugs a lot of memory (I can't even run it on me 4GB laptop), and conda env update either can't solve (as we have seen in the latest build fails on DockerHub) or it takes agessss (which may result in a fail to solve anyway) - creating the env with mamba is short and sweet, do we really need the env updater in or we can get away with a quick solve from scratch @bouweandela ?


Before you get started

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 馃洜 Technical or 馃И Scientific review.

@valeriupredoi
Copy link
Contributor Author

OK the local build fails with this PR too, with the same bugger as the DH one:

Step 8/11 : RUN Rscript ./esmvaltool/install/R/setup.R
 ---> Running in c170d17b5b56
ERROR conda.cli.main_run:execute(33): Subprocess for 'conda run ['/bin/bash', '-c', 'Rscript ./esmvaltool/install/R/setup.R']' command failed.  (See above for error)
/bin/bash: Rscript: command not found

so it's independent of the env build method (but I'd still keep the mamba env creation - super fast!). Looking at what's going one now...

@zklaus
Copy link
Contributor

zklaus commented Sep 14, 2021

Note that the setup.R thing is no longer needed.

@valeriupredoi
Copy link
Contributor Author

yes! was just Mark I eyeballing the script, good catch Klaus 馃嵑

@valeriupredoi
Copy link
Contributor Author

OK I fixed it 馃コ

Successfully built ESMValCore
Installing collected packages: ESMValCore
  Attempting uninstall: ESMValCore
    Found existing installation: ESMValCore 2.3.1
    Uninstalling ESMValCore-2.3.1:
      Successfully uninstalled ESMValCore-2.3.1
Successfully installed ESMValCore-2.3.1
Files removed: 224

Removing intermediate container 476c596fc6bb
 ---> 1070d72f9d60
Step 13/13 : ENTRYPOINT ["conda", "run", "--name", "esmvaltool", "esmvaltool"]
 ---> Running in c29c98cda5ba
Removing intermediate container c29c98cda5ba
 ---> 4356db36ef1b
Successfully built 4356db36ef1b
Successfully tagged esmvaltool:experimental

a positive result from a local build, and it's fast too, in under 40min, @bouweandela pls have a look, I've made some substantial changes to the .exp file 馃嵑

@valeriupredoi
Copy link
Contributor Author

valeriupredoi commented Sep 14, 2021

also sudo docker run -e HOME -v "$HOME":"$HOME" -v /data:/data esmvaltool:experimental run $HOME/ESMValTool/esmvaltool/recipes/examples/recipe_r.yml and sudo docker run -e HOME -v "$HOME":"$HOME" -v /data:/data esmvaltool:experimental run $HOME/ESMValTool/esmvaltool/recipes/examples/recipe_python.yml run nicely, the _ncl.yml one is not since NCL is not found - does NCL need to be in the environment too? _julia.yml dies with the now customary error we have seen across - no biggie, since we'll fix Julia 馃榿

ERROR: LoadError: ArgumentError: Package StringEncodings [69024149-9ee7-55f6-a4c4-859efe599b68] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

Stacktrace:
 [1] _require(::Base.PkgId) at ./loading.jl:929
 [2] require(::Base.PkgId) at ./loading.jl:858
 [3] require(::Module, ::Symbol) at ./loading.jl:853
 [4] include at ./boot.jl:317 [inlined]
 [5] include_relative(::Module, ::String) at ./loading.jl:1044
 [6] include(::Module, ::String) at ./sysimg.jl:29
 [7] top-level scope at none:2
 [8] eval at ./boot.jl:319 [inlined]
 [9] eval(::Expr) at ./client.jl:393
 [10] top-level scope at ./none:3
in expression starting at /home/valeriu/.julia/packages/YAML/IXUQ4/src/YAML.jl:11
ERROR: LoadError: Failed to precompile YAML [ddb6d928-2868-570f-bddf-ab3f9cf99eb6] to /home/valeriu/.julia/compiled/v1.0/YAML/LPCgu.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1203
 [3] _require(::Base.PkgId) at ./loading.jl:960
 [4] require(::Base.PkgId) at ./loading.jl:858
 [5] require(::Module, ::Symbol) at ./loading.jl:853
 [6] include at ./boot.jl:317 [inlined]
 [7] include_relative(::Module, ::String) at ./loading.jl:1044
 [8] include(::Module, ::String) at ./sysimg.jl:29
 [9] exec_options(::Base.JLOptions) at ./client.jl:266
 [10] _start() at ./client.jl:425
in expression starting at /src/ESMValTool/esmvaltool/diag_scripts/examples/diagnostic.jl:13

@bouweandela
Copy link
Member

@jvegasbsc Could you please review this? Because you created the original container?

@bouweandela
Copy link
Member

do we really need the env updater in or we can get away with a quick solve from scratch @bouweandela ?

I think this container is a bit broken by design, because the recipes in this repository are written to work with the latest released version of ESMValCore, while this container tries to build on top of the development version of ESMValCore. That may or may not be a real problem, but if there are changes to the ESMValCore environment you may run into trouble just creating a new environment from scratch. Having said that, I think things will probably work just fine most of the time.

@valeriupredoi
Copy link
Contributor Author

right! I asked that question yesterday when I knew less about this thing than today 馃榿 The problem is that an update via conda is slow as a slow train and I don't trust mamba update yet; maybe when mamba will be even more secure we can do that 馃憤

@valeriupredoi
Copy link
Contributor Author

absolutely friendly ping, no pressure at all, @jvegasbsc 馃槅

@jvegreg
Copy link
Contributor

jvegreg commented Sep 23, 2021

Environment changes in the core can be circumvent by adding the new dependencies to the tool, so they are not a dead end at least. I am ok with a more reliable build that may need extra tweaks from time to time if we add new conda dependencies to the core (as pip ones should be fine).

I am testing the build locally to give the green light for the merge. Please, be patient

@jvegreg
Copy link
Contributor

jvegreg commented Sep 23, 2021

Done! It looks fine, but it also makes me appreciate more #2299

@jvegreg jvegreg merged commit ea97c9a into main Sep 23, 2021
@jvegreg jvegreg deleted the fix_exp_container branch September 23, 2021 09:01
@valeriupredoi
Copy link
Contributor Author

cheers much @jvegasbsc 馃嵑

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants