Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Force latest Python in empty environment in conda install CI test #2069

Merged
merged 8 commits into from Mar 16, 2021

Conversation

valeriupredoi
Copy link
Contributor

@valeriupredoi valeriupredoi commented Mar 4, 2021

Closes #2063

Currently testing on me machine, env still solving but at least memory doesn't go up the roof anymore and stays stable at ~1.6GB
UPDATE after some 12min the darn thing solves and the installation goes ahead!

I think I know why we were getting hit badly - the Python version that comes in the empty environment right before installing esmvaltool is Python=2.7 (really, conda?)

.circleci/config.yml Outdated Show resolved Hide resolved
@valeriupredoi
Copy link
Contributor Author

OK conda install straight from conda now works fine (both on my machine and on the CI machine ) and I have updated the documentation with the instructions -> @bouweandela please review when you haz time 🍺

@valeriupredoi
Copy link
Contributor Author

frenly ping @bouweandela to approve this so we can have us a working conda install test plese 🍺

@valeriupredoi
Copy link
Contributor Author

@bouweandela it'd be really nice if we got this in today pls man, I want to show that (among other things) Circle tests pass OK during my presentation at the CRESCENDO GA tomorrow - not a nice note if our README badge shows FAILED 😁

@bouweandela
Copy link
Member

bouweandela commented Mar 15, 2021

Sorry for being slow to respond @valeriupredoi. I'm not entirely happy about these changes, because it shows that it has (again) become incredibly hard to solve the conda environment for a normal installation. If you have a look at the conda build, you'll see that it's been taking about 2 hours recently. I think it would be better to create a new conda package that is easier to solve.

However, if you prefer proceeding like this, I would recommend being at least a bit less specific about the Python version, e.g. use 3.9 (instead of 3.9.2), because if you're that specific about the version we would need to update the installation instructions every time there is a new minor Python release.

By the way, I noticed that the conda build is also broken since Friday for both the ESMValCore and ESMValTool packages.

@valeriupredoi
Copy link
Contributor Author

not a worry, bud, sorry for rushing you :) Yes, I completely agree with changing the forced version to a more generic Python 3.9, let me just commit that. About creating a new package - how would that change the situation? We still have the same deps and dep relations, unless you think of breaking the package down in smaller packages? Let's just go with this for now since this fixes the situation (or, was fixing it at the time of commit, lemme try again) and talk about the next steps after? 🍺

@bouweandela
Copy link
Member

About creating a new package - how would that change the situation? We still have the same deps and dep relations, unless you think of breaking the package down in smaller packages?

No, we would just need to add some more lower bounds on acceptable versions of dependencies and hope that makes the environment easier to solve, as usual.

Let's just go with this for now since this fixes the situation (or, was fixing it at the time of commit, lemme try again) and talk about the next steps after?

It doesn't solve the problem, because the conda build is broken too (takes forever), so just fixing this won't make CircleCI look green again.

@valeriupredoi
Copy link
Contributor Author

It doesn't solve the problem, because the conda build is broken too (takes forever), so just fixing this won't make CircleCI look green again.

hahaha, communist Circle it is 😁 sooo we can temporarily turn off the conda build test until we fix it 👿

@valeriupredoi
Copy link
Contributor Author

No, we would just need to add some more lower bounds on acceptable versions of dependencies and hope that makes the environment easier to solve, as usual.

Right you are! But beats me if I have the patience to do that anymore man 😝

@bouweandela
Copy link
Member

Same here..

@bouweandela
Copy link
Member

I just updated the installation instructions a bit and changed the command on CircleCI so it looks the same, hope that's OK with you.

@valeriupredoi
Copy link
Contributor Author

looks good, dude! Just approve this once the tests have passed and we should be OK for now 👍

@valeriupredoi
Copy link
Contributor Author

cheers for the changes and approval @bouweandela - looks like conda build is hanging again so I'll merge this (even if it hangs forever, at least conda install now works). We'll have to look into the build again (sigh!) 👍

@valeriupredoi valeriupredoi merged commit 67f355c into master Mar 16, 2021
@valeriupredoi valeriupredoi deleted the fix_conda_install_CI_test branch March 16, 2021 10:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Installation straight from conda fails from running out of memory on the CI machine
2 participants