Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conda Virtual Environments? #167

Open
Erstwild opened this issue Jul 6, 2016 · 32 comments
Open

Conda Virtual Environments? #167

Erstwild opened this issue Jul 6, 2016 · 32 comments

Comments

@Erstwild
Copy link

@Erstwild Erstwild commented Jul 6, 2016

Great project, btw! I see myself making liberal use of this.

I just wanted to check and see if there are any issues with using conda virtual environments? I successfully create a conda virtual environment (just flask and its dependencies), activate the new environment, and then maneuver to my project directory and then "zappa deploy project" where I have the zappa_settings.json configured correctly the best I can tell.

conda create -n enviro python=2.7 flask
source activate enviro
zappa deploy project

but then...

Packaging project as zip...
Zappa requires an active virtual environment.

Also, unrelated, when specifying a domain should it be the same as the hostname in app.py? For example "www.abc.com". I have my domain up on Route 53.

@Miserlou
Copy link
Owner

@Miserlou Miserlou commented Jul 6, 2016

There is an unmerged pull-request that adds Conda support. I've neglected it for too long already, so this is a good opportunity to re-address it: #108

The main thing is that I personally just don't know enough about Conda to evaluate this PR. It also needs tests. Do you think you can take a look at it and let me know what you think?

@Erstwild
Copy link
Author

@Erstwild Erstwild commented Jul 7, 2016

Sure, I'm glad to take a look. As an anaconda user, I'm partial to conda for convenience. I'll look for the right dev branch and see if I can do some functionality testing at least.

@Erstwild
Copy link
Author

@Erstwild Erstwild commented Jul 7, 2016

@Miserlou Is this currently in the "dev" branch?

@Miserlou
Copy link
Owner

@Miserlou Miserlou commented Jul 12, 2016

This is in the pull-request branch, not dev: https://github.com/mathieu1/Zappa/tree/conda_support

@milesgranger
Copy link

@milesgranger milesgranger commented Apr 3, 2017

Any update with this? :)

@solalatus
Copy link

@solalatus solalatus commented Apr 3, 2017

Would be really interesting to know!
I used this to install: https://anaconda.org/mathieu/zappa so there is some progress, but after deploy I ran into a bug very similar to this: #260
Does anyone have a clue about possible fixes?

@solalatus
Copy link

@solalatus solalatus commented Apr 3, 2017

One caveat: I did not use the latest version, so I did
conda install -c mathieu zappa
and not
conda install -c mathieu zappa=0.39.1.dev
(dependecies misssing...)

@solalatus
Copy link

@solalatus solalatus commented Apr 3, 2017

Update: upgraded to 0.39.1.dev, and can confirm issue #260 Sadly. :-(

@solalatus
Copy link

@solalatus solalatus commented Apr 3, 2017

And to be more precise: "module initialization error: Attempted relative import in non-package"

I use Flask 0.12 istalled via Conda main channel.

@raeidsaqur
Copy link

@raeidsaqur raeidsaqur commented Nov 19, 2017

Sorry, any update on using zappa with anaconda? Looks like there are still issues with this. Thanks.

@scalzee
Copy link

@scalzee scalzee commented Dec 19, 2017

same... having some issues. this would be really great.

@joaovcarvalho
Copy link

@joaovcarvalho joaovcarvalho commented Jan 19, 2018

Same here, conda environment is useful for windows users. It would be great to not have this issue.

@neelz040
Copy link

@neelz040 neelz040 commented Mar 13, 2018

@Miserlou Any update on this? :-)

@PeteW
Copy link
Contributor

@PeteW PeteW commented May 8, 2018

upvoting. zappa/miniconda compatibility on, like any OS would be a huge help.

@llealgt
Copy link

@llealgt llealgt commented May 16, 2018

Any news on this? or some status page we can follow?

@sohamnavadiya
Copy link

@sohamnavadiya sohamnavadiya commented Jun 18, 2018

Any update on this?

@milesgranger
Copy link

@milesgranger milesgranger commented Jun 19, 2018

With the addition of Conda Pack it seems like this might be an option to get conda environment up? I suppose there will be complications if the conda environment was created on Windows and then tried to be unpacked into the Lambda env.

@unixeO
Copy link

@unixeO unixeO commented Aug 23, 2018

I just set up a new conda env and the issue remains.

@johntiger1
Copy link

@johntiger1 johntiger1 commented Oct 20, 2018

Right now, conda install zappa fails

@johntiger1
Copy link

@johntiger1 johntiger1 commented Oct 20, 2018

It seems like this "hacky" solution works:
(just manually set the VIRTUAL_ENV environment variable)
#1030

@mark-hoffmann
Copy link
Contributor

@mark-hoffmann mark-hoffmann commented Nov 7, 2018

I just tried using a conda environment to install both conda and pip packages. When doing a zappa update with a slim handler, the tar.gz file ends up being more than 3x the size for some reason. It also takes much, much longer to process. In addition there is something going on because I keep getting Module not found errors for jinja2 (installed and working in the conda env) coming from an import flask statement when deployed. Everything is working locally though.

Has there been any update on fully adding support for conda environments? This would be very helpful to become a priority because it seems like many newer packages are opting to only release via conda channels and not pip.

@RAbraham
Copy link

@RAbraham RAbraham commented Nov 7, 2018

A question for conda users: Is this in the context of serving machine learning models on AWS Lambda?

If so, I wrote a library called Thampi which uploads your model and your conda environment to AWS Lambda and abstracts away the DevOps part of model serving. The caveat is that your conda requirements file has to be manually written.

Conceptually, you can work on Mac(Windows not tested, but seems possible) and Thampi will use docker to load your requirements file and zappa deploy for you.

This would not be possible without the amazing work of zappa. So thank you!

@solalatus
Copy link

@solalatus solalatus commented Nov 7, 2018

This looks indeed the thing I was aiming for! Nice job!

@nbeuchat
Copy link

@nbeuchat nbeuchat commented Feb 8, 2019

Upvoting this one for 2019!

@wilrho
Copy link

@wilrho wilrho commented Mar 1, 2019

Hi there it seems to be possible to use Zappa with Conda by following these instructions : #1030 (comment)

Apologies if I am missing the point of this issue.

@Miserlou could you please give your opinion on whether or not this workaround is wise?

@gregorypierce
Copy link

@gregorypierce gregorypierce commented Jun 23, 2019

I'll add in a request for support in 2019. Not having conda support makes the case for supporting zappa difficult :(

@lokeshh
Copy link

@lokeshh lokeshh commented Feb 29, 2020

As @johntiger1 says, its easy to fix. I just had to run the following command

conda create -n zzz python=3.7
conda activate zzz
# Install required libraries, test your application
(zzz) ➜  which python
/home/lokesh/miniconda3/envs/zzz/bin/python
# Remove the /bin/python and copy the rest
export VIRUTAL_ENV=/home/lokesh/miniconda3/envs/zzz

and it worked !

@jneves
Copy link
Collaborator

@jneves jneves commented Feb 29, 2020

I'm sorry, I never used conda, so I'll need a bit of guidance on this one if we're going to get the support in. Can someone point me to a tutorial and either describe me the issues or point me to the comments on this thread that identify the still existing ones?

@jneves
Copy link
Collaborator

@jneves jneves commented Mar 1, 2020

@lokeshh if you can install zappa from master, can you check if the newly merged --no_venv option removes the need to export the VIRTUAL_ENV?

@lokeshh
Copy link

@lokeshh lokeshh commented Mar 3, 2020

@jneves No, it doesn't work. I ran the latest zappa from master. Here's the error message I got https://gist.github.com/lokeshh/ba6eac37748bff04b5f3bdd1b8d0779f

@gloriamacia
Copy link

@gloriamacia gloriamacia commented Apr 26, 2020

@lokeshh if you can install zappa from master, can you check if the newly merged --no_venv option removes the need to export the VIRTUAL_ENV?

can you specify the command? I would like to try it out! FYI for me the option exporting the virtual environment worked!

@Mattobad
Copy link

@Mattobad Mattobad commented Jan 19, 2021

And for windows users

conda create -n ds python=3.7
conda activate ds
# Install required libraries, test your application
(ds) ➜  where python
C:\Users\user_name\Anaconda3\envs\ds\python.exe
set VIRUTAL_ENV=C:\Users\user_name\Anaconda3\envs\ds

and it worked!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet