set environment variables before running tests?

[ChillarAnand]

To run tests from terminal, I source environment variables source .env & run py.test which runs test suite.

But when I run tests from elpy(C-c C-t), I get lots of errors as environment variables are not available.

Is there a way to source environment variables before running tests?

[jorgenschaefer]

In general, I'd recommend adding the source .env part to virtualenvwrapper hooks so the environment is available whenever you activate the virtualenv. For example, I use this ~/.virtualenvs/postactive:

#!/bin/bash                                                                     
# This hook is run after every virtualenv is activated.                         

if [ -f "$VIRTUAL_ENV/$VIRTUALENVWRAPPER_PROJECT_FILENAME" ]
then
    if [ -f "$(cat "$VIRTUAL_ENV/$VIRTUALENVWRAPPER_PROJECT_FILENAME")/.env" ]
    then
        set -a
        . "$(cat "$VIRTUAL_ENV/$VIRTUALENVWRAPPER_PROJECT_FILENAME")/.env"
        set +a
    fi
fi

This will source and export any environment variables in an .env file in the virtualenvwrapper project directory (set with setvirtualenvproject) whenever it is activated. pyvenv-workon will detect the environment changes and export them back to Emacs.

Alternatively, you might be able to set elpy-test-pytest-runner-command to ("source" ".env" "&&" "py.test") – you might have to adjust the path to the .env file.

[ChillarAnand]

I have added postactivate hook.

In terminal, it is working fine.

>>> import os
>>> os.environ.get('DJANGO_SETTINGS_MODULE')
'project.settings.local'

In emacs, it is unable to fetch. I tried running it with/without restarting rpc after activating virtualenv.

import os
print(os.environ.get('DJANGO_SETTINGS_MODULE'))
None

Is there a way to make sure that variables are exported?

[jorgenschaefer]

Does M-: (getenv "DJANGO_SETTINGS_MODULE") return the correct virtualenv after switching to it using pyvenv-workon or pyvenv-activate?

[ChillarAnand]

no, its returning nil

[jorgenschaefer]

Then pyvenv is not seeing the environment variable changes you configured in the postactivate hook.

Does M-: (getenv "VIRTUALENVWRAPPER_HOOK_DIR") return the right directory?

Can the Python interpreter pointed to by pyvenv-virtualenvwrapper-python access the virtualenvwrapper module?

[ChillarAnand]

(getenv "VIRTUALENVWRAPPER_HOOK_DIR") returns nil

pyvenv-virtualenvwrapper-python is showing correct dir ("/home/k3/.virtualenvs/project/bin/python")

python-shell-virtualenv-path value is also nil.

[jorgenschaefer]

What version of virtualenvwrapper is that? VIRTUALENVWRAPPER_HOOK_DIR should be set to $HOME/.virtualenvs and exported as an environment variable.

[ChillarAnand]

I was using virtualenvwrapper==4.3.1 Just upgraded to latest version.

In terminal echo $VIRTUALENVWRAPPER_HOOK_DIR is showing correct dir (/home/anand/.virtualenvs), but in emacs print(os.environ.get('VIRTUALENVWRAPPER_HOOK_DIR')) is returning None.

[ChillarAnand]

emacs process-environment has

"VIRTUAL_ENV=/home/k3/.virtualenvs/pearl"
"PATH=/home/k3/.virtualenvs/pearl/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin"

but it doesn't have any variables related to venv wrapper?

[jorgenschaefer]

Hm. The newer versions of virtualenvwrapper seem to export fewer variables, which is a bit annoying. As a temporary solution, export VIRTUALENVWRAPPER_HOOK_DIR – it should work with that.

But this is a but in pyvenv, I'll look into it. Thanks!

[ChillarAnand]

Thank You Very Much!

[jorgenschaefer]

Could you upgrade to pyvenv 1.6 and see if that fixes this problem?

[ChillarAnand]

upgraded to 1.6. still unable to access env variables.

(getenv "DJANGO_SETTINGS_MODULE") - nil
(getenv "VIRTUALENVWRAPPER_HOOK_DIR") - nil
C-h v python-shell-virtualenv-path - nil
C-h v pyvenv-virtualenvwrapper-python - "/home/k3/.virtualenvs/pearl/bin/python"

[jorgenschaefer]

But the virtualenv is activated, so (getenv "VIRTUAL_ENV") returns the right directory, correct?
And /home/k3/.virtualenvs/pearl/bin/python can import the virtualenvwrapper module?

• What does M-: (pyvenv-hook-dir) return?
• When you run M-: (pyvenv-run-virtualenvwrapper-hook "post_activate") manually, does it set DJANGO_SETTINGS_MODULE? (It won't set VIRTUALENVWRAPPER_HOOK_DIR)
• Is there a *Virtualenvwrapper Hook Output* buffer?
• After /home/k3/.virtualenvs/pearl/bin/python -c "from virtualenvwrapper.hook_loader import main; main()" --script foo.txt post_activate what are the contents of foo.txt?

[ChillarAnand]

(getenv "VIRTUAL_ENV") returns correct dir "/home/k3/.virtualenvs/pearl/" & it can import virtualenvwrapper.
(pyvenv-hook-dir) - "/home/k3/.virtualenvs/"

After post-activate, it doesn't set DJANGO_SETTINGS_MODULE

No, no virtualenvwrapper hook output buffer

Contents of foo.txt

# post_activate
# project
#
# Change to the project directory
#
[ -f "$VIRTUAL_ENV/$VIRTUALENVWRAPPER_PROJECT_FILENAME" ] &&     virtualenvwrapper_cd         "$(cat "$VIRTUAL_ENV/$VIRTUALENVWRAPPER_PROJECT_FILENAME")"
# user_scripts
#
# Run user-provided scripts
#
[ -f "$VIRTUALENVWRAPPER_HOOK_DIR/postactivate" ] &&     source "$VIRTUALENVWRAPPER_HOOK_DIR/postactivate"
[ -f "$VIRTUAL_ENV/$VIRTUALENVWRAPPER_ENV_BIN_DIR/postactivate" ] &&     source "$VIRTUAL_ENV/$VIRTUALENVWRAPPER_ENV_BIN_DIR/postactivate"

[jorgenschaefer]

Thank you.

[ -f "$VIRTUALENVWRAPPER_HOOK_DIR/postactivate" ] && source "$VIRTUALENVWRAPPER_HOOK_DIR/postactivate"

Apparently, virtualenvwrapper requires VIRTUALENVWRAPPER_HOOK_DIR to be exported.

I just set up a local virtualenv with virtualenvwrapper 4.3.2. This does export VIRTUALENVWRAPPER_HOOK_DIR. Now the question is, why is it not exported for you.

When you are in a shell, does env | grep VIRTUALENVWRAPPER_HOOK_DIR give any output?

If yes (shot in the dark), are you using Mac OS X and are starting Emacs from the desktop and not the terminal?

[ChillarAnand]

env | grep VIRTUALENVWRAPPER_HOOK_DIR gives
VIRTUALENVWRAPPER_HOOK_DIR=/home/k3/.virtualenvs

i am using ubuntu & starting emacs from launcher NOT terminal.

[ChillarAnand]

oh.. thank you very much. if i start emacs from terminal, its working 👯

[jorgenschaefer]

Nice!

Ok. So the problem is that you do not actually load virtualenvwrapper in the environment Emacs is running in. You can fix that by setting the environment variables virtualenvwrapper requires manually, e.g. in your .emacs:

(let ((workon-home (expand-file-name "~/.virtualenvs/")))
  (setenv "WORKON_HOME" workon-home)
  (setenv "VIRTUALENVWRAPPER_HOOK_DIR" workon-home))

I'll see if I can add this as a feature to pyvenv, thank you for the patience in tracking this down!

[ChillarAnand]

sorry to bother again. i have added above lines in config & looks like there is one more issue.

see env variables

1. start emacs from launcher

   WORKON_HOME: "/home/k3/.virtualenvs/"
   VIRUTALENVWRAPPER_HOOK_DIR: "/home/k3/.virtualenvs/"
   DJANGO_SETTINGS_MODULE: nil
   VIRTUAL_ENV: nil

2. activate venv

   WORKON_HOME: "/home/k3/.virtualenvs/"
   VIRUTALENVWRAPPER_HOOK_DIR: "/home/k3/.virtualenvs/"
   DJANGO_SETTINGS_MODULE: nil
   VIRTUAL_ENV: "/home/k3/.virtualenvs/pearl"

previously when i said it was working, i have started terminal, activated venv & then restarted emacs. may be it was getting env variables from terminal

[jorgenschaefer]

As VIRTUALENVWRAPPER_HOOK_DIR is set, this is likely a different problem now (weird that it works when run from the terminal …).

• What is the value of pyvenv-virtualenvwrapper-python? Is it different if you start Emacs from the terminal?
• When you run M-: (pyvenv-run-virtualenvwrapper-hook "post_activate") manually, does it set DJANGO_SETTINGS_MODULE? (It won't set VIRTUALENVWRAPPER_HOOK_DIR)
• Is there a *Virtualenvwrapper Hook Output* buffer?
• After /home/k3/.virtualenvs/pearl/bin/python -c "from virtualenvwrapper.hook_loader import main; main()" --script foo.txt post_activate what are the contents of foo.txt?

[ChillarAnand]

when started from terminal pyvenv-virtualenvwrapper-python is "/home/k3/.virtualenvs/pearl/bin/python"

but when started from launcher it is "/usr/bin/python"

M-: (pyvenv-run-virtualenvwrapper-hook "post_activate") is not setting DJANGO_SETTINGS_MODULE.

there is no *Virtualenvwrapper Hook Output* buffer.

contents of foo.txt

# post_activate
# project
#
# Change to the project directory
#
[ -f "$VIRTUAL_ENV/$VIRTUALENVWRAPPER_PROJECT_FILENAME" ] &&     virtualenvwrapper_cd         "$(cat "$VIRTUAL_ENV/$VIRTUALENVWRAPPER_PROJECT_FILENAME")"
# user_scripts
#
# Run user-provided scripts
#
[ -f "$VIRTUALENVWRAPPER_HOOK_DIR/postactivate" ] &&     source "$VIRTUALENVWRAPPER_HOOK_DIR/postactivate"
[ -f "$VIRTUAL_ENV/$VIRTUALENVWRAPPER_ENV_BIN_DIR/postactivate" ] &&     source "$VIRTUAL_ENV/$VIRTUALENVWRAPPER_ENV_BIN_DIR/postactivate"

[ChillarAnand]

looks like pyvenv-virtualenvwrapper-python is getting changed somewhere when emacs is started from launcher.

C-h v pyvenv-virtualenvwrapper-python gives

pyvenv-virtualenvwrapper-python is a variable defined in `pyvenv.el'.
Its value is "/usr/bin/python"
Original value was 
"/home/k3/.virtualenvs/pearl/bin/python"

[jorgenschaefer]

This uses the value of the environment variable VIRTUALENVWRAPPER_PYTHON, or the current python executable – if your launcher does not set the environment variable, you have to set this correctly from your init file.

[ChillarAnand]

thank you :)

[jorgenschaefer]

You're welcome, glad it works :-)

[detrout]

I had the same problem where I wanted to set DJANGO_SETTINGS_MODULE and couldn't figure out how to get it to set through pre/postactivate scripts in a virtual environment, the only thing that really seemed to work for me was to set DJANGO_SETTINGS_MODULE before launching emacs.

For my use it was simpler to just set the DJANGO_SETTINGS_MODULE in the manage.py script created by django then set elpy-test-django-runner-command to ("./manage.py" "test" "--noinput")