Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Global variables from pytest_namespace hook doesnt seem to work when xdist is used. #1402

Closed
karthikborkar opened this Issue Feb 19, 2016 · 7 comments

Comments

Projects
None yet
2 participants
@karthikborkar
Copy link

karthikborkar commented Feb 19, 2016

To use global variables across my test i have used pytest_namespace hook, Updating these variables seem to work fine without xdist.But when i use xdist to run my test in parallel , These global variables are not updated

Here is my conftest.py file

def pytest_namespace():
     return  {'my_global_variable': 0}

`

Here is my test file test_dummy.py

def test_this(self):
     pytest.my_global_variable =  pytest.my_global_variable +1

def test_this1(self):
     pytest.my_global_variable =  pytest.my_global_variable +1

def test_this2(self):
     pytest.my_global_variable =  pytest.my_global_variable +1

def test_this3(self):
     pytest.my_global_variable =  pytest.my_global_variable +1

def test_this4(self):
     pytest.my_global_variable =  pytest.my_global_variable +1

def test_this5(self):
     pytest.my_global_variable = pytest.my_global_variable +1

So when i run
py.test -v -s path/test_dummy.py

i see the global variable my_global_variable getting updated correctly

but when i do

py.test -v -s path/test_dummy.py -n2

i see the global variable my_global_variable is not updated at all

what iam i missing here?

Python version : 2.7.9

pip list
`
pytest (2.7.0)
pytest-instafail (0.3.0)
pytest-rerunfailures (0.5)
pytest-timeout (0.3)
pytest-xdist (1.13.1)

`

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Feb 19, 2016

I tried your example and actually my_global_variable is updated, but perhaps not the way you expect: keep in mind that pytest-xdist with -n2 will spawn two workers in separate processes, and each one of them will have their own version of my_global_variable.

@karthikborkar

This comment has been minimized.

Copy link
Author

karthikborkar commented Feb 19, 2016

Thanks for the quick response.
So I was using global variable to print terminal summary (using another hook pytest_terminal_summary(terminalreporter)) and it wasn't
updating for me when running -n2.. I know it spans two separate
processes..is there a way to communicate all the result from subprocess
here to get final result ?
On 19-Feb-2016 8:05 PM, "Bruno Oliveira" notifications@github.com wrote:

I tried your example and actually my_global_variable is updated, but
perhaps not the way you expect: keep in mind that pytest-xdist with -n2
will spawn two workers in separate processes, and each one of them will
have their own version of my_global_variable.


Reply to this email directly or view it on GitHub
#1402 (comment).

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Feb 19, 2016

Hmm I'm afraid there's no easy way to send back information from the slaves unfortunately, others please correct me if I'm wrong (sending information from master to slaves can be done using the config.slaveinput config dictionary, though).

Could you describe the exact problem you are trying to accomplish? Perhaps with the actual problem others can suggest a different way to accomplish the same thing.

@karthikborkar

This comment has been minimized.

Copy link
Author

karthikborkar commented Feb 19, 2016

Basically i want to keep track of some attributes across my tests , i have many test_XXX.py file. and at the end after test run is done. i want to print it using pytest_terminal_summary hook

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Feb 19, 2016

Hmmm for this case in specific I would use a file on disk, by creating a temporary directory on the master node and forwarding that to the workers, which can then use that directory to write any data they want (for example in json). Then during pytest_terminal_summary you can read all generated files and compile them into a single piece of information.

Here's a quick howto create this shared directory and have a shared_directory fixture which can be used by both master and worker nodes equally.

def pytest_configure(config):        
    if is_master(config):
        config.shared_directory = tempdir.mktemp()

def pytest_unconfigure(config):        
    if is_master(config):
        shutil.rmtree(config.shared_directory)


def pytest_configure_node(self, node):
    """xdist hook"""
    node.slaveinput['shared_dir'] = node.config.shared_directory


@pytest.fixture
def shared_directory(request):
    if is_master(request.config):
        reutrn request.config.shared_directory
    else:
        return request.config.slaveinput['shared_dir']


def is_master(config):
    """True if the code running the given pytest.config object is running in a xdist master
    node or not running xdist at all.
    """
    return not hasattr(config, 'slaveinput')

HTH

@karthikborkar

This comment has been minimized.

Copy link
Author

karthikborkar commented Feb 22, 2016

Thanks, the above solution seem to work

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Feb 22, 2016

If there's nothing else, could you please close this issue? 😁

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.