Command line parsing fails when path is a value for an option

[isaulv]

On Mac OS X 10.10.4 using the latest homebrew Python 2.7.10, and py.test 2.7.2, if type on the command line:
py.test --app ~/path/to/app --host http://localhost:4444 tests/mobile

I get the following error:

usage: py.test [options] [file_or_dir] [file_or_dir] [...]
py.test: error: unrecognized arguments: --app --host http://localhost:4444 tests/mobile

This seems to be an issue regardless of OS or Python version.

This only works if I put the path as a last option. But this defeats the purpose of having options that can be set in "any" order.

This is a section of my conftest.py:

def pytest_addoption(parser):
    parser.addoption('--host', action='store', help='Webdriver Remote Host')
    parser.addoption('--base_url', action='store', help='base URL')
    parser.addoption('--app', action='store', help='path to iOS app')
    parser.addoption('--android_app', action='store', help='path to Android app')

Things to note: A full path name works okay, but it seems that a ~ and a $varialbe will fail to parse.

[RonnyPfannschmidt]

can you please show your folder layout as well including where you run py.test

[isaulv]

Running Python and Py.test in a virtualenv

automation / 
            conftest.py
            tests      /
                         mobile  /
                                  test_1.py
                                  test_2.py
                                  ...

I run py.test from at .../automation/ as my root folder.

[nicoddemus]

The problem is how conftest files and command-line options interact. This section explains in detail how plugins and conftest files are loaded during startup, but relevant to this issue is this note:

Note that pytest does not find conftest.py files in deeper nested sub directories at tool startup. It is usually a good idea to keep your conftest.py file in the top level test or project root directory.

So pytest starts and loads the initial plugins (registered using setuptools) and conftest files (just those found in cwd), reads command line options defined by plugins and conftest files found, parses the command line supplied by the user and starts collecting tests and other conftest files. That's the reason you see the error about the unknown command line option if the conftest file is not at the cwd, as it would be loaded only later during test collection.

To fix this I suggest the following options:

• Move your options to a "proper" setuptools plugin (docs here;

• If the conftest is inside your application's package, you can move it to a separate module and ask pytest to treat it as a plugin by using the --addopts option in a pytest.ini file:

  [pytest]
  addopts = -p automation.myplugin

  This will ensure pytest will load your plugin before actually parsing the command-line supplied by the user.

Having said all that, I think we should add something to the docs about this issue, as I believe it is a common source of confusion.

[isaulv]

I am not convinced your explanation is the reason why this fail. If I reverse the options to py.test --host http://localhost:4444 tests/mobile --app ~/path/to/app I have no issues.

[nicoddemus]

Hmmm sorry, I missed your note at the end:

Things to note: A full path name works okay, but it seems that a ~ and a $varialbe will fail to parse.

So, if you execute:

$ py.test --app ~/path/to/app --host http://localhost:4444 tests/mobile

You get:

usage: py.test [options] [file_or_dir] [file_or_dir] [...]
py.test: error: unrecognized arguments: --app --host http://localhost:4444 tests/mobile

But if you execute:

$ py.test --app /home/dudex/path/to/app --host http://localhost:4444 tests/mobile

or

$ py.test --host http://localhost:4444 tests/mobile --app ~/path/to/app

It works?

[isaulv]

That is correct.

[nicoddemus]

Thanks, sorry I misunderstood the problem initially.

This certainly looks like a bug, thanks for the report!

[nicoddemus]

@Dude-x I'm having trouble obtaining the same error using the conftest.py file you provided, on Windows and Ubuntu, pytest version 2.7.2 and master. Could you provide a minimal example which reproduces your problem?

[isaulv]

It seems issue #949 is somewhat related. I'll whip up a minimum viable test case later.

[nicoddemus]

Thanks!

[isaulv]

Here's a simple test case, using the latest stable release of py.test (2.7.2)
Directory structure:

test-case/
  conftest.py
  tests/
     test_1.py
     test_2.py # these tests have a simple test_one, test_two functions

Content of conftest.py

import pytest

def pytest_addoption(parser):
    parser.addoption('--host', action='store', help='Host')
    parser.addoption('--base_url', action='store', help='base URL')

def pytest_configure(config):
    if config.getoption('host'):
        print 'Host: {0}'.format(config.getoption('host'))
    if config.getoption('base_url'):
        print 'Base URL: {0}'.format(config.getoption('base_url'))

Command line at /Users/dude-x/development/test-case

py.test --host ~/development/ tests/

Output:

Host: /Users/dude-x/development/
===================================================== test session starts =====================================================
platform darwin -- Python 2.7.10 -- py-1.4.30 -- pytest-2.7.2
rootdir: /Users/dude-x/development, inifile:
plugins: flaky, timeout, xdist
collected 2 items

tests/test_1.py .
tests/test_2.py .

================================================== 2 passed in 0.01 seconds ==================================================

The thing to notice is that the rootdir is wrong, it should be /test-case, and not my parent.

Here's one where it fails to parse:

py.test --host ~/Desktop --base_url http://localhost:4444/wd/hub tests/
usage: py.test [options] [file_or_dir] [file_or_dir] [...]
py.test: error: unrecognized arguments: --host --base_url http://localhost:4444/wd/hub tests/

[nicoddemus]

Hi @Dude-x, thanks for posting the example. 😄

I think I know what's going on. During initialization, pytest tries to parse the arguments from the command-line. At this stage only builtin and plugin options are known, so the unknown args are kept aside for later inspection. After this, it tries to determine the rootdir by looking at all unknown arguments which are clearly not options (don't start with -). At this point, it doesn't know your --host option takes a parameter (it could be a flag for all pytest knows at this point) so it looks at the next argument and seeing a directory, tries to use it to determine rootdir. Only later it will load the conftest files and parse the remaining arguments.

So, it all boils down to the fact that conftests are loaded at a later stage than builtin and plugin options, which can affect which command line options will actually be validated and used as this mechanism depends entirely from where you execute py.test, or which arguments are passed on the command line.

If you change your conftest file into a plugin, it should work consistently no matter where you call py.test from, or what you pass on the command-line.

It is unfortunate (but understandable) that there's this gotcha when working with pytest_addoption and conftest files, because conftest initialization being lazy as it is now can lead to surprising errors. Perhaps a note to pytest_addoption discouraging its use from conftest files would at least warn users of these pitfall?

[nicoddemus]

I went to update pytest_addoption and noticed that it already mentions something to this effect:

This function must be implemented in a :ref:`plugin <pluginorder>` and is called once at the beginning of a test run.

<pluginorder> points to the aforementioned docs.

I will add a warning to not use that from conftest files.