Use py.test instead of cram

[jgonggrijp]

See #39.

[Jacksole]

Any headway on this issue?

[jgonggrijp]

The pytest documentation is a nice primer on pytest in general. I suggest you start reading there and try to convert one of our cram tests to pytest.

There is also #39. Someone in there posted a link to the functional tests of PyInstaller, which were written with pytest as well, so they may serve as example code.

[jgonggrijp]

To avoid confusion, there is currently one test file which contains multiple tests. This is one test:

pip-review/tests/review.t

Lines 19 to 22 in 04dcfe0

	Next, let's see what pip-review does:
	$ pip-review 2>&1 | egrep -v '^DEPRECATION:'
	python-dateutil==* is available (you have 1.5) (glob)

So you could start by porting just this test to pytest. Keep in mind that you will also need to port the setup code that comes before this test. Pytest has "fixtures" for this purpose.

[Pradhvan]

@jgonggrijp started porting the current test to pytest. 😄

I have two questions:

1. Why create $ cd $TESTDIR/.. is it being used somewhere in the test as I could not find it while reading the file?

2. Instead of using virtualenv pip install virtualenv can I move it to venv ?

Though for now, I am looking into creating a conftest.py for creating def playground(tmpdir) fixture that inherits from the tmpdir directory.

[jgonggrijp]

@Pradhvan Nice to know that you're working on this!

1. Why create $ cd $TESTDIR/.. is it being used somewhere in the test as I could not find it while reading the file?

That line doesn't create $TESTDIR, it moves into it. In other words, $TESTDIR becomes the current working directory, which it remains throughout all of the following tests. I'm not 100% sure why it is this way, because I'm not the original author of these tests, but I suspect that $TESTDIR is some kind of builtin variable that cram provides by default.

2. Instead of using virtualenv pip install virtualenv can I move it to venv ?

Sorry, I don't understand this question. Could you clarify?

Though for now, I am looking into creating a conftest.py for creating def playground(tmpdir) fixture that inherits from the tmpdir directory.

That sounds like a good idea.

[Pradhvan]

@jgonggrijp ahh okay got the use of $ cd $TESTDIR/ I guess tmpdir would do the same job when I port the code to Pytest

Instead of using virtualenv pip install virtualenv can I move it to venv ?

I meant instead of using virtualenv can't we use the venv given by the standard lib something like python -m venv

[jgonggrijp]

Today I learned that there is a venv module in the Python standard library! 😄

The answer to your question is "no", though (or at least "not yet"). Please keep using the third-party virtualenv for now. Two reasons:

• The standard library documentation clearly states that the module was introduced in Python 3.3. We're currently still supporting Python 2, so that makes it a no-go for the time being.
• The virtualenv documentation mentions a few things that it can do that venv cannot. While these differences may or may not be relevant to the test suite, let's give ourselves an easier time and make one transition at a time.

[5j9]

What do you think about creating offline tests using unittest.mock, patching subprocess.Popen, and checking logger calls instead of making actual subprocess calls to pip?
Such tests would run in an instant, but the downside is that if pip changes in a backward incompatible manner tests won't notice it.

[jgonggrijp]

@5j9 I think such an approach would make sense for unittests. Currently, we don't really have those; cram only allows you to write functional tests as it treats the tested tool as a black box.

In principle, I'm in favor of having both unittests to check our own internal mechanisms and a functional test (maybe a few) to check for compatibility with pip. Any test at all that runs in pytest is already a good start, though.