Add self-check to pytests

[elazarg]

Yet another item for #1673.

This PR uses the api for the self-check.

Example of a failed-run output:

~/workspace/mypy$ pytest -n0 -k test_mypy_package
================================ test session starts ================================
platform darwin -- Python 3.6.5, pytest-3.2.5, py-1.5.2, pluggy-0.4.0
rootdir: ~/workspace/mypy, inifile: pytest.ini
plugins: xdist-1.20.1, forked-0.2, cov-2.5.1
collected 6240 items                                                                 

mypy/test/testselfcheck.py F

===================================== FAILURES ======================================
_________________________ SelfCheckSuite.test_mypy_package __________________________
Self check failed
------------------------------- Captured stdout call --------------------------------
mypy/binder.py:254: error: No return value expected
=============================== 6239 tests deselected ===============================
==================== 1 failed, 6239 deselected in 23.75 seconds =====================

$ ./runtests.py self-check results in a similar output, though slightly more verbose, as usual.

[elazarg]

3.4 fails the type check with this error:

mypy/test/data.py:232: error: "TemporaryDirectory" expects no type arguments, but 1 given

I have no idea why did it pass before. (See short discussion here)

[JelleZijlstra]

Maybe we just didn't run a self-check with 3.4 before? That error is consistent with the stubs (https://github.com/python/typeshed/blob/master/stdlib/3/tempfile.pyi): TemporaryDirectory is generic only in 3.5+.

[ilevkivskyi]

I think self check is only run on 3.6. Btw @elazarg I added a new label topic-tests for test related issues.

[elazarg]

I see. I've changed the name of the self testing from package to self-check so the -x flag doesn't work. Otherwise the checks pass.

[ethanhs]

Please make sure to update the appveyor/travis configs to replace package with self-check if we are going ahead with that.

[elazarg]

Where's the appveyor check? I've only touched .travis.yml.

[ethanhs]

@elazarg it is excluded on this line https://github.com/python/mypy/blob/master/appveyor.yml#L24

[elazarg]

@ethanhs is it OK now?

[ethanhs]

The CI configs look good!