Tests failing on Arch

[WhyNotHugo]

When updating the Arch package to 0.16.0, I noticed some tests failing locally. We're currently at python 3.6.2.

______________________________________________________________________________________ test_request_ssl _______________________________________________________________________________________
/usr/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py:441: in wrap_socket
    cnx.do_handshake()
/usr/lib/python3.6/site-packages/OpenSSL/SSL.py:1716: in do_handshake
    self._raise_ssl_error(self._ssl, result)
/usr/lib/python3.6/site-packages/OpenSSL/SSL.py:1456: in _raise_ssl_error
    _raise_current_error()
/usr/lib/python3.6/site-packages/OpenSSL/_util.py:54: in exception_from_error_queue
    raise exception_type(errors)
E   OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]

During handling of the above exception, another exception occurred:
/usr/lib/python3.6/site-packages/urllib3/connectionpool.py:601: in urlopen
    chunked=chunked)
/usr/lib/python3.6/site-packages/urllib3/connectionpool.py:346: in _make_request
    self._validate_conn(conn)
/usr/lib/python3.6/site-packages/urllib3/connectionpool.py:850: in _validate_conn
    conn.connect()
/usr/lib/python3.6/site-packages/urllib3/connection.py:326: in connect
    ssl_context=context)
/usr/lib/python3.6/site-packages/urllib3/util/ssl_.py:329: in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
/usr/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py:448: in wrap_socket
    raise ssl.SSLError('bad handshake: %r' % e)
E   ssl.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)

During handling of the above exception, another exception occurred:
/usr/lib/python3.6/site-packages/requests/adapters.py:440: in send
    timeout=timeout
/usr/lib/python3.6/site-packages/urllib3/connectionpool.py:639: in urlopen
    _stacktrace=sys.exc_info()[2])
/usr/lib/python3.6/site-packages/urllib3/util/retry.py:388: in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
E   urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='127.0.0.1', port=42479): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))

During handling of the above exception, another exception occurred:
tests/system/utils/test_main.py:32: in test_request_ssl
    http.request('GET', httpsserver.url)
vdirsyncer/http.py:164: in request
    r = func(method, url, **kwargs)
/usr/lib/python3.6/site-packages/requests/sessions.py:502: in request
    resp = self.send(prep, **send_kwargs)
/usr/lib/python3.6/site-packages/requests/sessions.py:612: in send
    r = adapter.send(request, **kwargs)
/usr/lib/python3.6/site-packages/requests/adapters.py:504: in send
    raise ConnectionError(e, request=request)
E   requests.exceptions.ConnectionError: HTTPSConnectionPool(host='127.0.0.1', port=42479): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))
_________________________________________________ test_request_ssl_fingerprints[94:FD:7A:CB:50:75:A4:69:82:0A:F8:23:DF:07:FC:69:3E:CD:90:CA] __________________________________________________
/usr/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py:441: in wrap_socket
    cnx.do_handshake()
/usr/lib/python3.6/site-packages/OpenSSL/SSL.py:1716: in do_handshake
    self._raise_ssl_error(self._ssl, result)
/usr/lib/python3.6/site-packages/OpenSSL/SSL.py:1456: in _raise_ssl_error
    _raise_current_error()
/usr/lib/python3.6/site-packages/OpenSSL/_util.py:54: in exception_from_error_queue
    raise exception_type(errors)
E   OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]

During handling of the above exception, another exception occurred:
/usr/lib/python3.6/site-packages/urllib3/connectionpool.py:601: in urlopen
    chunked=chunked)
/usr/lib/python3.6/site-packages/urllib3/connectionpool.py:346: in _make_request
    self._validate_conn(conn)
/usr/lib/python3.6/site-packages/urllib3/connectionpool.py:850: in _validate_conn
    conn.connect()
/usr/lib/python3.6/site-packages/urllib3/connection.py:326: in connect
    ssl_context=context)
/usr/lib/python3.6/site-packages/urllib3/util/ssl_.py:329: in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
/usr/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py:448: in wrap_socket
    raise ssl.SSLError('bad handshake: %r' % e)
E   ssl.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)

During handling of the above exception, another exception occurred:
/usr/lib/python3.6/site-packages/requests/adapters.py:440: in send
    timeout=timeout
/usr/lib/python3.6/site-packages/urllib3/connectionpool.py:639: in urlopen
    _stacktrace=sys.exc_info()[2])
/usr/lib/python3.6/site-packages/urllib3/util/retry.py:388: in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
E   urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='127.0.0.1', port=43889): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))

During handling of the above exception, another exception occurred:
tests/system/utils/test_main.py:57: in test_request_ssl_fingerprints
    verify_fingerprint=fingerprint)
vdirsyncer/http.py:164: in request
    r = func(method, url, **kwargs)
/usr/lib/python3.6/site-packages/requests/sessions.py:502: in request
    resp = self.send(prep, **send_kwargs)
/usr/lib/python3.6/site-packages/requests/sessions.py:612: in send
    r = adapter.send(request, **kwargs)
/usr/lib/python3.6/site-packages/requests/adapters.py:504: in send
    raise ConnectionError(e, request=request)
E   requests.exceptions.ConnectionError: HTTPSConnectionPool(host='127.0.0.1', port=43889): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))
------------------------------------------------------------------------------------ Captured stderr call -------------------------------------------------------------------------------------
127.0.0.1 - - [08/Aug/2017 10:23:51] "GET / HTTP/1.1" 200 -
_______________________________________________________ test_request_ssl_fingerprints[19:90:F7:23:94:F2:EF:AB:2B:64:2D:57:3D:25:95:2D] ________________________________________________________
/usr/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py:441: in wrap_socket
    cnx.do_handshake()
/usr/lib/python3.6/site-packages/OpenSSL/SSL.py:1716: in do_handshake
    self._raise_ssl_error(self._ssl, result)
/usr/lib/python3.6/site-packages/OpenSSL/SSL.py:1456: in _raise_ssl_error
    _raise_current_error()
/usr/lib/python3.6/site-packages/OpenSSL/_util.py:54: in exception_from_error_queue
    raise exception_type(errors)
E   OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]

During handling of the above exception, another exception occurred:
/usr/lib/python3.6/site-packages/urllib3/connectionpool.py:601: in urlopen
    chunked=chunked)
/usr/lib/python3.6/site-packages/urllib3/connectionpool.py:346: in _make_request
    self._validate_conn(conn)
/usr/lib/python3.6/site-packages/urllib3/connectionpool.py:850: in _validate_conn
    conn.connect()
/usr/lib/python3.6/site-packages/urllib3/connection.py:326: in connect
    ssl_context=context)
/usr/lib/python3.6/site-packages/urllib3/util/ssl_.py:329: in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
/usr/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py:448: in wrap_socket
    raise ssl.SSLError('bad handshake: %r' % e)
E   ssl.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)

During handling of the above exception, another exception occurred:
/usr/lib/python3.6/site-packages/requests/adapters.py:440: in send
    timeout=timeout
/usr/lib/python3.6/site-packages/urllib3/connectionpool.py:639: in urlopen
    _stacktrace=sys.exc_info()[2])
/usr/lib/python3.6/site-packages/urllib3/util/retry.py:388: in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
E   urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='127.0.0.1', port=33183): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))

During handling of the above exception, another exception occurred:
tests/system/utils/test_main.py:57: in test_request_ssl_fingerprints
    verify_fingerprint=fingerprint)
vdirsyncer/http.py:164: in request
    r = func(method, url, **kwargs)
/usr/lib/python3.6/site-packages/requests/sessions.py:502: in request
    resp = self.send(prep, **send_kwargs)
/usr/lib/python3.6/site-packages/requests/sessions.py:612: in send
    r = adapter.send(request, **kwargs)
/usr/lib/python3.6/site-packages/requests/adapters.py:504: in send
    raise ConnectionError(e, request=request)
E   requests.exceptions.ConnectionError: HTTPSConnectionPool(host='127.0.0.1', port=33183): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))
------------------------------------------------------------------------------------ Captured stderr call -------------------------------------------------------------------------------------
127.0.0.1 - - [08/Aug/2017 10:23:52] "GET / HTTP/1.1" 200 -
====================================================================================== warnings summary =======================================================================================
None
  pytest_funcarg__httpserver: declaring fixtures using "pytest_funcarg__" prefix is deprecated and scheduled to be removed in pytest 4.0.  Please remove the prefix and use the @pytest.fixture decorator instead.
  pytest_funcarg__httpsserver: declaring fixtures using "pytest_funcarg__" prefix is deprecated and scheduled to be removed in pytest 4.0.  Please remove the prefix and use the @pytest.fixture decorator instead.
  pytest_funcarg__smtpserver: declaring fixtures using "pytest_funcarg__" prefix is deprecated and scheduled to be removed in pytest 4.0.  Please remove the prefix and use the @pytest.fixture decorator instead.

-- Docs: http://doc.pytest.org/en/latest/warnings.html
=============================================================== 3 failed, 333 passed, 178 skipped, 3 warnings in 16.36 seconds ================================================================
make: *** [Makefile:37: test] Error 1
=

[XenGi]

Works on my arch install. Just installed the package from AUR.
I only see 3 warnings like this:

pytest_funcarg__smtpserver: declaring fixtures using "pytest_funcarg__" prefix is deprecated and scheduled to be removed in pytest 4.0.  Please remove the prefix and use the @pytest.fixture decorator instead.

When I checkout the git in a fresh virtualenv I get the following error:

$ python setup.py test
[...]

Installed /home/xengi/vdirsyncer/.eggs/chardet-3.0.4-py3.6.egg
running egg_info
creating vdirsyncer.egg-info
writing vdirsyncer.egg-info/PKG-INFO
writing dependency_links to vdirsyncer.egg-info/dependency_links.txt
writing entry points to vdirsyncer.egg-info/entry_points.txt
writing requirements to vdirsyncer.egg-info/requires.txt
writing top-level names to vdirsyncer.egg-info/top_level.txt
writing manifest file 'vdirsyncer.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'docs/_build'
warning: no previously-included files matching '*.py[cdo]' found anywhere in distribution
warning: no previously-included files matching '__pycache__' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
warning: no previously-included files matching '*.pyd' found anywhere in distribution
writing manifest file 'vdirsyncer.egg-info/SOURCES.txt'
running build_ext
tests (unittest.loader._FailedTest) ... ERROR

======================================================================
ERROR: tests (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: tests
Traceback (most recent call last):
  File "/usr/lib64/python3.6/unittest/loader.py", line 462, in _find_test_path
    package = self._get_module_from_name(name)
  File "/usr/lib64/python3.6/unittest/loader.py", line 369, in _get_module_from_name
    __import__(name)
  File "/home/xengi/vdirsyncer/tests/__init__.py", line 6, in <module>
    import hypothesis.strategies as st
ModuleNotFoundError: No module named 'hypothesis'


----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (errors=1)
Test failed: <unittest.runner.TextTestResult run=1 errors=1 failures=0>
error: Test failed: <unittest.runner.TextTestResult run=1 errors=1 failures=0>

After installing the requirements with pip install -r test-requirements.txt I get this:

$ python setup.py test
[...]

Using /home/xengi/vdirsyncer/.eggs/idna-2.5-py3.6.egg
Searching for chardet<3.1.0,>=3.0.2
Best match: chardet 3.0.4
Processing chardet-3.0.4-py3.6.egg

Using /home/xengi/vdirsyncer/.eggs/chardet-3.0.4-py3.6.egg
running egg_info
writing vdirsyncer.egg-info/PKG-INFO
writing dependency_links to vdirsyncer.egg-info/dependency_links.txt
writing entry points to vdirsyncer.egg-info/entry_points.txt
writing requirements to vdirsyncer.egg-info/requires.txt
writing top-level names to vdirsyncer.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'docs/_build'
warning: no previously-included files matching '*.py[cdo]' found anywhere in distribution
warning: no previously-included files matching '__pycache__' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
warning: no previously-included files matching '*.pyd' found anywhere in distribution
writing manifest file 'vdirsyncer.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
  File "setup.py", line 103, in <module>
    'Topic :: Utilities',
  File "/usr/lib64/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib64/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/xengi/.pyvenvs/tempenv-113e191276de/lib/python3.6/site-packages/setuptools/command/test.py", line 215, in run
    self.run_tests()
  File "/home/xengi/.pyvenvs/tempenv-113e191276de/lib/python3.6/site-packages/setuptools/command/test.py", line 238, in run_tests
    **exit_kwarg
  File "/usr/lib64/python3.6/unittest/main.py", line 94, in __init__
    self.parseArgs(argv)
  File "/usr/lib64/python3.6/unittest/main.py", line 124, in parseArgs
    self._do_discovery([])
  File "/usr/lib64/python3.6/unittest/main.py", line 229, in _do_discovery
    self.test = loader.discover(self.start, self.pattern, self.top)
  File "/usr/lib64/python3.6/unittest/loader.py", line 341, in discover
    tests = list(self._find_tests(start_dir, pattern))
  File "/usr/lib64/python3.6/unittest/loader.py", line 398, in _find_tests
    full_path, pattern, namespace)
  File "/usr/lib64/python3.6/unittest/loader.py", line 475, in _find_test_path
    tests = self.loadTestsFromModule(package, pattern=pattern)
  File "/home/xengi/.pyvenvs/tempenv-113e191276de/lib/python3.6/site-packages/setuptools/command/test.py", line 43, in loadTestsFromModule
    tests.append(self.loadTestsFromName(submodule))
  File "/usr/lib64/python3.6/unittest/loader.py", line 190, in loadTestsFromName
    return self.loadTestsFromModule(obj)
  File "/home/xengi/.pyvenvs/tempenv-113e191276de/lib/python3.6/site-packages/setuptools/command/test.py", line 43, in loadTestsFromModule
    tests.append(self.loadTestsFromName(submodule))
  File "/usr/lib64/python3.6/unittest/loader.py", line 153, in loadTestsFromName
    module = __import__(module_name)
  File "/home/xengi/vdirsyncer/tests/storage/dav/__init__.py", line 20, in <module>
    dav_server = os.environ['DAV_SERVER']
  File "/home/xengi/.pyvenvs//tempenv-113e191276de/lib/python3.6/os.py", line 669, in __getitem__
    raise KeyError(key) from None
KeyError: 'DAV_SERVER'

Then I saw the Makefile and tried:

$ make test
py.test  tests/
Traceback (most recent call last):
  File "/home/xengi/.pyvenvs/tempenv-113e191276de/lib/python3.6/site-packages/_pytest/config.py", line 342, in _getconftestmodules
    return self._path2confmods[path]
KeyError: local('/home/xengi/vdirsyncer/tests')

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/xengi/.pyvenvs/tempenv-113e191276de/lib/python3.6/site-packages/_pytest/config.py", line 373, in _importconftest
    return self._conftestpath2mod[conftestpath]
KeyError: local('/home/xengi/vdirsyncer/tests/conftest.py')

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/xengi/.pyvenvs/tempenv-113e191276de/lib/python3.6/site-packages/_pytest/config.py", line 379, in _importconftest
    mod = conftestpath.pyimport()
  File "/home/xengi/.pyvenvs/tempenv-113e191276de/lib/python3.6/site-packages/py/_path/local.py", line 662, in pyimport
    __import__(modname)
  File "/home/xengi/.pyvenvs/tempenv-113e191276de/lib/python3.6/site-packages/_pytest/assertion/rewrite.py", line 212, in load_module
    py.builtin.exec_(co, mod.__dict__)
  File "/home/xengi/vdirsyncer/tests/conftest.py", line 8, in <module>
    import click_log
ModuleNotFoundError: No module named 'click_log'
ERROR: could not load /home/xengi/vdirsyncer/tests/conftest.py

It would be nice if there could be some docs in the README that describes how to run the tests etc.

[untitaker]

@XenGi There is the CONTRIBUTING section in the docs describing everything.

@hobarrera Yeah, this is fixed in master. Will release soon.

[WhyNotHugo]

Thanks!

[XenGi]

@untitaker thx! I'll have a look at that. Would still be nice to have that in the readme.