Some tests fail on FreeBSD

[yurivict]

On FreeBSD 10.3 I am getting these failures. One hard failure, still attaching the full log:

testing/python/show_fixtures_per_test.py .....
========================================================================= short test summary info ==========================================================================
XFAIL testing/acceptance_test.py::TestInvocationVariants::()::test_noclass_discovery_if_not_testcase
  decide: feature or bug
XFAIL testing/test_capture.py::TestPerTestCapturing::()::test_capture_scope_cache
  unimplemented feature
XFAIL testing/test_collection.py::TestPrunetraceback::()::test_collect_report_postprocessing
  other mechanism for adding to reporting needed
XFAIL testing/test_config.py::TestParseIni::()::test_confcutdir
  probably not needed
XFAIL testing/test_junitxml.py::test_runs_twice_xdist
  reason: [NOTRUN] hangs
XFAIL testing/test_mark.py::TestFunctional::()::test_merging_markers_deep
  unfixed
XFAIL testing/test_mark.py::TestFunctional::()::test_mark_should_not_pass_to_siebling_class
  markers smear on methods of base classes
XFAIL testing/test_mark.py::TestFunctional::()::test_mark_from_parameters
  callspec2.setmulti misuses keywords
XFAIL testing/test_mark.py::TestKeywordSelection::()::test_keyword_extra_dash
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction_capture
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction_exception
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction_on_collection_issue181
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction_on_internal_error
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction_capturing_simple
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_set_trace_interception
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_and_capsys
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_pdb.py::TestPDB::()::test_set_trace_capturing_afterwards
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction_doctest
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_interaction_capturing_twice
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_used_outside_test
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_pdb.py::TestPDB::()::test_pdb_used_in_generate_tests
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_pdb.py::TestPDB::()::test_enter_pdb_hook_is_called
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_pytester.py::test_make_hook_recorder
  reason: internal reportrecorder tests need refactoring
XFAIL testing/test_runner.py::test_runtest_in_module_ordering
XFAIL testing/test_terminal.py::TestTerminal::()::test_runtest_location_shown_before_test_starts
  reason: pexpect does not work reliably on freebsd
XFAIL testing/test_unittest.py::TestTrialUnittest::()::test_trial_pdb
  reason: pexpect does not work reliably on freebsd
XFAIL testing/python/fixture.py::TestAutouseDiscovery::()::test_setup_enabled_functionnode
  'enabled' feature not implemented
XFAIL testing/python/metafunc.py::TestMarkersWithParametrization::()::test_nested_marks
  is this important to support??
SKIP [1] testing/test_tmpdir.py:172: win only
SKIP [1] /usr/ports/devel/py-pytest/work/pytest-3.0.0/testing/python/integration.py:120: could not import 'unittest.mock'
SKIP [1] testing/test_assertrewrite.py:518: packages without __init__.py not supported on python 2
SKIP [1] /usr/ports/devel/py-pytest/work/pytest-3.0.0/testing/code/test_excinfo.py:233: could not import 'decorator'
SKIP [1] testing/test_assertrewrite.py:284: condition: sys.version_info < (3,5)
SKIP [1] /usr/ports/devel/py-pytest/work/pytest-3.0.0/testing/test_parseopt.py:282: bash provided no output, argcomplete not available?
SKIP [1] testing/code/test_excinfo.py:975: condition: sys.version_info[0] < 3
SKIP [1] /usr/ports/devel/py-pytest/work/pytest-3.0.0/testing/python/integration.py:106: could not import 'unittest.mock'

================================================================================= FAILURES =================================================================================
___________________________________________________________________ TestNewAPI.test_cache_failure_warns ____________________________________________________________________

self = <test_cache.TestNewAPI instance at 0x809eff878>, testdir = <Testdir local('/tmp/pytest-of-yuri/pytest-121/testdir/test_cache_failure_warns0')>

    @pytest.mark.skipif(sys.platform.startswith('win'), reason='no chmod on windows')
    def test_cache_failure_warns(self, testdir):
        testdir.tmpdir.ensure_dir('.cache').chmod(0)
        testdir.makepyfile("""
                def test_error():
                    raise Exception

            """)
        result = testdir.runpytest('-rw')
        assert result.ret == 1
        result.stdout.fnmatch_lines([
            "*could not create cache path*",
>           "*1 pytest-warnings*",
        ])
E       Failed: nomatch: '*could not create cache path*'
E           and: u'============================= test session starts =============================='
            and: u'platform freebsd10 -- Python 2.7.12, pytest-3.0.0, py-1.4.31, pluggy-0.3.1'
E           and: u'rootdir: /tmp/pytest-of-yuri/pytest-121/testdir/test_cache_failure_warns0, inifile: '
E           and: u'plugins: xdist-1.13.1, hypothesis-1.18.1'
E           and: u'collected 1 items'
E           and: u''
E           and: u'test_cache_failure_warns.py F'
E           and: u''
E           and: u'=================================== FAILURES ==================================='
E           and: u'__________________________________ test_error __________________________________'
E           and: u''
E           and: u'    def test_error():'
E           and: u'>       raise Exception'
E           and: u'E       Exception'
E           and: u''
E           and: u'test_cache_failure_warns.py:2: Exception'
E           and: u'=========================== 1 failed in 0.01 seconds ==========================='
E           and: u''
E       remains unmatched: '*could not create cache path*'

/usr/ports/devel/py-pytest/work/pytest-3.0.0/testing/test_cache.py:57: Failed
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
============================= test session starts ==============================
platform freebsd10 -- Python 2.7.12, pytest-3.0.0, py-1.4.31, pluggy-0.3.1
rootdir: /tmp/pytest-of-yuri/pytest-121/testdir/test_cache_failure_warns0, inifile: 
plugins: xdist-1.13.1, hypothesis-1.18.1
collected 1 items

test_cache_failure_warns.py F

=================================== FAILURES ===================================
__________________________________ test_error __________________________________

    def test_error():
>       raise Exception
E       Exception

test_cache_failure_warns.py:2: Exception
=========================== 1 failed in 0.01 seconds ===========================
====================================================== 1 failed, 1653 passed, 8 skipped, 29 xfailed in 152.07 seconds ======================================================
*** Error code 1

[nicoddemus]

Thanks for the report.

Actually a single test has failed, TestNewAPI.test_cache_failure_warns.

The problem here is that, for some reason, even after calling testdir.tmpdir.ensure_dir('.cache').chmod(0) pytest was still able to create subdirectories in the cache dir.

Could you run this in your system? The second ensure_dir call should fail.

import py.path
py.path.local().ensure_dir('cache').chmod(0)
py.path.local().ensure_dir('cache/foo')

[yurivict]

$ python test.py
Traceback (most recent call last):
  File "x.py", line 3, in <module>
    py.path.local().ensure_dir('cache/foo')
  File "/usr/local/lib/python2.7/site-packages/py/_path/common.py", line 233, in ensure_dir
    return self.ensure(*args, **{"dir": True})
  File "/usr/local/lib/python2.7/site-packages/py/_path/local.py", line 512, in ensure
    return p._ensuredirs()
  File "/usr/local/lib/python2.7/site-packages/py/_path/local.py", line 497, in _ensuredirs
    self.mkdir()
  File "/usr/local/lib/python2.7/site-packages/py/_path/local.py", line 447, in mkdir
    py.error.checked_call(os.mkdir, getattr(p, "strpath", p))
  File "/usr/local/lib/python2.7/site-packages/py/_error.py", line 84, in checked_call
    raise cls("%s%r" % (func.__name__, args))
py.error.EACCES: [Permission denied]: mkdir('/tmp/cache/foo',)

[nicoddemus]

Thanks for running the sample.

Not sure what might be the problem then, perhaps others can chime in?

[The-Compiler]

If you do this in a temporary directory:

$ mkdir .cache
$ chmod 0 .cache
$ cat > test_foo.py
def test_foo():
    raise Exception
<Hit Ctrl-D>
$ py.test -rw

do you get a warning?

[yurivict]

collected 0 items / 1 errors 

================================================================================== ERRORS ==================================================================================
____________________________________________________________________________ ERROR collecting  _____________________________________________________________________________
/usr/local/lib/python2.7/site-packages/py/_path/common.py:332: in visit
    for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
/usr/local/lib/python2.7/site-packages/py/_path/common.py:378: in gen
    for p in self.gen(subdir):
/usr/local/lib/python2.7/site-packages/py/_path/common.py:363: in gen
    entries = path.listdir()
/usr/local/lib/python2.7/site-packages/py/_path/local.py:379: in listdir
    names = py.error.checked_call(os.listdir, self.strpath)
/usr/local/lib/python2.7/site-packages/py/_error.py:84: in checked_call
    raise cls("%s%r" % (func.__name__, args))
E   EACCES: [Permission denied]: listdir('/tmp/ar6c0Bg5',)
========================================================================== pytest-warning summary ==========================================================================
WI9 None could not create cache path /tmp/.cache/v/cache/lastfailed
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================ 1 pytest-warnings, 1 error in 0.41 seconds ================================================================

[The-Compiler]

Probably it failed there because it tried to collect some tests from /tmp subdirectories. However, the warning is there, and it isn't in the test... No idea why!

[nicoddemus]

Any news regarding this issue or we can close it?

[jezifm]

Experiencing same issue running tests using root

testdir.tmpdir.ensure_dir(".pytest_cache").chmod(0) was not able to prevent creation of subdirectory.

Test was done using docker

Dockerfile

FROM python:3.6
RUN pip install --upgrade --pre tox
RUN pip install -U setuptools setuptools_scm
RUN pip install pre-commit
COPY pytest /pytest
WORKDIR /pytest
RUN pre-commit install-hooks
CMD ["tox","--recreate"]

[nicoddemus]

Closing as this has not seen activity in awhile.