Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tests fail with python3.8, latest version pulled from master (Linux Mint / Ubuntu) #581

Closed
nicolashainaux opened this issue Jan 9, 2021 · 14 comments
Closed

Tests fail with python3.8, latest version pulled from master (Linux Mint / Ubuntu) #581

nicolashainaux opened this issue Jan 9, 2021 · 14 comments
Labels
bug packaging

Comments

@nicolashainaux
Copy link

Hello,

Trying to update the (unofficial) Ubuntu 20.04 package on my PPA to a newer version, I got an error in the tests.
I reproduced it at home (Linux Mint), just running python3.8 -m pytest:
(NB I tried also with the 4.3.3 release (pointing git's HEAD to 3bc6f0c Release 4.3.3) and got the exact same output)

$ python3.8 -m pytest
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-4.6.9, py-1.9.0, pluggy-0.13.1
rootdir: /home/nico/.local/dev/pyfakefs
plugins: cov-2.8.1, mock-1.10.4, pyfakefs-4.2.dev0
collected 2024 items                                                           

pyfakefs/pytest_tests/pytest_check_failed_plugin_test.py s               [  0%]
pyfakefs/pytest_tests/pytest_fixture_param_test.py x..                   [  0%]
pyfakefs/pytest_tests/pytest_fixture_test.py x..                         [  0%]
pyfakefs/pytest_tests/pytest_plugin_test.py ...                          [  0%]
pyfakefs/tests/dynamic_patch_test.py .....                               [  0%]
pyfakefs/tests/example_test.py .......s                                  [  1%]
pyfakefs/tests/fake_filesystem_glob_test.py .........                    [  1%]
pyfakefs/tests/fake_filesystem_shutil_test.py ...............s...sssssss [  2%]
ssssssssssssss...........sssssssssss                                     [  4%]
pyfakefs/tests/fake_filesystem_test.py ................................. [  6%]
........................................................................ [  9%]
................................................................s....... [ 13%]
......................................................................   [ 16%]
pyfakefs/tests/fake_filesystem_unittest_test.py ........................ [ 18%]
.x...............................x..........................             [ 20%]
pyfakefs/tests/fake_filesystem_vs_real_test.py ......................... [ 22%]
..                                                                       [ 22%]
pyfakefs/tests/fake_open_test.py ....................................... [ 24%]
................................ssssssssssssssssssssssssssssssssssssssss [ 27%]
sssssssssssssssssssssssssssssss...sss..sss............ssssssssssss...... [ 31%]
....ssssssssss.......sssssss.....sssss.....sss..ss.....sssss............ [ 34%]
......ssssssssssssssssss                                                 [ 36%]
pyfakefs/tests/fake_os_test.py ......................................... [ 38%]
........................................................................ [ 41%]
...........................F............................................ [ 45%]
........................................................................ [ 48%]
....................ssssssssssssssssssssssssssssssssssssssssssssssssssss [ 52%]
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 55%]
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 59%]
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 63%]
sssssssss............................................................... [ 66%]
....................ssssssssssssssssssssssssssssssssssssssssssssssssssss [ 70%]
sssssssssssssssssssssssssssssss...................s..................... [ 73%]
..............s......sssssssssssssssssssssssssssssssssssssssssssssssss.. [ 77%]
.........sssssssssss......................................ssssssssssssss [ 80%]
............ssssssssssssss............ssssssssssssss............ssssssss [ 84%]
ssssss.......sss                                                         [ 85%]
pyfakefs/tests/fake_pathlib_test.py ....s.ssssssssssssss.s......ssssssss [ 86%]
...........s.sss...sssssssssssssssssss...s.....s..............ssssssssss [ 90%]
ssssssssssssss.....................................sssssssssssssssssssss [ 94%]
ssssssssssssssss.....                                                    [ 95%]
pyfakefs/tests/fake_stat_time_test.py .......sssssss........ssssssss.... [ 96%]
...sssssss........ssssssss.....sssss......ssssss                         [ 99%]
pyfakefs/tests/fake_tempfile_test.py ........                            [ 99%]
pyfakefs/tests/mox3_stubout_test.py .........                            [100%]

=================================== FAILURES ===================================
___________________ FakeOsModuleTest.test_open_existing_pipe ___________________

self = <pyfakefs.tests.fake_os_test.FakeOsModuleTest testMethod=test_open_existing_pipe>

    def test_open_existing_pipe(self):
        if self.is_pypy:
            raise unittest.SkipTest('Does not work correctly with PyPy')
        read_fd, write_fd = self.os.pipe()
        with self.open(write_fd, 'wb') as f:
>           self.assertEqual(4, f.write(b'test'))

pyfakefs/tests/fake_os_test.py:2709: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pyfakefs.fake_filesystem.FakePipeWrapper object at 0x7fe042b61d30>
contents = b'test'

    def write(self, contents):
        """Write to the real pipe."""
>       return os.write(self.fd, contents)
E       OSError: [Errno 9] Bad file descriptor

pyfakefs/fake_filesystem.py:5156: OSError
======== 1 failed, 1212 passed, 807 skipped, 4 xfailed in 6.16 seconds =========
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3/dist-packages/pytest.py", line 102, in <module>
    raise SystemExit(pytest.main())
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 82, in main
    return config.hook.pytest_cmdline_main(config=config)
  File "/home/nico/.local/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/nico/.local/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/nico/.local/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/home/nico/.local/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/home/nico/.local/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/nico/.local/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 243, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/usr/lib/python3/dist-packages/_pytest/main.py", line 238, in wrap_session
    config._ensure_unconfigure()
  File "/usr/lib/python3/dist-packages/_pytest/config/__init__.py", line 732, in _ensure_unconfigure
    fin()
  File "/usr/lib/python3/dist-packages/_pytest/capture.py", line 127, in stop_global_capturing
    self._global_capturing.stop_capturing()
  File "/usr/lib/python3/dist-packages/_pytest/capture.py", line 532, in stop_capturing
    self.in_.done()
  File "/usr/lib/python3/dist-packages/_pytest/capture.py", line 611, in done
    _attempt_to_close_capture_file(self.tmpfile)
  File "/usr/lib/python3/dist-packages/_pytest/capture.py", line 850, in _attempt_to_close_capture_file
    f.close()
OSError: [Errno 9] Bad file descriptor

Environment details

Python 3.8.5 (default, Jul 28 2020, 12:59:40) 
[GCC 9.3.0]
pyfakefs 4.4dev
@mrbean-bremen
Copy link
Member

Thank you - this passes in Ubuntu 18.04. This is what GH actions uses at the moment as Ubuntu-latest, but they will switch to 20.4 soon, so this will come up anyway... has to be some change in the pipe implementation.

@mrbean-bremen
Copy link
Member

Hm, with Ubuntu 20.04 the error does not occur in CI, didn't test Mint... maybe I misunderstood and you used a version newer than 20.04?

@nicolashainaux
Copy link
Author

No, this is 20.04. My Linux Mint at home is based on 20.04 (I checked again, this is sure) and the tests on Launchpad are run to build a focal package, so it must be on a 20.04 virtual machine too.

I used git checkout to travel through recent changes in history and found the exact commit that triggers the error, it's this one:

6f8d80a Allow to open existing pipe file descriptor - fixes #493

Now, this would be interesting to find out the exact difference that makes the test fail on my machine and on Launchpad, but not in CI. I don't know where is the build log for the 20.04 VM you've used. I've checked a random recent build log and seen that the versions you're using, even on Ubuntu 18.04, look more recent than mine (at home and in the launchpad VM: platform linux -- Python 3.8.5, pytest-4.6.9, in your CI tests: platform linux -- Python 3.8.6, pytest-6.2.1). I don't know why it is so outdated. I will try to update this all and run the tests again. Once done I'll tell you.

Here is the link to the launchpad build log that failed (it's about one week ago, so a little bit older than on my machine, but it's the same test failing).

@mrbean-bremen
Copy link
Member

mrbean-bremen commented Jan 9, 2021 •
edited

Thanks! Here is the build log of the GHActions build for comparison. It uses Ubuntu 20.04.1 and Python 3.8.6 (the pytest version doesn't matter, as pytest is not used for the failing test).

Allow to open existing pipe file descriptor

Yes, the test is a check for that commit that improves handling of pipes. It partly relies on pipes in the real file system, so a change in the pipe implementation might cause the problem.

@nicolashainaux
Copy link
Author

Well, I installed python 3.8.6 via pyenv (and the latest pytest), but the tests fail with the same error, plus a bunch of warnings. I am a bit stuck... Same python version but the tests fail on one machine and not another...

As to the pipe implementation, I don't know the details about this, but could that mean the linux kernel version could be of importance? (If so, my current local version is 5.4.0-60-generic; launchpad's VM one is 4.15.0-128-generic; I couldn't find the one used in the GH's virtual environment).

I don't know if the warnings are relevant. Here is the log, in case it's useful:

pyfakefs [:6f8d80a|*1] $ pyenv global 3.8.6
pyfakefs [:6f8d80a|*1] $ pytest
============================= test session starts ==============================
platform linux -- Python 3.8.6, pytest-6.2.1, py-1.9.0, pluggy-0.13.1
rootdir: /home/nico/.local/dev/pyfakefs
collected 1967 items                                                           

pyfakefs/pytest_tests/pytest_check_failed_plugin_test.py s               [  0%]
pyfakefs/pytest_tests/pytest_fixture_param_test.py x..                   [  0%]
pyfakefs/pytest_tests/pytest_fixture_test.py x..                         [  0%]
pyfakefs/pytest_tests/pytest_plugin_test.py ...                          [  0%]
pyfakefs/tests/dynamic_patch_test.py .....                               [  0%]
pyfakefs/tests/example_test.py .......s                                  [  1%]
pyfakefs/tests/fake_filesystem_glob_test.py .........                    [  1%]
pyfakefs/tests/fake_filesystem_shutil_test.py ...............s...sssssss [  2%]
ssssssssssssss...........sssssssssss                                     [  4%]
pyfakefs/tests/fake_filesystem_test.py ................................. [  6%]
........................................................................ [ 10%]
................................................................s....... [ 13%]
....................................................................     [ 17%]
pyfakefs/tests/fake_filesystem_unittest_test.py .....................x.. [ 18%]
...x..................                                                   [ 19%]
pyfakefs/tests/fake_filesystem_vs_real_test.py ......................... [ 20%]
..                                                                       [ 20%]
pyfakefs/tests/fake_open_test.py ....................................... [ 22%]
................................ssssssssssssssssssssssssssssssssssssssss [ 26%]
sssssssssssssssssssssssssssssss............ssssssssssss..........sssssss [ 30%]
sss.......sssssss.....sssss.....sss..ss.....sssss..................sssss [ 33%]
sssssssssssss                                                            [ 34%]
pyfakefs/tests/fake_os_test.py ......................................... [ 36%]
........................................................................ [ 40%]
...........................F............................................ [ 43%]
........................................................................ [ 47%]
....................ssssssssssssssssssssssssssssssssssssssssssssssssssss [ 51%]
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 54%]
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 58%]
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 62%]
sssssssss............................................................... [ 65%]
....................ssssssssssssssssssssssssssssssssssssssssssssssssssss [ 69%]
sssssssssssssssssssssssssssssss...................s..................... [ 73%]
..............s......sssssssssssssssssssssssssssssssssssssssssssssssss.. [ 76%]
........ssssssssss......................................ssssssssssssss.. [ 80%]
..........ssssssssssssss............ssssssssssssss............ssssssssss [ 84%]
ssss.......sss                                                           [ 84%]
pyfakefs/tests/fake_pathlib_test.py ....s.ssssssssssssss.s......ssssssss [ 86%]
...........s.sss...sssssssssssssssssss...s..................ssssssssssss [ 90%]
ssssssssss.....................................sssssssssssssssssssssssss [ 94%]
ssssssssssss.....                                                        [ 94%]
pyfakefs/tests/fake_stat_time_test.py .......sssssss........ssssssss.... [ 96%]
...sssssss........ssssssss.....sssss......ssssss                         [ 99%]
pyfakefs/tests/fake_tempfile_test.py ........                            [ 99%]
pyfakefs/tests/mox3_stubout_test.py .........                            [100%]

=================================== FAILURES ===================================
___________________ FakeOsModuleTest.test_open_existing_pipe ___________________

self = <pyfakefs.tests.fake_os_test.FakeOsModuleTest testMethod=test_open_existing_pipe>

    def test_open_existing_pipe(self):
        if self.is_pypy:
            raise unittest.SkipTest('Does not work correctly with PyPy')
        read_fd, write_fd = self.os.pipe()
        with self.open(write_fd, 'wb') as f:
>           self.assertEqual(4, f.write(b'test'))

pyfakefs/tests/fake_os_test.py:2709: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pyfakefs.fake_filesystem.FakePipeWrapper object at 0x7f39b6bbe550>
contents = b'test'

    def write(self, contents):
        """Write to the real pipe."""
>       return os.write(self.fd, contents)
E       OSError: [Errno 9] Bad file descriptor

pyfakefs/fake_filesystem.py:5025: OSError
=============================== warnings summary ===============================
pyfakefs/pytest_tests/pytest_plugin_test.py::test_pause_resume
  /home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7f39b81894c0>
  
  Traceback (most recent call last):
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 440, in __del__
      self.close()
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 436, in close
      unlink(self.name)
  FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpq_kjfhoq'
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

pyfakefs/pytest_tests/pytest_plugin_test.py::test_pause_resume_contextmanager
  /home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7f39b81894c0>
  
  Traceback (most recent call last):
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 440, in __del__
      self.close()
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 436, in close
      unlink(self.name)
  FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpfeiukrq7'
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

pyfakefs/tests/fake_filesystem_unittest_test.py::PauseResumeTest::test_pause_resume
  /home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7f39b81894c0>
  
  Traceback (most recent call last):
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 440, in __del__
      self.close()
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 436, in close
      unlink(self.name)
  FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp61mr70ev'
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

pyfakefs/tests/fake_filesystem_unittest_test.py::PauseResumeTest::test_pause_resume_contextmanager
  /home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7f39b81894c0>
  
  Traceback (most recent call last):
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 440, in __del__
      self.close()
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 436, in close
      unlink(self.name)
  FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpgbgqe4u1'
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

pyfakefs/tests/fake_filesystem_unittest_test.py::PauseResumeTest::test_pause_resume_fs
  /home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7f39b81894c0>
  
  Traceback (most recent call last):
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 440, in __del__
      self.close()
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 436, in close
      unlink(self.name)
  FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp33ijz4r6'
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

pyfakefs/tests/fake_filesystem_unittest_test.py::PauseResumeTest::test_pause_resume_fs_contextmanager
  /home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7f39b81894c0>
  
  Traceback (most recent call last):
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 440, in __del__
      self.close()
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 436, in close
      unlink(self.name)
  FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp76p_uvk4'
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

pyfakefs/tests/fake_filesystem_unittest_test.py::PauseResumePatcherTest::test_pause_resume
  /home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7f39b81894c0>
  
  Traceback (most recent call last):
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 440, in __del__
      self.close()
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 436, in close
      unlink(self.name)
  FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp5fv_nchn'
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

pyfakefs/tests/fake_filesystem_unittest_test.py::PauseResumePatcherTest::test_pause_resume_contextmanager
  /home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7f39b81894c0>
  
  Traceback (most recent call last):
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 440, in __del__
      self.close()
    File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/tempfile.py", line 436, in close
      unlink(self.name)
  FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp0exi5hac'
  
    warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED pyfakefs/tests/fake_os_test.py::FakeOsModuleTest::test_open_existing_pipe
===== 1 failed, 1165 passed, 797 skipped, 4 xfailed, 8 warnings in 20.84s ======
Traceback (most recent call last):
  File "/home/nico/.pyenv/versions/3.8.6/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/config/__init__.py", line 185, in console_main
    code = main()
  File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/config/__init__.py", line 162, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/home/nico/.local/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/home/nico/.local/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/home/nico/.local/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/home/nico/.local/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/home/nico/.local/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/home/nico/.local/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/main.py", line 316, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/main.py", line 311, in wrap_session
    config._ensure_unconfigure()
  File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/config/__init__.py", line 991, in _ensure_unconfigure
    fin()
  File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/capture.py", line 694, in stop_global_capturing
    self._global_capturing.stop_capturing()
  File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/capture.py", line 610, in stop_capturing
    self.in_.done()
  File "/home/nico/.pyenv/versions/3.8.6/lib/python3.8/site-packages/_pytest/capture.py", line 443, in done
    self.tmpfile.close()
OSError: [Errno 9] Bad file descriptor

@mrbean-bremen
Copy link
Member

mrbean-bremen commented Jan 10, 2021 •
edited

Thank you for the investigation, much appreciated! The warnings are unrelated, but I will have a look into them later (they belong to the pause/resume functionality) - good to know.
As for the pipe - I will try later to reproduce this in Ubunto 20.04 locally (currently working under Windows). As for now, I have no real idea. As I said, we use the real os pipe in this code, including the file descriptors, so the kernel version may certainly play a role here. Which kernel version is used in the CI is indeed not clear, I had a look at it yesterday, too.

UPDATE: uname -a in the CI enviroment shows:

Linux fv-az212-589 5.4.0-1032-azure #33-Ubuntu SMP Fri Nov 13 14:23:34 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

This is some Azure tailored kernel, not sure how this compares to the standard kernel versions.

This may take a bit, especially if I'm not able to reproduce it. I assume this is not an urgent problem for you (you are not trying to packing pyfakefs with that version or something)?

@nicolashainaux
Copy link
Author

No, I do not need the newest features. I will package from a commit that works (c8a0f0c).
If we find where the problem comes from, this would probably not help much, unfortunately, because I don't think I have any control over the software or kernel versions in the Launchpad VM, which is required to use in order to publish the package. This would mean wait and publish a newer python3-pyfakefs package for a future Ubuntu version only.
Thank you for your quick answers!

@mrbean-bremen
Copy link
Member

Ok, sorry for all the confusion - nothing to do with the kernel version, but with the test framework! 😶
In the CI, the tests are run using unittest, but python setup.py test uses pytest. I had actually forgotten this... I had changed this myself some time ago because of some packaging issues.
So this even fails under Ubuntu 18.04 (which I still have installed, as it turns out). I still don't have an idea as to why there is this difference, but it may have to do with pytest manipulating/using the pipe itself.

@mrbean-bremen
Copy link
Member

I will probably just skip the test for now if it is run from pytest, as I suspect that this is actually a test problem.

mrbean-bremen added a commit that referenced this issue Jan 10, 2021
@mrbean-bremen
Skip test_open_existing_pipe test under pytest
7311044 
- does not work under pytest, see #581
- skip temporary until fixed
@mrbean-bremen
Copy link
Member

Did this in master for the time being. I will check how to fix this properly later.
Replacing the pipe with a fake pipe would be one possibility, but this would probably break other code not related to the file system. While pipes are files, they do not belong to the file system, so I'm a bit undecided how to handle this best. There has been no real use case so far - the original issue that caused this had been using multiprocessing, which will not work well with pyfakefs anyway (this is just for my own reference).

@mrbean-bremen
Copy link
Member

Ok, this looks like a plain bug in pyfakefs now. Bit embarrasing - could have seen this earlier... The reason why it passes in unittest seems to be that the fake file descriptor is mapped to the same real file descriptor, while pytest opens some file on its own before the test, so the descriptors no longer match.

mrbean-bremen added a commit to mrbean-bremen/pyfakefs that referenced this issue Jan 10, 2021
@mrbean-bremen
Fix handling of pipe file descriptors in fake fs
875c500 
- see pytest-dev#581
mrbean-bremen added a commit that referenced this issue Jan 10, 2021
@mrbean-bremen
Add running python setup.py test to CI tests
3b54026 
- may catch additional problems, see #581
@mrbean-bremen
Copy link
Member

@nicolashainaux - the bug is fixed in master, and I have added python setup.py test to the CI build to catch this kind of problems in the future. The warnings you have shown do not show up in the CI build, though.

@mrbean-bremen
Copy link
Member

Ok, after updating to Ubuntu 20.04.1 and installing Python 3.8.6 I still cannot reproduce the warnings, so I'm going to ignore them for now, as they also don't show up in CI. Closing this as fixed - feel free to reopen it if needed. Thanks again for the report!

@nicolashainaux
Copy link
Author

nicolashainaux commented Jan 11, 2021 •
edited

Many thanks for investigating this so quickly and efficiently 👍 Moreover, the warnings are gone, too.
I should be able to package a newer version, then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug packaging
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mrbean-bremen @nicolashainaux