test_handling_bad_descriptor fails on macos 10.14

[sobolevn]

I am trying to run test_pathlib locally on my machine. One test fails:

» ./python.exe -m test -v test_pathlib -m test_handling_bad_descriptor
== CPython 3.12.0a1+ (heads/main:b27b57c6e4, Oct 28 2022, 20:34:51) [Clang 11.0.0 (clang-1100.0.33.16)]
== macOS-10.14.6-x86_64-i386-64bit little-endian
== cwd: /Users/sobolev/Desktop/cpython/build/test_python_5120æ
== CPU count: 4
== encodings: locale=UTF-8, FS=utf-8
0:00:00 load avg: 1.86 Run tests sequentially
0:00:00 load avg: 1.86 [1/1] test_pathlib
test_handling_bad_descriptor (test.test_pathlib.PosixPathTest.test_handling_bad_descriptor) ... ERROR

======================================================================
ERROR: test_handling_bad_descriptor (test.test_pathlib.PosixPathTest.test_handling_bad_descriptor)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sobolev/Desktop/cpython/Lib/test/test_pathlib.py", line 2869, in test_handling_bad_descriptor
    file_descriptors = list(pathlib.Path('/dev/fd').rglob("*"))[3:]
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/pathlib.py", line 894, in rglob
    for p in selector.select_from(self):
  File "/Users/sobolev/Desktop/cpython/Lib/pathlib.py", line 334, in _select_from
    for p in successor_select(starting_point, is_dir, exists, scandir):
  File "/Users/sobolev/Desktop/cpython/Lib/pathlib.py", line 279, in _select_from
    with scandir(parent_path) as scandir_it:
         ^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/pathlib.py", line 864, in _scandir
    return os.scandir(self)
           ^^^^^^^^^^^^^^^^
NotADirectoryError: [Errno 20] Not a directory: '/dev/fd/36'

----------------------------------------------------------------------
Ran 1 test in 0.015s

Here's how it is defined:

cpython/Lib/test/test_pathlib.py

Lines 2865 to 2886 in 7ee3aca

	@unittest.skipIf(sys.platform != "darwin",
	"Bad file descriptor in /dev/fd affects only macOS")
	def test_handling_bad_descriptor(self):
	try:
	file_descriptors = list(pathlib.Path('/dev/fd').rglob("*"))[3:]
	if not file_descriptors:
	self.skipTest("no file descriptors - issue was not reproduced")
	# Checking all file descriptors because there is no guarantee
	# which one will fail.
	for f in file_descriptors:
	f.exists()
	f.is_dir()
	f.is_file()
	f.is_symlink()
	f.is_block_device()
	f.is_char_device()
	f.is_fifo()
	f.is_socket()
	except OSError as e:
	if e.errno == errno.EBADF:
	self.fail("Bad file descriptor not handled.")
	raise

My macos version: 10.14.x
Python is built with: ./configure --with-pydebug && make

Looks like something is off, I will try to investigate and send a PR.

[sobolevn]

This is not just about tests:

» ./python.exe                                                        
Python 3.12.0a1+ (heads/main:7ea10567af, Oct 29 2022, 13:15:59) [Clang 11.0.0 (clang-1100.0.33.16)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathlib
>>> list(pathlib.Path('/dev/fd').rglob("*"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/sobolev/Desktop/cpython/Lib/pathlib.py", line 912, in rglob
    for p in selector.select_from(self):
  File "/Users/sobolev/Desktop/cpython/Lib/pathlib.py", line 334, in _select_from
    for p in successor_select(starting_point, is_dir, exists, scandir):
  File "/Users/sobolev/Desktop/cpython/Lib/pathlib.py", line 279, in _select_from
    with scandir(parent_path) as scandir_it:
         ^^^^^^^^^^^^^^^^^^^^
  File "/Users/sobolev/Desktop/cpython/Lib/pathlib.py", line 882, in _scandir
    return os.scandir(self)
           ^^^^^^^^^^^^^^^^
NotADirectoryError: [Errno 20] Not a directory: '/dev/fd/36'

It fails in runtime too.

[sobolevn]

Original PR: #8542

[sobolevn]

My tree results:

» tree /dev/fd
/dev/fd
├── 0
├── 1
├── 2
├── 25
├── 27
├── 28
├── 29
├── 3
├── 30
├── 31
├── 32
├── 33
├── 34
├── 36 [error opening dir]
├── 37 [error opening dir]
├── 38
├── 39
├── 4
├── 40
├── 41
├── 42
├── 44
├── 45
├── 46
├── 47
├── 48
├── 49
├── 5 [error opening dir]
├── 50
├── 51
├── 52
├── 53
├── 54
├── 55
├── 62
├── 63
├── 64
├── 65
├── 66
└── 73

And ls -alh:

» la /dev/fd
total 53M
drw-r--r--  34 root    wheel  1.1K May 28 19:29 36/
drw-r--r-- 121 root    wheel  3.8K Oct 13 21:51 37/
dr--r--r--   1 root    wheel     0 Oct 28 21:14 5/
crw--w----   1 sobolev tty   16, 1 Oct 29 13:24 0
crw--w----   1 sobolev tty   16, 1 Oct 29 13:24 1
crw--w----   1 sobolev tty   16, 1 Oct 29 13:24 2
-rw-r--r--   1 sobolev staff   15M Aug 22 08:12 25
-rw-r--r--   1 sobolev staff   79K Aug 22 08:12 27
prw-rw----   0 sobolev staff     0 Oct 28 22:00 28|
prw-rw----   0 sobolev staff     0 Oct 28 22:00 29|
-rw-r--r--   1 sobolev staff   10M Aug 22 08:12 3
-rw-r--r--   1 sobolev staff  109K Aug 22 08:12 30
-rw-r--r--   1 sobolev staff  224K Aug 22 08:12 31
-rw-r--r--   1 sobolev staff  4.9M Aug 22 08:12 32
prw-rw----   0 sobolev staff     0 Oct 29 12:58 33|
prw-rw----   0 sobolev staff     0 Oct 29 12:58 34|
-rw-r--r--   1 root    wheel   367 Oct 19  2020 38
-rw-r--r--   1 sobolev staff   15M Aug 22 08:12 39
-rw-r--r--   1 sobolev staff  8.5M Aug 22 08:12 4
-rw-------   1 sobolev staff   308 Oct 28 22:00 40
-rw-------   1 sobolev staff     0 Jul 19  2021 41
-rw-------   1 sobolev staff    41 Jul 19  2021 42
-rw-------   1 sobolev staff  257K Oct 11 21:34 44
-rw-------   1 sobolev staff     0 Jul 19  2021 45
-rw-------   1 sobolev staff  8.0K Apr 17  2018 46
-rw-------   1 sobolev staff  264K Oct 29 13:22 47
-rw-------   1 sobolev staff  8.0K Apr 17  2018 48
-rw-------   1 sobolev staff  8.0K Apr 17  2018 49
-rw-r--r--   1 root    wheel   99K Apr  5  2019 50
-rw-r--r--   1 root    wheel  310K Apr  5  2019 51
-rw-r--r--   1 sobolev staff  1.0K Aug 16  2021 52
-rw-r--r--   1 sobolev staff   64K Aug 16  2021 53
-rw-r--r--   1 sobolev staff  1.5K Aug 16  2021 54
-rw-r--r--   1 sobolev staff   64K Aug 16  2021 55
-rw-------   1 sobolev staff   13K Oct 28 22:00 62
-rw-r--r--   1 root    wheel  325K Aug 21  2019 63
-rw-------   1 sobolev staff   56K Oct 29 13:24 64
-rw-------   1 sobolev staff  1.4K Oct 29 13:23 65
crw-rw-rw-   1 root    tty   15, 0 Oct 29 13:24 66
srw-rw-rw-   0 sobolev staff     0 Jan  1  1970 73=

[sobolevn]

Looks like this is specific to my machine. I cannot reproduce it on my other mac.