Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use fchmod #214

Merged
merged 1 commit into from Mar 27, 2023
Merged

use fchmod #214

merged 1 commit into from Mar 27, 2023

Conversation

jfennick
Copy link
Contributor

@jfennick jfennick commented Mar 26, 2023

I'm getting a TypeError when py-filelock calls os.chmod(). The problem occurs when using pypy3.9, but not python3.11 I'm running into this in the context of trying to install pre-commit, but the problem appears to be in py-filelock.

$ mamba create --name filelock
$ mamba activate filelock
$ mamba install pypy "python<3.10"
$ pip install pre-commit
$ pre-commit
An unexpected error has occurred: CalledProcessError: command: ('/Users/jakefennick/mambaforge-pypy3/envs/filelock/bin/python3.9', '-mvirtualenv', '/Users/jakefennick/.cache/pre-commit/repocvpepigy/py_env-python3.9')
return code: 1
stdout:
    RuntimeError: failed to build image pip, wheel, setuptools because:
    Traceback (most recent call last):
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/virtualenv/seed/embed/via_app_data/via_app_data.py", line 52, in _install
        with parent.non_reentrant_lock_for_key(wheel_img.name):
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/contextlib.py", line 119, in __enter__
        return next(self.gen)
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/virtualenv/util/lock.py", line 146, in non_reentrant_lock_for_key
        with _CountedFileLock(str(self.path / f"{name}.lock")):
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/filelock/_api.py", line 227, in __enter__
        self.acquire()
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/virtualenv/util/lock.py", line 29, in acquire
        super().acquire(timeout, poll_interval)
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/filelock/_api.py", line 182, in acquire
        self._acquire()
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/filelock/_unix.py", line 38, in _acquire
        os.chmod(fd, self._mode)
    TypeError: expected str, bytes or os.PathLike object, not int
    
    Traceback (most recent call last):
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/virtualenv/seed/embed/via_app_data/via_app_data.py", line 52, in _install
        with parent.non_reentrant_lock_for_key(wheel_img.name):
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/contextlib.py", line 119, in __enter__
        return next(self.gen)
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/virtualenv/util/lock.py", line 146, in non_reentrant_lock_for_key
        with _CountedFileLock(str(self.path / f"{name}.lock")):
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/filelock/_api.py", line 227, in __enter__
        self.acquire()
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/virtualenv/util/lock.py", line 29, in acquire
        super().acquire(timeout, poll_interval)
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/filelock/_api.py", line 182, in acquire
        self._acquire()
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/filelock/_unix.py", line 38, in _acquire
        os.chmod(fd, self._mode)
    TypeError: expected str, bytes or os.PathLike object, not int
    
    Traceback (most recent call last):
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/virtualenv/seed/embed/via_app_data/via_app_data.py", line 52, in _install
        with parent.non_reentrant_lock_for_key(wheel_img.name):
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/contextlib.py", line 119, in __enter__
        return next(self.gen)
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/virtualenv/util/lock.py", line 146, in non_reentrant_lock_for_key
        with _CountedFileLock(str(self.path / f"{name}.lock")):
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/filelock/_api.py", line 227, in __enter__
        self.acquire()
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/virtualenv/util/lock.py", line 29, in acquire
        super().acquire(timeout, poll_interval)
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/filelock/_api.py", line 182, in acquire
        self._acquire()
      File "/Users/jakefennick/mambaforge-pypy3/envs/filelock/lib/pypy3.9/site-packages/filelock/_unix.py", line 38, in _acquire
        os.chmod(fd, self._mode)
    TypeError: expected str, bytes or os.PathLike object, not int
stderr: (none)
Check the log at /Users/jakefennick/.cache/pre-commit/pre-commit.log

https://docs.python.org/3/library/os.html#os.chmod

Although the documentation claims that os.chmod accepts both a path-like object as well as an integer file descriptor, it does not seem to work here. However, in this case we 100% know we have a file descriptor, so directly calling os.fchmod should always work.

@gaborbernat
Copy link
Member

cc @jahrules

@jahrules
Copy link
Contributor

The docs claim that as of python 3.3 fchmod and chmod are equivalent.

But if using fchmod resolves some issue, it shouldn't be a problem.

https://docs.python.org/3/library/os.html?highlight=fchmod#os.fchmod

@jfennick
Copy link
Contributor Author

Note that (if this PR is correct) the CI failure is spurious.

https://results.pre-commit.ci/run/github/17098256/1679850526.TbItmJeCQ3eodvVdUw9VBw

@jahrules
Copy link
Contributor

Note that (if this PR is correct) the CI failure is spurious.

https://results.pre-commit.ci/run/github/17098256/1679850526.TbItmJeCQ3eodvVdUw9VBw

you just need to add fchmod to the whitelist and it will pass.

@jfennick
Copy link
Contributor Author

Note that (if this PR is correct) the CI failure is spurious.
https://results.pre-commit.ci/run/github/17098256/1679850526.TbItmJeCQ3eodvVdUw9VBw

you just need to add fchmod to the whitelist and it will pass.

Done!

Copy link
Member

@gaborbernat gaborbernat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a test if they are not equivalent.

@jahrules
Copy link
Contributor

@jfennick have you confirmed that this change fixes your issue?

@jfennick
Copy link
Contributor Author

Let's add a test if they are not equivalent.

Could you be more specific? I'm not sure what kind of a test you're looking for.

@jfennick
Copy link
Contributor Author

jfennick commented Mar 26, 2023

@jfennick have you confirmed that this change fixes your issue?

Yes, this PR fixes the issue on my machine.

Have you tried running my minimal reproducible example?

@jfennick
Copy link
Contributor Author

$ mamba create --name filelock_src
$ mamba activate filelock_src
$ mamba install pypy "python<3.10"
$ pip install pre-commit
$ pip install filelock@git+https://github.com/jfennick/py-filelock.git
$ pre-commit
[INFO] Installing environment for https://github.com/asottile/add-trailing-comma.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/asottile/pyupgrade.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/PyCQA/isort.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/psf/black.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/asottile/blacken-docs.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/tox-dev/tox-ini-fmt.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/tox-dev/pyproject-fmt.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/PyCQA/flake8.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
check python ast.....................................(no files to check)Skipped
check builtin type constructor use...................(no files to check)Skipped
check docstring is first.............................(no files to check)Skipped
check for merge conflicts............................(no files to check)Skipped
check yaml...........................................(no files to check)Skipped
check toml...........................................(no files to check)Skipped
debug statements (python)............................(no files to check)Skipped
fix end of files.....................................(no files to check)Skipped
trim trailing whitespace.............................(no files to check)Skipped
Add trailing commas..................................(no files to check)Skipped
pyupgrade............................................(no files to check)Skipped
isort................................................(no files to check)Skipped
black................................................(no files to check)Skipped
blacken-docs.........................................(no files to check)Skipped
rst ``code`` is two backticks........................(no files to check)Skipped
tox-ini-fmt..........................................(no files to check)Skipped
pyproject-fmt........................................(no files to check)Skipped
flake8...............................................(no files to check)Skipped
prettier.............................................(no files to check)Skipped
markdownlint.........................................(no files to check)Skipped
Check hooks apply to the repository..................(no files to check)Skipped
Check for useless excludes...........................(no files to check)Skipped

@gaborbernat
Copy link
Member

Let's add a test if they are not equivalent.

Could you be more specific? I'm not sure what kind of a test you're looking for.

A test that would replicate this issue in the CI and shows that this fix fixes it. Generally, you commit first to the test and then to the solution.

@gaborbernat
Copy link
Member

Although the documentation claims that os.chmod accepts both a path-like object as well as an integer file descriptor, it does not seem to work here. However, in this case we 100% know we have a file descriptor, so directly calling os.fchmod should always work.

If this is a pypy issue, how come to our CI did not catch it? Also, if this is a pypy bug why are we fixing it in this codebase and not within pypy? Also, did you submit and link a pypy bug report? I'll not accept PRs that are fixing upstream issues without a very good reason.

@jahrules
Copy link
Contributor

Although the documentation claims that os.chmod accepts both a path-like object as well as an integer file descriptor, it does not seem to work here. However, in this case we 100% know we have a file descriptor, so directly calling os.fchmod should always work.

If this is a pypy issue, how come to our CI did not catch it? Also, if this is a pypy bug why are we fixing it in this codebase and not within pypy? Also, did you submit and link a pypy bug report? I'll not accept PRs that are fixing upstream issues without a very good reason.

I highly suspect that this is down the the OS or filesystem being used. I don't know enough about how the os module interacts with the OS functions to know which is more likely.

@jfennick can you share what OS/filesystem you're using?

@jfennick
Copy link
Contributor Author

The only reason I can think of that it's failing on my machine and not the CI is that I'm on macos Ventura 13.2.1 and the CI has macos 12. My filesystem is the default Apple Filesystem APFS.

@TheMatt2
Copy link
Contributor

While I don't see any issue with using os.fchmod, I can not replicate this bug.

What is the Pypy version? It should be 7.x.x something.
Here is the version string I get when setting up Mamba (in a Docker container)

Python 3.9.16 | packaged by conda-forge | (feeb267e, Jan 18 2023, 16:13:01)
[PyPy 7.3.11 with GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>> 

Considering Pypy is "Python in Python", perhaps this is an old Pypy version, and a compatibility bug in pypy?

@jfennick
Copy link
Contributor Author

Should be the latest and greatest:

$ pypy --version
Python 3.9.16 | packaged by conda-forge | (feeb267e, Jan 18 2023, 16:19:00)
[PyPy 7.3.11 with GCC Clang 14.0.6]

@TheMatt2
Copy link
Contributor

Strange.

Also, it turns out that that error message is not what you would expect from Python < 3.3 either.
The error Python 3.2 gives is as follows:

Python 3.2.6 (default, Mar 26 2023, 23:08:31) 
[GCC 11.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> fd = os.open("test", os.O_CREAT)
>>> os.chmod(fd, 0o644)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Can't convert 'int' object to str implicitly
>>> 

@jfennick
Copy link
Contributor Author

Very strange indeed:

(filelock) jakefennick$ python
Python 3.9.16 | packaged by conda-forge | (feeb267e, Jan 18 2023, 16:19:00)
[PyPy 7.3.11 with GCC Clang 14.0.6] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>> import os
>>>> fd = os.open("test", os.O_CREAT)
>>>> os.chmod(fd, 0o644)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: expected str, bytes or os.PathLike object, not int
>>>> os.fchmod(fd, 0o644)

@TheMatt2
Copy link
Contributor

TheMatt2 commented Mar 27, 2023

I think you got a Pypy bug there... one way or the other...

Python 3.9.16 (feeb267ead3e6771d3f2f49b83e1894839f64fb7, Dec 29 2022, 14:23:21)
[PyPy 7.3.11 with GCC 10.2.1 20210130 (Red Hat 10.2.1-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>> import os
>>>> fd = os.open("test", os.O_CREAT)
>>>> os.chmod(fd, 0o644)
>>>> 

Python 3.9.16 | packaged by conda-forge | (feeb267e, Jan 18 2023, 16:13:01)
[PyPy 7.3.11 with GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>> import os
>>>> fd = os.open("test", os.O_CREAT)
>>>> os.chmod(fd, 0o644)
>>>> 

I also tried your minimal reproducible example (albeit, in a Docker container), and could not reproduce the error.

@jfennick
Copy link
Contributor Author

Here's the relevant section from my os.py

    if _exists("statvfs") and _exists("fstatvfs"): # mac os x10.3
        _add("HAVE_FSTATVFS", "statvfs")
    supports_fd = _set

    _set = set()
    _add("HAVE_FACCESSAT",  "access")
    # Some platforms don't support lchmod().  Often the function exists
    # anyway, as a stub that always returns ENOSUP or perhaps EOPNOTSUPP.
    # (No, I don't know why that's a good design.)  ./configure will detect
    # this and reject it--so HAVE_LCHMOD still won't be defined on such
    # platforms.  This is Very Helpful.
    #
    # However, sometimes platforms without a working lchmod() *do* have
    # fchmodat().  (Examples: Linux kernel 3.2 with glibc 2.15,
    # OpenIndiana 3.x.)  And fchmodat() has a flag that theoretically makes
    # it behave like lchmod().  So in theory it would be a suitable
    # replacement for lchmod().  But when lchmod() doesn't work, fchmodat()'s
    # flag doesn't work *either*.  Sadly ./configure isn't sophisticated
    # enough to detect this condition--it only determines whether or not
    # fchmodat() minimally works.
    #
    # Therefore we simply ignore fchmodat() when deciding whether or not
    # os.chmod supports follow_symlinks.  Just checking lchmod() is
    # sufficient.  After all--if you have a working fchmodat(), your
    # lchmod() almost certainly works too.
    #
    # _add("HAVE_FCHMODAT",   "chmod")
    _add("HAVE_FCHOWNAT",   "chown")
    _add("HAVE_FSTATAT",    "stat")
    _add("HAVE_LCHFLAGS",   "chflags")
    _add("HAVE_LCHMOD",     "chmod")
    if _exists("lchown"): # mac os x10.3
        _add("HAVE_LCHOWN", "chown")

I'm not exactly sure what's going on here, but it sounds like the story around os.chmod isn't so simple. There's a lot of macos 10.3 specific logic here, so maybe they forgot to add similar logic for chmod.

Anyhow, I think the point of this PR is that why are we even giving pypy, etc the chance to mess this up? At this particular call site we 100% know for sure fd is an integer, so why call an unnecessarily general function? Why not minimize your attack surface?

@TheMatt2
Copy link
Contributor

TheMatt2 commented Mar 27, 2023

I also tried your minimal reproducible example (albeit, in a Docker container), and could not reproduce the error.

Setup on an actual Mac.
MacOS 12.6.3

I was able to replicate the error.
It's real.

% mamba create --name filelock_src
% mamba activate filelock_src
% mamba install pypy "python<3.10"
% pip install pre-commit
% pip install filelock
$ git clone https://github.com/tox-dev/py-filelock.git
% pre-commit
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Initializing environment for https://github.com/asottile/add-trailing-comma.
[INFO] Initializing environment for https://github.com/asottile/pyupgrade.
[INFO] Initializing environment for https://github.com/PyCQA/isort.
[INFO] Initializing environment for https://github.com/psf/black.
[INFO] Initializing environment for https://github.com/asottile/blacken-docs.
[INFO] Initializing environment for https://github.com/asottile/blacken-docs:black==23.1.
[INFO] Initializing environment for https://github.com/pre-commit/pygrep-hooks.
[INFO] Initializing environment for https://github.com/tox-dev/tox-ini-fmt.
[INFO] Initializing environment for https://github.com/tox-dev/pyproject-fmt.
[INFO] Initializing environment for https://github.com/PyCQA/flake8.
[INFO] Initializing environment for https://github.com/PyCQA/flake8:flake8-bugbear==23.3.12,flake8-comprehensions==3.11.1,flake8-pytest-style==1.7.2,flake8-spellcheck==0.28,flake8-unused-arguments==0.0.13,flake8-noqa==1.3.1,pep8-naming==0.13.3,flake8-pyproject==1.2.3.
[INFO] Initializing environment for https://github.com/pre-commit/mirrors-prettier.
[INFO] Initializing environment for https://github.com/pre-commit/mirrors-prettier:prettier@2.7.1,@prettier/plugin-xml@2.2.
[INFO] Initializing environment for https://github.com/igorshubovych/markdownlint-cli.
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('/Users/Guest/mambaforge/envs/filelock_src/bin/python3.9', '-mvirtualenv', '/Users/Guest/.cache/pre-commit/repok_yeqbq1/py_env-python3.9')
return code: 1
stdout:
    RuntimeError: failed to build image setuptools, wheel, pip because:
    Traceback (most recent call last):
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/virtualenv/seed/embed/via_app_data/via_app_data.py", line 52, in _install
        with parent.non_reentrant_lock_for_key(wheel_img.name):
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/contextlib.py", line 119, in __enter__
        return next(self.gen)
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/virtualenv/util/lock.py", line 146, in non_reentrant_lock_for_key
        with _CountedFileLock(str(self.path / f"{name}.lock")):
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/filelock/_api.py", line 227, in __enter__
        self.acquire()
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/virtualenv/util/lock.py", line 29, in acquire
        super().acquire(timeout, poll_interval)
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/filelock/_api.py", line 182, in acquire
        self._acquire()
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/filelock/_unix.py", line 38, in _acquire
        os.chmod(fd, self._mode)
    TypeError: expected str, bytes or os.PathLike object, not int
    
    Traceback (most recent call last):
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/virtualenv/seed/embed/via_app_data/via_app_data.py", line 52, in _install
        with parent.non_reentrant_lock_for_key(wheel_img.name):
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/contextlib.py", line 119, in __enter__
        return next(self.gen)
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/virtualenv/util/lock.py", line 146, in non_reentrant_lock_for_key
        with _CountedFileLock(str(self.path / f"{name}.lock")):
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/filelock/_api.py", line 227, in __enter__
        self.acquire()
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/virtualenv/util/lock.py", line 29, in acquire
        super().acquire(timeout, poll_interval)
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/filelock/_api.py", line 182, in acquire
        self._acquire()
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/filelock/_unix.py", line 38, in _acquire
        os.chmod(fd, self._mode)
    TypeError: expected str, bytes or os.PathLike object, not int
    
    Traceback (most recent call last):
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/virtualenv/seed/embed/via_app_data/via_app_data.py", line 52, in _install
        with parent.non_reentrant_lock_for_key(wheel_img.name):
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/contextlib.py", line 119, in __enter__
        return next(self.gen)
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/virtualenv/util/lock.py", line 146, in non_reentrant_lock_for_key
        with _CountedFileLock(str(self.path / f"{name}.lock")):
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/filelock/_api.py", line 227, in __enter__
        self.acquire()
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/virtualenv/util/lock.py", line 29, in acquire
        super().acquire(timeout, poll_interval)
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/filelock/_api.py", line 182, in acquire
        self._acquire()
      File "/Users/Guest/mambaforge/envs/filelock_src/lib/pypy3.9/site-packages/filelock/_unix.py", line 38, in _acquire
        os.chmod(fd, self._mode)
    TypeError: expected str, bytes or os.PathLike object, not int
stderr: (none)
Check the log at /Users/Guest/.cache/pre-commit/pre-commit.log

@gaborbernat
Copy link
Member

gaborbernat commented Mar 27, 2023

I'd like to notify the right people of their bug before we accept a temporary workaround on our side. So I'm holding this PR until we figure out how is this a problem and fill.in a bug issue there. Because per the official documentation this should work.

@jfennick
Copy link
Contributor Author

Excellent, sounds good to me. I can work off my fork in the meantime.

@TheMatt2
Copy link
Contributor

TheMatt2 commented Mar 27, 2023

I think I tracked down part of the issue.

The official Python docs say that os.chmod(fd, mode) will work for Python > 3.2

Pypy, on the other hand, says the ability use a file descriptor "may be unavailable" https://foss.heptapod.net/pypy/pypy/-/blob/59ace91749d15ffc3b60510ae727de51e21f2728/pypy/module/posix/interp_posix.py#L1183-L1199

Change the access permissions of a file.

path may always be specified as a string.
On some platforms, path may also be specified as an open file descriptor.
 If this functionality is unavailable, using it raises an exception.

With the normal Pypy for MacOS, os.chmod() works as you would expect, but when Pypy is installed via Mamba, you see this error.

I note that the Python build information is [PyPy 7.3.11 with GCC Clang 14.0.6]. Normal python builds for MacOS (in my experience) either say "[Clang 14...]" or "[Clang Apple LLVM...]". I've never seen "GCC Clang".

So I guess an issue needs to be raised with Pypy that their chmod() does not always succeed, even if the platform does actually support it? Working out precisely why this is happening... is still a bit beyond me.

I suspect the build server didn't catch it because it may be using micromamba, not the full distribution but I'm not sure.

@gaborbernat
Copy link
Member

I don't think we use mamba to provision Python. It uses the GitHub action. Can someone fill in an issue to pypy and link it?

@jahrules
Copy link
Contributor

@TheMatt2
Copy link
Contributor

TheMatt2 commented Mar 27, 2023

PYPY maintainers please read!!! (Added as a comment to Pypy issue)

This issue applies to the version of pypy as installed in Anaconda on macOS according to the official installation instructions:

The native pypy build for MacOS seems unaffected.

https://www.pypy.org/posts/2022/11/pypy-and-conda-forge.html

(filelock) jakefennick$ python
Python 3.9.16 | packaged by conda-forge | (feeb267e, Jan 18 2023, 16:19:00)
[PyPy 7.3.11 with GCC Clang 14.0.6] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>> import os
>>>> fd = os.open("test", os.O_CREAT)
>>>> os.chmod(fd, 0o644)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: expected str, bytes or os.PathLike object, not int
>>>> os.fchmod(fd, 0o644)

@gaborbernat gaborbernat merged commit c216922 into tox-dev:main Mar 27, 2023
28 checks passed
matfax pushed a commit to matfax/mutapath that referenced this pull request Jul 9, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [filelock](https://togithub.com/tox-dev/py-filelock) | `3.0.12` ->
`3.12.2` |
[![age](https://badges.renovateapi.com/packages/pypi/filelock/3.12.2/age-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://badges.renovateapi.com/packages/pypi/filelock/3.12.2/adoption-slim)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://badges.renovateapi.com/packages/pypi/filelock/3.12.2/compatibility-slim/3.0.12)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://badges.renovateapi.com/packages/pypi/filelock/3.12.2/confidence-slim/3.0.12)](https://docs.renovatebot.com/merge-confidence/)
|

---

### Release Notes

<details>
<summary>tox-dev/py-filelock (filelock)</summary>

###
[`v3.12.2`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.12.2)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.12.1...3.12.2)

#### What's Changed

- Restore 'if TYPE_CHECKING' syntax for FileLock definition by
[@&#8203;dlax](https://togithub.com/dlax) in
[tox-dev/filelock#245

#### New Contributors

- [@&#8203;dlax](https://togithub.com/dlax) made their first
contribution in
[tox-dev/filelock#245

**Full Changelog**:
tox-dev/filelock@3.12.1...3.12.2

###
[`v3.12.1`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.12.1)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.12.0...3.12.1)

##### What's Changed

- Add trusted-publish by
[@&#8203;gaborbernat](https://togithub.com/gaborbernat) in
[tox-dev/filelock#236
- Add 3.12 support by
[@&#8203;gaborbernat](https://togithub.com/gaborbernat) in
[tox-dev/filelock#237
- Bump pypa/gh-action-pypi-publish from 1.8.5 to 1.8.6 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[tox-dev/filelock#239
- git ls-files -z -- .github/workflows/check.yml | xargs -0 sed -i
's|3.12.0-alpha.7|3.12.0-beta.1|g' by
[@&#8203;gaborbernat](https://togithub.com/gaborbernat) in
[tox-dev/filelock#243
- Use ruff by [@&#8203;gaborbernat](https://togithub.com/gaborbernat) in
[tox-dev/filelock#244
- Fix test_bad_lock_file for other OSes by
[@&#8203;TheMatt2](https://togithub.com/TheMatt2) in
[tox-dev/filelock#242

**Full Changelog**:
tox-dev/filelock@3.12.0...3.12.1

###
[`v3.12.0`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.12.0)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.11.0...3.12.0)

#### What's Changed

- Fix:
[#&#8203;225](https://togithub.com/tox-dev/py-filelock/issues/225) :Get
rid of warning about inability to link to \_thread.\_local. by
[@&#8203;csm10495](https://togithub.com/csm10495) in
[tox-dev/filelock#226
- Bump deps and tools by
[@&#8203;gaborbernat](https://togithub.com/gaborbernat) in
[tox-dev/filelock#228
- Add umask check to tests so umask 002 is valid by
[@&#8203;TheMatt2](https://togithub.com/TheMatt2) in
[tox-dev/filelock#227
- Fix lock hang on Windows by
[@&#8203;TheMatt2](https://togithub.com/TheMatt2) in
[tox-dev/filelock#231
- Conditionally disable/enable thread-local lock behavior. by
[@&#8203;csm10495](https://togithub.com/csm10495) in
[tox-dev/filelock#232

**Full Changelog**:
tox-dev/filelock@3.11.0...3.12.0

###
[`v3.11.0`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.11.0)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.10.7...3.11.0)

#### What's Changed

- Bump pypa/gh-action-pypi-publish from 1.8.3 to 1.8.5 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[tox-dev/filelock#218
- Fix
[#&#8203;220](https://togithub.com/tox-dev/py-filelock/issues/220):
Allow filelock test thread to catch any exceptions by
[@&#8203;TheMatt2](https://togithub.com/TheMatt2) in
[tox-dev/filelock#221
- Bump deps and tools by
[@&#8203;gaborbernat](https://togithub.com/gaborbernat) in
[tox-dev/filelock#222
- Run more pypy versions in CI but without coverage by
[@&#8203;gaborbernat](https://togithub.com/gaborbernat) in
[tox-dev/filelock#224
- Make the lock a thread local variable by
[@&#8203;csm10495](https://togithub.com/csm10495) in
[tox-dev/filelock#219

#### New Contributors

- [@&#8203;csm10495](https://togithub.com/csm10495) made their first
contribution in
[tox-dev/filelock#219

**Full Changelog**:
tox-dev/filelock@3.10.7...3.11.0

###
[`v3.10.7`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.10.7)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.10.6...3.10.7)

#### What's Changed

- use fchmod by [@&#8203;jfennick](https://togithub.com/jfennick) in
[tox-dev/filelock#214

#### New Contributors

- [@&#8203;jfennick](https://togithub.com/jfennick) made their first
contribution in
[tox-dev/filelock#214

**Full Changelog**:
tox-dev/filelock@3.10.6...3.10.7

###
[`v3.10.6`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.10.6)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.10.5...3.10.6)

#### What's Changed

- Bugfix/147 by [@&#8203;jahrules](https://togithub.com/jahrules) in
[tox-dev/filelock#213

**Full Changelog**:
tox-dev/filelock@3.10.5...3.10.6

###
[`v3.10.5`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.10.5)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.10.4...3.10.5)

#### What's Changed

- proposed fix for issue
[#&#8203;67](https://togithub.com/tox-dev/py-filelock/issues/67) by
[@&#8203;jahrules](https://togithub.com/jahrules) in
[tox-dev/filelock#212

**Full Changelog**:
tox-dev/filelock@3.10.4...3.10.5

###
[`v3.10.4`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.10.4)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.10.3...3.10.4)

#### What's Changed

- updated os.open to preserve mode by
[@&#8203;jahrules](https://togithub.com/jahrules) in
[tox-dev/filelock#211

**Full Changelog**:
tox-dev/filelock@3.10.3...3.10.4

###
[`v3.10.3`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.10.3)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.10.2...3.10.3)

#### What's Changed

- Bump pypa/gh-action-pypi-publish from 1.8.1 to 1.8.3 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[tox-dev/filelock#207
- bug fix by [@&#8203;jahrules](https://togithub.com/jahrules) in
[tox-dev/filelock#209

**Full Changelog**:
tox-dev/filelock@3.10.2...3.10.3

###
[`v3.10.2`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.10.2)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.10.1...3.10.2)

#### What's Changed

- changed from os.umask to os.chmod by
[@&#8203;jahrules](https://togithub.com/jahrules) in
[tox-dev/filelock#206

**Full Changelog**:
tox-dev/filelock@3.10.1...3.10.2

###
[`v3.10.1`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.10.1)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.10.0...3.10.1)

#### What's Changed

- Bump pypa/gh-action-pypi-publish from 1.7.1 to 1.8.1 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[tox-dev/filelock#200
- Bump deps and tools by
[@&#8203;gaborbernat](https://togithub.com/gaborbernat) in
[tox-dev/filelock#201
- Properly pickle of Timeout objects + test cases by
[@&#8203;TheMatt2](https://togithub.com/TheMatt2) in
[tox-dev/filelock#203

#### New Contributors

- [@&#8203;TheMatt2](https://togithub.com/TheMatt2) made their first
contribution in
[tox-dev/filelock#203

**Full Changelog**:
tox-dev/filelock@3.10.0...3.10.1

###
[`v3.10.0`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.10.0)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.9.1...3.10.0)

#### What's Changed

- added multiuser support and associated tests by
[@&#8203;jahrules](https://togithub.com/jahrules) in
[tox-dev/filelock#192

#### New Contributors

- [@&#8203;jahrules](https://togithub.com/jahrules) made their first
contribution in
[tox-dev/filelock#192

**Full Changelog**:
tox-dev/filelock@3.9.1...3.10.0

###
[`v3.9.1`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.9.1)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.9.0...3.9.1)

#### What's Changed

- \[pre-commit.ci] pre-commit autoupdate by
[@&#8203;pre-commit-ci](https://togithub.com/pre-commit-ci) in
[tox-dev/filelock#188
- Bump deps and tools by
[@&#8203;gaborbernat](https://togithub.com/gaborbernat) in
[tox-dev/filelock#193
- Bump deps and tools by
[@&#8203;gaborbernat](https://togithub.com/gaborbernat) in
[tox-dev/filelock#197
- Bump pypa/gh-action-pypi-publish from 1.6.4 to 1.7.1 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[tox-dev/filelock#198
- use time.perf_counter instead of time.monotonic by
[@&#8203;zpz](https://togithub.com/zpz) in
[tox-dev/filelock#194

#### New Contributors

- [@&#8203;zpz](https://togithub.com/zpz) made their first contribution
in
[tox-dev/filelock#194

**Full Changelog**:
tox-dev/filelock@3.9.0...3.9.1

###
[`v3.9.0`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.9.0)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.8.2...3.9.0)

#### What's Changed

- Move to hatchling build backend by
[@&#8203;gaborbernat](https://togithub.com/gaborbernat) in
[tox-dev/filelock#185

**Full Changelog**:
tox-dev/filelock@3.8.2...3.9.0

###
[`v3.8.2`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.8.2)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.8.1...3.8.2)

#### What's Changed

- Bump pypa/gh-action-pypi-publish from 1.5.1 to 1.6.1 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[tox-dev/filelock#178
- Update the license classifier to "Unlicense" by
[@&#8203;jond01](https://togithub.com/jond01) in
[tox-dev/filelock#180

#### New Contributors

- [@&#8203;jond01](https://togithub.com/jond01) made their first
contribution in
[tox-dev/filelock#180

**Full Changelog**:
tox-dev/filelock@3.8.1...3.8.2

###
[`v3.8.1`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.8.1)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.8.0...3.8.1)

#### What's Changed

- \[pre-commit.ci] pre-commit autoupdate by
[@&#8203;pre-commit-ci](https://togithub.com/pre-commit-ci) in
[tox-dev/filelock#166
- link to flufl.lock by [@&#8203;dholth](https://togithub.com/dholth) in
[tox-dev/filelock#167
- \[pre-commit.ci] pre-commit autoupdate by
[@&#8203;pre-commit-ci](https://togithub.com/pre-commit-ci) in
[tox-dev/filelock#168
- \[pre-commit.ci] pre-commit autoupdate by
[@&#8203;pre-commit-ci](https://togithub.com/pre-commit-ci) in
[tox-dev/filelock#169
- \[pre-commit.ci] pre-commit autoupdate by
[@&#8203;pre-commit-ci](https://togithub.com/pre-commit-ci) in
[tox-dev/filelock#170
- fix BaseFileLock.timeout's getter/setter being obscured by itself by
[@&#8203;dearfl](https://togithub.com/dearfl) in
[tox-dev/filelock#172
- Fix mypy fails understanding FileLock by
[@&#8203;gaborbernat](https://togithub.com/gaborbernat) in
[tox-dev/filelock#177

#### New Contributors

- [@&#8203;dholth](https://togithub.com/dholth) made their first
contribution in
[tox-dev/filelock#167
- [@&#8203;dearfl](https://togithub.com/dearfl) made their first
contribution in
[tox-dev/filelock#172

**Full Changelog**:
tox-dev/filelock@3.8.0...3.8.1

###
[`v3.8.0`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.8.0)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.7.1...3.8.0)

#### What's Changed

- \[pre-commit.ci] pre-commit autoupdate by
[@&#8203;pre-commit-ci](https://togithub.com/pre-commit-ci) in
[tox-dev/filelock#149
- Bump actions/upload-artifact from 2 to 3 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[tox-dev/filelock#154
- Bump actions/download-artifact from 2 to 3 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[tox-dev/filelock#152
- Bump pre-commit/action from 2.0.3 to 3.0.0 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[tox-dev/filelock#151
- Bump actions/checkout from 2 to 3 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[tox-dev/filelock#153
- Bump actions/setup-python from 2 to 4 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[tox-dev/filelock#150
- Add timeout unit to docstrings by
[@&#8203;jnordberg](https://togithub.com/jnordberg) in
[tox-dev/filelock#148
- Unify badges style by
[@&#8203;DeadNews](https://togithub.com/DeadNews) in
[tox-dev/filelock#155
- \[pre-commit.ci] pre-commit autoupdate by
[@&#8203;pre-commit-ci](https://togithub.com/pre-commit-ci) in
[tox-dev/filelock#156
- \[pre-commit.ci] pre-commit autoupdate by
[@&#8203;pre-commit-ci](https://togithub.com/pre-commit-ci) in
[tox-dev/filelock#157
- Check 3.11 support by
[@&#8203;gaborbernat](https://togithub.com/gaborbernat) in
[tox-dev/filelock#158
- \[pre-commit.ci] pre-commit autoupdate by
[@&#8203;pre-commit-ci](https://togithub.com/pre-commit-ci) in
[tox-dev/filelock#159
- Bump dependencies by
[@&#8203;gaborbernat](https://togithub.com/gaborbernat) in
[tox-dev/filelock#160
- \[pre-commit.ci] pre-commit autoupdate by
[@&#8203;pre-commit-ci](https://togithub.com/pre-commit-ci) in
[tox-dev/filelock#162

#### New Contributors

- [@&#8203;dependabot](https://togithub.com/dependabot) made their first
contribution in
[tox-dev/filelock#154
- [@&#8203;jnordberg](https://togithub.com/jnordberg) made their first
contribution in
[tox-dev/filelock#148
- [@&#8203;DeadNews](https://togithub.com/DeadNews) made their first
contribution in
[tox-dev/filelock#155

**Full Changelog**:
tox-dev/filelock@3.7.1...3.8.0

###
[`v3.7.1`](https://togithub.com/tox-dev/py-filelock/compare/3.7.0...3.7.1)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.7.0...3.7.1)

###
[`v3.7.0`](https://togithub.com/tox-dev/py-filelock/compare/3.6.0...3.7.0)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.6.0...3.7.0)

###
[`v3.6.0`](https://togithub.com/tox-dev/py-filelock/compare/3.5.1...3.6.0)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.5.1...3.6.0)

###
[`v3.5.1`](https://togithub.com/tox-dev/py-filelock/compare/3.5.0...3.5.1)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.5.0...3.5.1)

###
[`v3.5.0`](https://togithub.com/tox-dev/py-filelock/compare/3.4.2...3.5.0)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.4.2...3.5.0)

###
[`v3.4.2`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.4.2):
Drop Python 3.6 support

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.4.1...3.4.2)

###
[`v3.4.1`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.4.1):
Add stacklevel to deprecation warnings for argument name change

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.4.0...3.4.1)

###
[`v3.4.0`](https://togithub.com/tox-dev/py-filelock/compare/3.3.2...3.4.0)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.3.2...3.4.0)

###
[`v3.3.2`](https://togithub.com/tox-dev/py-filelock/compare/3.3.1...3.3.2)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.3.1...3.3.2)

###
[`v3.3.1`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.3.1):
Keep filelock logger as not set

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.3.0...3.3.1)

###
[`v3.3.0`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.3.0):
Drop python 2.7+3.5 support and add type annotations

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.2.1...3.3.0)

###
[`v3.2.1`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.2.1):
New documentation and enable logging of our logger on debug level

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.2.0...3.2.1)

###
[`v3.2.0`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.2.0)

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/3.1.0...3.2.0)

1. [#&#8203;96](https://togithub.com/tox-dev/py-filelock/issues/96) -
Raise when trying to acquire in R/O or missing folder
2. [#&#8203;95](https://togithub.com/tox-dev/py-filelock/issues/95) -
Move log from info to debug

###
[`v3.1.0`](https://togithub.com/tox-dev/py-filelock/releases/tag/3.1.0):
Move from module file to package

[Compare
Source](https://togithub.com/tox-dev/py-filelock/compare/v3.0.12...3.1.0)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/matfax/mutapath).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi41LjMiLCJ1cGRhdGVkSW5WZXIiOiIzNi41LjMiLCJ0YXJnZXRCcmFuY2giOiJtYWluIn0=-->

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants