Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python rounds down atime_ns and mtime_ns to the nearest microsecond on macOS #143987

Open
Luflosi opened this issue Oct 31, 2021 · 3 comments
Open
Labels
0.kind: bug Something is broken 6.topic: darwin Running or building packages on Darwin

Comments

@Luflosi
Copy link
Contributor

Luflosi commented Oct 31, 2021

Describe the bug

Executing the python code

import os
import tempfile
a = tempfile.mkdtemp()
os.utime(a, ns=(12, 34))
os.stat(a).st_mtime_ns
os.rmdir(a)

with Python 3.9 on macOS shows an mtime (st_mtime_ns) of 0 ns when it should be 34 nanoseconds. When replacing 34 with 999, st_mtime_ns is still 0 but when replacing it with 1000 or 1001, st_mtime_ns is 1000.
This does not happen with Python 3.9 installed via Homebrew, where the timestamp correctly keeps the 1 nanosecond resolution.
Changing the file path from using tempfile.mkdtemp() to hardcoding a path to a file in the home directory does not change anything.
The Nixpkgs Python version shows this behaviour on two different macOS installations.
On Linux this works correctly when using Python from Nixpkgs.
I determined that os.utime() causes the problem while os.stat() works correctly.

Steps To Reproduce

Steps to reproduce the behavior:

  1. Use macOS
  2. Execute nix-shell -p python3
  3. Execute python3
  4. Copy & paste the above python code
  5. See that st_mtime_ns is 0 instead of 34.

Expected behavior

I expect os.utime() to have nanosecond resolution as with the Homebrew version and on Linux.

Additional context

This causes a test failure with the latest (unreleased) master version of kitty: kovidgoyal/kitty#4175.

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

 - system: `"x86_64-darwin"`
 - host os: `Darwin 19.6.0, macOS 10.15.7`
 - multi-user?: `no`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.5pre20211007_844dd90`
 - channels(user): `"nixpkgs-21.11pre325602.51acb65b302"`
 - nixpkgs: `/Users/user/.nix-defexpr/channels/nixpkgs`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:
@Luflosi Luflosi added the 0.kind: bug Something is broken label Oct 31, 2021
@tpwrules
Copy link
Contributor

It looks like this is because x86_64-darwin currently builds against the macOS 10.12 SDK [1], which does not have available the futimens and utimensat functions [2] that os.utime() needs [3] to set the time with nanosecond resolution. If not present, it falls back to microsecond resolution.

1: #116341
2: https://hydra.nixos.org/log/p0027bpyrykzxn9wj6k31iq2qnz1vws1-python3-3.9.6.drv (search for utimensat)
3: https://github.com/python/cpython/blob/bc8fd7c9c24eb71217ebdb8a12cf38e9dc4215b2/Modules/posixmodule.c#L5168

@veprbl veprbl added the 6.topic: darwin Running or building packages on Darwin label Oct 31, 2021
@Luflosi
Copy link
Contributor Author

Luflosi commented Nov 1, 2021

Awesome, thanks for digging deep and figuring this out.
Should we close this issue since this should be resolved when the SDK is updated?

@r-burns
Copy link
Contributor

r-burns commented Nov 5, 2021

I think it will be useful to be able to refer to this issue until the SDK is updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: bug Something is broken 6.topic: darwin Running or building packages on Darwin
Projects
None yet
Development

No branches or pull requests

4 participants