Skip os.utime() test on NFS?

[smontanaro]

BPO	21483
Nosy	@smontanaro, @pitrou, @larryhastings, @berkerpeksag
Files	test_import.patch: Patch to skip os.utime() when utime returns EINVAL test_import.patch: Fixed truncated line

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-04-22.08:32:41.533>
created_at = <Date 2014-05-12.16:48:28.930>
labels = []
title = 'Skip os.utime() test on NFS?'
updated_at = <Date 2015-04-22.08:32:41.531>
user = 'https://github.com/smontanaro'

bugs.python.org fields:

activity = <Date 2015-04-22.08:32:41.531>
actor = 'berker.peksag'
assignee = 'none'
closed = True
closed_date = <Date 2015-04-22.08:32:41.533>
closer = 'berker.peksag'
components = []
creation = <Date 2014-05-12.16:48:28.930>
creator = 'skip.montanaro'
dependencies = []
files = ['38291', '38346']
hgrepos = []
issue_num = 21483
keywords = ['patch']
message_count = 11.0
messages = ['218336', '218417', '218420', '218431', '236998', '237306', '241735', '241738', '241740', '241788', '241789']
nosy_count = 6.0
nosy_names = ['skip.montanaro', 'pitrou', 'larry', 'python-dev', 'berker.peksag', 'ischwabacher']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue21483'
versions = []

[smontanaro]

I got a strange error during "make test" in a fresh build (hg clone ;
./configure ; make ; make test)

Traceback (most recent call last):
  File "Lib/test/test_import.py", line 293, in test_timestamp_overflow
    os.utime(source, (2 ** 33 - 5, 2 ** 33 - 5))
OSError: [Errno 22] Invalid argument

It took me a little while to figure out what was wrong. A simple check
succeeded:

>>> import os
>>> os.utime("/tmp/trash", (2 ** 33 - 5, 2 ** 33 - 5))
>>> 2**33-5
8589934587

After a bit of poking around, I realized test.support.TESTFN is on an
NFS filesystem in my work environment:

>>> import importlib
>>> from test.support import TESTFN
>>> TESTFN
'@test_31576_tmp'
>>> import sys
>>> sys.path.insert(0, os.curdir)
>>> source = TESTFN + ".py"
>>> compiled = importlib.util.cache_from_source(source)
>>> compiled
'__pycache__/@test_31576_tmp.cpython-35.pyc'
>>> open(source, 'w')
<_io.TextIOWrapper name='@test_31576_tmp.py' mode='w' encoding='ANSI_X3.4-1968'>
>>> source
'@test_31576_tmp.py'
>>> os.path.getmtime(source)
1399912946.9349897
>>> 2**33-5
8589934587
>>> os.utime(source, (2 ** 33 - 5, 2 ** 33 - 5))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument
>>> os.utime("/tmp/trash", (2 ** 33 - 5, 2 ** 33 - 5))
>>> os.curdir
'.'
>>> os.getcwd()
'/home/skipm/3rdParty/python/cpython'
>>>
blade% df -h .
Filesystem                          Size  Used Avail Use% Mounted on
nfshost3:/global/export/home/skipm  2.1T  1.5T  639G  71% /home/skipm

Should this test be skipped on NFS-mounted filesystems? Or should the
test environment try to insure TESTFN is created on a local
filesystem?

Skip

[pitrou]

Or we could simply catch the EINVAL failure and then skip the test.
I suppose the filesystem doesn't support timestamps > 2**32 ?

[larryhastings]

Antoine's plan sounds good to me. And, is it true that NFS only supports four-byte timestamps?

[smontanaro]

Or we could simply catch the EINVAL failure and then skip the test.
I suppose the filesystem doesn't support timestamps > 2**32 ?

Just in that one test? If I passed -1 that would probably also
generate EINVAL (haven't tested).

I'll have to check with the admins to see if this particular
filesystem is NFS3 or NFS4. I'm guessing 2**32 might be the max for
the former. The latter causes us all sorts of other heartburn though,
so even though it might be "more modern" in this regard, we don't use
it much.

[ischwabacher]

Patch to do precisely this. Wish I'd spent more time searching for this thread and less time debugging; it would have saved me a lot of trouble.

[ischwabacher]

...and fixed a spot where git diff + copy/paste truncated a long line. /sheepish

[smontanaro]

Any chance this will make it into 3.5? It's not a big deal, but I notice that it's still present in 3.5.0a4.

[larryhastings]

Issac doesn't have commit bit...

... but you do.

[smontanaro]

On Tue, Apr 21, 2015 at 2:56 PM, Larry Hastings <report@bugs.python.org> wrote:

Issac doesn't have commit bit...

... but you do.

Alas, my commit bit long ago fell into disuse. I haven't checked
anything in since long before the days of Mercurial and the current
workflow. I would almost certainly mess something up. In fact, to
avoid confusion in the future, removal of my commit bit might not be a
bad idea.

[python-dev]

New changeset aa60f4d1a569 by Berker Peksag in branch 'default':
Issue bpo-21483: Skip test_timestamp_overflow on NFS.
https://hg.python.org/cpython/rev/aa60f4d1a569

[berkerpeksag]

Thanks Isaac.