test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots

[vstinner]

BPO	37400
Nosy	@vstinner, @koobs, @miss-islington
PRs	bpo-37400: pythoninfo logs getpwuid and getgrouplist #14373 bpo-37400: Fix test_os.test_chown() #14374 [3.8] bpo-37400: Fix test_os.test_chown() (GH-14374) #14377 [3.7] bpo-37400: Fix test_os.test_chown() (GH-14374) #14378

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-06-25.20:26:55.446>
created_at = <Date 2019-06-25.11:07:09.612>
labels = ['3.8', '3.7', 'tests', '3.9']
title = 'test_os: test_chown() started to fail on AMD64 FreeBSD 10-STABLE Non-Debug 3.* buildbots'
updated_at = <Date 2019-06-25.20:26:55.443>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2019-06-25.20:26:55.443>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-06-25.20:26:55.446>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2019-06-25.11:07:09.612>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 37400
keywords = ['patch']
message_count = 10.0
messages = ['346508', '346510', '346512', '346515', '346526', '346528', '346532', '346535', '346556', '346557']
nosy_count = 3.0
nosy_names = ['vstinner', 'koobs', 'miss-islington']
pr_nums = ['14373', '14374', '14377', '14378']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue37400'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']

[vstinner]

https://buildbot.python.org/all/#/builders/167/builds/1265

test_chown (test.test_os.ChownFileTests) ... ERROR

======================================================================
ERROR: test_chown (test.test_os.ChownFileTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd10.nondebug/build/Lib/test/test_os.py", line 1327, in test_chown
    os.chown(support.TESTFN, uid, gid_1)
PermissionError: [Errno 1] Operation not permitted: '@test_95158_tmp'

Extract of the test:

    import grp
    groups = [g.gr_gid for g in grp.getgrall() if getpass.getuser() in g.gr_mem]
    if hasattr(os, 'getgid'):
        process_gid = os.getgid()
        if process_gid not in groups:
            groups.append(process_gid)

@unittest.skipUnless(len(groups) > 1, "test needs more than one group")
def test_chown(self):
    gid_1, gid_2 = groups[:2]
    uid = os.stat(support.TESTFN).st_uid
    os.chown(support.TESTFN, uid, gid_1)  # <======== FAIL HERE
    gid = os.stat(support.TESTFN).st_gid
    self.assertEqual(gid, gid_1)
    os.chown(support.TESTFN, uid, gid_2)
    gid = os.stat(support.TESTFN).st_gid
    self.assertEqual(gid, gid_2)

Extract of test.pythoninfo:

os.uid: 1002
os.umask: 077
os.gid: 1002
os.groups: 1002

I'm not sure that the code to manually get groups from grp.getgrall() is correct. Why not relying on the *current* groups, os.getgroups()?

Note: there is also os.getgrouplist(), I'm not sure of the difference between os.getgroups() and os.getgrouplist(). I know that os.getgrouplist() was modified recently to use:

/*
 * NGROUPS_MAX is defined by POSIX.1 as the maximum
 * number of supplimental groups a users can belong to.
 * We have to increment it by one because
 * getgrouplist() returns both the supplemental groups
 * and the primary group, i.e. all of the groups the
 * user belongs to.
 */
ngroups = 1 + MAX_GROUPS;

[koobs]

This looks like a reincarnation of bpo-27838

[koobs]

And I just remembered that I had to restart the build worker service on that host (koobs-freebsd10) this morning as I was receiving messaging that the worker had gone missing.

I ran [koobs@10-STABLE-amd64:~] sudo /usr/local/etc/rc.d/buildslave restart

which is the same as the prevailing reproduction case in bpo-27838

I could restart the worker to make the issue go away, but I think the underlying issue should be fixed instead.

Should we close this as a dupe and reopen the original?

[vstinner]

New changeset 9cb2741 by Victor Stinner in branch 'master':
bpo-37400: pythoninfo logs getpwuid and getgrouplist (GH-14373)
9cb2741

[vstinner]

I merged my pythoninfo change, more complete pythoninfo from the buildbot:

os.getgid: 1002
os.getgrouplist: 1002
os.getgroups: 1002
os.getuid: 1002

pwd.getpwuid(1002): pwd.struct_passwd(pw_name='buildbot', pw_passwd='*', pw_uid=1002, pw_gid=1002, pw_gecos='FreeBSD BuildBot', pw_dir='/home/buildbot', pw_shell='/bin/sh')

IMHO the test is wrong: it should rely on os.getgroups() rather tan "discover" groups from pw.getpwall().

[vstinner]

I'm connected to the FreeBSD CURRENT buildbot as the user "haypo". I'm unable to reproduce the bug:

CURRENT-amd64% id
uid=1003(haypo) gid=1003(haypo) groups=1003(haypo)

CURRENT-amd64% ./python -m test test_os -m test_chown
(...)
Tests result: SUCCESS

Moreover, it seems like something changes on the buildbot, since the user "buildbot" now has 0 group!?

$ ./python
>>> [g.gr_gid for g in grp.getgrall() if 'buildbot' in g.gr_mem]
[]

Or maybe as the user "haypo", I cannot see *all* groups.

In case of doubt, I will blindly apply my fix PR 14374.

[vstinner]

New changeset d7c87d9 by Victor Stinner in branch 'master':
bpo-37400: Fix test_os.test_chown() (GH-14374)
d7c87d9

[miss-islington]

New changeset 12d174b by Miss Islington (bot) in branch '3.8':
bpo-37400: Fix test_os.test_chown() (GH-14374)
12d174b

[vstinner]

New changeset 1d4b6ba by Victor Stinner (Miss Islington (bot)) in branch '3.7':
bpo-37400: Fix test_os.test_chown() (GH-14374) (GH-14378)
1d4b6ba

[vstinner]

I merged my fix. I close the issue and hope that the test will not fail again ;-)

Python 2.7 is not affected: it doesn't have test_chown().