Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

posix.sched_{get|set}affinity(-1) no longer returns ProcessLookup causing test failures on FreeBSD #91361

Open
tiran opened this issue Apr 3, 2022 · 9 comments
Labels
3.9 only security fixes 3.10 only security fixes 3.11 bug and security fixes OS-freebsd tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error

Comments

@tiran
Copy link
Member

tiran commented Apr 3, 2022

BPO 47205
Nosy @tiran, @koobs, @miss-islington, @Fidget-Spinner
PRs
  • bpo-47205: Skip error check of sched_get/setaffinity on FreeBSD (GH-32285) #32285
  • [3.10] bpo-47205: Skip error check of sched_get/setaffinity on FreeBSD (GH-32285) #32286
  • [3.9] bpo-47205: Skip error check of sched_get/setaffinity on FreeBSD (GH-32285) #32287
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2022-04-03.12:36:46.409>
    labels = ['type-bug', 'tests', '3.9', '3.10', '3.11']
    title = 'posix.sched_{get|set}affinity(-1) no longer returns ProcessLookup causing test failures on FreeBSD'
    updated_at = <Date 2022-04-05.23:07:44.290>
    user = 'https://github.com/tiran'

    bugs.python.org fields:

    activity = <Date 2022-04-05.23:07:44.290>
    actor = 'koobs'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Tests', 'FreeBSD']
    creation = <Date 2022-04-03.12:36:46.409>
    creator = 'christian.heimes'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 47205
    keywords = ['patch']
    message_count = 9.0
    messages = ['416616', '416620', '416621', '416623', '416624', '416627', '416640', '416647', '416830']
    nosy_count = 4.0
    nosy_names = ['christian.heimes', 'koobs', 'miss-islington', 'kj']
    pr_nums = ['32285', '32286', '32287']
    priority = 'normal'
    resolution = None
    stage = None
    status = 'open'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue47205'
    versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

    @tiran
    Copy link
    Member Author

    tiran commented Apr 3, 2022

    Two test cases have been failing on FreeBSD buildbot for a while. Let's skip the error check on FreeBSD for now.

    0:27:24 load avg: 1.97 Re-running test_posix in verbose mode (matching: test_sched_getaffinity, test_sched_setaffinity)
    test_sched_getaffinity (test.test_posix.PosixTester.test_sched_getaffinity) ... FAIL
    test_sched_setaffinity (test.test_posix.PosixTester.test_sched_setaffinity) ... FAIL
    ======================================================================
    FAIL: test_sched_getaffinity (test.test_posix.PosixTester.test_sched_getaffinity)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/usr/home/buildbot/python/pull_request.koobs-freebsd-564d/build/Lib/test/test_posix.py", line 1197, in test_sched_getaffinity
        self.assertRaises(OSError, posix.sched_getaffinity, -1)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    AssertionError: OSError not raised by sched_getaffinity

    ======================================================================
    FAIL: test_sched_setaffinity (test.test_posix.PosixTester.test_sched_setaffinity)
    ----------------------------------------------------------------------

    Traceback (most recent call last):
      File "/usr/home/buildbot/python/pull_request.koobs-freebsd-564d/build/Lib/test/test_posix.py", line 1215, in test_sched_setaffinity
        self.assertRaises(OSError, posix.sched_setaffinity, -1, mask)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    AssertionError: OSError not raised by sched_setaffinity

    Ran 2 tests in 0.015s

    @tiran tiran added 3.9 only security fixes 3.10 only security fixes 3.11 bug and security fixes tests Tests in the Lib/test dir labels Apr 3, 2022
    @tiran
    Copy link
    Member Author

    tiran commented Apr 3, 2022

    Koobs, please take a look.

    @Fidget-Spinner
    Copy link
    Member

    FWIW, seems like the buildbot started failing at either commit cb495a1 or 7088120. But I have no clue how commits changing the Windows build config can lead to test_posix failing.

    @tiran
    Copy link
    Member Author

    tiran commented Apr 3, 2022

    Koob's build bot is a FreeBSD 14 host. FreeBSD 14 is under development and has not reached stable yet. I bet that the issue was caused by a recent system and Kernel upgrade.

    @tiran
    Copy link
    Member Author

    tiran commented Apr 3, 2022

    New changeset b82cdd1 by Christian Heimes in branch 'main':
    bpo-47205: Skip error check of sched_get/setaffinity on FreeBSD (GH-32285)
    b82cdd1

    @miss-islington
    Copy link
    Contributor

    New changeset 490ccbd by Miss Islington (bot) in branch '3.9':
    bpo-47205: Skip error check of sched_get/setaffinity on FreeBSD (GH-32285)
    490ccbd

    @miss-islington
    Copy link
    Contributor

    New changeset 470dfe2 by Miss Islington (bot) in branch '3.10':
    bpo-47205: Skip error check of sched_get/setaffinity on FreeBSD (GH-32285)
    470dfe2

    @koobs
    Copy link

    koobs commented Apr 4, 2022

    Investigating

    @koobs koobs added type-bug An unexpected behavior, bug, or error labels Apr 4, 2022
    @koobs
    Copy link

    koobs commented Apr 5, 2022

    From one of our base/kernel developers:

    --------------------------------------------------------------

    koobs wrote:

    I don't grok the system call semantics, but it appears the issue is calls
    with the '-1' argument:

    mask = posix.sched_getaffinity(0)
    ...
    self.assertRaises(OSError, posix.sched_getaffinity, -1)
    ...
    self.assertRaises(OSError, posix.sched_setaffinity, -1, mask)

    This line was added 10 years ago in:

    8486987

    It wants an OSError [1] and presumably was getting it before recent
    failures, but isn't anymore:

    [1] https://docs.python.org/3/library/exceptions.html#OSError

    So this is the difference:

    Python 3.11.0a6+ (heads/main:38ae5b8c0c, Apr  5 2022, 03:27:23) [GCC 11.2.0] on freebsd13
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import posix
    >>> mask = posix.sched_getaffinity(-1)
    >>> mask
    {0, 1, 2, 3, 4, 5, 6, 7}
    
    Python 3.10.4 (main, Mar 25 2022, 00:00:00) [GCC 11.2.1 20220127 (Red Hat 11.2.1-9)] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import posix
    >>> mask = posix.sched_getaffinity(-1)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ProcessLookupError: [Errno 3] No such process

    FreeBSD always accepted -1 as denoting the current process, while Linux
    does not. In fact, Linux uses 0 as current process pid alias.

    For FreeBSD, both -1 and 0 works as getpid(), and I do not see it right
    to break our traditional API conventions. I believe the test should be
    fixed.

    --------------------------------------------------------------

    I am not sure what changed between 3.10 and main that changed/removed
    ProcessLookupError as the return for posix.sched_{get|set}affinity
    with -1 as the argument, but that appears to be the cause of the current test failure

    @koobs koobs changed the title test_sched_getaffinity and setaffinity fail on FreeBSD buildbit posix.sched_{get|set}affinity(-1) no longer returns ProcessLookup causing test failures on FreeBSD Apr 5, 2022
    @koobs koobs changed the title test_sched_getaffinity and setaffinity fail on FreeBSD buildbit posix.sched_{get|set}affinity(-1) no longer returns ProcessLookup causing test failures on FreeBSD Apr 5, 2022
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.9 only security fixes 3.10 only security fixes 3.11 bug and security fixes OS-freebsd tests Tests in the Lib/test dir type-bug An unexpected behavior, bug, or error
    Projects
    None yet
    Development

    No branches or pull requests

    5 participants