Skip to content

x/sys: SchedGetaffinity fails on Linux machines with enough (potential) CPUs #23639

Closed

Description

@siebenmann

What version of Go are you using (go version)?

go version devel +926f278 Tue Jan 30 15:22:28 2018 +0000 linux/amd64

And also the current version of golang.org/x/sys from git.

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOOS="linux"

What did you do?

Here is a trivial program to call unix.SchedGetaffinity() to count the number of currently usable CPUs in the system:

package main

import (
        "fmt"
        "os"

        "golang.org/x/sys/unix"
)

func main() {
        var cpuset unix.CPUSet
        err := unix.SchedGetaffinity(0, &cpuset)
        if err != nil {
                fmt.Printf("numcpus: cannot get affinity: %s\n", err)
                os.Exit(1)
        }
        fmt.Printf("%d\n", cpuset.Count())
}

On many of our amd64 machines this will work fine. However, on some of them (with large CPU counts) it will fail with:

$ numcpus
numcpus: cannot get affinity: invalid argument

Using strace says that sched_getaffinity() is being called as follows:

sched_getaffinity(0, 8, 0xc420075ed8)   = -1 EINVAL (Invalid argument)

The second argument here is the number of bytes in the cpuset. Eight bytes is not very many and looking at how unix.CPUSet is defined in the source code suggests that there is supposed to be room for 1024 CPUs. However, inspection of the actual code making the system call in affinity_linux.go suggests that it is making a size of pointer versus size of object pointed to confusion. The set argument is a pointer to a CPUSet not a CPUSet, so unsafe.Sizeof(set) is the size of the pointer (8 bytes on a 64-bit machine) instead of the much larger size of the CPUSet.

Activity

  1. added this to the Unreleased milestone on Jan 31, 2018
  2. ianlancetaylor commented on Jan 31, 2018

    @ianlancetaylor
    Contributor
  3. added
    NeedsFixThe path to resolution is known, but the work has not been done.
    on Jan 31, 2018
  4. gopherbot commented on Feb 1, 2018

    @gopherbot
    Contributor

    Change https://golang.org/cl/91375 mentions this issue: unix: fix cpuset size argument in sched_affinity syscall

  5. tklauser commented on Feb 1, 2018

    @tklauser
    Member

    @siebenmann Thanks for the detailed report an analysis. I sent golang.org/cl/91375 with a fix. Care to give it a try?

  6. siebenmann commented on Feb 1, 2018

    @siebenmann
    Author

    I can confirm that this change/fix fixes the issue for me.

  7. locked and limited conversation to collaborators on Feb 1, 2019
  8. added a commit that references this issue on Jun 7, 2024
    8f27ce8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    FrozenDueToAgeNeedsFixThe path to resolution is known, but the work has not been done.

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions