[not our bug] nixpkgs: Minio do not build on macOS with Go modules

[hauleth]

Current Behavior

Fails to build with:

# github.com/shirou/gopsutil/host
vendor/github.com/shirou/gopsutil/host/host_darwin_cgo.go:9:11: fatal error: 'include/smc.c' file not found
 #include "include/smc.c"
          ^~~~~~~~~~~~~~~
1 error generated.

Possible Solution

Update Minio to use newest version of gopsutil to include this patch shirou/gopsutil#889

Steps to Reproduce (for bugs)

Try to build Minio on macOS with Nix.

Your Environment

• Operating System and version (uname -a): Darwin 19.6.0 Darwin Kernel Version 19.6.0: Sun Jul 5 00:43:10 PDT 2020; root:xnu-6153.141.1~9/RELEASE_X86_64 x86_64 i386 MacBookPro15,1 Darwin

[harshavardhana]

github.com/shirou/gopsutil/host

vendor/github.com/shirou/gopsutil/host/host_darwin_cgo.go:9:11: fatal error: 'include/smc.c' file not found
#include "include/smc.c"
^~~~~~~~~~~~~~~
1 error generated.

We build without CGO @hauleth you should set

GO111MODULE=on CGO_ENABLED=0 go get -tags kqueue github.com/minio/minio

also why are you trying to build from source? isn't our release binary enough? https://dl.minio.io/server/minio/release/darwin-amd64/

[hauleth]

@harshavardhana I am using Nix package manager that builds it for me.

[hauleth]

With CGO_ENABLED=0 it fails on building notify:

github.com/rjeczalik/notify
# github.com/rjeczalik/notify
vendor/github.com/rjeczalik/notify/watcher_fsevents.go:49:11: undefined: stream
vendor/github.com/rjeczalik/notify/watcher_fsevents.go:200:13: undefined: newStream

[harshavardhana]

github.com/rjeczalik/notify

github.com/rjeczalik/notify

vendor/github.com/rjeczalik/notify/watcher_fsevents.go:49:11: undefined: stream
vendor/github.com/rjeczalik/notify/watcher_fsevents.go:200:13: undefined: newStream

You missed passing -tags kqueue @hauleth

[harshavardhana]

@harshavardhana I am using Nix package manager that builds it for me.

can this not package pre-existing binaries? - building binaries outside of our builds will not be supported by MinIO @hauleth because we sign our binaries and they are trusted.

[hauleth]

@harshavardhana Nix requires reproducibility and I do not know if this is guaranteed. I am also using the version from the official repo at https://github.com/nixos/nixpkgs which seems to using building from source approach instead of fetching already compiled binaries (as with most of the other packages) to be able to make it point to proper external libraries when needed.

[harshavardhana]

@harshavardhana Nix requires reproducibility and I do not know if this is guaranteed. I am also using the version from the official repo at https://github.com/nixos/nixpkgs which seems to using building from source approach instead of fetching already compiled binaries (as with most of the other packages) to be able to make it point to proper external libraries when needed.

That may be for other packages, but that is a guarantee we cannot provide our users. These binaries cannot be trusted by us and not verified by us.

If necessary we can take the ownership here and provide supported builds.

[harshavardhana]

@harshavardhana Nix requires reproducibility and I do not know if this is guaranteed. I am also using the version from the official repo at https://github.com/nixos/nixpkgs which seems to using building from source approach instead of fetching already compiled binaries (as with most of the other packages) to be able to make it point to proper external libraries when needed.

@hauleth why would you not use brew install - Nix packages are meant for Linux and the Nix package manager might not support macOS properly. Why not stick to most common way to install things on macOS? is there a specific requirement that you wish to use Nix package manager?

[hauleth]

Because HomeBrew is an abomination of package manager. Being popular doesn't mean that it is a good solution. Nix gives me features that HomeBrew cannot and will not support (for example nix-shell, easily supporting different versions of applications, environment isolation, reproducible builds or declarative management of whole system).

I can, and I have, written quick derivation to use precompiled binary, however I would still like to find a solution to this problem for the future.

[harshavardhana]

Because HomeBrew is an abomination of package manager. Being popular doesn't mean that it is a good solution. Nix gives me features that HomeBrew cannot and will not support (for example nix-shell, easily supporting different versions of applications, environment isolation, reproducible builds or declarative management of whole system).

That is quite a loaded opinion @hauleth - I don't think we have an interest in supporting source compiled packages which we cannot certify for our users.

I can, and I have, written quick derivation to use precompiled binary, however I would still like to find a solution to this problem for the future.

The problem is simple use go build tags with --tags kqueue - that is how we release our binaries anyways.