macOS kapitan release proposal

[royari]

Related to issue #368 and #274

Issues with making the helm binding pipeline for macOS

• What works:
  • The .so file is generated without any error
  • Individual helm unittests run without any error
• What doesn't work:
  • While trying to run all the unittests together, the program gets stuck with fatal: morestack g0error message
  • Upon searching the issue up, It seems to be a prevailing kernel issue for macOS
  • Currently, I'm trying to find a workaround by changing the stack size

My environment

OS: macOS 10.15.5

$ go version
go version go1.14.5 darwin/amd64

$ python3 --version
Python 3.7.7

$ go env

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/roy/Library/Caches/go-build"
GOENV="/Users/roy/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/roy/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.14.5/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14.5/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS="-Wno-error -Wno-nullability-completeness -Wno-expansion-to-defined -Wno-builtin-requires-header"
CGO_CXXFLAGS="-I/usr/local/opt/libjpeg/include -I/usr/local/opt/libpng/include -I/usr/local/opt/libtiff/include -Dcimg_no_system_calls"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-L/usr/local/opt/libjpeg/lib -L/usr/local/opt/libpng/lib -L/usr/local/opt/libtiff/lib"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/t0/zssh__gn7pq9hgwc4rd5r5jm0000gn/T/go-build805186882=/tmp/go-build -gno-record-gcc-switches -fno-common"

[uberspot]

Hi @AlphaRoy14 :)

Thanks for putting these in a structured doc. 👍

I'll cc @sebradloff and @ramaro since both of them have been looking at parts of the Helm bindings on OSX, for their input here too.

As part of GSoC I think we could definitely start with the 3rd task for now (in a separate PR), while we discuss ideas for the binding build jobs here.

Travis could probably handle at least part of the build, with multiplatform builds.
Another idea which I'm not sure if it's feasible is running github actions that would build the osx bindings as it seems to support osx runners (https://docs.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners)?
Github actions might be simpler to 'chain' as steps, so we'd build the bindings in the first, test them in the next one and package them all in one action/job. I haven't looked into this enough thought and I'm not sure if it's simpler than travis.

[royari]

A small update:

the stack size of the go executable can be changed with the following command

$ go build -a -ldflags='-extldflags "-Wl,-stack_size -Wl,0x200000"' helm_fetch.go

However, since we are interested to make a shared object (.so) file, I guess there's no point in looking at the issue from the stack size angle(?).

[sebradloff]

@AlphaRoy14 yea I kind of gave up trying to solve this issue because no matter what I did it seemed I kept hitting the fatal: morestack g0 error message. This might be a bad idea, but I'm wondering if maybe we create a .so for each process we want to run on mac OSX. If we create a limit of 4 processes for Mac and create 4 .so files, would that potentially work? The multiprocess runs on a per "target" while one process runs sequentially through each "input" so I'm thinking this might be viable. Then again, I'm not sure how that will work with the unit tests.

[royari]

Hi @sebradloff, I tried modifying the code for helm fetch and helm input to run sequentially using just a for loop and something weird happened: the $ python3 -m unittest tests.test_dependency_manager.DependencyManagerTest.test_compile_fetch ran successfully which wasn't running earlier, tests.test_dependency_manager.DependencyManagerTest panicked with

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x10abb695c]

The above error message has been reported a few times for darwin and go, but I didn't get any solution unfortunately :/
I tried running $ python3 -m unittest tests.test_dependency_manager.DependencyManagerTest.test_compile_fetch again and it was back to throwing fatal: morestack g0 error.

So I'm not sure if using 4 .so files would make any difference. I might be missing something though.