make release times out

[roberthbailey]

After 6 minutes or so of using >50% my 4-core MBP CPU and chewing up a ton of memory, make release gives up with a timeout.

$ make release
build/release.sh
+++ [0401 22:10:18] Verifying Prerequisites....
+++ [0401 22:10:18] Setting boot2docker clock
+++ [0401 22:10:19] Building Docker image kube-build:cross.
+++ [0401 22:10:21] Building Docker image kube-build:build-2d62e.
+++ [0401 22:10:42] Running build command....
+++ [0401 22:10:42] Creating data container
+++ [0402 05:10:43] Building go targets for linux/amd64 in parallel (output will appear in a burst when complete):
    cmd/kube-proxy
    cmd/kube-apiserver
    cmd/kube-controller-manager
    cmd/kubelet
    cmd/hyperkube
    cmd/kubernetes
    plugin/cmd/kube-scheduler
+++ [0402 05:10:43] linux/amd64: Parallel go build started
+++ [0402 05:10:56] linux/amd64: Parallel go build finished
+++ [0402 05:10:56] Building go targets for linux/amd64
    linux/386
    linux/arm
    darwin/amd64
    darwin/386
    windows/amd64 in parallel (output will appear in a burst when complete):
    cmd/kubectl
    cmd/e2e
    cmd/integration
    cmd/gendocs
    cmd/genman
    examples/k8petstore/web-server
+++ [0402 05:10:56] linux/amd64: Parallel go build started
go build github.com/GoogleCloudPlatform/kubernetes/cmd/integration: /usr/src/go/pkg/tool/linux_amd64/6l: signal: killed
!!! Error in /go/src/github.com/GoogleCloudPlatform/kubernetes/hack/lib/golang.sh:363
  'go install "${goflags[@]:+${goflags[@]}}" -ldflags "${version_ldflags}" "${nonstatics[@]:+${nonstatics[@]}}"' exited with status 1
Call stack:
  1: /go/src/github.com/GoogleCloudPlatform/kubernetes/hack/lib/golang.sh:363 kube::golang::build_binaries(...)
  2: hack/build-cross.sh:31 main(...)
Exiting with status 1
+++ [0402 05:10:56] linux/386: Parallel go build started
go build github.com/GoogleCloudPlatform/kubernetes/cmd/integration: /usr/src/go/pkg/tool/linux_amd64/8l: signal: killed
!!! Error in /go/src/github.com/GoogleCloudPlatform/kubernetes/hack/lib/golang.sh:363
  'go install "${goflags[@]:+${goflags[@]}}" -ldflags "${version_ldflags}" "${nonstatics[@]:+${nonstatics[@]}}"' exited with status 1
Call stack:
  1: /go/src/github.com/GoogleCloudPlatform/kubernetes/hack/lib/golang.sh:363 kube::golang::build_binaries(...)
  2: hack/build-cross.sh:31 main(...)
Exiting with status 1
+++ [0402 05:10:56] linux/arm: Parallel go build started
+++ [0402 05:11:30] linux/arm: Parallel go build finished
+++ [0402 05:10:56] darwin/amd64: Parallel go build started
go build github.com/GoogleCloudPlatform/kubernetes/cmd/integration: /usr/src/go/pkg/tool/linux_amd64/6l: signal: killed
!!! Error in /go/src/github.com/GoogleCloudPlatform/kubernetes/hack/lib/golang.sh:363
  'go install "${goflags[@]:+${goflags[@]}}" -ldflags "${version_ldflags}" "${nonstatics[@]:+${nonstatics[@]}}"' exited with status 1
Call stack:
  1: /go/src/github.com/GoogleCloudPlatform/kubernetes/hack/lib/golang.sh:363 kube::golang::build_binaries(...)
  2: hack/build-cross.sh:31 main(...)
Exiting with status 1
+++ [0402 05:10:56] darwin/386: Parallel go build started
go build github.com/GoogleCloudPlatform/kubernetes/cmd/genman: /usr/src/go/pkg/tool/linux_amd64/8l: signal: killed
!!! Error in /go/src/github.com/GoogleCloudPlatform/kubernetes/hack/lib/golang.sh:363
  'go install "${goflags[@]:+${goflags[@]}}" -ldflags "${version_ldflags}" "${nonstatics[@]:+${nonstatics[@]}}"' exited with status 1
Call stack:
  1: /go/src/github.com/GoogleCloudPlatform/kubernetes/hack/lib/golang.sh:363 kube::golang::build_binaries(...)
  2: hack/build-cross.sh:31 main(...)
Exiting with status 1
+++ [0402 05:10:56] windows/amd64: Parallel go build started
+++ [0402 05:16:55] windows/amd64: Parallel go build finished
!!! Error in /go/src/github.com/GoogleCloudPlatform/kubernetes/hack/lib/golang.sh:273
  '( kube::golang::setup_env; local version_ldflags; version_ldflags=$(kube::version::ldflags); local host_platform; host_platform=$(kube::golang::host_platform); local goflags; eval "goflags=(${KUBE_GOFLAGS:-})"; local use_go_build; local -a targets=(); local arg; for arg in "$@";
do
    if [[ "${arg}" == "--use_go_build" ]]; then
        use_go_build=true;
    else
        if [[ "${arg}" == -* ]]; then
            goflags+=("${arg}");
        else
            targets+=("${arg}");
        fi;
    fi;
done; if [[ ${#targets[@]} -eq 0 ]]; then
    targets=("${KUBE_ALL_TARGETS[@]}");
fi; local -a platforms=("${KUBE_BUILD_PLATFORMS[@]:+${KUBE_BUILD_PLATFORMS[@]}}"); if [[ ${#platforms[@]} -eq 0 ]]; then
    platforms=("${host_platform}");
fi; local binaries; binaries=($(kube::golang::binaries_from_targets "${targets[@]}")); kube::log::status "Building go targets for ${platforms[@]} in parallel (output will appear in a burst when complete):" "${targets[@]}"; local platform; for platform in "${platforms[@]}";
do
    ( kube::golang::set_platform_envs "${platform}"; kube::log::status "${platform}: Parallel go build started"; local -a statics=(); local -a nonstatics=(); for binary in "${binaries[@]}";
    do
        if kube::golang::is_statically_linked_library "${binary}"; then
            kube::golang::exit_if_stdlib_not_installed; statics+=($binary);
        else
            nonstatics+=($binary);
        fi;
    done; if [[ -n ${use_go_build:-} ]]; then
        local output_path="${KUBE_GOPATH}/bin"; if [[ $platform != $host_platform ]]; then
            output_path="${output_path}/${platform//\//_}";
        fi; for binary in "${binaries[@]}";
        do
            local bin=$(basename "${binary}"); if [[ ${GOOS} == "windows" ]]; then
                bin="${bin}.exe";
            fi; if kube::golang::is_statically_linked_library "${binary}"; then
                kube::golang::exit_if_stdlib_not_installed; CGO_ENABLED=0 go build -installsuffix cgo -o "${output_path}/${bin}" "${goflags[@]:+${goflags[@]}}" -ldflags "${version_ldflags}" "${binary}";
            else
                go build -o "${output_path}/${bin}" "${goflags[@]:+${goflags[@]}}" -ldflags "${version_ldflags}" "${binary}";
            fi;
        done;
    else
        if [[ "${#nonstatics[@]}" != 0 ]]; then
            go install "${goflags[@]:+${goflags[@]}}" -ldflags "${version_ldflags}" "${nonstatics[@]:+${nonstatics[@]}}";
        fi; if [[ "${#statics[@]}" != 0 ]]; then
            CGO_ENABLED=0 go install -installsuffix cgo "${goflags[@]:+${goflags[@]}}" -ldflags "${version_ldflags}" "${statics[@]:+${statics[@]}}";
        fi;
    fi; kube::log::status "${platform}: Parallel go build finished" ) &> "/tmp//${platform//\//_}.build" &
done; local fails=0; for job in $(jobs -p);
do
    wait ${job} || let "fails+=1";
done; for platform in "${platforms[@]}";
do
    cat "/tmp//${platform//\//_}.build";
done; exit ${fails} )' exited with status 4
Call stack:
  1: /go/src/github.com/GoogleCloudPlatform/kubernetes/hack/lib/golang.sh:273 kube::golang::build_binaries(...)
  2: hack/build-cross.sh:31 main(...)
Exiting with status 1
!!! Error in build/../build/common.sh:398
  '"${docker_cmd[@]}" "$@"' exited with status 1
Call stack:
  1: build/../build/common.sh:398 kube::build::run_build_command(...)
  2: build/release.sh:32 main(...)
Exiting with status 1
make: *** [release] Error 1

[roberthbailey]

This is on my master branch (no local changes) after rebasing to head this evening. It was working fine earlier today so #6196 seems like the likely culprit.

/cc @zmerlynn

[zmerlynn]

It's certainly my changes. I'm on a low function device, feel free to
revert.

This is only an issue for low memory systems, I suspect. :/
On Apr 1, 2015 10:24 PM, "Robert Bailey" notifications@github.com wrote:

This is on my master branch (no local changes) after rebasing to head this
evening. It was working fine earlier today so #6196
#6196 seems like
the likely culprit.

/cc @zmerlynn https://github.com/zmerlynn

—
Reply to this email directly or view it on GitHub
#6346 (comment)
.

[zmerlynn]

(This is functioning fine on Jenkins and my desktop, FWIW.)
On Apr 1, 2015 10:36 PM, "Zachary Loafman" zml@google.com wrote:

It's certainly my changes. I'm on a low function device, feel free to
revert.

This is only an issue for low memory systems, I suspect. :/
On Apr 1, 2015 10:24 PM, "Robert Bailey" notifications@github.com wrote:

This is on my master branch (no local changes) after rebasing to head
this evening. It was working fine earlier today so #6196
#6196 seems like
the likely culprit.

/cc @zmerlynn https://github.com/zmerlynn

—
Reply to this email directly or view it on GitHub
#6346 (comment)
.