Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Publish binaries to GitHub Releases #766

Merged
merged 10 commits into from
Apr 21, 2018
Merged

Publish binaries to GitHub Releases #766

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
bc13eff
add make and travis options for building for windows
grounded042 Apr 4, 2018
0120ea5
add darwin build
grounded042 Apr 5, 2018
896fa37
use make for windows in the travis setup since darwin uses make as well
grounded042 Apr 5, 2018
d98cde9
add script to package files for deployment
grounded042 Apr 5, 2018
19ca6e6
add linux make target
grounded042 Apr 5, 2018
222e388
missing 'then'
grounded042 Apr 5, 2018
527a042
cleanup white space and wrong data
grounded042 Apr 5, 2018
cf9f34d
Merge branch 'master' into binary_distro
yurishkuro Apr 10, 2018
0810615
Merge branch 'master' into binary_distro
yurishkuro Apr 12, 2018
4488a72
decrease duplication
grounded042 Apr 20, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,15 @@ matrix:
- go: 1.9
env:
- HOTROD=true
- go: 1.9
env:
- LINUX=true
- go: 1.9
env:
- WINDOWS=true
- go: 1.9
env:
- DARWIN=true

services:
- docker
Expand Down Expand Up @@ -54,9 +63,34 @@ script:
- if [ "$DOCKER" == true ]; then bash ./scripts/travis/build-docker-images.sh ; else echo 'skipping docker images'; fi
- if [ "$ES_INTEGRATION_TEST" == true ]; then bash ./scripts/travis/es-integration-test.sh ; else echo 'skipping elastic search integration test'; fi
- if [ "$HOTROD" == true ]; then bash ./scripts/travis/hotrod-integration-test.sh ; else echo 'skipping hotrod example'; fi
- if [ "$LINUX" == true ]; then make build-linux ; else echo 'skipping linux'; fi
- if [ "$WINDOWS" == true ]; then make build-windows ; else echo 'skipping windows'; fi
- if [ "$DARWIN" == true ]; then make build-darwin ; else echo 'skipping darwin'; fi
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall this looks good, but this 3 additions concern me as they are going to significantly increase Travis build times. I'd rather have a single DEPLOY=true step that would build files for all targets, which would be a lot faster since we don't have to wait for Travis VM/container to initialize.

Also, there's now a huge amount of duplication in the make file, I think it could be simplified a lot through variables, similar to the deploy.sh file, e.g.

linux-binaries: 
    make build-agent build-collector ... GOOS=darwin

build-agent:
    CGO_ENABLED=0 GOOS=$(GOOS) ...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in fact, DEPLOY=true doesn't even need to be a new matrix entry, we can co-locate it with the DOCKER=true entry (since it requires linux binaries anyway).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the input @yurishkuro! I'll have time closer to the middle of this week to make these changes.


after_success:
- if [ "$COVERAGE" == true ]; then travis_retry goveralls -coverprofile=cover.out -service=travis-ci || true ; else echo 'skipping coverage'; fi

after_failure:
- if [ "$CROSSDOCK" == true ]; then make crossdock-logs ; else echo 'skipping crossdock'; fi

before_deploy:
- bash ./scripts/travis/package-deploy.sh

deploy:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not familiar with travis deploy workflow - where do these files end up? And do we need a different naming conventions? E.g. https://prometheus.io/download/:

prometheus-2.2.1.darwin-amd64.tar.gz
prometheus-2.2.1.linux-amd64.tar.gz
prometheus-2.2.1.windows-amd64.tar.gz

Also, since these are archive files, do we want to bundle all binaries into one archive?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, we should change these commands to only build and deploy these binaries on landing the PR rather than every commit (unless this already does this)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those files will end up as individual assets over at the release message (e.g. https://github.com/jaegertracing/jaeger/releases/tag/v1.3.0). That's why we need someone to supply a token at YOUR_API_KEY_ENCRYPTED - it allows pushing the files back to GitHub.

Because we have on.tags: true Travis will only push files back to a release message when the commit has a tag. We could also add branch: master to further pin down when it deploys.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also add branch: master to further pin down when it deploys.

I don't think that's what we want:

GitHub Releases uses git tags. If the build commit does not have any tags, one will be created in the form of untagged-*, where * is a random hex string.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpkrohling I think that's only if you have on.branch: true. I was thinking that if you had multiple on options it would only deploy if all conditions were met. I'll look into it.

Copy link
Author

@grounded042 grounded042 Apr 5, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified from the Travis docs:

When all conditions specified in the on: section are met, your build will deploy.

So if we have on.tags: true and on.branch: master it will only deploy when it's a commit on master that has a tag already.

provider: releases
api_key:
secure: YOUR_API_KEY_ENCRYPTED
file:
- "cmd/standalone/standalone-linux"
- "cmd/standalone/standalone-windows.exe"
- "cmd/agent/agent-linux"
- "cmd/agent/agent-windows.exe"
- "cmd/query/query-linux"
- "cmd/query/query-windows.exe"
- "cmd/collector/collector-linux"
- "cmd/collector/collector-windows.exe"
skip_cleanup: true
on:
tags: true
repo: jaegertracing/jaeger
branch: master
45 changes: 45 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,18 +147,50 @@ build_ui:
build-all-in-one-linux: build_ui
CGO_ENABLED=0 GOOS=linux installsuffix=cgo go build -o ./cmd/standalone/standalone-linux $(BUILD_INFO) ./cmd/standalone/main.go

.PHONY: build-all-in-one-windows
build-all-in-one-windows: build_ui
CGO_ENABLED=0 GOOS=windows installsuffix=cgo go build -o ./cmd/standalone/standalone-windows.exe $(BUILD_INFO) ./cmd/standalone/main.go

.PHONY: build-all-in-one-darwin
build-all-in-one-darwin: build_ui
CGO_ENABLED=0 GOOS=darwin installsuffix=cgo go build -o ./cmd/standalone/standalone-darwin $(BUILD_INFO) ./cmd/standalone/main.go

.PHONY: build-agent-linux
build-agent-linux:
CGO_ENABLED=0 GOOS=linux installsuffix=cgo go build -o ./cmd/agent/agent-linux $(BUILD_INFO) ./cmd/agent/main.go

.PHONY: build-agent-windows
build-agent-windows:
CGO_ENABLED=0 GOOS=windows installsuffix=cgo go build -o ./cmd/agent/agent-windows.exe $(BUILD_INFO) ./cmd/agent/main.go
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason not to call this agent.exe? Is there any other platform that uses .exe extension?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just following convention since the agent for linux is called agent-linux. I'll change it up to be agent.exe.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yurishkuro what if we named the different binaries for windows along the lines of jaeger.<product>.exe and dropped the -windows on all of them?

Copy link
Member

@yurishkuro yurishkuro Apr 4, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my preference would be to have identical names but group them into dirs/archives by target, e.g.

jaeger-1.4.0-darwin-amd64.tar.gz
  - jaeger-agent
  - jaeger-collector
  - etc
jaeger-1.4.0-linux-amd64.tar.gz
  - jaeger-agent
  - jaeger-collector
  - etc
jaeger-1.4.0-windows-amd64.tar.gz
  - jaeger-agent.exe
  - jaeger-collector.exe
  - etc

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it. I'll work on getting that setup @yurishkuro.


.PHONY: build-agent-darwin
build-agent-darwin:
CGO_ENABLED=0 GOOS=darwin installsuffix=cgo go build -o ./cmd/agent/agent-darwin $(BUILD_INFO) ./cmd/agent/main.go

.PHONY: build-query-linux
build-query-linux:
CGO_ENABLED=0 GOOS=linux installsuffix=cgo go build -o ./cmd/query/query-linux $(BUILD_INFO) ./cmd/query/main.go

.PHONY: build-query-windows
build-query-windows:
CGO_ENABLED=0 GOOS=windows installsuffix=cgo go build -o ./cmd/query/query-windows.exe $(BUILD_INFO) ./cmd/query/main.go

.PHONY: build-query-darwin
build-query-darwin:
CGO_ENABLED=0 GOOS=darwin installsuffix=cgo go build -o ./cmd/query/query-darwin $(BUILD_INFO) ./cmd/query/main.go

.PHONY: build-collector-linux
build-collector-linux:
CGO_ENABLED=0 GOOS=linux installsuffix=cgo go build -o ./cmd/collector/collector-linux $(BUILD_INFO) ./cmd/collector/main.go

.PHONY: build-collector-windows
build-collector-windows:
CGO_ENABLED=0 GOOS=windows installsuffix=cgo go build -o ./cmd/collector/collector-windows.exe $(BUILD_INFO) ./cmd/collector/main.go

.PHONY: build-collector-darwin
build-collector-darwin:
CGO_ENABLED=0 GOOS=darwin installsuffix=cgo go build -o ./cmd/collector/collector-darwin $(BUILD_INFO) ./cmd/collector/main.go

.PHONY: docker-no-ui
docker-no-ui: build-agent-linux build-collector-linux build-query-linux build-crossdock-linux
mkdir -p jaeger-ui-build/build/
Expand All @@ -167,6 +199,15 @@ docker-no-ui: build-agent-linux build-collector-linux build-query-linux build-cr
.PHONY: docker
docker: build_ui docker-no-ui

.PHONY: build-linux
build-linux: build-agent-linux build-collector-linux build-query-linux build-all-in-one-linux

.PHONY: build-windows
build-windows: build-agent-windows build-collector-windows build-query-windows build-all-in-one-windows

.PHONY: build-darwin
build-darwin: build-agent-darwin build-collector-darwin build-query-darwin build-all-in-one-darwin

.PHONY: docker-images-only
docker-images-only:
cp -r jaeger-ui-build/build/ cmd/query/jaeger-ui-build
Expand Down Expand Up @@ -262,3 +303,7 @@ install-mockery:
.PHONY: generate-mocks
generate-mocks: install-mockery
$(MOCKERY) -all -dir ./pkg/es/ -output ./pkg/es/mocks && rm pkg/es/mocks/ClientBuilder.go

.PHONY: echo-version
echo-version:
@echo $(GIT_CLOSEST_TAG)
80 changes: 80 additions & 0 deletions scripts/travis/package-deploy.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
#!/bin/bash

# stage-file copies the file $1 with the specified path $2
# if no file exists it will silently continue
function stage-file {
if [ -f $1 ]; then
echo "Copying $1 to $2"
cp $1 $2
else
echo "$1 does not exist. Continuing on."
fi
}

# stage-platform-files stages the different the platform ($1) into the package
# staging dir ($2). If you pass in a file extension ($3) it will be used when
# copying on both the target and the source
function stage-platform-files {
local PLATFORM=$1
local PACKAGE_STAGING_DIR=$2
local FILE_EXTENSION=$3

stage-file ./cmd/standalone/standalone-$PLATFORM$FILE_EXTENSION $PACKAGE_STAGING_DIR/jaeger-standalone$FILE_EXTENSION
stage-file ./cmd/agent/agent-$PLATFORM$FILE_EXTENSION $PACKAGE_STAGING_DIR/jaeger-agent$FILE_EXTENSION
stage-file ./cmd/query/query-$PLATFORM$FILE_EXTENSION $PACKAGE_STAGING_DIR/jaeger-query$FILE_EXTENSION
stage-file ./cmd/collector/collector-$PLATFORM$FILE_EXTENSION $PACKAGE_STAGING_DIR/jaeger-collector$FILE_EXTENSION
}

# package pulls built files for the platform ($1). If you pass in a file
# extension ($2) it will be used on the binaries
function package {
local PLATFORM=$1
local FILE_EXTENSION=$2

local PACKAGE_STAGING_DIR=$DEPLOY_STAGING_DIR/$PLATFORM
mkdir $PACKAGE_STAGING_DIR

stage-platform-files $PLATFORM $PACKAGE_STAGING_DIR $FILE_EXTENSION

local PACKAGE_FILES=$(ls -A $PACKAGE_STAGING_DIR/*) 2>/dev/null

if [ "$PACKAGE_FILES" ]; then
local ARCHIVE_NAME="jaeger-$VERSION-$PLATFORM-amd64.tar.gz"
echo "Packaging the following files into $ARCHIVE_NAME:"
echo $PACKAGE_FILES
tar -czvf ./deploy/$ARCHIVE_NAME $PACKAGE_FILES
else
echo "Will not package or deploy $PLATFORM files as there are no files to package!"
fi
}

# script start

DEPLOY_STAGING_DIR=./deploy-staging
VERSION="$(make echo-version | awk 'match($0, /([0-9]*\.[0-9]*\.[0-9]*)$/) { print substr($0, RSTART, RLENGTH) }')"
echo "Working on version: $VERSION"

# make needed directories
mkdir deploy
mkdir $DEPLOY_STAGING_DIR

# package linux
if [ "$LINUX" = true ]; then
package linux
else
echo "Skipping the packaging of linux binaries as \$LINUX was not true."
fi

# package darwin
if [ "$DARWIN" = true ]; then
package darwin
else
echo "Skipping the packaging of darwin binaries as \$DARWIN was not true."
fi

# package windows
if [ "$WINDOWS" = true ]; then
package windows .exe
else
echo "Skipping the packaging of windows binaries as \$WINDOWS was not true."
fi