Skip to content

Commit

Permalink
Feature: Packaging (#28)
Browse files Browse the repository at this point in the history
* Feature: Packaging

This changeset introduces initial packaging for Gust, and for apps
that use Gust. Changes so far:

- [x] Fixup package for eventual publish to `npm`
- [x] Add support for GitHub packages
- [x] Add support for `rules_docker` targets
- [x] Add support for tarball targets

* Update Micronaut -> 1.3.0

* Upgrade Yarn dependencies

* Upgrade Bazel dependencies

* Fixes and functionality for crosslib builds

- Fix defs when used in a cross-lib context
- Fix DOM test
- Tighten up DualStackTest

* Initial support for container targets

- Add Java container target
- Add example push command (currently blocked on
  bazelbuild/rules_docker#1413)

* Re-enable container tag via BUILD_SCM_VERSION

* Fix workspace status command file

* Working container build-and-run flow

- Ability to build Java app containers, and push them to GCR
- Tested containers locally (manually for now)

* Update fork of rules_closure

* Remove forced JDK8 build flags

* Update J2CL with static bootclasspath

* Update rules_graal with new attrs for debugging

* Initial build flow for Graal native binary targets

- Setup build lib
- Attach to Graal image
- Override default compiler path (workaround for Bazel bug)

* Re-enable disk caching in CI

* Native binary support, with container image

- Support building via `native-image`
- Support pushing to container repo for native image
- No cross-platform build support yet (oracle/graal#407)

* Fix injected testdeps

* Don't enable native targets without a flag

* Add CI routine for baking samples

* Cleaner CI flow

* Attempt GCR auth fix, flush keys

* Another credentials fix

* Fix Bazel label in regular CI job

* Try auth via gcloud

* Explicitly activate gcloud key

* Apply new auth to sample image build

* Fixup key paths in CI

* Move GCP install step up in CI flow

* Cleanup CI configs

* Use relative access to service key from env

* Unify image bake, restore Docker index pull

* Add base container build file

* Use new base container
  • Loading branch information
sgammon committed Feb 6, 2020
1 parent 1f258a4 commit d7770b0
Show file tree
Hide file tree
Showing 29 changed files with 1,114 additions and 725 deletions.
8 changes: 5 additions & 3 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,18 @@ build --watchfs
build --symlink_prefix=dist/
build --nolegacy_external_runfiles
build --disk_cache=~/.cache/bazel-disk-cache
build --workspace_status_command=./tools/bazel_stamp_vars.sh
build --incompatible_strict_action_env
build --javacopt="-source 8 -target 8 -encoding UTF-8"
build --javacopt="-encoding UTF-8"
build --strict_java_deps=strict
build --use_ijars
build --interface_shared_objects
build --java_toolchain=@bazel_tools//tools/jdk:toolchain_java11
build --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_java11
build --workspace_status_command=./tools/bazel_stamp_vars.sh
build --embed_label=alpha

run --incompatible_strict_action_env
run --workspace_status_command=./tools/bazel_stamp_vars.sh

build:ci --spawn_strategy=local
build:ci --javabase=//defs/toolchain/java:java_runtime
Expand All @@ -35,7 +37,7 @@ build:dev --define=jdk=zulu
build:dev --define=ZULUBASE=/Library/Java/JavaVirtualMachines/zulu-12.jdk/Contents/Home
build:dev --javabase=//defs/toolchain/java:java_runtime

run:dev --define=VERBOSE_LOGS=1 -- --node_options=--inspect-brk
#run:dev --define=VERBOSE_LOGS=1 -- --node_options=--inspect-brk

query --output=label_kind

Expand Down
32 changes: 23 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,32 +12,46 @@ jobs:

strategy:
matrix:
#os: [["Ubuntu", "ubuntu-latest", "linux-amd64"], ["macOS Catalina", "macos-latest", "darwin-amd64"]]
#bazel: ["2.0.0", "1.0.0"]
os: [["Ubuntu", "ubuntu-latest", "linux-amd64"]]
os: [["Ubuntu", "ubuntu-latest", "linux-amd64", "linux-x86_64"]]
bazel: ["2.0.0"]
gcloud: ["279.0.0"]

runs-on: ${{ matrix.os[1] }}

steps:
- name: Checkout Sources
uses: actions/checkout@v1

- name: Install Bazelisk
- name: Install GCP SDK
run: |
curl -LO "https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-${{ matrix.gcloud }}-${{ matrix.os[3] }}.tar.gz"
mv google-cloud-sdk-${{ matrix.gcloud }}-${{ matrix.os[3] }}.tar.gz google-cloud-sdk.tar.gz
tar -xf google-cloud-sdk.tar.gz
echo "$SERVICE_KEY" > "${GITHUB_WORKSPACE}/service-key.json"
./google-cloud-sdk/bin/gcloud auth activate-service-account --key-file=`pwd`/service-key.json
./google-cloud-sdk/bin/gcloud auth configure-docker
env:
SERVICE_KEY: ${{ secrets.GOOGLE_CREDENTIALS }}
GOOGLE_APPLICATION_CREDENTIALS: service-key.json

- name: Install Bazelisk (Bazel v${{ matrix.bazel }})
run: |
curl -LO "https://github.com/bazelbuild/bazelisk/releases/download/${BAZELISK_VERSION}/bazelisk-${{ matrix.os[2] }}"
mkdir -p "${GITHUB_WORKSPACE}/bin/"
mv bazelisk-${{ matrix.os[2] }} "${GITHUB_WORKSPACE}/bin/bazelisk"
chmod +x "${GITHUB_WORKSPACE}/bin/bazelisk"
- name: Select Bazel v${{ matrix.bazel }}
run: |
echo "${{ matrix.bazel }}" > .bazelversion
- name: Prepare Cache
id: cache-ci
uses: actions/cache@v1
with:
path: "/home/runner/.cache/bazel-disk-cache"
key: bazel-{{ matrix.bazel }}-{{ matrix.os[1] }}-v1

- name: Build & Test
run: |
echo "$SERVICE_KEY" > service-key.json
export
export PATH="$PATH:${GITHUB_WORKSPACE}/google-cloud-sdk/bin"
make build BAZELISK="${GITHUB_WORKSPACE}/bin/bazelisk" CI=yes CACHE=yes
make test BAZELISK="${GITHUB_WORKSPACE}/bin/bazelisk" CI=yes CACHE=yes
env:
Expand Down
51 changes: 51 additions & 0 deletions .github/workflows/images.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Samples

on:
push: {}
#push:
# branches: ["master"]

env:
BAZELISK_VERSION: v1.2.1

jobs:
build:
name: "Sample Images"

strategy:
matrix:
os: [["Ubuntu", "ubuntu-latest", "linux-amd64", "linux-x86_64"]]
bazel: ["2.0.0"]
gcloud: ["279.0.0"]

runs-on: ${{ matrix.os[1] }}

steps:
- name: Checkout Sources
uses: actions/checkout@v1

- name: Prepare Cache
id: cache-ci
uses: actions/cache@v1
with:
path: "/home/runner/.cache/bazel-disk-cache"
key: bazel-{{ matrix.bazel }}-{{ matrix.os[1] }}-v1

- name: Bake Images
run: |
curl -LO "https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-${{ matrix.gcloud }}-${{ matrix.os[3] }}.tar.gz"
mv google-cloud-sdk-${{ matrix.gcloud }}-${{ matrix.os[3] }}.tar.gz google-cloud-sdk.tar.gz
tar -xf google-cloud-sdk.tar.gz
echo "$SERVICE_KEY" > "${GITHUB_WORKSPACE}/service-key.json"
./google-cloud-sdk/bin/gcloud auth activate-service-account --key-file=`pwd`/service-key.json
./google-cloud-sdk/bin/gcloud auth configure-docker
curl -LO "https://github.com/bazelbuild/bazelisk/releases/download/${BAZELISK_VERSION}/bazelisk-${{ matrix.os[2] }}"
mkdir -p "${GITHUB_WORKSPACE}/bin/"
mv bazelisk-${{ matrix.os[2] }} "${GITHUB_WORKSPACE}/bin/bazelisk"
chmod +x "${GITHUB_WORKSPACE}/bin/bazelisk"
echo "${{ matrix.bazel }}" > .bazelversion
export PATH="$PATH:${GITHUB_WORKSPACE}/google-cloud-sdk/bin"
make samples BAZELISK="${GITHUB_WORKSPACE}/bin/bazelisk" CI=yes CACHE=yes
env:
SERVICE_KEY: ${{ secrets.GOOGLE_CREDENTIALS }}
GOOGLE_APPLICATION_CREDENTIALS: service-key.json
9 changes: 7 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ BUILD_ARGS ?=

BAZELISK ?= $(shell which bazelisk)
BAZELISK_ARGS ?=
BASE_ARGS ?= --google_default_credentials=true
BASE_ARGS ?= --google_default_credentials=true --define project=$(PROJECT)


# Flag: `COVERAGE`
Expand Down Expand Up @@ -80,6 +80,11 @@ r run: ## Run the specified target.
c clean: ## Clean ephemeral targets.
$(BAZELISK) $(BAZELISK_ARGS) clean

samples: ## Build and push sample app images.
$(BAZELISK) $(BAZELISK_ARGS) run $(TAG) $(BASE_ARGS) $(BUILD_ARGS) //javatests/server:BasicTestApplication-image-push
$(BAZELISK) $(BAZELISK_ARGS) run $(TAG) $(BASE_ARGS) $(BUILD_ARGS) //javatests/server:BasicTestApplication-native-image-push
$(BAZELISK) $(BAZELISK_ARGS) run $(TAG) $(BASE_ARGS) $(BUILD_ARGS) //javatests/ssr:SSRTestApplication-image-push

distclean: ## Clean targets, caches and dependencies.
$(BAZELISK) $(BAZELISK_ARGS) clean --expunge_async

Expand All @@ -105,5 +110,5 @@ update-deps: ## Re-seal and update all dependencies.
@echo "Re-pinning Maven dependencies..."
$(BAZELISK) $(BAZELISK_ARGS) run @unpinned_maven//:pin

.PHONY: build test help
.PHONY: build test help samples

30 changes: 24 additions & 6 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -90,17 +90,18 @@ load("@io_bazel_rules_webtesting//web/versioned:browsers-0.3.2.bzl", "browser_re
browser_repositories(chromium=CHROMIUM, firefox=FIREFOX, sauce=SAUCE)

## Docker
load("@io_bazel_rules_docker//repositories:repositories.bzl",
container_repositories = "repositories")
load("@io_bazel_rules_docker//container:container.bzl",
"container_pull")
load("@io_bazel_rules_docker//java:image.bzl",
_java_image_repos = "repositories")
load("@io_bazel_rules_docker//repositories:repositories.bzl", container_repositories = "repositories")
load("@io_bazel_rules_docker//java:image.bzl", _java_image_repos = "repositories")
load("@io_bazel_rules_docker//go:image.bzl", _go_image_repos = "repositories")
load("@io_bazel_rules_docker//python:image.bzl", _py_image_repos = "repositories")

load("@io_bazel_rules_docker//repositories:deps.bzl",
container_deps = "deps")

container_repositories()
container_deps()
_go_image_repos()
_py_image_repos()
_java_image_repos()

## JS Interop
Expand Down Expand Up @@ -164,3 +165,20 @@ pip_import(

load("//defs/toolchain/python:repos.bzl", "gust_python_repositories")
gust_python_repositories()

## Java Containers
load("@io_bazel_rules_docker//container:container.bzl", "container_pull")

container_pull(
name = "java_base",
registry = "gcr.io",
repository = "distroless/java",
digest = "sha256:0ce06c40e99e0dce26bdbcec30afe7a890a57bbd250777bd31ff2d1b798c7809",
)

container_pull(
name = "native_base",
registry = "us.gcr.io",
repository = "bloom-sandbox/base/alpine",
digest = "sha256:decbf1b8ba41c556941f2fbd82811822f7b9622cbd3a17d5d4041cb5438bae2d",
)
50 changes: 50 additions & 0 deletions base/alpine/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
FROM alpine:3.11

ENV LANG=C.UTF-8

# Here we install GNU libc (aka glibc) and set C.UTF-8 locale as default. (originally from `frolvlad`)

RUN ALPINE_GLIBC_BASE_URL="https://github.com/sgerrand/alpine-pkg-glibc/releases/download" && \
ALPINE_GLIBC_PACKAGE_VERSION="2.30-r0" && \
ALPINE_GLIBC_BASE_PACKAGE_FILENAME="glibc-$ALPINE_GLIBC_PACKAGE_VERSION.apk" && \
ALPINE_GLIBC_BIN_PACKAGE_FILENAME="glibc-bin-$ALPINE_GLIBC_PACKAGE_VERSION.apk" && \
ALPINE_GLIBC_I18N_PACKAGE_FILENAME="glibc-i18n-$ALPINE_GLIBC_PACKAGE_VERSION.apk" && \
apk add --no-cache --virtual=.build-dependencies wget ca-certificates && \
echo \
"-----BEGIN PUBLIC KEY-----\
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEApZ2u1KJKUu/fW4A25y9m\
y70AGEa/J3Wi5ibNVGNn1gT1r0VfgeWd0pUybS4UmcHdiNzxJPgoWQhV2SSW1JYu\
tOqKZF5QSN6X937PTUpNBjUvLtTQ1ve1fp39uf/lEXPpFpOPL88LKnDBgbh7wkCp\
m2KzLVGChf83MS0ShL6G9EQIAUxLm99VpgRjwqTQ/KfzGtpke1wqws4au0Ab4qPY\
KXvMLSPLUp7cfulWvhmZSegr5AdhNw5KNizPqCJT8ZrGvgHypXyiFvvAH5YRtSsc\
Zvo9GI2e2MaZyo9/lvb+LbLEJZKEQckqRj4P26gmASrZEPStwc+yqy1ShHLA0j6m\
1QIDAQAB\
-----END PUBLIC KEY-----" | sed 's/ */\n/g' > "/etc/apk/keys/sgerrand.rsa.pub" && \
wget \
"$ALPINE_GLIBC_BASE_URL/$ALPINE_GLIBC_PACKAGE_VERSION/$ALPINE_GLIBC_BASE_PACKAGE_FILENAME" \
"$ALPINE_GLIBC_BASE_URL/$ALPINE_GLIBC_PACKAGE_VERSION/$ALPINE_GLIBC_BIN_PACKAGE_FILENAME" \
"$ALPINE_GLIBC_BASE_URL/$ALPINE_GLIBC_PACKAGE_VERSION/$ALPINE_GLIBC_I18N_PACKAGE_FILENAME" && \
apk add --no-cache \
"$ALPINE_GLIBC_BASE_PACKAGE_FILENAME" \
"$ALPINE_GLIBC_BIN_PACKAGE_FILENAME" \
"$ALPINE_GLIBC_I18N_PACKAGE_FILENAME" && \
\
rm "/etc/apk/keys/sgerrand.rsa.pub" && \
/usr/glibc-compat/bin/localedef --force --inputfile POSIX --charmap UTF-8 "$LANG" || true && \
echo "export LANG=$LANG" > /etc/profile.d/locale.sh && \
\
apk del glibc-i18n && \
\
rm "/root/.wget-hsts" && \
apk del .build-dependencies && \
rm \
"$ALPINE_GLIBC_BASE_PACKAGE_FILENAME" \
"$ALPINE_GLIBC_BIN_PACKAGE_FILENAME" \
"$ALPINE_GLIBC_I18N_PACKAGE_FILENAME"

# Extra packages

RUN apk update && \
apk add libstdc++ && \
rm -rf /var/cache/apk/*

Loading

0 comments on commit d7770b0

Please sign in to comment.