Skip to content

Commit

Permalink
Fix build & CI pipelines
Browse files Browse the repository at this point in the history
This retains the original HashiCorp upstream build & test pipelines,
cleaning them up for OpenBao and removing HashiCorp internal tooling
references that aren't necessary for us.

The CI pipeline currently fails with test errors and commenting will
need to be tested on the main repository with an appropriately scoped
token. However, builds pass and produce usable, unsigned artifacts.

This can form the basis of a proper (signed) release pipeline
eventually, taking actions from the build stage of the tagged
release commit and signing and verifying them.

In order to fix CI, some changes to the Go modules were done, removing
redundant tooling packages and re-adding the kubernetes integration
tests. This also fixes CI to correctly run api & sdk tests, fixing #61
again.

Removed, unnecessary actions:

 - actionlint was used to allow-list actions upstream,
 - add-hashicorp-contributed-label was used to add a label to internal
   PRs for visibility,
 - backport was the tool to automatically backport PRs,
 - milestone-checker was used to ensure PRs had appropriate milestones
   prior to merge,
 - oss was used to classify issues against the specified label category
 - remove-labels was used to clean up issues & PRs
 - security-scan requires internal tooling not made public
 - test-ci-bootstrap & test-ci-cleanup are both part of the complex Enos
   integration tests, which were removed in
   85455fb due to resource
   requirements.

Resolves: #31
Resolves: #42
Resolves: #152
Related: #153

Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
  • Loading branch information
cipherboy authored and naphelps committed Mar 5, 2024
1 parent 3300aa1 commit 50d31a1
Show file tree
Hide file tree
Showing 59 changed files with 238 additions and 2,540 deletions.
4 changes: 0 additions & 4 deletions .github/actions/set-up-go/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,6 @@ name: Set up Go with a shared module cache
description: Set up Go with a shared module cache

inputs:
github-token:
description: "An elevated Github token to access private modules if necessary"
type: string
no-restore:
description: "Whether or not to restore the Go module cache on a cache hit"
type: boolean
Expand Down Expand Up @@ -61,7 +58,6 @@ runs:
name: Download go modules
shell: bash
run: |
git config --global url."https://${{ inputs.github-token }}@github.com".insteadOf https://github.com
for mod in $(find . -type f -name go.mod); do
pushd "$(dirname $mod)"
go list ./...
Expand Down
8 changes: 4 additions & 4 deletions .github/scripts/gh_comment.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,23 +16,23 @@ function update_or_create_comment {
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
--paginate \
/repos/hashicorp/"$REPO"/issues/"$PR_NUMBER"/comments | jq -r --arg SEARCH_KEY "$SEARCH_KEY" '.[] | select (.body | contains($SEARCH_KEY)) | .id')
/repos/openbao/"$REPO"/issues/"$PR_NUMBER"/comments | jq -r --arg SEARCH_KEY "$SEARCH_KEY" '.[] | select (.body | contains($SEARCH_KEY)) | .id')

if [[ "$comment_id" != "" ]]; then
# update the comment with the new body
gh api \
--method PATCH \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
/repos/hashicorp/"$REPO"/issues/comments/"$comment_id" \
/repos/openbao/"$REPO"/issues/comments/"$comment_id" \
-f body="$BODY"
else
# create a comment with the new body
gh api \
--method POST \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
/repos/hashicorp/"$REPO"/issues/"$PR_NUMBER"/comments \
/repos/openbao/"$REPO"/issues/"$PR_NUMBER"/comments \
-f body="$BODY"
fi
}
}
4 changes: 1 addition & 3 deletions .github/scripts/report_failed_builds.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,9 @@ set -e
[ ${BUILD_DARWIN:?} ]
[ ${BUILD_DOCKER:?} ]
[ ${BUILD_UBI:?} ]
[ ${TEST:?} ]
[ ${TEST_DOCKER_K8S:?} ]

# listing out all of the jobs with the status
jobs=( "build-other:$BUILD_OTHER" "build-linux:$BUILD_LINUX" "build-darwin:$BUILD_DARWIN" "build-docker:$BUILD_DOCKER" "build-ubi:$BUILD_UBI" "test:$TEST" "test-docker-k8s:$TEST_DOCKER_K8S" )
jobs=( "build-other:$BUILD_OTHER" "build-linux:$BUILD_LINUX" "build-darwin:$BUILD_DARWIN" "build-docker:$BUILD_DOCKER" "build-ubi:$BUILD_UBI" )

# there is a case where even if a job is failed, it reports as cancelled. So, we look for both.
failed_jobs=()
Expand Down
6 changes: 3 additions & 3 deletions .github/scripts/report_failed_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ else
# Remove any rows that don't have a test name
# Only keep the test type, test package, test name, and logs column
# Remove the scroll emoji
# Remove "github.com/hashicorp/vault" from the package name
TABLE_DATA=$(echo "$TABLE_DATA" | awk -F\| '{if ($4 != " - ") { print "|" $2 "|" $3 "|" $4 "|" $7 }}' | sed -r 's/ :scroll://' | sed -r 's/github.com\/hashicorp\/vault\///')
# Remove "github.com/openbao/openbao" from the package name
TABLE_DATA=$(echo "$TABLE_DATA" | awk -F\| '{if ($4 != " - ") { print "|" $2 "|" $3 "|" $4 "|" $7 }}' | sed -r 's/ :scroll://' | sed -r 's/github.com\/openbao\/openbao\///')
NUM_FAILURES=$(wc -l <<< "$TABLE_DATA")

# Check if the number of failures is greater than the maximum tests to display
Expand All @@ -39,4 +39,4 @@ fi

source ./.github/scripts/gh_comment.sh

update_or_create_comment "$REPO" "$PR_NUMBER" "CI Results:" "$BODY"
update_or_create_comment "$REPO" "$PR_NUMBER" "CI Results:" "$BODY"
17 changes: 0 additions & 17 deletions .github/workflows/actionlint.yml

This file was deleted.

26 changes: 0 additions & 26 deletions .github/workflows/add-hashicorp-contributed-label.yml

This file was deleted.

23 changes: 0 additions & 23 deletions .github/workflows/backport.yml

This file was deleted.

27 changes: 13 additions & 14 deletions .github/workflows/build-vault-ce.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ on:
type: string
package-name:
type: string
default: vault
vault-version:
default: bao
bao-version:
type: string
required: true
web-ui-cache-key:
Expand All @@ -35,13 +35,11 @@ on:

jobs:
build:
runs-on: custom-linux-xl-vault-latest
name: Vault ${{ inputs.goos }} ${{ inputs.goarch }} v${{ inputs.vault-version }}
runs-on: ubuntu-latest
name: OpenBao ${{ inputs.goos }} ${{ inputs.goarch }} v${{ inputs.bao-version }}
steps:
- uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
- uses: ./.github/actions/set-up-go
with:
github-token: ${{ secrets.ELEVATED_GITHUB_TOKEN }}
- name: Restore UI from cache
uses: actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 # v3.3.1
with:
Expand All @@ -50,22 +48,22 @@ jobs:
fail-on-cache-miss: true
path: http/web_ui
key: ${{ inputs.web-ui-cache-key }}
- name: Build Vault
- name: Build OpenBao
env:
GO_TAGS: ${{ inputs.go-tags }}
CGO_ENABLED: ${{ inputs.cgo-enabled }}
GOARCH: ${{ inputs.goarch }}
GOOS: ${{ inputs.goos }}
VERSION: ${{ inputs.vault-version }}
VERSION: ${{ inputs.bao-version }}
run:
make ci-build
- name: Determine artifact basename
env:
GOARCH: ${{ inputs.goarch }}
GOOS: ${{ inputs.goos }}
VERSION: ${{ inputs.vault-version }}
VERSION: ${{ inputs.bao-version }}
run: echo "ARTIFACT_BASENAME=$(make ci-get-artifact-basename)" >> "$GITHUB_ENV"
- name: Bundle Vault
- name: Bundle OpenBao
env:
BUNDLE_PATH: out/${{ env.ARTIFACT_BASENAME }}.zip
run: make ci-bundle
Expand All @@ -78,11 +76,12 @@ jobs:
uses: hashicorp/actions-packaging-linux@v1
with:
name: ${{ github.event.repository.name }}
description: Vault is a tool for secrets management, encryption as a service, and privileged access management.
description: OpenBao exists to provide a software solution to manage, store, and distribute sensitive data including secrets, certificates, and keys.
arch: ${{ inputs.goarch }}
version: ${{ inputs.vault-version }}
maintainer: HashiCorp
homepage: https://github.com/hashicorp/vault
version: ${{ inputs.bao-version }}
vendor: OpenBao
maintainer: OpenBao
homepage: https://github.com/openbao/openbao
license: MPL-2.0
binary: dist/${{ inputs.package-name }}
deb_depends: openssl
Expand Down

0 comments on commit 50d31a1

Please sign in to comment.