Skip to content

Commit

Permalink
HBASE-27935 Introduce Jenkins PR job for hbase-kustomize (addendum)
Browse files Browse the repository at this point in the history
- add executable permissions to the shell scripts.
- disable docker buildkit in yetus.
- use Yetus's default dockerfile
- make yetus a little more forgiving
- correct pass-through of github credentials
- address yetus nits
- enable writing comments back to the PR (access doesn't permissions don't allow writing to Checks
  API ?)

Signed-off-by: Sean Busbey <busbey@apache.org>
  • Loading branch information
ndimiduk committed Jun 19, 2023
1 parent 781615f commit ae11ed7
Show file tree
Hide file tree
Showing 5 changed files with 31 additions and 61 deletions.
18 changes: 18 additions & 0 deletions .yetus/detsecrets-ignored-hashes.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# the word `GITHUB_TOKEN`
c267b646441a206d44803d8cb20896c4a166cac2
49 changes: 0 additions & 49 deletions dev-support/jenkins/Dockerfile

This file was deleted.

4 changes: 1 addition & 3 deletions dev-support/jenkins/Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,13 @@ pipeline {
YETUS_REL = 'yetus'
// Branch or tag name. Yetus release tags are 'rel/X.Y.Z'
YETUS_VERSION = 'rel/0.14.1'
DOCKERFILE_REL = "${SRC_REL}/dev-support/jenkins/Dockerfile"
YETUS_DRIVER_REL = "${SRC_REL}/dev-support/jenkins/jenkins_precommit_github_yetus.sh"
ARCHIVE_PATTERN_LIST = 'TEST-*.xml'
BUILD_URL_ARTIFACTS = "artifact/${WORKDIR_REL}/${PATCH_REL}"
WORKDIR_REL = 'yetus-precommit-check'
WORKDIR = "${WORKSPACE}/${WORKDIR_REL}"
SOURCEDIR = "${WORKDIR}/${SRC_REL}"
PATCHDIR = "${WORKDIR}/${PATCH_REL}"
DOCKERFILE = "${WORKDIR}/${DOCKERFILE_REL}"
YETUS_DRIVER = "${WORKDIR}/${YETUS_DRIVER_REL}"
YETUSDIR = "${WORKDIR}/${YETUS_REL}"
PLUGINS = 'all'
Expand Down Expand Up @@ -73,7 +71,7 @@ pipeline {
withCredentials([
usernamePassword(
credentialsId: 'apache-hbase-at-github.com',
passwordVariable: 'GITHUB_PASSWORD',
passwordVariable: 'GITHUB_TOKEN',
usernameVariable: 'GITHUB_USER'
)]) {
sh label: 'test-patch', script: '''#!/bin/bash -e
Expand Down
Empty file modified dev-support/jenkins/gather_machine_environment.sh
100644 → 100755
Empty file.
21 changes: 12 additions & 9 deletions dev-support/jenkins/jenkins_precommit_github_yetus.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,7 @@ declare -a required_envs=(
# these ENV variables define the required API with Jenkinsfile_GitHub
"ARCHIVE_PATTERN_LIST"
"BUILD_URL_ARTIFACTS"
"DOCKERFILE"
"GITHUB_PASSWORD"
"GITHUB_USER"
"GITHUB_TOKEN"
"PATCHDIR"
"PLUGINS"
"SOURCEDIR"
Expand Down Expand Up @@ -81,16 +79,18 @@ fi
if [ -n "${JAVA_HOME}" ]; then
unset JAVA_HOME
fi
YETUS_ARGS+=('--project=hbase-kustomize')
YETUS_ARGS+=("--patch-dir=${PATCHDIR}")
# where the source is located
YETUS_ARGS+=("--basedir=${SOURCEDIR}")
# lots of different output formats
YETUS_ARGS+=("--brief-report-file=${PATCHDIR}/brief.txt")
YETUS_ARGS+=("--console-report-file=${PATCHDIR}/console.txt")
YETUS_ARGS+=("--html-report-file=${PATCHDIR}/report.html")
# enable writing back to Github
YETUS_ARGS+=("--github-password=${GITHUB_PASSWORD}")
YETUS_ARGS+=("--github-user=${GITHUB_USER}")
# don't complain about issues on source branch
YETUS_ARGS+=('--continuous-improvement=true')
# don't worry about unrecognized options
YETUS_ARGS+=('--ignore-unknown-options=true')
# auto-kill any surefire stragglers during unit test runs
YETUS_ARGS+=("--reapermode=kill")
# set relatively high limits for ASF machines
Expand All @@ -106,17 +106,20 @@ YETUS_ARGS+=("--build-url-artifacts=${BUILD_URL_ARTIFACTS}")
# plugins to enable
YETUS_ARGS+=("--plugins=${PLUGINS}")
YETUS_ARGS+=("--tests-filter=test4tests")
# run in docker mode and specifically point to our
# Dockerfile since we don't want to use the auto-pulled version.
# run in docker mode
YETUS_ARGS+=("--docker")
YETUS_ARGS+=("--dockerfile=${DOCKERFILE}")
# our jenkins workers don't have buildkit installed (INFRA-24704)
YETUS_ARGS+=('--docker-buildkit=false')
# effectively treat dev-support as a custom maven module
YETUS_ARGS+=("--skip-dirs=dev-support")
# help keep the ASF boxes clean
YETUS_ARGS+=("--sentinel")
YETUS_ARGS+=("--github-token=${GITHUB_TOKEN}")
# use emoji vote so it is easier to find the broken line
YETUS_ARGS+=("--github-use-emoji-vote")
YETUS_ARGS+=("--github-repo=apache/hbase-kustomize")
# enable writing back to Github
YETUS_ARGS+=('--github-write-comment')
# increasing proc limit to avoid OOME: unable to create native threads
YETUS_ARGS+=("--proclimit=5000")

Expand Down

0 comments on commit ae11ed7

Please sign in to comment.