Skip to content

testing: migrate to trampoline_v2 #3860

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tmatsuo merged 38 commits into from
May 29, 2020
Merged

testing: migrate to trampoline_v2 #3860

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
47701bc
testing: prototype trampoline_v2
May 22, 2020
5b858a3
add RUN_TESTS_DIRS envvar
May 22, 2020
1481b8e
add a script for running tests locally
May 22, 2020
2446587
disable docker build for run_tests_local.sh
May 22, 2020
6dad190
don't rely on git, remove noxfile.py only if we copied it
May 22, 2020
c601780
fix tab
May 22, 2020
3433282
remove the tmpdir at the end
May 22, 2020
65b49ad
add TRAMPOLINE_IMAGE_UPLOAD envvar
May 22, 2020
a93c319
add example
May 22, 2020
bc6390a
add .trampolinerc_tmpl
May 22, 2020
13c2031
allow docker in docker
May 23, 2020
c4057c6
ignore failure when creating a group
May 23, 2020
a7f2b11
use /v for project root
May 23, 2020
11b92ba
use $PROJECT_ROOT instead of $PWD
May 23, 2020
3bc8822
temporarily remove the github/python-docs-samples prefix
May 23, 2020
fd88e4b
also pass down KOKORO_KEYSTORE_DIR
May 23, 2020
dfb7fca
pass down Kokoro dynamic variables
May 23, 2020
12c0e58
introduce TRAMPOLINE_CI and TRAMPOLINE_V2 envvar
May 24, 2020
a8bf076
use --entrypoint instead of giving the command
May 25, 2020
5c40d90
add kbiulder's uid as the default arg
May 25, 2020
fcdf8af
do not show the docker commands by default
May 26, 2020
dc426b8
make the mount point configurable
May 26, 2020
41d5ff8
change TRAMPOLINE_IMAGE_SOURCE to TRAMPOLINE_DOCKERFILE
May 26, 2020
0ec909a
remove unnecessary workaround
May 27, 2020
110817a
Trial to run the trampoline_v2 in Kokoro.
May 23, 2020
e9e23a1
summon BCB again
May 27, 2020
9cec5d3
only decrypt the secret when the key exists
May 27, 2020
8b53a48
remove secret if we used decrypt-secrets.sh before the test
May 27, 2020
ecb4b2e
add 2 more envvars for build cop bot
May 27, 2020
c755d53
use user's gid
May 28, 2020
c42e4cc
setup kokoro service account only on kokoro
May 28, 2020
498514a
address review comments
May 28, 2020
dc7c012
correctly report the exit value
May 28, 2020
84da914
temporarily limit the test to vision/automl
May 28, 2020
0b158dd
remove unnecessary build args
May 28, 2020
2479339
use docker gid
May 28, 2020
b13de77
test everything
May 28, 2020
74fd55f
download cloud_sql_proxy to ${HOME} again
May 29, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .kokoro/docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -161,4 +161,19 @@ ENV PATH /google-cloud-sdk/bin:$PATH
# Enable redis-server on boot.
RUN sudo systemctl enable redis-server.service

# Create a user and allow sudo

# kbuilder uid on the default Kokoro image
ARG UID=1000
ARG USERNAME=kbuilder

# Add a new user to the container image.
# This is needed for ssh and sudo access.

# Add a new user with the caller's uid and the username.
RUN useradd -d /h -u ${UID} ${USERNAME}

# Allow nopasswd sudo
RUN echo "${USERNAME} ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
Comment on lines +166 to +177
Copy link
Copy Markdown
Contributor

@kurtisvg kurtisvg May 29, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I still think it would be better to used a fixed non-root user, but we can revisit later


CMD ["python3.6"]
7 changes: 6 additions & 1 deletion .kokoro/lint/common.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ env_vars: {
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/trampoline"

# Use the trampoline script to run in docker.
build_file: "python-docs-samples/.kokoro/trampoline.sh"
build_file: "python-docs-samples/.kokoro/trampoline_v2.sh"

# Download secrets from Cloud Storage.
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/python-docs-samples"
Expand All @@ -41,3 +41,8 @@ env_vars: {
key: "RUN_TESTS_SESSION"
value: "lint"
}

env_vars: {
key: "TRAMPOLINE_DOCKERFILE"
value: ".kokoro/docker/Dockerfile"
}
2 changes: 1 addition & 1 deletion .kokoro/lint/continuous.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@
# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests_diff_head.sh"
value: ".kokoro/tests/run_tests_diff_head.sh"
}
2 changes: 1 addition & 1 deletion .kokoro/lint/periodic.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@
# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests.sh"
value: ".kokoro/tests/run_tests.sh"
}
2 changes: 1 addition & 1 deletion .kokoro/lint/presubmit.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@
# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests_diff_master.sh"
value: ".kokoro/tests/run_tests_diff_master.sh"
}
10 changes: 7 additions & 3 deletions .kokoro/python2.7/common.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ env_vars: {
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/trampoline"

# Use the trampoline script to run in docker.
build_file: "python-docs-samples/.kokoro/trampoline.sh"
build_file: "python-docs-samples/.kokoro/trampoline_v2.sh"

# Download secrets from Cloud Storage.
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/python-docs-samples"
Expand All @@ -44,9 +44,13 @@ env_vars: {
value: "py-2.7"
}

# Declare build specific Cloud project. It still uses the common one,
# but we'll update the value once we have more Cloud projects.
# Declare build specific Cloud project.
env_vars: {
key: "BUILD_SPECIFIC_GCLOUD_PROJECT"
value: "python-docs-samples-tests"
}

env_vars: {
key: "TRAMPOLINE_DOCKERFILE"
value: ".kokoro/docker/Dockerfile"
}
2 changes: 1 addition & 1 deletion .kokoro/python2.7/continuous.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@
# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests_diff_head.sh"
value: ".kokoro/tests/run_tests_diff_head.sh"
}
4 changes: 2 additions & 2 deletions .kokoro/python2.7/periodic.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@
# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests.sh"
value: ".kokoro/tests/run_tests.sh"
}

env_vars: {
key: "REPORT_TO_BUILD_COP_BOT"
value: "True"
value: "true"
}
2 changes: 1 addition & 1 deletion .kokoro/python2.7/presubmit.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@
# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests_diff_master.sh"
value: ".kokoro/tests/run_tests_diff_master.sh"
}
7 changes: 6 additions & 1 deletion .kokoro/python3.6/common.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ env_vars: {
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/trampoline"

# Use the trampoline script to run in docker.
build_file: "python-docs-samples/.kokoro/trampoline.sh"
build_file: "python-docs-samples/.kokoro/trampoline_v2.sh"

# Download secrets from Cloud Storage.
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/python-docs-samples"
Expand All @@ -50,3 +50,8 @@ env_vars: {
key: "BUILD_SPECIFIC_GCLOUD_PROJECT"
value: "python-docs-samples-tests-py36"
}

env_vars: {
key: "TRAMPOLINE_DOCKERFILE"
value: ".kokoro/docker/Dockerfile"
}
2 changes: 1 addition & 1 deletion .kokoro/python3.6/continuous.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@
# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests_diff_head.sh"
value: ".kokoro/tests/run_tests_diff_head.sh"
}
4 changes: 2 additions & 2 deletions .kokoro/python3.6/periodic.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@
# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests.sh"
value: ".kokoro/tests/run_tests.sh"
}

env_vars: {
key: "REPORT_TO_BUILD_COP_BOT"
value: "True"
value: "true"
}
2 changes: 1 addition & 1 deletion .kokoro/python3.6/presubmit.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@
# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests_diff_master.sh"
value: ".kokoro/tests/run_tests_diff_master.sh"
}
7 changes: 6 additions & 1 deletion .kokoro/python3.7/common.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ env_vars: {
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/trampoline"

# Use the trampoline script to run in docker.
build_file: "python-docs-samples/.kokoro/trampoline.sh"
build_file: "python-docs-samples/.kokoro/trampoline_v2.sh"

# Download secrets from Cloud Storage.
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/python-docs-samples"
Expand All @@ -50,3 +50,8 @@ env_vars: {
key: "BUILD_SPECIFIC_GCLOUD_PROJECT"
value: "python-docs-samples-tests-py37"
}

env_vars: {
key: "TRAMPOLINE_DOCKERFILE"
value: ".kokoro/docker/Dockerfile"
}
2 changes: 1 addition & 1 deletion .kokoro/python3.7/continuous.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@
# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests_diff_head.sh"
value: ".kokoro/tests/run_tests_diff_head.sh"
}
4 changes: 2 additions & 2 deletions .kokoro/python3.7/periodic.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@
# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests.sh"
value: ".kokoro/tests/run_tests.sh"
}

env_vars: {
key: "REPORT_TO_BUILD_COP_BOT"
value: "True"
value: "true"
}
3 changes: 1 addition & 2 deletions .kokoro/python3.7/presubmit.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,7 @@

# Format: //devtools/kokoro/config/proto/build.proto

# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests_diff_master.sh"
value: ".kokoro/tests/run_tests_diff_master.sh"
}
7 changes: 6 additions & 1 deletion .kokoro/python3.8/common.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ env_vars: {
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/trampoline"

# Use the trampoline script to run in docker.
build_file: "python-docs-samples/.kokoro/trampoline.sh"
build_file: "python-docs-samples/.kokoro/trampoline_v2.sh"

# Download secrets from Cloud Storage.
gfile_resources: "/bigstore/cloud-devrel-kokoro-resources/python-docs-samples"
Expand All @@ -50,3 +50,8 @@ env_vars: {
key: "BUILD_SPECIFIC_GCLOUD_PROJECT"
value: "python-docs-samples-tests-py38"
}

env_vars: {
key: "TRAMPOLINE_DOCKERFILE"
value: ".kokoro/docker/Dockerfile"
}
2 changes: 1 addition & 1 deletion .kokoro/python3.8/continuous.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@
# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests_diff_head.sh"
value: ".kokoro/tests/run_tests_diff_head.sh"
}
4 changes: 2 additions & 2 deletions .kokoro/python3.8/periodic.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,10 @@
# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests.sh"
value: ".kokoro/tests/run_tests.sh"
}

env_vars: {
key: "REPORT_TO_BUILD_COP_BOT"
value: "True"
value: "true"
}
2 changes: 1 addition & 1 deletion .kokoro/python3.8/presubmit.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,5 @@
# Tell the trampoline which build file to use.
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/python-docs-samples/.kokoro/tests/run_tests_diff_master.sh"
value: ".kokoro/tests/run_tests_diff_master.sh"
}
Loading