-
Notifications
You must be signed in to change notification settings - Fork 10.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[PSM Interop] Simplify PSM Interop Kokoro buildscripts #36450
Closed
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sergiitk
added
lang/c++
area/psm interop
python
Pull requests that update Python code
labels
Apr 25, 2024
sergiitk
force-pushed
the
psm-interop-common-prod-tests
branch
from
April 25, 2024 02:55
409e104
to
60fde67
Compare
grpc-checks
bot
added
bloat/none
per-call-memory/neutral
per-channel-memory/neutral
per-call-memory/increase
and removed
per-call-memory/neutral
labels
Apr 25, 2024
sergiitk
force-pushed
the
psm-interop-common-prod-tests
branch
from
April 25, 2024 20:22
60fde67
to
cb787e9
Compare
sergiitk
force-pushed
the
psm-interop-common-prod-tests
branch
from
April 27, 2024 02:24
c0e0f4b
to
e30d7e4
Compare
grpc-checks
bot
added
per-call-memory/neutral
per-call-memory/increase
and removed
per-call-memory/increase
per-call-memory/neutral
labels
Apr 27, 2024
Using grpc/psm-interop#73 / grpc/psm-interop@107a1a0. Full LB suite run: URL Map py: |
sergiitk
added a commit
to grpc/psm-interop
that referenced
this pull request
May 1, 2024
#73) Major rework of the shared Kokoro PSM Interop install library. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to psm_interop_kokoro_lib.sh: 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Added new options to control PSM interop test driver execution via environment variables: ```sh # Overrides the list of test to run. A whitespace-separated string. Example: # PSM_TESTS="app_net_test baseline_test" readonly PSM_TESTS="${PSM_TESTS:-}" # A space-separated string with extra flags to append to the test driver arguments. # Can be used to execute a run and test a new flag value, f.e.: # PSM_EXTRA_FLAGS="--noenable_workload_identity --td_bootstrap_image=us-docker.pkg.dev/new-image..." # In addition, can be used to run a single test in a suite. # F.e. to run only test_mtls in security_test: # PSM_TESTS="security_test" PSM_EXTRA_FLAGS="SecurityTest.test_mtls" readonly PSM_EXTRA_FLAGS="${PSM_EXTRA_FLAGS:-}" ``` Improved logging: 1. Buildscript test logs are now prepended with the date: - when a shell command passed to `psm::tools::run_verbose` - `psm::tools::log` is used in place of `echo` 2. Individual install and build steps partially or fully redirected to the dedicated log files: - `install-apt.log` - `install-pip.log` - `build-docker.log` 3. Various other log improvements: printing tests in the suite, printing flags passed to the frameworks, etc Corresponding per-language buildscript PRs: 1. java - grpc/grpc-java#11121 2. cpp - grpc/grpc#36450 3. go - grpc/grpc-go#7171 4. node - grpc/grpc-node#2729 Notes for the reviewer: 1. The new script is backward-compatible with the old-style buildscripts. 2. Per-language buildscript PRs will be backported to all supported branches, and this should be the very last backport chore we'll ever need. 3. A minor change to the subsetting test was needed to solve [a one-off exception](https://github.com/grpc/grpc/blob/6e981d7460af561230d455c3623774e76e18cbfe/tools/internal_ci/linux/grpc_xds_k8s_lb.sh#L123-L127) from the normal `run_test` flow. ref b/288578634
@yashykt please review, but don't merge it yet. I'd like to wait until I got approvals on the same change from the rest of the repos. So we don't have to do another set of backports if anything comes up. |
sergiitk
added
the
release notes: no
Indicates if PR should not be in release notes
label
May 2, 2024
sergiitk
added a commit
to sergiitk/grpc
that referenced
this pull request
May 6, 2024
Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79 Closes grpc#36450 COPYBARA_INTEGRATE_REVIEW=grpc#36450 from sergiitk:psm-interop-common-prod-tests 4b977b7 PiperOrigin-RevId: 630502807
sergiitk
added a commit
to sergiitk/grpc
that referenced
this pull request
May 6, 2024
Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79 Closes grpc#36450 COPYBARA_INTEGRATE_REVIEW=grpc#36450 from sergiitk:psm-interop-common-prod-tests 4b977b7 PiperOrigin-RevId: 630502807
This was referenced May 6, 2024
sergiitk
added a commit
to sergiitk/grpc
that referenced
this pull request
May 6, 2024
Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79 Closes grpc#36450 COPYBARA_INTEGRATE_REVIEW=grpc#36450 from sergiitk:psm-interop-common-prod-tests 4b977b7 PiperOrigin-RevId: 630502807
sergiitk
added a commit
to sergiitk/grpc
that referenced
this pull request
May 6, 2024
Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79 Closes grpc#36450 COPYBARA_INTEGRATE_REVIEW=grpc#36450 from sergiitk:psm-interop-common-prod-tests 4b977b7 PiperOrigin-RevId: 630502807
sergiitk
added a commit
to sergiitk/grpc
that referenced
this pull request
May 6, 2024
Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79 Closes grpc#36450 COPYBARA_INTEGRATE_REVIEW=grpc#36450 from sergiitk:psm-interop-common-prod-tests 4b977b7 PiperOrigin-RevId: 630502807
sergiitk
added a commit
to sergiitk/grpc
that referenced
this pull request
May 6, 2024
Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79 Closes grpc#36450 COPYBARA_INTEGRATE_REVIEW=grpc#36450 from sergiitk:psm-interop-common-prod-tests 4b977b7 PiperOrigin-RevId: 630502807
sergiitk
added a commit
to sergiitk/grpc
that referenced
this pull request
May 6, 2024
Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79 Closes grpc#36450 COPYBARA_INTEGRATE_REVIEW=grpc#36450 from sergiitk:psm-interop-common-prod-tests 4b977b7 PiperOrigin-RevId: 630502807
sergiitk
added a commit
to sergiitk/grpc
that referenced
this pull request
May 6, 2024
Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79 Closes grpc#36450 COPYBARA_INTEGRATE_REVIEW=grpc#36450 from sergiitk:psm-interop-common-prod-tests 4b977b7 PiperOrigin-RevId: 630502807
sergiitk
added a commit
to sergiitk/grpc
that referenced
this pull request
May 6, 2024
Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79 Closes grpc#36450 COPYBARA_INTEGRATE_REVIEW=grpc#36450 from sergiitk:psm-interop-common-prod-tests 4b977b7 PiperOrigin-RevId: 630502807
yashykt
pushed a commit
that referenced
this pull request
May 6, 2024
…ort) (#36542) Backport of #36450 to v1.55.x. --- Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79
yashykt
pushed a commit
that referenced
this pull request
May 6, 2024
…ort) (#36541) Backport of #36450 to v1.56.x. --- Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79
yashykt
pushed a commit
that referenced
this pull request
May 6, 2024
…ort) (#36540) Backport of #36450 to v1.57.x. --- Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79
yashykt
pushed a commit
that referenced
this pull request
May 6, 2024
…ort) (#36539) Backport of #36450 to v1.58.x. --- Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79
yashykt
pushed a commit
that referenced
this pull request
May 6, 2024
…ort) (#36538) Backport of #36450 to v1.59.x. --- Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79
yashykt
pushed a commit
that referenced
this pull request
May 6, 2024
…ort) (#36535) Backport of #36450 to v1.60.x. --- Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79
yashykt
pushed a commit
that referenced
this pull request
May 6, 2024
…ort) (#36534) Backport of #36450 to v1.61.x. --- Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79
yashykt
pushed a commit
that referenced
this pull request
May 6, 2024
…ort) (#36533) Backport of #36450 to v1.62.x. --- Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79
yashykt
pushed a commit
that referenced
this pull request
May 6, 2024
…ort) (#36532) Backport of #36450 to v1.63.x. --- Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73. Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to [psm_interop_kokoro_lib.sh](https://github.com/grpc/psm-interop/blob/main/.kokoro/psm_interop_kokoro_lib.sh): 1. The list of tests in the each test suite 2. Per-test-suite flag customization 3. `run_test` methods 4. `build_docker_images_if_needed` methods 5. Generic `build_test_app_docker_images` methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow. Now all PSM Interop jobs share the same buildscripts by all test suites: 1. buildscript that invokes the test: `psm-interop-test-{language}.sh` (configured as `build_file` in the build cfg) 2. buildscript that builds the xDS test client/server and publishes them as a Docker image: `psm-interop-build-{language}.sh` (conventional name called from `psm_interop_kokoro_lib.sh`) `psm-interop-test-{language}.sh`: 1. Sets `GRPC_LANGUAGE`, `BUILD_SCRIPT_DIR` environment variables. 2. Downloads the shared `psm_interop_kokoro_lib.sh` from the main branch of the psm-interop repo. 3. Sources `psm-interop-build-{language}.sh` 4. Calls `psm::run "${PSM_TEST_SUITE}"` (`PSM_TEST_SUITE` configured in the cfg file). `psm-interop-build-{language}.sh`: 1. Defines `psm::lang::build_docker_images` which is called from `psm_interop_kokoro_lib.sh`. 2. Invokes any repo-specific logic. 3. May use `psm::build::docker_images_generic` for generic Docker build, tag, push, or provide implement its own build/publish method. References: - b/288578634 - See the full list of the new features at grpc/psm-interop#73. - Additional fixes to the shared lib: grpc/psm-interop#78, grpc/psm-interop#79
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
area/psm interop
bloat/none
lang/c++
per-call-memory/decrease
per-channel-memory/neutral
python
Pull requests that update Python code
release notes: no
Indicates if PR should not be in release notes
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73.
Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to psm_interop_kokoro_lib.sh:
run_test
methodsbuild_docker_images_if_needed
methodsbuild_test_app_docker_images
methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow.Now all PSM Interop jobs share the same buildscripts by all test suites:
psm-interop-test-{language}.sh
(configured asbuild_file
in the build cfg)psm-interop-build-{language}.sh
(conventional name called frompsm_interop_kokoro_lib.sh
)psm-interop-test-{language}.sh
:GRPC_LANGUAGE
,BUILD_SCRIPT_DIR
environment variables.psm_interop_kokoro_lib.sh
from the main branch of the psm-interop repo.psm-interop-build-{language}.sh
psm::run "${PSM_TEST_SUITE}"
(PSM_TEST_SUITE
configured in the cfg file).psm-interop-build-{language}.sh
:psm::lang::build_docker_images
which is called frompsm_interop_kokoro_lib.sh
.psm::build::docker_images_generic
for generic Docker build, tag, push, or provide implement its own build/publish method.References: