Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

e2e scenario test: Build with workspace #478

Merged
merged 1 commit into from
Feb 27, 2023
Merged

e2e scenario test: Build with workspace #478

merged 1 commit into from
Feb 27, 2023

Conversation

justinsb
Copy link
Member

@justinsb justinsb commented Feb 24, 2023 •
edited
Loading

We tweak the scenarios and make it easier to run with a go workspace, including building with the latest k/k.

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Feb 24, 2023
@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Feb 24, 2023
@justinsb
Copy link
Member Author

justinsb commented Feb 24, 2023 •
edited
Loading

Assuming we are in ~/testit/cloud-provider-gcp, and k8s is checked out into ~/testit/kubernetes ...

> e2e/setup-workspace.sh
...
> IMAGE_TAG=latest IMAGE_REPO=ko.local e2e/scenarios/kops-simple 
...
> docker run -it ko.local/cloud-controller-manager:latest --help | grep concu
      --concurrent-node-syncs int32   Number of workers concurrently synchronizing nodes. (default 1)
      --concurrent-service-syncs int32   The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1)

@aojea
Copy link
Member

aojea commented Feb 24, 2023

Assuming we are in ~/testit/cloud-provider-gcp, and k8s is checked out into ~/testit/kubernetes ...

> e2e/setup-workspace.sh
...
> IMAGE_TAG=latest IMAGE_REPO=ko.local e2e/scenarios/kops-simple 
...
> docker run -it ko.local/cloud-controller-manager:latest --help | grep concu
      --concurrent-node-syncs int32   Number of workers concurrently synchronizing nodes. (default 1)
      --concurrent-service-syncs int32   The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load (default 1)

lovely

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Feb 24, 2023
@justinsb justinsb changed the title WIP: e2e scenario test: Build with workspace e2e scenario test: Build with workspace Feb 26, 2023
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 26, 2023
justinsb
justinsb commented Feb 26, 2023
View reviewed changes
e2e/scenarios/kops-simple
@@ -86,7 +86,9 @@ if [[ -z "${ADMIN_ACCESS}" ]]; then
fi
echo "ADMIN_ACCESS=${ADMIN_ACCESS}"

create_args="--networking gce"
# cilium does not yet pass conformance tests (shared hostport test)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how we're getting around this elsewhere... this is the seemingly well-known cilium/cilium#14287

e2e/scenarios/kops-simple
@@ -103,13 +105,21 @@ echo "KOPS_FEATURE_FLAGS=${KOPS_FEATURE_FLAGS}"
# Note that these arguments for kubetest2 and kOps, not (for example) the arguments passed to the cloud-provider-gcp
KUBETEST2_ARGS="kops --kops-binary-path=${BINDIR}/kops"
KUBETEST2_ARGS="${KUBETEST2_ARGS} -v=2 --cloud-provider=${CLOUD_PROVIDER}"
KUBETEST2_ARGS="${KUBETEST2_ARGS} --cluster-name=${CLUSTER_NAME:-} --kops-root=${REPO_ROOT}"
KUBETEST2_ARGS="${KUBETEST2_ARGS} --cluster-name=${CLUSTER_NAME:-} --kops-root=${WORKSPACE}/kops"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Fixing an error in the original script)

e2e/scenarios/kops-simple
KUBETEST2_ARGS="${KUBETEST2_ARGS} --admin-access=${ADMIN_ACCESS:-} --env=KOPS_FEATURE_FLAGS=${KOPS_FEATURE_FLAGS}"

if [[ -n "${GCP_PROJECT:-}" ]]; then
KUBETEST2_ARGS="${KUBETEST2_ARGS} --gcp-project=${GCP_PROJECT}"
fi

if [[ -n "${STAGE_LOCATION:-}" ]]; then
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are useful when debugging locally.

I'm using a command like

export KUBE_SSH_USER=${USER}; SSH_PRIVATE_KEY=~/.ssh/google_compute_engine STAGE_LOCATION=gs://kops-dev-$(gcloud config get project)-${USER}/ e2e/scenarios/kops-simple 2>&1 | tee log

e2e/scenarios/kops-simple
@@ -125,18 +135,17 @@ fi

kubetest2 ${KUBETEST2_ARGS} \
--up \
\
--build \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(--build will build kOps, which is needed because otherwise we can be mixing versions here, between the binary we built above and supporting binaries like the on-node agent)

e2e/scenarios/kops-simple
kubetest2 ${KUBETEST2_ARGS} \
--test=kops \
-- \
--test-package-version="${K8S_VERSION}" \
--parallel=25 \
--parallel=30 \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to match what you did in your example :-)

e2e/setup-kops.sh Outdated
@@ -0,0 +1,29 @@
#!/usr/bin/env bash
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file isn't strictly necessary, but included for homogeneity with setup-kube.sh

e2e/setup-kube.sh Outdated
# Clone kubernetes into ${WORKSPACE}/kubernetes
cd "${WORKSPACE}"
KUBE_VERSION=${KUBE_VERSION:-master}
git clone --depth 1 --branch ${KUBE_VERSION} https://github.com/kubernetes/kubernetes kubernetes
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should assume this has already been done, either by prow with extra_refs, or by a user that is developing and setting up a local development workspace.

In the local-development scenario, I imagine we want the go work... commands (and workarounds for go.mod), but maybe these don't belong under e2e?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should assume this has already been done, either by prow with extra_refs, or by a user that is developing and setting up a local development workspace.

yes, I think that is the best approach

@justinsb
Copy link
Member Author

/assign @aojea

@justinsb @aojea
e2e scenario test: Build with workspaces
0110c88 
We tweak the scenarios and make it easier to run with a go workspace,
including building with the latest k/k.

Co-authored-by: Antonio Ojea <aojea@google.com>
@aojea
Copy link
Member

aojea commented Feb 27, 2023

/approve
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 27, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aojea, justinsb

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 27, 2023
@k8s-ci-robot k8s-ci-robot merged commit c6ff3a5 into kubernetes:master Feb 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aojea aojea Awaiting requested review from aojea

@jpbetz jpbetz Awaiting requested review from jpbetz

Assignees

@aojea aojea

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@justinsb @aojea @k8s-ci-robot