Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Creating squid AMI in VPC for CI aws proxy testing #4719

Merged
merged 1 commit into from Nov 5, 2019
Merged

Creating squid AMI in VPC for CI aws proxy testing #4719

merged 1 commit into from Nov 5, 2019

Conversation

ewolinetz
Copy link
Contributor

@ewolinetz ewolinetz commented Aug 12, 2019 •
edited

This implements phase 2 of https://jira.coreos.com/browse/DPTP-475 where we move to using a UPI install and blackhole the private subnet so it does not have external internet access directly.

This PR also collects the proxy access logs and stores them in proxy/squid.service during teardown.

@openshift-ci-robot openshift-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 12, 2019
@ewolinetz ewolinetz force-pushed the proxy_phase2_ci branch 2 times, most recently from 2e05907 to 6fe8fae Compare August 12, 2019 21:51
@ewolinetz
Copy link
Contributor Author

/test pj-rehearse

2 similar comments
@ewolinetz
Copy link
Contributor Author

/test pj-rehearse

@ewolinetz
Copy link
Contributor Author

/test pj-rehearse

@ewolinetz ewolinetz force-pushed the proxy_phase2_ci branch 2 times, most recently from 2e10b6b to c42d04a Compare August 13, 2019 22:29
@openshift-ci-robot openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 13, 2019
@openshift-ci-robot openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 14, 2019
@openshift-ci-robot openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 14, 2019
@ewolinetz ewolinetz force-pushed the proxy_phase2_ci branch 3 times, most recently from c831b68 to 8bbd55f Compare August 14, 2019 16:33
@ewolinetz
Copy link
Contributor Author

/test pj-rehearse

1 similar comment
@ewolinetz
Copy link
Contributor Author

/test pj-rehearse

@ewolinetz
Copy link
Contributor Author

/refresh

@ewolinetz ewolinetz closed this Aug 14, 2019
@ewolinetz ewolinetz reopened this Aug 14, 2019
@ewolinetz
Copy link
Contributor Author

/test pj-rehearse

1 similar comment
@ewolinetz
Copy link
Contributor Author

/test pj-rehearse

@ewolinetz ewolinetz force-pushed the proxy_phase2_ci branch 3 times, most recently from d41b377 to e2d7ac9 Compare August 14, 2019 19:05
@ewolinetz
Copy link
Contributor Author

/test pj-rehearse

1 similar comment
@ewolinetz
Copy link
Contributor Author

/test pj-rehearse

@wking
Copy link
Member

wking commented Oct 17, 2019

We seem to be out of the registry-CI breakage now.

/test pj-rehearse

@wking
Copy link
Member

wking commented Oct 18, 2019

/test pj-rehearse

3 similar comments
@ewolinetz
Copy link
Contributor Author

/test pj-rehearse

@danehans
Copy link

/test pj-rehearse

@danehans
Copy link

/test pj-rehearse

@ewolinetz
Copy link
Contributor Author

@abhinavdahiya can someone from the installer team please review this?

@ewolinetz
Copy link
Contributor Author

Not sure why this isn't being aggregated in the job details but the following tests failed and seem to just be flakes:

failed: (5m12s) 2019-10-21T20:48:15 "[sig-network] Networking Granular Checks: Services should function for node-Service: udp [Suite:openshift/conformance/parallel] [Suite:k8s]"

failed: (9m44s) 2019-10-21T20:30:23 "[sig-network] Networking Granular Checks: Services should function for endpoint-Service: udp [Suite:openshift/conformance/parallel] [Suite:k8s] [Skipped:Network/OVNKubernetes]"

failed: (9m46s) 2019-10-21T20:30:26 "[sig-network] Networking Granular Checks: Services should function for pod-Service: udp [Suite:openshift/conformance/parallel] [Suite:k8s]"

failed: (43.2s) 2019-10-21T20:40:50 "[sig-storage] In-tree Volumes [Driver: aws] [Testpattern: Pre-provisioned PV (ext3)] volumes should store data [Suite:openshift/conformance/parallel] [Suite:k8s]"

@abhinavdahiya
Copy link
Contributor

The proxy job is failing currently? Is that a flake?

sdodson
sdodson reviewed Oct 24, 2019
View reviewed changes
ci-operator/templates/openshift/installer/cluster-launch-installer-upi-e2e.yaml
@@ -550,7 +1236,8 @@ objects:

# begin bootstrapping
if [[ "${CLUSTER_TYPE}" == "aws" ]]; then
RHCOS_AMI=ami-0df3f99538fbef10f # FIXME: assumes AWS_REGION is us-east-1
# RHCOS_AMI=ami-0df3f99538fbef10f # 4.1 ami FIXME: assumes AWS_REGION is us-east-1
RHCOS_AMI=ami-0ae2df22579e00be5 # 4.2 ami FIXME: assumes AWS_REGION is us-east-1
Copy link
Member

@sdodson sdodson Oct 24, 2019 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it normal for AWS UPI jobs to require this? Seems like this will require maintenance that we should avoid.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I'm not sure if this value could be provided via a parameter but that would be more ideal.
It was required to bump it from the 4.1 AMI because it didn't have the logic to update the machine's trusted cert.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I'm not sure if this value could be provided via a parameter but that would be more ideal.

Other platforms use rhcos.json (e.g. here). But I dislike using an installer-internal file for our CI, and would rather stay closer to our official docs to ensure we bump AMIs in the docs when we make some change that breaks vs. the currently-documented AMIs. But this is complicated by us documenting different AMIs for different 4.y, and I don't have an easy way around that yet. I'd really like openshift/installer#2092 or one of those family to give users the same access to the RHCOS JSON that we enjoy in CI.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this value could be provided via a parameter but that would be more ideal.

I guess providing via a job-specific parameter would be the way to get per-4.y AMIs in in the absence of something like openshift/installer#2092.

@ewolinetz
Copy link
Contributor Author

The proxy job is failing currently? Is that a flake?

Yes, it seems to be a flake [1]. I've seen it pass two times, though in those instances it with a noProxy hack where we provided the cluster's base domain (though it was still flaky). @danehans had confirmed that hack is not required.

[1] #4719 (comment)

@openshift-ci-robot openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Oct 24, 2019
@ewolinetz
Copy link
Contributor Author

/test pj-rehearse

@danehans
Copy link

danehans commented Nov 5, 2019

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Nov 5, 2019
@ewolinetz
Copy link
Contributor Author

/test pj-rehearse

@abhinavdahiya
Copy link
Contributor

/approve

@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: abhinavdahiya, danehans, ewolinetz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 5, 2019
@openshift-merge-robot openshift-merge-robot merged commit f34ec4a into openshift:master Nov 5, 2019
@openshift-ci-robot
Copy link
Contributor

@ewolinetz: Updated the following 3 configmaps:

  • job-config-master configmap in namespace ci using the following files:
    • key openshift-installer-master-presubmits.yaml using file ci-operator/jobs/openshift/installer/openshift-installer-master-presubmits.yaml
  • prow-job-cluster-launch-installer-upi-e2e configmap in namespace ci using the following files:
    • key cluster-launch-installer-upi-e2e.yaml using file ci-operator/templates/openshift/installer/cluster-launch-installer-upi-e2e.yaml
  • prow-job-cluster-launch-installer-upi-e2e configmap in namespace ci-stg using the following files:
    • key cluster-launch-installer-upi-e2e.yaml using file ci-operator/templates/openshift/installer/cluster-launch-installer-upi-e2e.yaml

In response to this:

This implements phase 2 of https://jira.coreos.com/browse/DPTP-475 where we move to using a UPI install and blackhole the private subnet so it does not have external internet access directly.

This PR also collects the proxy access logs and stores them in proxy/squid.service during teardown.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ewolinetz ewolinetz mentioned this pull request Nov 5, 2019
Merged
@openshift-ci-robot
Copy link
Contributor

@ewolinetz: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
ci/rehearse/openshift/installer/master/e2e-gcp-upi 27c9755 link /test pj-rehearse
ci/rehearse/openshift/installer/master/e2e-vsphere 27c9755 link /test pj-rehearse
ci/rehearse/openshift/installer/master/e2e-aws-proxy 27c9755 link /test pj-rehearse
ci/prow/pj-rehearse 27c9755 link /test pj-rehearse

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sdodson sdodson sdodson left review comments

@wking wking wking left review comments

@danehans danehans danehans left review comments

@crawford crawford Awaiting requested review from crawford

Assignees

@danehans danehans

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
7 participants
@ewolinetz @openshift-merge-robot @danehans @wking @abhinavdahiya @openshift-ci-robot @sdodson