Bug 1817028: *: add an init container to stop the pod with bad revision

[alaypatel07]

Add an init container that fails the pod if it has stale revision data.
This can especially occur during DR as explained in #278.

This is a two-pronged solution:

1. Add the init container that fails on bad env data
2. Modify the clustermembercontroller to not add the member unless all
   init containers have exit 0.

[hexfusion]

Alay this will always fail see notes.

[hexfusion]

{"component":"entrypoint","file":"prow/entrypoint/run.go:168","func":"k8s.io/test-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Entrypoint received interrupt: terminated","time":"2020-03-27T20:04:41Z"}

prow blew up

/test all

[alaypatel07]

/retest

[hexfusion]

/test all

[alaypatel07]

/retest

[hexfusion]

/test all

[hexfusion]

sigh why are we failing AWS

[hexfusion]

        s: "promQL query: count_over_time(ALERTS{alertname!~\"Watchdog|AlertmanagerReceiversNotConfigured|KubeAPILatencyHigh\",alertstate=\"firing\",severity!=\"info\"}[2h]) >= 1 had reported incorrect results:\n[{\"metric\":{\"alertname\":\"TargetDown\",\"alertstate\":\"firing\",\"job\":\"metrics\",\"namespace\":\"openshift-console-operator\",\"service\":\"metrics\",\"severity\":\"warning\"},\"value\":[1585834134.713,\"51\"]}]",

we need to get the bottom of this

[deads2k]

I can't read this. I expected to see a downward API injection of IP address and then a bash comparison between the computed IP and the downward API IP.

[alaypatel07]

/retest

[openshift-ci-robot]

@alaypatel07: The following tests failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/e2e-aws-disruptive	52e4742	link	/test e2e-aws-disruptive
ci/prow/e2e-azure	52e4742	link	/test e2e-azure

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[hexfusion]

/lgtm

manual tested in DR

[hexfusion]

/skip

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alaypatel07, hexfusion

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [alaypatel07,hexfusion]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

@alaypatel07: All pull requests linked via external trackers have merged: openshift/library-go#760, openshift/cluster-etcd-operator#284. Bugzilla bug 1817028 has been moved to the MODIFIED state.

In response to this:

Bug 1817028: *: add an init container to stop the pod with bad revision

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[alaypatel07]

/cherrypick release-4.4

[openshift-cherrypick-robot]

@alaypatel07: new pull request created: #293

In response to this:

/cherrypick release-4.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.