Skip to content

Commit

Permalink
blocked-edges: Details on bugs for 4.3.2 and 4.3.3
Browse files Browse the repository at this point in the history
Expanding on the links from b1465b7 (Information on why 4.3.2 and
4.3.3 not in fast and stable channels of 4.3, 2020-03-03, #87).

Port bug:

* 4.4.0: https://bugzilla.redhat.com/show_bug.cgi?id=1801300
  openshift/cluster-version-operator#322
  Which made it into 4.4.0-rc.0:

  $ oc adm release info --commits quay.io/openshift-release-dev/ocp-release:4.4.0-rc.0-x86_64 | grep cluster-version
    cluster-version-operator                       https://github.com/openshift/cluster-version-operator                       23856901003b95b559087b8e83bffdee82872b2b
  $ git --no-pager log --oneline --first-parent -6 origin/release-4.4
  2385690 (origin/release-4.4) Merge pull request openshift#332 from openshift-cherrypick-robot/cherry-pick-328-to-release-4.4
  ...
  2df3d56 Merge pull request openshift#322 from vrutkovs/remove-outdated-ports

* 4.3.z: https://bugzilla.redhat.com/show_bug.cgi?id=1802710
  openshift/cluster-version-operator#323
  Which made it into 4.3.3:

  $ oc adm release info --commits quay.io/openshift-release-dev/ocp-release:4.3.3-x86_64 | grep cluster-version
    cluster-version-operator                      https://github.com/openshift/cluster-version-operator                      210b4b1e6b1b7f53b5dc0d935de9c5d27058280c
  $ oc adm release info --commits quay.io/openshift-release-dev/ocp-release:4.3.2-x86_64 | grep cluster-version
    cluster-version-operator                      https://github.com/openshift/cluster-version-operator                      beee410fc8780e5613c09fc2690716b711747041
  $ git --no-pager log --oneline --first-parent -5 origin/release-4.3
  210b4b1 (origin/release-4.3) Merge pull request openshift#321 from openshift-cherrypick-robot/cherry-pick-319-to-release-4.3
  5057680 Merge pull request openshift#323 from vrutkovs/4.3-container-ports
  ...
  beee410 Merge pull request openshift#290 from wking/no-ephemeral-storage-in-4.2-so-4.3-cannot-rely-on-it

* 4.2.z: https://bugzilla.redhat.com/show_bug.cgi?id=1802248
  openshift/cluster-version-operator#325
  Which made it into 4.2.21:

  $ oc adm release info --commits quay.io/openshift-release-dev/ocp-release:4.2.21-x86_64 | grep cluster-version
    cluster-version-operator                      https://github.com/openshift/cluster-version-operator                      ccbed39b6faab201a1bafc49a7f519194d5dd548
  $ oc adm release info --commits quay.io/openshift-release-dev/ocp-release:4.2.20-x86_64 | grep cluster-version
    cluster-version-operator                      https://github.com/openshift/cluster-version-operator                      9f4f04e736a0bbc61323593fbb62874570f07762
  $ git --no-pager log --oneline --first-parent -4 origin/release-4.2
  4b39863 (origin/release-4.2) Merge pull request openshift#314 from wking/resource-merge-index-mutating-existing-4.2
  ccbed39 Merge pull request openshift#300 from openshift-cherrypick-robot/cherry-pick-298-to-release-4.2
  a8ed501 Merge pull request openshift#325 from vrutkovs/4.2-container-ports
  9f4f04e Merge pull request openshift#288 from wking/no-ephemeral-storage-in-4.2

* autoscaler and machine-API operator both removed their metrics port
  in 4.2 -> 4.3 [1].  So 4.2 clusters which update to 4.3 < 4.3.5 will
  hit this.

* ingress operator removed its metrics port in 4.1 -> 4.2 [2], so 4.1
  clusters which update to 4.2 < 4.2.21 will hit this.

Multus bug:

* 4.5.0: https://bugzilla.redhat.com/show_bug.cgi?id=1805987

* 4.4.0: https://bugzilla.redhat.com/show_bug.cgi?id=1805774
  openshift/cluster-network-operator#507
  openshift/multus-cni#49
  Which made it into 4.4.0-rc.0:

  $ oc adm release info --commits quay.io/openshift-release-dev/ocp-release:4.4.0-rc.0-x86_64 | grep 'network-operator\|multus-cni'
    cluster-network-operator                       https://github.com/openshift/cluster-network-operator                       983f31b7c4a207542bb71b8221addf82a954c6e0
    multus-cni                                     https://github.com/openshift/multus-cni                                     384bc19c84d2da109f9a5e30367c86bf32ad4e51
  cluster-network-operator$ git --no-pager log --oneline --first-parent -5 origin/release-4.4
  c4ee5bbd (origin/release-4.4) Merge pull request openshift#518 from openshift-cherrypick-robot/cherry-pick-516-to-release-4.4
  983f31b7 Merge pull request openshift#514 from pecameron/cp500.4.4
  ...
  53afa956 Merge pull request openshift#507 from danwinship/add-readiness-indicator-4.4
  multus-cni$ git --no-pager log --oneline --first-parent -1 origin/release-4.4
  384bc19c (origin/release-4.4) Merge pull request #53 from dougbtv/no-config-invalidation-44

* 4.3.z: https://bugzilla.redhat.com/show_bug.cgi?id=1805444
  openshift/cluster-network-operator#485
  openshift/multus-cni#48

  Which made it into 4.3.5 (4.3.4 doesn't exist):

  $ oc adm release info --commits quay.io/openshift-release-dev/ocp-release:4.3.5-x86_64 | grep 'network-operator\|multus-cni'
    cluster-network-operator                      https://github.com/openshift/cluster-network-operator                      93c9431b9f5160cc620bba93a6f59e2525a54e2c
    multus-cni                                    https://github.com/openshift/multus-cni                                    50acccef844042b0bc3ffe5f243cb2e13647d07d
  $ oc adm release info --commits quay.io/openshift-release-dev/ocp-release:4.3.3-x86_64 | grep 'network-operator\|multus-cni'
    cluster-network-operator                      https://github.com/openshift/cluster-network-operator                      3e76a1ef1dccac9b44bdbd00ff145b2a1abe611a
    multus-cni                                    https://github.com/openshift/multus-cni                                    1cb7d0f9c0a336ba7f33a0e2800c12205f10878c
  cluster-network-operator$ git --no-pager log --oneline --first-parent -9 origin/release-4.3
  3af6eeb4 (origin/release-4.3) Merge pull request openshift#526 from dougbtv/reintroduce-whereabouts-routeoverride-43
  ...
  93c9431b Merge pull request openshift#504 from juanluisvaladas/fix-noproxy-43
  7ae0b7e8 Merge pull request openshift#485 from dougbtv/add-readiness-indicator-43
  ...
  3e76a1ef Merge pull request openshift#470 from alexanderConstantinescu/bugfix/fix_ovn_controller_rbac
  multus-cni$ git --no-pager log --oneline --first-parent -3 origin/release-4.3
  9e85cb1b (origin/release-4.3) Merge pull request #52 from dougbtv/no-config-invalidation-43
  50acccef Merge pull request #48 from dougbtv/readiness-indicator-poll-43
  1cb7d0f9 Merge pull request #40 from dougbtv/rhel8-set-commit

* Not backported to 4.2.z.  It's not clear to me what the situation on
  4.2 is.  Also not clear to me what sort of updates would trigger
  this issue; whether it is all *-> into broken releases?  Just 4.2 ->
  broken 4.3?  Broken 4.2 -> any 4.3?  But we don't need to bottom
  that out, because of the next point.

* The impact is only a minute or so of unreachable workloads, after
  which the issue resolves automatically.  The UpgradeBlocker was
  dropped from the 4.3 bug in [3].  So I'm just removing references to
  this series from blocked-edges.

Memory leak:

* 4.5.0: MCO    https://bugzilla.redhat.com/show_bug.cgi?id=1800319
           openshift/machine-config-operator#1450
           openshift/machine-config-operator#1476
         origin https://bugzilla.redhat.com/show_bug.cgi?id=1802687
           openshift/origin#24568

* 4.4.0: origin https://bugzilla.redhat.com/show_bug.cgi?id=1806786
           openshift/origin#24596
  The 4.5 MCO changes were not backported to 4.4.
  The origin change made it into 4.4.0-rc.0:

  $ oc adm release info --commits quay.io/openshift-release-dev/ocp-release:4.4.0-rc.0-x86_64 | grep hyperkube
    hyperkube                                      https://github.com/openshift/ose                                            261a15f4558887a0e53b4d5ad3e4e82f6d37b59f
  $ git --no-pager log -50 --format='%h %b' origin/enterprise-4.4 | grep -n '261a15f455\|1806786'
  5:261a15f455
  42:2850b41468 [release-4.4] Bug 1806786: UPSTREAM: 88251: Partially fix incorrect configuration of kubepods.slice unit by kubelet

* 4.3.z: origin https://bugzilla.redhat.com/show_bug.cgi?id=1808429
           openshift/origin#24611
         https://bugzilla.redhat.com/show_bug.cgi?id=1801824 ASSIGNED
           I'm not sure what this one is; possibly a dup of 1808429.
  The origin change did not make it into 4.3.5:

  $ oc adm release info --commits quay.io/openshift-release-dev/ocp-release:4.3.5-x86_64 | grep hyperkube
    hyperkube                                     https://github.com/openshift/ose                                           b3bfb5a8f782055fe16140fc3a670dd9d27271a7
  $ git --no-pager log --format='%h %b' origin/enterprise-4.3 | grep -n 'b3bfb5a8f7\|1808429'
  14:ec856fba4d [release-4.3] Bug 1808429: UPSTREAM: 88251: Partially fix incorrect configuration of kubepods.slice unit by kubelet
  16:b3bfb5a8f7 Created by command:

* 4.2.z: https://bugzilla.redhat.com/show_bug.cgi?id=1801826 Closed WONTFIX.
         https://bugzilla.redhat.com/show_bug.cgi?id=1810136 s380x NEW
  Not sure if that's "does not affect 4.2.z" or "not important enough
  to fix on 4.2.z".

* This doesn't seem to be a regression, and doesn't seem to be
  update-specific.  UpgradeBlocker was dropped from the 4.3 bug in
  [4].  So I'm just removing references to this series from
  blocked-edges.

Also, no need to talk about these in the channel YAML files, since we
aren't tombstoning the releases (we discovered the bug after marking
the releases supported by tagging them into fast channels).

Also block * -> 4.3 for 4.3 before 4.3.5, now that we have the Multus
bug to worry about.  4.2 -> 4.3.1 had been blocked before, although
that block was lifted in c641bbd (channels/fast-4.2: Promote 4.2.18
(and 4.2.18+amd64 to fast-4.3), 2020-02-19, #60).  This isolates the
two 4.3 RCs, but we don't commit to providing updates out of candidate
releases anway.  If we get pushback on that (unlikely now that 4.3 has
been GA for so long), we can add rc -> 4.3.6 update edges or some
such.

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1801300#c29
[2]: https://bugzilla.redhat.com/show_bug.cgi?id=1802248#c3
[3]: https://bugzilla.redhat.com/show_bug.cgi?id=1805444#c9
[4]: https://bugzilla.redhat.com/show_bug.cgi?id=1808429#c6
  • Loading branch information
wking committed Mar 19, 2020
1 parent 0e8eda4 commit dd23858
Show file tree
Hide file tree
Showing 6 changed files with 7 additions and 4 deletions.
1 change: 1 addition & 0 deletions blocked-edges/4.3.0.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
to: 4.3.0
from: 4\.2\..*
# 4.2 -> 4.3 updates hit TargetDown for cluster-autoscaler-operator, fixed in 4.3.5, https://bugzilla.redhat.com/show_bug.cgi?id=1801300
# 4.2 -> 4.3 updates occasionally hit RequiredPoolsFailed, fixed in 4.2.18 and rc.0, but not in 4.2.16: https://bugzilla.redhat.com/show_bug.cgi?id=1782152 https://bugzilla.redhat.com/show_bug.cgi?id=1782149
# 4.2 -> 4.3 updates on GCP occasionally have unreachable workloads: https://bugzilla.redhat.com/show_bug.cgi?id=1793635
4 changes: 4 additions & 0 deletions blocked-edges/4.3.1.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
to: 4.3.1
from: .*
# 4.2 -> 4.3 updates hit TargetDown for cluster-autoscaler-operator, fixed in 4.3.5, https://bugzilla.redhat.com/show_bug.cgi?id=1801300
# 4.2 -> 4.3 updates on GCP occasionally have unreachable workloads: https://bugzilla.redhat.com/show_bug.cgi?id=1793635
2 changes: 1 addition & 1 deletion blocked-edges/4.3.2.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# https://bugzilla.redhat.com/show_bug.cgi?id=1802248, https://bugzilla.redhat.com/show_bug.cgi?id=1805444, https://bugzilla.redhat.com/show_bug.cgi?id=1808429
to: 4.3.2
from: .*
# 4.2 -> 4.3 updates hit TargetDown for cluster-autoscaler-operator, fixed in 4.3.5, https://bugzilla.redhat.com/show_bug.cgi?id=1801300
2 changes: 1 addition & 1 deletion blocked-edges/4.3.3.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
# https://bugzilla.redhat.com/show_bug.cgi?id=1802248, https://bugzilla.redhat.com/show_bug.cgi?id=1805444, https://bugzilla.redhat.com/show_bug.cgi?id=1808429
to: 4.3.3
from: .*
# 4.2 -> 4.3 updates hit TargetDown for cluster-autoscaler-operator, fixed in 4.3.5, https://bugzilla.redhat.com/show_bug.cgi?id=1801300
1 change: 0 additions & 1 deletion channels/fast-4.3.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ versions:
# No 4.2.24 because of the service CA rotation issue: https://bugzilla.redhat.com/show_bug.cgi?id=1810036 https://bugzilla.redhat.com/show_bug.cgi?id=1801573
- 4.3.0
- 4.3.1
# Upgrade edges disabled for 4.3.2 and 4.3.3 because of bugs https://bugzilla.redhat.com/show_bug.cgi?id=1802248, https://bugzilla.redhat.com/show_bug.cgi?id=1805444, https://bugzilla.redhat.com/show_bug.cgi?id=1808429
- 4.3.2
- 4.3.3
# No 4.3.4 because of https://bugzilla.redhat.com/show_bug.cgi?id=1805726
Expand Down
1 change: 0 additions & 1 deletion channels/stable-4.3.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@ versions:
# 4.2.21 edge into stable-4.3 pending further review
- 4.3.0
- 4.3.1
# Upgrade edges disabled for 4.3.2 and 4.3.3 because of bugs https://bugzilla.redhat.com/show_bug.cgi?id=1802248, https://bugzilla.redhat.com/show_bug.cgi?id=1805444, https://bugzilla.redhat.com/show_bug.cgi?id=1808429
- 4.3.2
- 4.3.3
# No 4.3.4 because of https://bugzilla.redhat.com/show_bug.cgi?id=1805726
Expand Down

0 comments on commit dd23858

Please sign in to comment.