Skip to content

Conversation

@camilamacedo86
Copy link
Contributor

@camilamacedo86 camilamacedo86 commented Sep 15, 2021

Description
Fix Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2001940

What was the problem
The wrong syntax with inner join returns the values for ANY head of channel and not only when it is the head of the channel which is the default of the package.

Problem faced (Example scenario)

Unable to deprecate the following bundles from the image registry.redhat.io/redhat/community-operator-index:v4.8.

apicurio-registry.v0.0.1 - defaultChannel:NO
apicurio-registry.v0.0.3-v1.2.3.final - defaultChannel:NO
apicurio-registry.v0.0.4-v1.3.2.final - defaultChannel:NO

When should deprecate since the image has the bundle: apicurio-registry-operator.v1.0.0-v2.0.0.final which is the head of the default channel 2.x which was not deprecated.

Testing the changes made in this PR

  1. Runing the command :
./bin/opm index deprecatetruncate --bundles="quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e,quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671,quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d" --from-index=registry.redhat.io/redhat/community-operator-index:v4.8  --tag "quay/cmacedo/community-operator-index:apicu" --allow-package-removal --container-tool=docker
Password:
Sorry, try again.
Password:
WARN[0000] DEPRECATION NOTICE:
Sqlite-based catalogs and their related subcommands are deprecated. Support for
them will be removed in a future release. Please migrate your catalog workflows
to the new file-based catalog format. 
INFO[0000] deprecating bundles from the index            bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0000] Pulling previous image registry.redhat.io/redhat/community-operator-index:v4.8 to get metadata  bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0000] running /usr/local/bin/docker pull registry.redhat.io/redhat/community-operator-index:v4.8  bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0004] running /usr/local/bin/docker pull registry.redhat.io/redhat/community-operator-index:v4.8  bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0006] Getting label data from previous image        bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0006] running docker inspect                        bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0006] running docker create                         bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0006] running docker cp                             bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0017] running docker rm                             bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0019] allow-package-removal enabled: checking default channel heads for package removal  bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0019] no head of default channel found - skipping package removal  bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0019] deprecating bundles                           bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0019] Generating dockerfile                         bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0019] writing dockerfile: index.Dockerfile351495467  bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0019] running docker build                          bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"
INFO[0019] [docker build -f index.Dockerfile351495467 -t quay/cmacedo/community-operator-index:apicu .]  bundles="[quay.io/openshift-community-operators/apicurio-registry@sha256:661b4538ba60b7e25754d82132590a9e10182f38679db1011abe81c161e8903e quay.io/openshift-community-operators/apicurio-registry@sha256:85f51fd88be01507d8f8ae9032066dee232960715a8b6e97f2b4c6d5d3cc9671 quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d]"

  1. Checking if all bundles were deprecated, the alpha channel was removed and the head of the default package channel was not removed as its channel:
docker create --name test quay/cmacedo/community-operator-index:apicu "yes"
docker cp test:/database/index.db .
open index.db

Screen Shot 2021-09-16 at 15 59 34

Screen Shot 2021-09-16 at 15 59 16

Screen Shot 2021-09-16 at 15 57 44

Regards usage of the method changed in this pr and possible impact which would require regression tests:

  • The places changed in this PR are only used in the specific context of deprecating bundle and I could not find any usage of them in any other operation
  • They are covered with tests and the checked scenario was added too.

Screen Shot 2021-09-16 at 16 47 40

Screen Shot 2021-09-16 at 16 48 30

@camilamacedo86
Copy link
Contributor Author

@codecov
Copy link

codecov bot commented Sep 15, 2021

Codecov Report

Merging #780 (ea3bd9f) into master (195bc03) will decrease coverage by 0.26%.
The diff coverage is 50.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #780      +/-   ##
==========================================
- Coverage   50.90%   50.64%   -0.27%     
==========================================
  Files         102      102              
  Lines        8753     8848      +95     
==========================================
+ Hits         4456     4481      +25     
- Misses       3458     3523      +65     
- Partials      839      844       +5     
Impacted Files Coverage Δ
pkg/sqlite/query.go 64.53% <0.00%> (-0.08%) ⬇️
pkg/sqlite/load.go 38.18% <100.00%> (+0.07%) ⬆️
alpha/declcfg/diff_include.go 31.91% <0.00%> (-58.72%) ⬇️
alpha/declcfg/diff.go 77.14% <0.00%> (-2.17%) ⬇️
alpha/action/diff.go 75.86% <0.00%> (+11.15%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 195bc03...ea3bd9f. Read the comment docs.

@camilamacedo86 camilamacedo86 changed the title fix: Unable to deprecate bundles that are head of a NOT default channel with opm deprecatetruncate wip : fix: Unable to deprecate bundles that are head of a NOT default channel with opm deprecatetruncate Sep 16, 2021
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 16, 2021
…el with opm deprecatetruncate

Signed-off-by: Camila Macedo <cmacedo@redhat.com>
…ot the pkg default

Signed-off-by: Camila Macedo <cmacedo@redhat.com>

(test): check that allows deprecate when the channel is not the default head

Signed-off-by: Camila Macedo <cmacedo@redhat.com>
@camilamacedo86 camilamacedo86 changed the title wip : fix: Unable to deprecate bundles that are head of a NOT default channel with opm deprecatetruncate fix: Unable to deprecate bundles that are head of a NOT default channel with opm deprecatetruncate Sep 16, 2021
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 16, 2021
SELECT package_name FROM package, channel
WHERE channel.package_name == package.name
AND package.default_channel == channel.name
AND channel.head_operatorbundle_name = (SELECT name FROM operatorbundle WHERE bundlepath=? LIMIT 1) `
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See that it was returning the package when the bundle is HEAD of channel but the channel is NOT the default channel of the package, e.g:

SELECT package_name FROM package 
	INNER JOIN channel ON channel.name = package.default_channel
	WHERE channel.head_operatorbundle_name = (SELECT name FROM operatorbundle WHERE bundlepath='quay.io/openshift-community-operators/apicurio-registry@sha256:239525b1a4a7030fb0184bea77a1e56b9bb652bbbb78e3facdedf829ee2f7e9d' LIMIT 1)

Screen Shot 2021-09-16 at 16 32 57

The select with inner join was wrong.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same thing here, the inner join was missing a compare on channel.package_name = package.name, this looks good.

isDefaultChannelHead := `SELECT head_operatorbundle_name FROM package, channel
WHERE channel.package_name == package.name
AND package.default_channel == channel.name
AND channel.head_operatorbundle_name = ?`
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see that the select with the inner join is wrong and is returning the head_operatorbundle_name when the bundle is head of a channel that is NOT the default channel of the package:

Screen Shot 2021-09-16 at 16 36 19

Screen Shot 2021-09-16 at 16 36 04

Screen Shot 2021-09-16 at 16 35 28

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified this, the inner join was missing a compare on package name.
SELECT head_operatorbundle_name FROM channel INNER JOIN package ON channel.name = package.default_channel AND channel.package_name = package.name WHERE channel.head_operatorbundle_name = ?

Copy link
Contributor

@ankitathomas ankitathomas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this, the change looks good!

isDefaultChannelHead := `SELECT head_operatorbundle_name FROM package, channel
WHERE channel.package_name == package.name
AND package.default_channel == channel.name
AND channel.head_operatorbundle_name = ?`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified this, the inner join was missing a compare on package name.
SELECT head_operatorbundle_name FROM channel INNER JOIN package ON channel.name = package.default_channel AND channel.package_name = package.name WHERE channel.head_operatorbundle_name = ?

SELECT package_name FROM package, channel
WHERE channel.package_name == package.name
AND package.default_channel == channel.name
AND channel.head_operatorbundle_name = (SELECT name FROM operatorbundle WHERE bundlepath=? LIMIT 1) `
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same thing here, the inner join was missing a compare on channel.package_name = package.name, this looks good.

@ankitathomas
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 16, 2021
isDefaultChannelHead := `SELECT head_operatorbundle_name FROM channel
INNER JOIN package ON channel.name = package.default_channel
WHERE channel.head_operatorbundle_name = ?`
isDefaultChannelHead := `SELECT head_operatorbundle_name FROM package, channel
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took this issue as an opportunity to dust off my SQL sk1llz and here's what I've found:

The issue in the original query seems to be that the inner join is only made against the channel name and there are two channels (2.x and alpha). So, it's also joining the alpha rows with packages that have a default channel of alpha. An alternative query here would be to join on both the package name and the default channel. E.g.

SELECT head_operatorbundle_name FROM channel
INNER JOIN package ON channel.package_name = package.name AND channel.name = package.default_channel
WHERE channel.head_operatorbundle_name = ?

I believe the above and what you propose are equivalent in performance and intent. However, this StackOverflow entry seems to suggest that the inner join syntax is recommended. Not that this means you should change anything. Just relating what I've found ^^

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I didn't see the previous discussion on this. Feel free to ignore the comments above ^^

@@ -0,0 +1,9 @@
apiVersion: v1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a question to help my understanding: why was it necessary to add all of these manifest and bundle files? why were they not there in the first place?

Copy link
Contributor Author

@camilamacedo86 camilamacedo86 Sep 16, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the scenario for we are able to add a new test to ensure that after the change we not only break anything as it is working as supposed to be. You can see that are the same bundles from the community image.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of these files don't directly impact the test at all.

@camilamacedo86 could you minimize the set of files added in this PR. This should only require CSVs and annotation files, right?

@@ -0,0 +1,9 @@
apiVersion: v1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of these files don't directly impact the test at all.

@camilamacedo86 could you minimize the set of files added in this PR. This should only require CSVs and annotation files, right?

@kevinrizza
Copy link
Member

This change lgtm minus Nick's comments around testing

@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Sep 22, 2021
@camilamacedo86
Copy link
Contributor Author

HI @njhale and @kevinrizza,

That the manifests were removed from the mocks. Feel free to check.

Signed-off-by: Camila Macedo <cmacedo@redhat.com>
@kevinrizza
Copy link
Member

/approve

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 23, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ankitathomas, camilamacedo86, kevinrizza

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 23, 2021
@ankitathomas
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 23, 2021
@openshift-merge-robot openshift-merge-robot merged commit ed832fd into operator-framework:master Sep 23, 2021
akihikokuroda pushed a commit to akihikokuroda/operator-registry that referenced this pull request Sep 30, 2021
…el with opm deprecatetruncate (operator-framework#780)

* fix: Unable to deprecate bundles that are head of a NOT default channel with opm deprecatetruncate

Signed-off-by: Camila Macedo <cmacedo@redhat.com>

* add test to ensure that the bundles are deprecated when the head is not the pkg default

Signed-off-by: Camila Macedo <cmacedo@redhat.com>

(test): check that allows deprecate when the channel is not the default head

Signed-off-by: Camila Macedo <cmacedo@redhat.com>

* removing uncessary manifests from the mock testdata

Signed-off-by: Camila Macedo <cmacedo@redhat.com>
Signed-off-by: akihikokuroda <akihikokuroda2020@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants