Bug 1823714: Update PkgManifest upon catsrc update #1482

Bowenislandsong · 2020-04-24T17:56:08Z

The catsrc update sync triggred by the informer does not update pkgmanifest. This commit addes update mechanism.

Description of the change:

Motivation for the change:

Reviewer Checklist

Implementation matches the proposed design, or proposal is updated to match implementation
Sufficient unit test coverage
Sufficient end-to-end test coverage
Docs updated or added to /docs
Commit messages sensible and descriptive

openshift-ci-robot · 2020-04-24T17:56:14Z

@Bowenislandsong: This pull request references Bugzilla bug 1823714, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.5.0) matches configured target release for branch (4.5.0)
bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1823714: Update PkgManifest upon catsrc update

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Bowenislandsong · 2020-04-24T22:23:58Z

/retest

Bowenislandsong · 2020-04-27T12:27:29Z

/retest

awgreene

Hey @Bowenislandsong - nice work, this looks good. I have a few questions that I'd like you to address before I LGTM. Thanks!

awgreene · 2020-04-27T12:38:09Z

pkg/package-server/provider/registry.go

@@ -189,6 +189,15 @@ func (p *RegistryProvider) syncCatalogSource(obj interface{}) (syncError error)
 		Namespace: source.GetNamespace(),


You commit message has misspelled words: triggred andaddes. You should consider enabling spellcheck in VIM or whichever editor you use to create your commit messages.

neat! did not know that's an option

funny, i went and looked. turns out i had it on. Somehow i dont see notifiations. I'll check what was wrong with that.

awgreene · 2020-04-27T12:48:46Z

pkg/package-server/provider/registry.go

+
+	logger.Infof("updating PackageManifest based on CatalogSource changes: %v", key)
+	timeout, cancel := context.WithTimeout(context.Background(), cacheTimeout)
+	defer cancel()
+	client, err := p.registryClient(key)
+	if err == nil {
+		err = p.refreshCache(timeout, client)
+	}


I see that your adding this code to check if the list of packages has changed. The code looks correct in that it will update the list of packagemanifests but I have two questions:

Could you make sure that the error is captured when the PackageServer is unable to refresh the cache? Either as a log or returning the error?

It looks like this could be moved into the if statement below to prevent it from being called when sources don't already exist. Could you investigate?

wow that was a very nice catch, I assumed the return was err instead of syncError. Kinda a weird way to express dislike for this kind of error return method.

good call, that would reduce the traffic. I somehow thought about it backward initially.

harishsurf · 2020-04-27T15:31:28Z

test/e2e/packagemanifest_e2e_test.go

@@ -168,6 +169,74 @@ var _ = Describe("Package Manifest", func() {
 			require.True(GinkgoT(), ok, "Expect this image %s to exist in the related images list\n", v)
 		}
 	})
+	It("updating CatalogSource", func() {


Could the test be more structured to use ginkgo's container blocks and replacing assertions with gomega matcher library? You could refer to #1424 as a reference

you could refer to this cheatsheet

You are right, that would be much better. I did think about it, but may I suggest that we do that in a separate PR and have this test arranged exactly like the other tests in this file? IMO refactoring should be its own PR rather than smuggles in bug fixes.

I updated the test to at least not to use assert and conform to the other tests.

This is a new test -- why create more future work for ourselves? If this bug were on fire, I can see the argument for addressing the test feedback in a follow-up, but it doesn't seem to be urgent enough to do that.

Lets wait for https://github.com/operator-framework/operator-lifecycle-manager/pull/1439/files to merge first.

I agree with @benluddy

Yes, I agree, but I am saying waiting for https://github.com/operator-framework/operator-lifecycle-manager/pull/1439/files will resolve all of our concerns. That PR is very close to merging.

/hold

pkg/package-server/provider/registry.go

SSshahan · 2020-05-09T03:16:21Z

/test e2e-aws-olm

awgreene · 2020-05-13T12:52:06Z

/retest

awgreene

one or two comments.

awgreene · 2020-05-13T12:54:34Z

pkg/package-server/provider/registry.go

+		client, err := p.registryClient(key)
+		if err != nil {
+			syncError = err
+			return


Suggested change

client, err := p.registryClient(key)

if err != nil {

syncError = err

return

client, syncError := p.registryClient(key)

if syncErr != nil {

return

}

Thanks! I will update once the dependent PR is merged!

Just realized, i can't do this, and was delieberatly avoiding it. This reinitialized syncError.

Can we just return err on line 199?

@Bowenislandsong just change := to = and decalre the client variable before the function call

Bowenislandsong · 2020-05-21T13:50:36Z

/hold cancel

harishsurf

Looks good. I just had one comment for the e2e test.

test/e2e/packagemanifest_e2e_test.go

Bowenislandsong · 2020-05-26T19:39:27Z

/retest

Bowenislandsong · 2020-05-26T19:39:51Z

/retest e2e-aws-olm

awgreene

A few non-blocking nits, nice work.

awgreene · 2020-05-27T17:43:22Z

pkg/package-server/provider/registry.go

+		client, err := p.registryClient(key)
+		if err != nil {
+			syncError = err
+			return


@Bowenislandsong just change := to = and decalre the client variable before the function call

awgreene · 2020-05-27T17:46:02Z

test/e2e/packagemanifest_e2e_test.go

+				catalogSource, err = crc.OperatorsV1alpha1().CatalogSources(testNamespace).Get(context.TODO(), catalogSource.GetName(), metav1.GetOptions{})
+				Expect(err).NotTo(HaveOccurred(), "error getting catalogSource")
+
+				displayName := "updated Name"


shouldn't this be = not :=

awgreene · 2020-05-27T20:25:42Z

/lgtm

The catsrc update sync triggered by the informer does not update pkgmanifest. This commit adds an update mechanism.

dinhxuanvu

/lgtm

dinhxuanvu · 2020-05-27T22:06:01Z

/approve

kevinrizza · 2020-05-27T22:07:39Z

/approve

openshift-ci-robot · 2020-05-27T22:07:43Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: awgreene, Bowenislandsong, dinhxuanvu, kevinrizza

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [kevinrizza]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-bot · 2020-05-27T22:54:43Z

/retest