Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 1868125: [opm] Add valid bundles to index when permissive enabled #517

Closed

Conversation

anik120
Copy link
Contributor

@anik120 anik120 commented Nov 13, 2020

Description of the change:

When the opm index add was used with the permissive flag, the index
was being created, but the valid bundles from the list of bundles provided
with the command was not being added. Instead, an empty index was being
created. This PR fixes the issue by adding the valid bundles to the index,
while ignoring the invalid ones.

Motivation for the change:

Reviewer Checklist

  • Implementation matches the proposed design, or proposal is updated to match implementation
  • Sufficient unit test coverage
  • Sufficient end-to-end test coverage
  • Docs updated or added to /docs
  • Commit messages sensible and descriptive

When the `opm index add` was used with the `permissive` flag, the index
was being created, but the valid bundles from the list of bundles provided
with the command was not being added. Instead, an empty index was being
created. This PR fixes the issue by adding the valid bundles to the index,
while ignoring the invalid ones.
@openshift-ci-robot openshift-ci-robot added the bugzilla/severity-low Referenced Bugzilla bug's severity is low for the branch this PR is targeting. label Nov 13, 2020
@openshift-ci-robot
Copy link

@anik120: This pull request references Bugzilla bug 1868125, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.7.0) matches configured target release for branch (4.7.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1868125: [opm] Add valid bundles to index when permissive enabled

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Nov 13, 2020
@kevinrizza
Copy link
Member

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Nov 13, 2020
Copy link
Member

@njhale njhale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One comment on the design.

Can we also add a test to make sure this works as expected?

}

func NewDirectoryPopulator(loader Load, graphLoader GraphLoader, querier Query, imageDirMap map[image.Reference]string, overwriteDirMap map[string]map[image.Reference]string, overwrite bool) *DirectoryPopulator {
func NewDirectoryPopulator(loader Load, graphLoader GraphLoader, querier Query, imageDirMap map[image.Reference]string, overwriteDirMap map[string]map[image.Reference]string, overwrite, permissive bool) *DirectoryPopulator {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to propagate this down so long as we always aggregate errors and continue instead of exiting immediately.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@njhale I'm trying to think why we might prefer to aggregate errors instead of propagating it all the way down.
The reason I originally decided to take it all the way down is coz

a) If we have introduced a concept of permissive for opm it seems natural to have the same concept for our DirectoryPopulator too. Basically if some other implementation instead of opm want to use DirectoryPopulator it should be aware/have access to permissive too.

b) If you scale this to n bundles where n is large, it seems more efficient to fail fast if permissive isn't enabled instead of churning all the way through.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like the error-permissive concept would benefit from a non-permissive error type, since there are error instances that shouldn't be permitted in any case.

type ErrNonPermissive struct {
	err error
}

func foo(l string) error {
	if err = nonPermissiveErrorReturner(l); err != nil {
		return ErrNonPermissive{err}
	}
	if err = permissiveErrorReturner(l); err != nil {
		return err
	}
	return nil
}

func bar(list []string, permissive bool) error {
	var errs []error
	for _, l := range list {
		err := foo(l)
		if err != nil {
			if errors.As(err, &ErrNonPermissive{}) || !permissive {
				return err
			}
			errs = append(errs, err)
		}
	}
	return utilerrors.NewAggregate(errs)
}

Then you wouldn't have to propagate permissive down as far, and you could have any procedure that returns an error "know" about permissiveness without implementing (too much) of the handling logic itself

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you scale this to n bundles where n is large, it seems more efficient to fail fast if permissive isn't enabled instead of churning all the way through.

fair enough, but these function signatures are already unwieldy as they are today. For the most part, I think we could benefit from not copying all this data around by making populate a method of DirectoryPopulator (unless it's used by more than one type of populator).

I'm also of the opinion that a lot of this code will get tossed soon anyway, so it may not be super important to keep things pretty now, but we should definitely think about it for the next iteration of operator-registry.

@openshift-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: anik120
To complete the pull request process, please assign kevinrizza after the PR has been reviewed.
You can assign the PR to them by writing /assign @kevinrizza in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

remainingImages = append(remainingImages, pkgImage)
if !i.permissive {
pkgRemainingImages++
remainingImages = append(remainingImages, pkgImage)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are remainingImages not used by callers who want permissive population?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@estroz remainingImages are the pkgImage s that are not "valid bundles" and could not be added to foundImages. The reason we need to not add "invalid bundles" to remainingImages when permissive mode is enabled is to make sure that we don't loop infinitely because imagesToAdd always has those "invalid images" in them

@njhale
Copy link
Member

njhale commented Nov 16, 2020

/retest

1 similar comment
@anik120
Copy link
Contributor Author

anik120 commented Nov 17, 2020

/retest

@openshift-merge-robot
Copy link
Contributor

@anik120: The following test failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/e2e-aws 0778a09 link /test e2e-aws

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@openshift-ci-robot openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 18, 2020
@openshift-ci-robot
Copy link

@anik120: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@anik120
Copy link
Contributor Author

anik120 commented Nov 18, 2020

#503 and #522 fixes the issue.

@anik120 anik120 closed this Nov 18, 2020
@openshift-ci-robot
Copy link

@anik120: This pull request references Bugzilla bug 1868125. The bug has been updated to no longer refer to the pull request using the external bug tracker. All external bug links have been closed. The bug has been moved to the NEW state.

In response to this:

Bug 1868125: [opm] Add valid bundles to index when permissive enabled

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugzilla/severity-low Referenced Bugzilla bug's severity is low for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants