Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 1920205: use secure local registry for e2e tests #552

Merged

Conversation

exdx
Copy link
Member

@exdx exdx commented Jan 13, 2021

Description of the change:
Use local docker registry instead of quay.io for pushing images during the course of opm e2e tests

Motivation for the change:
The e2e tests are currently skipped on forks because the quay credentials cannot be securely provided to the fork. The e2e image has been made public and a local registry is spun up during the course of the test instead.

Reviewer Checklist

  • Implementation matches the proposed design, or proposal is updated to match implementation
  • Sufficient unit test coverage
  • Sufficient end-to-end test coverage
  • Docs updated or added to /docs
  • Commit messages sensible and descriptive

@codecov
Copy link

codecov bot commented Jan 13, 2021

Codecov Report

Merging #552 (088e862) into master (8e6c13f) will increase coverage by 0.03%.
The diff coverage is 44.18%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #552      +/-   ##
==========================================
+ Coverage   49.45%   49.49%   +0.03%     
==========================================
  Files          90       91       +1     
  Lines        6317     6302      -15     
==========================================
- Hits         3124     3119       -5     
+ Misses       2497     2492       -5     
+ Partials      696      691       -5     
Impacted Files Coverage Δ
pkg/appregistry/dbloader.go 0.00% <0.00%> (ø)
pkg/lib/indexer/indexer.go 6.93% <0.00%> (ø)
pkg/sqlite/query.go 67.08% <0.00%> (ø)
pkg/sqlite/remove.go 0.00% <ø> (ø)
pkg/sqlite/stranded.go 0.00% <0.00%> (ø)
pkg/sqlite/load.go 41.67% <46.87%> (-0.12%) ⬇️
pkg/sqlite/db.go 50.00% <50.00%> (ø)
pkg/appregistry/builder.go 31.73% <100.00%> (ø)
pkg/mirror/mirror.go 34.14% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8e6c13f...6e870ac. Read the comment docs.

@exdx exdx force-pushed the feat/e2e-registry branch 6 times, most recently from 53a5893 to e8c2e84 Compare January 14, 2021 19:23
@exdx exdx force-pushed the feat/e2e-registry branch 11 times, most recently from 23afb54 to c7051c4 Compare January 22, 2021 20:04
@exdx
Copy link
Member Author

exdx commented Jan 22, 2021

The e2e tests are now passing -- all tests are being run with podman, docker, and containerd enabled. The goal of the PR was to enable testing against a local registry without make any substantial changes to the tests themselves.

The only change to code was the potential for a data race that was uncovered in the containerd resolver.

 WARNING: DATA RACE
Write at 0x000003cea580 by goroutine 39:
  github.com/operator-framework/operator-registry/pkg/image/containerdregistry.NewResolver()
      /home/runner/work/operator-registry/operator-registry/pkg/image/containerdregistry/resolver.go:44 +0x51b
  github.com/operator-framework/operator-registry/pkg/image/containerdregistry.NewRegistry()
      /home/runner/work/operator-registry/operator-registry/pkg/image/containerdregistry/options.go:93 +0x69c
  github.com/operator-framework/operator-registry/pkg/lib/bundle.(*BundleExporter).Export()
      /home/runner/work/operator-registry/operator-registry/pkg/lib/bundle/exporter.go:47 +0x7bd
  github.com/operator-framework/operator-registry/pkg/lib/indexer.ImageIndexer.ExportFromIndex.func1()
      /home/runner/work/operator-registry/operator-registry/pkg/lib/indexer/indexer.go:547 +0x266
Previous write at 0x000003cea580 by goroutine 34:
  github.com/operator-framework/operator-registry/pkg/image/containerdregistry.NewResolver()
      /home/runner/work/operator-registry/operator-registry/pkg/image/containerdregistry/resolver.go:44 +0x51b
  github.com/operator-framework/operator-registry/pkg/image/containerdregistry.NewRegistry()
      /home/runner/work/operator-registry/operator-registry/pkg/image/containerdregistry/options.go:93 +0x69c
  github.com/operator-framework/operator-registry/pkg/lib/bundle.(*BundleExporter).Export()
      /home/runner/work/operator-registry/operator-registry/pkg/lib/bundle/exporter.go:47 +0x7bd
  github.com/operator-framework/operator-registry/pkg/lib/indexer.ImageIndexer.ExportFromIndex.func1()
      /home/runner/work/operator-registry/operator-registry/pkg/lib/indexer/indexer.go:547 +0x266

Having this PR in the codebase will enable registry e2e tests to run on every commit and help ensure stability for large changes to the registry like in the declarative index proposal.

@exdx exdx requested review from hasbro17 and removed request for jmrodri January 22, 2021 20:20
Copy link
Member

@dinhxuanvu dinhxuanvu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. Just a nit.

test/e2e/opm_test.go Outdated Show resolved Hide resolved
@exdx exdx changed the title feat: use secure local registry for e2e tests Bug 1920205: use secure local registry for e2e tests Jan 25, 2021
@openshift-ci-robot openshift-ci-robot added the bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. label Jan 25, 2021
@openshift-ci-robot openshift-ci-robot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Jan 25, 2021
@openshift-ci-robot
Copy link

@exdx: This pull request references Bugzilla bug 1920205, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.7.0) matches configured target release for branch (4.7.0)
  • bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1920205: use secure local registry for e2e tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@exdx
Copy link
Member Author

exdx commented Jan 25, 2021

/bugzilla refresh

@openshift-ci-robot
Copy link

@exdx: This pull request references Bugzilla bug 1920205, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.7.0) matches configured target release for branch (4.7.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Signed-off-by: Daniel Sover <dsover@redhat.com>
@kevinrizza
Copy link
Member

/approve

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 25, 2021
Copy link
Member

@dinhxuanvu dinhxuanvu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a quick question.

test/e2e/opm_test.go Show resolved Hide resolved
Copy link
Member

@dinhxuanvu dinhxuanvu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 25, 2021
@openshift-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dinhxuanvu, exdx, kevinrizza

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [dinhxuanvu,kevinrizza]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit 16422cd into operator-framework:master Jan 25, 2021
@openshift-ci-robot
Copy link

@exdx: All pull requests linked via external trackers have merged:

Bugzilla bug 1920205 has been moved to the MODIFIED state.

In response to this:

Bug 1920205: use secure local registry for e2e tests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@jianzhangbjz
Copy link
Contributor

/label qe-approved

@openshift-ci-robot openshift-ci-robot added the qe-approved Signifies that QE has signed off on this PR label Jan 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged. qe-approved Signifies that QE has signed off on this PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants