Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 1703941: use image-registry-certificates for system and user certificates #272

Merged
merged 1 commit into from
May 3, 2019

Conversation

dmage
Copy link
Contributor

@dmage dmage commented Apr 30, 2019

A config map with the annotation service.beta.openshift.io/inject-cabundle can contain only service-ca.crt. We need to combine user-provided certificates with this service-ca.crt save the result as image-registry-certificates.

https://bugzilla.redhat.com/show_bug.cgi?id=1703941

@openshift-ci-robot openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 30, 2019
@openshift-ci-robot openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Apr 30, 2019
@bparees
Copy link
Contributor

bparees commented Apr 30, 2019

A config map with the annotation service.beta.openshift.io/inject-cabundle can contain only service-ca.crt

was this a recent change? we had tested this previously right?

@adambkaplan does this affect the configmap that the buildcontroller creates for builds, also?

@adambkaplan
Copy link
Contributor

/hold

@bparees this affects builds as well. Observing this in my testing.

@openshift/sig-auth is the API for the annotation frozen? If possible we shouldn't ship GA code with beta APIs.

@openshift-ci-robot openshift-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 30, 2019
}

cm.Data[internalHostname] = cert
} else {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 - @dmage this will also help us drop the port on the internal registry's service.

@bparees
Copy link
Contributor

bparees commented Apr 30, 2019

This looks like the right direction, thanks @dmage
Can you confirm for me that when the service-ca.crt key gets updated the CM, we will

  1. observe the change
  2. update the merged CM
  3. rollout both the registry pods

(the node-ca pods will pick up the updated value from (2) automatically since they just reread it periodically)

@adambkaplan
Copy link
Contributor

/hold cancel

Annotation will be using service.beta.openshift.io for GA.

@openshift-ci-robot openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 1, 2019
@dmage
Copy link
Contributor Author

dmage commented May 2, 2019

@bparees yes

  1. we will receive an event and run the entire re-sync cycle
  2. it will run generatorCaConfig.Update, which will update the config map image-registry-certificates accordingly to the result of generatorCaConfig.expected
  3. and we have this resource as a dependency for the Deployment
  4. we inject into the pod template the checksum of all dependencies

@bparees
Copy link
Contributor

bparees commented May 2, 2019

we will receive an event and run the entire re-sync cycle

if we miss the event(for example, because we were down at the time), will we also detect the change on a relist?

everything else sounds good.

@dmage
Copy link
Contributor Author

dmage commented May 3, 2019

@bparees yes, we will.

@dmage dmage changed the title [WIP] Use image-registry-certificates for system and user certificates Use image-registry-certificates for system and user certificates May 3, 2019
@openshift-ci-robot openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 3, 2019
@dmage
Copy link
Contributor Author

dmage commented May 3, 2019

This PR removes support of image-registry.openshift-image-registry.svc.cluster.local:5000. Maybe we want it back, though.

@dmage
Copy link
Contributor Author

dmage commented May 3, 2019

/retest

@dmage
Copy link
Contributor Author

dmage commented May 3, 2019

/assign @adambkaplan

@bparees
Copy link
Contributor

bparees commented May 3, 2019

This PR removes support of image-registry.openshift-image-registry.svc.cluster.local:5000. Maybe we want it back, though.

yeah let's not rock the boat if we don't have to.

else
rm /etc/docker/certs.d/image-registry.openshift-image-registry.svc.cluster.local:5000/service-ca.crt
rm /etc/docker/certs.d/${internalRegistryHostname}/service-ca.crt
fi
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmage If I read the PR correctly - we don't need this any more because the ServiceCA ConfigMap is created first, and then the certificate data is copied into the registry's "global" CA keyed on the internal registry hostname.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, but we don't put image-registry.openshift-image-registry.svc.cluster.local:5000 to this configmap.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

per @bparees - let's restore this for now. We can revisit in 4.2 when we hopefully are able to drop the port number.

@adambkaplan
Copy link
Contributor

@dmage I assume we already have logic that forces a rollout of the nodeca dameon when the ConfigMap containing the CAs is updated?

@dmage
Copy link
Contributor Author

dmage commented May 3, 2019

@adambkaplan afaik kubelet will update pods filesystem and we do not need to restart it.

@bparees
Copy link
Contributor

bparees commented May 3, 2019

@dmage is correct. the node-ca run-script just re-reads the filesystem every minute or so. And kube will update the filesystem when the configmap changes.

@adambkaplan
Copy link
Contributor

@dmage so we're only worried about the registry pods, which per your previous comment is already taken care of.

Latest update adds the local service name to the mounted ConfigMap.

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label May 3, 2019
@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adambkaplan, dmage

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@dmage dmage changed the title Use image-registry-certificates for system and user certificates Bug 1703941: use image-registry-certificates for system and user certificates May 3, 2019
@dmage
Copy link
Contributor Author

dmage commented May 3, 2019

/retest

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-ci-robot openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label May 3, 2019
@openshift-ci-robot
Copy link
Contributor

New changes are detected. LGTM label has been removed.

@dmage
Copy link
Contributor Author

dmage commented May 3, 2019

/retest

@dmage dmage added the lgtm Indicates that a PR is ready to be merged. label May 3, 2019
@openshift-merge-robot openshift-merge-robot merged commit eedb26c into openshift:master May 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants