-
Notifications
You must be signed in to change notification settings - Fork 1.8k
GH#36350: Create conneted to disconnected cluster document #36351
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
mburke5678
merged 1 commit into
openshift:main
from
mburke5678:connected-to-disconnected
Nov 3, 2021
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,190 @@ | ||
// Module included in the following assemblies: | ||
// | ||
// * post_installation_configuration/connected-to-disconnected.adoc | ||
|
||
[id="connected-to-disconnected-config-registry_{context}"] | ||
= Configuring the cluster for the mirror registry | ||
|
||
After creating and mirroring the images to the mirror registry, you must modify your cluster so that pods can pull images from the mirror registry. | ||
|
||
You must: | ||
|
||
* Add the mirror registry credentials to the global pull secret. | ||
* Add the mirror registry server certificate to the cluster. | ||
* Create an `ImageContentSourcePolicy` custom resource (ICSP), which associates the mirror registry with the source registry. | ||
|
||
|
||
|
||
. Add mirror registry credential to the cluster global pull-secret: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc set data secret/pull-secret -n openshift-config --from-file=.dockerconfigjson=<pull_secret_location> <1> | ||
---- | ||
<1> Provide the path to the new pull secret file. | ||
+ | ||
For example: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc set data secret/pull-secret -n openshift-config --from-file=.dockerconfigjson=.mirrorsecretconfigjson | ||
---- | ||
|
||
. Add the CA-signed mirror registry server certificate to the nodes in the cluster: | ||
|
||
.. Create a config map that includes the server certificate for the mirror registry | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc create configmap <config_map_name> --from-file=<mirror_address_host>..<port>=$path/ca.crt -n openshift-config | ||
---- | ||
+ | ||
For example: | ||
+ | ||
[source,terminal] | ||
---- | ||
S oc create configmap registry-config --from-file=mirror.registry.com..443=/root/certs/ca-chain.cert.pem -n openshift-config | ||
---- | ||
|
||
.. Use the config map to update the `image.config.openshift.io/cluster` custom resource (CR). {product-title} applies the changes to this CR to all nodes in the cluster: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc patch image.config.openshift.io/cluster --patch '{"spec":{"additionalTrustedCA":{"name":"<config_map_name>"}}}' --type=merge | ||
---- | ||
+ | ||
For example: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc patch image.config.openshift.io/cluster --patch '{"spec":{"additionalTrustedCA":{"name":"registry-config"}}}' --type=merge | ||
---- | ||
|
||
. Create an ICSP to redirect container pull requests from the online registries to the mirror registry: | ||
|
||
.. Create the `ImageContentSourcePolicy` custom resource: | ||
+ | ||
[source,yaml] | ||
---- | ||
apiVersion: operator.openshift.io/v1alpha1 | ||
kind: ImageContentSourcePolicy | ||
metadata: | ||
name: mirror-ocp | ||
spec: | ||
repositoryDigestMirrors: | ||
- mirrors: | ||
- mirror.registry.com:443/ocp/release <1> | ||
source: quay.io/openshift-release-dev/ocp-release <2> | ||
- mirrors: | ||
- mirror.registry.com:443/ocp/release | ||
source: quay.io/openshift-release-dev/ocp-v4.0-art-dev | ||
---- | ||
<1> Specifies the name of the mirror image registry and repository. | ||
<2> Specifies the online registry and repository containing the content that is mirrored. | ||
|
||
.. Create the ICSP object: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc create -f registryrepomirror.yaml | ||
---- | ||
+ | ||
.Example output | ||
[source,terminal] | ||
---- | ||
imagecontentsourcepolicy.operator.openshift.io/mirror-ocp created | ||
---- | ||
+ | ||
{product-title} applies the changes to this CR to all nodes in the cluster. | ||
|
||
. Verify that the credentials, CA, and ICSP for mirror registry were added: | ||
|
||
.. Log into a node: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc debug node/<node_name> | ||
---- | ||
|
||
.. Set `/host` as the root directory within the debug shell: | ||
+ | ||
[source,terminal] | ||
---- | ||
sh-4.4# chroot /host | ||
---- | ||
|
||
.. Check the `config.json` file for the credentials: | ||
+ | ||
[source,terminal] | ||
---- | ||
sh-4.4# cat /var/lib/kubelet/config.json | ||
---- | ||
+ | ||
.Example output | ||
[source,terminal] | ||
---- | ||
{"auths":{"brew.registry.redhat.io":{"xx=="},"brewregistry.stage.redhat.io":{"auth":"xxx=="},"mirror.registry.com:443":{"auth":"xx="}}} <1> | ||
---- | ||
<1> Ensure that the mirror registry and credentials are present. | ||
|
||
.. Change to the `certs.d` directory | ||
+ | ||
[source,terminal] | ||
---- | ||
sh-4.4# cd /etc/docker/certs.d/ | ||
---- | ||
|
||
.. List the certificates in the `certs.d` directory: | ||
+ | ||
[source,terminal] | ||
---- | ||
sh-4.4# ls | ||
---- | ||
+ | ||
.Example output | ||
---- | ||
image-registry.openshift-image-registry.svc.cluster.local:5000 | ||
image-registry.openshift-image-registry.svc:5000 | ||
mirror.registry.com:443 <1> | ||
---- | ||
<1> Ensure that the mirror registry is in the list. | ||
|
||
.. Check that the ICSP added the mirror registry to the `registries.conf` file: | ||
+ | ||
[source,terminal] | ||
---- | ||
sh-4.4# cat /etc/containers/registries.conf | ||
---- | ||
+ | ||
.Example output | ||
+ | ||
[source,terminal] | ||
---- | ||
unqualified-search-registries = ["registry.access.redhat.com", "docker.io"] | ||
|
||
[[registry]] | ||
prefix = "" | ||
location = "quay.io/openshift-release-dev/ocp-release" | ||
mirror-by-digest-only = true | ||
|
||
[[registry.mirror]] | ||
location = "mirror.registry.com:443/ocp/release" | ||
|
||
[[registry]] | ||
prefix = "" | ||
location = "quay.io/openshift-release-dev/ocp-v4.0-art-dev" | ||
mirror-by-digest-only = true | ||
|
||
[[registry.mirror]] | ||
location = "mirror.registry.com:443/ocp/release" | ||
---- | ||
+ | ||
The `registry.mirror` parameters indicate that the mirror registry is searched before the original registry. | ||
|
||
.. Exit the node. | ||
+ | ||
[source,terminal] | ||
---- | ||
sh-4.4# exit | ||
---- | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
// Module included in the following assemblies: | ||
// | ||
// * post_installation_configuration/connected-to-disconnected.adoc | ||
|
||
[id="connected-to-disconnected-disconnect_{context}"] | ||
= Disconnect the cluster from the network | ||
|
||
After mirroring all the required repositories and configuring your cluster to work as a disconnected cluster, you can disconnect the cluster from the network. | ||
|
||
|
||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,110 @@ | ||
// Module included in the following assemblies: | ||
// | ||
// * post_installation_configuration/connected-to-disconnected.adoc | ||
|
||
[id="connected-to-disconnected-mirror-images_{context}"] | ||
= Mirroring the images | ||
|
||
After the cluster is properly configured, you can mirror the images from your external repositories to the mirror repository. | ||
|
||
.Procedure | ||
|
||
. Mirror the Operator Lifecycle Manager (OLM) images: | ||
// copied from olm-mirroring-catalog.adoc | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc adm catalog mirror registry.redhat.io/redhat/redhat-operator-index:v{product-version} <mirror_registry>:<port>/olm -a <reg_creds> | ||
---- | ||
+ | ||
-- | ||
where: | ||
|
||
`product-version`:: Specifies the tag that corresponds to the version of {product-title} to install, such as `4.8`. | ||
`mirror_registry`:: Specifies the target registry and namespace to mirror the Operator content to, where `<namespace>` is any existing namespace on the registry. | ||
`reg_creds`:: Specifies the location of your modified `.dockerconfigjson` file. | ||
-- | ||
+ | ||
For example: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc adm catalog mirror registry.redhat.io/redhat/redhat-operator-index:v4.8 mirror.registry.com:443/olm -a ./.dockerconfigjson --index-filter-by-os='.*' | ||
---- | ||
|
||
. Mirror the content for any other Red Hat-provided Operator: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc adm catalog mirror <index_image> <mirror_registry>:<port>/<namespace> -a <reg_creds> | ||
---- | ||
+ | ||
-- | ||
where: | ||
|
||
`index_image`:: Specifies the index image for the catalog you want to mirror. For example, this might be a pruned index image that you created previously, or one of the source index images for the default catalogs, such as `{index-image-pullspec}`. | ||
`mirror_registry`:: Specifies the target registry and namespace to mirror the Operator content to, where `<namespace>` is any existing namespace on the registry. | ||
`reg_creds`:: Optional: Specifies the location of your registry credentials file, if required. | ||
-- | ||
+ | ||
For example: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc adm catalog mirror registry.redhat.io/redhat/community-operator-index:v4.8 mirror.registry.com:443/olm -a ./.dockerconfigjson --index-filter-by-os='.*' | ||
---- | ||
|
||
. Mirror the {product-title} image repository: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc adm release mirror -a .dockerconfigjson --from=quay.io/openshift-release-dev/ocp-release:v<product-version>-<architecture> --to=<local_registry>/<local_repository> --to-release-image=<local_registry>/<local_repository>:v<product-version>-<architecture> | ||
---- | ||
+ | ||
-- | ||
where: | ||
|
||
`product-version`:: Specifies the tag that corresponds to the version of {product-title} to install, such as `4.8.15-x86_64`. | ||
`architecture`:: Specifies the type of architecture for your server, such as `x86_64`. | ||
`local_registry`:: Specifies the registry domain name for your mirror repository. | ||
`local_repository`:: Specifies the name of the repository to create in your registry, such as `ocp4/openshift4`. | ||
-- | ||
+ | ||
For example: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc adm release mirror -a .dockerconfigjson --from=quay.io/openshift-release-dev/ocp-release:4.8.15-x86_64 --to=mirror.registry.com:443/ocp/release --to-release-image=mirror.registry.com:443/ocp/release:4.8.15-x86_64 | ||
---- | ||
+ | ||
.Example output | ||
+ | ||
[source,terminal] | ||
+ | ||
---- | ||
info: Mirroring 109 images to mirror.registry.com/ocp/release ... | ||
mirror.registry.com:443/ | ||
ocp/release | ||
manifests: | ||
sha256:086224cadce475029065a0efc5244923f43fb9bb3bb47637e0aaf1f32b9cad47 -> 4.8.15-x86_64-thanos | ||
sha256:0a214f12737cb1cfbec473cc301aa2c289d4837224c9603e99d1e90fc00328db -> 4.8.15-x86_64-kuryr-controller | ||
sha256:0cf5fd36ac4b95f9de506623b902118a90ff17a07b663aad5d57c425ca44038c -> 4.8.15-x86_64-pod | ||
sha256:0d1c356c26d6e5945a488ab2b050b75a8b838fc948a75c0fa13a9084974680cb -> 4.8.15-x86_64-kube-client-agent | ||
|
||
….. | ||
sha256:66e37d2532607e6c91eedf23b9600b4db904ce68e92b43c43d5b417ca6c8e63c mirror.registry.com:443/ocp/release:4.5.41-multus-admission-controller | ||
sha256:d36efdbf8d5b2cbc4dcdbd64297107d88a31ef6b0ec4a39695915c10db4973f1 mirror.registry.com:443/ocp/release:4.5.41-cluster-kube-scheduler-operator | ||
sha256:bd1baa5c8239b23ecdf76819ddb63cd1cd6091119fecdbf1a0db1fb3760321a2 mirror.registry.com:443/ocp/release:4.5.41-aws-machine-controllers | ||
info: Mirroring completed in 2.02s (0B/s) | ||
|
||
Success | ||
Update image: mirror.registry.com:443/ocp/release:4.5.41-x86_64 | ||
Mirror prefix: mirror.registry.com:443/ocp/release | ||
---- | ||
|
||
. Mirror any other registries, as needed: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc image mirror <online_registry>/my/image:latest <mirror_registry> | ||
---- |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
// Module included in the following assemblies: | ||
// | ||
// * post_installation_configuration/connected-to-disconnected.adoc | ||
|
||
[id="connected-to-disconnected-prepare-mirror_{context}"] | ||
= Preparing the cluster for mirroring | ||
|
||
Before disconnecting your cluster, you must mirror, or copy, the images to a mirror registry that is reachable by every node in your disconnected cluster. In order to mirror the images, you must prepare your cluster by: | ||
|
||
* Adding the mirror registry certificates to the list of trusted CAs on your host. | ||
* Creating a `.dockerconfigjson` file that contains your image pull secret, which is from the `cloud.openshift.com` token. | ||
|
||
.Procedure | ||
|
||
. Configuring credentials that allow image mirroring: | ||
|
||
.. Add the CA certificate for the mirror registry, in the simple PEM or DER file formats, to the list of trusted CAs. For example: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ cp </path/to/cert.crt> /usr/share/pki/ca-trust-source/anchors/ | ||
---- | ||
+ | ||
-- | ||
where:: | ||
+ | ||
`</path/to/cert.crt>`:: Specifies the path to the certificate on your local file system. | ||
-- | ||
|
||
.. Update the CA trust. For example, in Linux: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ update-ca-trust | ||
---- | ||
|
||
.. Extract the `.dockerconfigjson` file from the global pull secret: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc extract secret/pull-secret -n openshift-config --confirm --to=. | ||
---- | ||
+ | ||
.Example output | ||
[source,terminal] | ||
---- | ||
.dockerconfigjson | ||
---- | ||
|
||
.. Edit the `.dockerconfigjson` file to add your mirror registry and authentication credentials and save it as a new file: | ||
// copied from olm-accessing-images-private-registries | ||
+ | ||
[source,terminal] | ||
---- | ||
{"auths":{"<local_registry>": {"auth": "<credentials>","email": "you@example.com"}}},"<registry>:<port>/<namespace>/":{"auth":"<token>"}}} | ||
---- | ||
+ | ||
where: | ||
+ | ||
`<local_registry>`:: Specifies the registry domain name, and optionally the port, that your mirror registry uses to serve content. | ||
`auth`:: Specifies the base64-encoded user name and password for your mirror registry. | ||
`<registry>:<port>/<namespace>`:: Specifies the mirror registry details. | ||
`<token>`:: Specifies the base64-encoded `username:password` for your mirror registry. | ||
+ | ||
For example: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ {"auths":{"cloud.openshift.com":{"auth":"b3BlbnNoaWZ0Y3UjhGOVZPT0lOMEFaUjdPUzRGTA==","email":"user@example.com"}, | ||
"quay.io":{"auth":"b3BlbnNoaWZ0LXJlbGVhc2UtZGOVZPT0lOMEFaUGSTd4VGVGVUjdPUzRGTA==","email":"user@example.com"}, | ||
"registry.connect.redhat.com"{"auth":"NTE3MTMwNDB8dWhjLTFEZlN3VHkxOSTd4VGVGVU1MdTpleUpoYkdjaUailA==","email":"user@example.com"}, | ||
"registry.redhat.io":{"auth":"NTE3MTMwNDB8dWhjLTFEZlN3VH3BGSTd4VGVGVU1MdTpleUpoYkdjaU9fZw==","email":"user@example.com"}, | ||
"registry.svc.ci.openshift.org":{"auth":"dXNlcjpyWjAwWVFjSEJiT2RKVW1pSmg4dW92dGp1SXRxQ3RGN1pwajJhN1ZXeTRV"},"my-registry:5000/my-namespace/":{"auth":"dXNlcm5hbWU6cGFzc3dvcmQ="}}} | ||
---- |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.