Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

prefer podman authentication file locations #37726

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
#!/bin/bash

export HOME=/tmp/home
mkdir -p "$HOME/.docker"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}/containers"
cd "$HOME" || exit 1

# If this is a periodic type job then we need to populate repo metadata from the JOB_SPEC
Expand Down Expand Up @@ -64,16 +66,16 @@ echo "INFO Image tag is $IMAGE_TAG"
# Setup registry credentials
REGISTRY_TOKEN_FILE="$SECRETS_PATH/$REGISTRY_SECRET/$REGISTRY_SECRET_FILE"

config_file="$HOME/.docker/config.json"
config_file="${XDG_RUNTIME_DIR}/containers/auth.json"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assisted-baremetal-images-publish, assisted-baremetal-operator-publish, opendatahub-io-ci-image-mirror and red-hat-data-services-ci-image-mirror seem to be used only up to 4.10 so these should be okay to change

cat "$REGISTRY_TOKEN_FILE" > "$config_file" || {
echo "ERROR Could not read registry secret file"
echo " From: $REGISTRY_TOKEN_FILE"
echo " To : $config_file"
}

if [[ ! -r "$REGISTRY_TOKEN_FILE" ]]; then
echo "ERROR Registry config file not found: $REGISTRY_TOKEN_FILE"
echo " Is the docker/config.json in a different location?"
echo "ERROR Registry authentication file not found: $REGISTRY_TOKEN_FILE"
echo " Is the auth.json in a different location?"
exit 1
fi

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,16 +35,19 @@ GITHUB_TOKEN=$(cat "$GITHUB_TOKEN_FILE")
# Setup registry credentials
REGISTRY_TOKEN_FILE="$SECRETS_PATH/$REGISTRY_SECRET/$REGISTRY_SECRET_FILE"
echo "## Setting up registry credentials."
mkdir -p "$HOME/.docker"
config_file="$HOME/.docker/config.json"
export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}/containers"
config_file="${XDG_RUNTIME_DIR}/containers/auth.json"
cat "$REGISTRY_TOKEN_FILE" > "$config_file" || {
echo "ERROR Could not read registry secret file"
echo " From: $REGISTRY_TOKEN_FILE"
echo " To : $config_file"
}
if [[ ! -r "$REGISTRY_TOKEN_FILE" ]]; then
echo "ERROR Registry config file not found: $REGISTRY_TOKEN_FILE"
echo " Is the docker/config.json in a different location?"
echo "ERROR Registry authentication file not found: $REGISTRY_TOKEN_FILE"
echo " Is the auth.json in a different location?"
exit 1
fi
oc registry login
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this PR changes only places that depend on ~/.docker/config.json, eg. plain oc registry login. Other invocations of oc registry login that write to a different file were not changed in this PR as they are not required to be updated to comply with openshift/oc#1376.

There are still many mention of docker/config.json and we can update these in a followup as these will be mostly cosmetic changes.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ fi
MIRROR_REGISTRY_HOST=`head -n 1 "${SHARED_DIR}/mirror_registry_url"`
echo "MIRROR_REGISTRY_HOST: $MIRROR_REGISTRY_HOST"

export HOME="${HOME:-/tmp/home}"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if HOME variable is always exposed in all of these so I am adding it to make sure we always have it.

export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

oc registry login

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@ export AWS_SHARED_CREDENTIALS_FILE="${CLUSTER_PROFILE_DIR}/.awscred"

REGION="${LEASED_RESOURCE}"

export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

function run_command() {
local cmd="$1"
echo "Running Command: ${cmd}"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@ MPREFIX="${SHARED_DIR}/manifest"
cluster_id="${NAMESPACE}-${JOB_NAME_HASH}"
export ALIBABA_CLOUD_CREDENTIALS_FILE="${SHARED_DIR}/alibabacreds.ini"

export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

# extract ccoctl from the release image
oc registry login
# extract alibabacloud credentials requests from the release image
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,11 @@ aws --region $REGION ec2 describe-images --owners aws-marketplace \
# Select proper version.

# Get readable version from image, e.g. 4.8.49, 4.12.0-0.nightly-2022-09-05-090751
export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

oc registry login
version=$(oc adm release info ${RELEASE_IMAGE_LATEST} -ojson | jq -r '.metadata.version')
image_name_prefix="rhcos-`echo ${version} | awk -F '.' '{print $1$2}'`" # e.g. rhcos-48, rhcos-412
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@ export AWS_SHARED_CREDENTIALS_FILE="${CLUSTER_PROFILE_DIR}/.awscred"
REGION="${LEASED_RESOURCE}"

# extract aws credentials requests from the release image
export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

oc registry login
oc adm release extract --credentials-requests --cloud=aws --to="/tmp/credrequests" "$RELEASE_IMAGE_LATEST"

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ set -o nounset
set -o errexit
set -o pipefail

export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

function getVersion() {
local release_image=""
if [ -n "${RELEASE_IMAGE_INITIAL-}" ]; then
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ fi

# Set PATH to include YQ, installed via pip in the image
export PATH="$PATH:/usr/local/bin"
export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

CLUSTER_NAME=${NAMESPACE}-${JOB_NAME_HASH}
RESOURCE_GROUP=${CLUSTER_NAME}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ set -o nounset
set -o errexit
set -o pipefail

export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

MPREFIX="${SHARED_DIR}/manifest"
TPREFIX="${SHARED_DIR}/tls"
infra_name=${NAMESPACE}-${JOB_NAME_HASH}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ set -o nounset
set -o errexit
set -o pipefail

export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

CONFIG="${SHARED_DIR}/install-config.yaml"
function getVersion() {
local release_image=""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ set -o pipefail

source "${SHARED_DIR}/nutanix_context.sh"

export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

CR_DIR="/tmp/credentials_request"
mkdir -p "${CR_DIR}"

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ set -o pipefail

trap 'CHILDREN=$(jobs -p); if test -n "${CHILDREN}"; then kill ${CHILDREN} && wait; fi' TERM

export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

mirror_output="${SHARED_DIR}/mirror_output"
pull_secret_filename="new_pull_secret"
new_pull_secret="${SHARED_DIR}/${pull_secret_filename}"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ set -o pipefail

trap 'CHILDREN=$(jobs -p); if test -n "${CHILDREN}"; then kill ${CHILDREN} && wait; fi' TERM

export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

mirror_output="${SHARED_DIR}/mirror_output"
new_pull_secret="${SHARED_DIR}/new_pull_secret"
install_config_icsp_patch="${SHARED_DIR}/install-config-icsp.yaml.patch"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
#!/bin/bash

export HOME=/tmp/home
mkdir -p "$HOME/.docker"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}/containers"
cd "$HOME" || exit 1

# log function
Expand Down Expand Up @@ -100,7 +102,7 @@ if [[ ! -r "$REGISTRY_TOKEN_FILE" ]]; then
exit 1
fi

config_file="$HOME/.docker/config.json"
config_file="${XDG_RUNTIME_DIR}/containers/auth.json"
cp $REGISTRY_TOKEN_FILE $config_file || {
log "ERROR Could not create registry secret file"
log " From: $REGISTRY_TOKEN_FILE"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ set -o nounset
set -o errexit
set -o pipefail

export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

fail() {
echo "CredentialsRequest manifests in ${OPENSHIFT_LATEST_RELEASE_IMAGE} diverge from ${OPENSHIFT_FROZEN_RELEASE_IMAGE}. This can cause trouble for Manual credentialsMode clusters ( https://docs.openshift.com/container-platform/4.9/installing/installing_aws/manually-creating-iam.html , and similarly for other clouds) perfoming patch updates (4.y.z -> 4.y.z'), because current Manual-mode guards only apply to minor updates (4.y.z -> 4.(y+1).z'). Find the team who introduced the change, and discuss whether the change is required (and acceptably documented in release notes for folks running Manual-mode clusters), in which case bump the oldest-supported-credentials-request config for the job to freeze on the new state ( grep for oldest-supported-credentials-request to find config locations like https://github.com/openshift/release/blob/0c7ecf26dfc04f9c64632e34d7a5ebcf42d69b99/ci-operator/config/openshift/release/openshift-release-master__nightly-4.10.yaml#L74-L77 ). Until https://issues.redhat.com/browse/DPTP-2731 is implemented, you will need to wait for a new candidate payload that includes your changes to be available before bumping this value. If you decide the change is not required, have the relevant team revert their change."
return 1
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ set -o pipefail

trap 'CHILDREN=$(jobs -p); if test -n "${CHILDREN}"; then kill ${CHILDREN} && wait; fi' TERM

export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

mirror_output="${SHARED_DIR}/mirror_output"
new_pull_secret="${SHARED_DIR}/new_pull_secret"
install_config_icsp_patch="${SHARED_DIR}/install-config-icsp.yaml.patch"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ set -o pipefail
oc version
openshift-install version

export HOME="${HOME:-/tmp/home}"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}"

mirror_output="${SHARED_DIR}/mirror_output"
new_pull_secret="${SHARED_DIR}/new_pull_secret"
install_config_icsp_patch="${SHARED_DIR}/install-config-icsp.yaml.patch"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
#!/bin/bash

export HOME=/tmp/home
mkdir -p "$HOME/.docker"
export XDG_RUNTIME_DIR="${HOME}/run"
export REGISTRY_AUTH_PREFERENCE=podman # TODO: remove later, used for migrating oc from docker to podman
mkdir -p "${XDG_RUNTIME_DIR}/containers"
cd "$HOME" || exit 1

# log function
Expand Down Expand Up @@ -100,7 +102,7 @@ if [[ ! -r "$REGISTRY_TOKEN_FILE" ]]; then
exit 1
fi

config_file="$HOME/.docker/config.json"
config_file="${XDG_RUNTIME_DIR}/containers/auth.json"
cp $REGISTRY_TOKEN_FILE $config_file || {
log "ERROR Could not create registry secret file"
log " From: $REGISTRY_TOKEN_FILE"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,8 @@ objects:
value: /tmp/artifacts
- name: HOME
value: /tmp/home
- name: XDG_RUNTIME_DIR
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these changes should be identical to changes here openshift/ci-tools#3345

value: /tmp/home/run
- name: IMAGE_FORMAT
value: ${IMAGE_FORMAT}
- name: KUBECONFIG
Expand Down Expand Up @@ -228,7 +230,7 @@ objects:
done
}

mkdir -p "${HOME}"
mkdir -p "${HOME}" "${XDG_RUNTIME_DIR}"

# wait for the API to come up
while true; do
Expand Down Expand Up @@ -390,6 +392,8 @@ objects:
value: test
- name: HOME
value: /tmp
- name: XDG_RUNTIME_DIR
value: /tmp/run
- name: INSTALL_INITIAL_RELEASE
- name: RELEASE_IMAGE_INITIAL
command:
Expand Down Expand Up @@ -422,14 +426,15 @@ objects:

cp "$(command -v openshift-install)" /tmp/shared
mkdir ${ARTIFACT_DIR}/installer
mkdir -p "${XDG_RUNTIME_DIR}"

if [[ -n "${INSTALL_INITIAL_RELEASE}" && -n "${RELEASE_IMAGE_INITIAL}" ]]; then
echo "Installing from initial release ${RELEASE_IMAGE_INITIAL}"
OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE="${RELEASE_IMAGE_INITIAL}"
elif [[ "${CLUSTER_VARIANT}" =~ "mirror" ]]; then
# mirror the release image and override the release image to point to the mirrored one
mkdir /tmp/.docker && cp /etc/openshift-installer/pull-secret /tmp/.docker/config.json
oc registry login
oc registry login --to /tmp/.docker/config.json
MIRROR_BASE=$( oc get is release -o 'jsonpath={.status.publicDockerImageRepository}' )
oc adm release new --from-release ${RELEASE_IMAGE_LATEST} --to-image ${MIRROR_BASE}-scratch:release --mirror ${MIRROR_BASE}-scratch || echo 'ignore: the release could not be reproduced from its inputs'
oc adm release mirror --from ${MIRROR_BASE}-scratch:release --to ${MIRROR_BASE} --to-release-image ${MIRROR_BASE}:mirrored
Expand Down Expand Up @@ -741,6 +746,8 @@ objects:
value: /etc/openshift-installer/osServicePrincipal.json
- name: GOOGLE_CLOUD_KEYFILE_JSON
value: /etc/openshift-installer/gce.json
- name: XDG_RUNTIME_DIR
value: /tmp/run
- name: KUBECONFIG
value: /tmp/artifacts/installer/auth/kubeconfig
command:
Expand Down Expand Up @@ -772,7 +779,7 @@ objects:
export PATH=$PATH:/tmp/shared

echo "Gathering artifacts ..."
mkdir -p ${ARTIFACT_DIR}/pods ${ARTIFACT_DIR}/nodes ${ARTIFACT_DIR}/metrics ${ARTIFACT_DIR}/bootstrap ${ARTIFACT_DIR}/network
mkdir -p ${ARTIFACT_DIR}/pods ${ARTIFACT_DIR}/nodes ${ARTIFACT_DIR}/metrics ${ARTIFACT_DIR}/bootstrap ${ARTIFACT_DIR}/network "${XDG_RUNTIME_DIR}"

oc --insecure-skip-tls-verify --request-timeout=5s get nodes -o jsonpath --template '{range .items[*]}{.metadata.name}{"\n"}{end}' > /tmp/nodes
oc --insecure-skip-tls-verify --request-timeout=5s get pods --all-namespaces --template '{{ range .items }}{{ $name := .metadata.name }}{{ $ns := .metadata.namespace }}{{ range .spec.containers }}-n {{ $ns }} {{ $name }} -c {{ .name }}{{ "\n" }}{{ end }}{{ range .spec.initContainers }}-n {{ $ns }} {{ $name }} -c {{ .name }}{{ "\n" }}{{ end }}{{ end }}' > /tmp/containers
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,8 @@ objects:
value: /tmp/artifacts
- name: HOME
value: /tmp/home
- name: XDG_RUNTIME_DIR
value: /tmp/home/run
- name: IMAGE_FORMAT
value: ${IMAGE_FORMAT}
- name: KUBECONFIG
Expand Down Expand Up @@ -241,7 +243,7 @@ objects:
done
}

mkdir -p "${HOME}"
mkdir -p "${HOME}" "${XDG_RUNTIME_DIR}"

# Share oc with other containers
cp "$(command -v oc)" /tmp/shared
Expand Down Expand Up @@ -399,6 +401,8 @@ objects:
value: test
- name: HOME
value: /tmp
- name: XDG_RUNTIME_DIR
value: /tmp/run
- name: INSTALL_INITIAL_RELEASE
- name: RELEASE_IMAGE_INITIAL
command:
Expand All @@ -412,6 +416,7 @@ objects:
trap 'CHILDREN=$(jobs -p); if test -n "${CHILDREN}"; then kill ${CHILDREN} && wait; fi' TERM
cp "$(command -v openshift-install)" /tmp
mkdir ${ARTIFACT_DIR}/installer
mkdir -p "${XDG_RUNTIME_DIR}"

function has_variant() {
regex="(^|,)$1($|,)"
Expand All @@ -430,7 +435,7 @@ objects:

# mirror the release image and override the release image to point to the mirrored one
mkdir /tmp/.docker && cp /etc/openshift-installer/pull-secret /tmp/.docker/config.json
oc registry login
oc registry login --to /tmp/.docker/config.json
MIRROR_BASE=$( oc get is release -o 'jsonpath={.status.publicDockerImageRepository}' )
oc adm release new --from-release ${RELEASE_IMAGE_LATEST} --to-image ${MIRROR_BASE}-scratch:release --mirror ${MIRROR_BASE}-scratch || echo 'ignore: the release could not be reproduced from its inputs'
oc adm release mirror --from ${MIRROR_BASE}-scratch:release --to ${MIRROR_BASE} --to-release-image ${MIRROR_BASE}:mirrored
Expand Down Expand Up @@ -1012,6 +1017,8 @@ objects:
value: test
- name: HOME
value: /tmp
- name: XDG_RUNTIME_DIR
value: /tmp/run
- name: LC_ALL
value: en_US.UTF-8
command:
Expand Down Expand Up @@ -1043,7 +1050,7 @@ objects:
export PATH=$PATH:/tmp/shared

echo "Gathering artifacts ..."
mkdir -p ${ARTIFACT_DIR}/pods ${ARTIFACT_DIR}/nodes ${ARTIFACT_DIR}/metrics ${ARTIFACT_DIR}/bootstrap ${ARTIFACT_DIR}/network
mkdir -p ${ARTIFACT_DIR}/pods ${ARTIFACT_DIR}/nodes ${ARTIFACT_DIR}/metrics ${ARTIFACT_DIR}/bootstrap ${ARTIFACT_DIR}/network "${XDG_RUNTIME_DIR}"

oc --insecure-skip-tls-verify --request-timeout=5s get nodes -o jsonpath --template '{range .items[*]}{.metadata.name}{"\n"}{end}' > /tmp/nodes
oc --insecure-skip-tls-verify --request-timeout=5s get nodes -o jsonpath --template '{range .items[*]}{.spec.providerID}{"\n"}{end}' | sed 's|.*/||' > /tmp/node-provider-IDs
Expand Down