Short-lived Credentials with AWS Security Token Service using AWS CloudFront and private S3 bucket

NOTE: The steps described in this document, create a private S3 Bucket using CloudFront Distribution, was introduced to ccoctl using the flag --create-private-s3-bucket. We are keeping this document to provide an overview of the steps used in this solution.

To create an IAM OpenID Connect identity provider you should expose the OIDC config using a public HTTPS endpoint. The steps described here will guide you to create one CloudFront Distribution to expose the HTTPS endpoint, serving objects from a private S3 Bucket accessed by Origin Access Identity.

Summary:

Create
Delete
- Delete the CloudFront Distribution
- Delete the Origin Access Identity

Requirements

oc
ccoctl
aws-cli
You followed the Steps 1 to in the section "Steps to install an OpenShift Cluster with STS"

Steps to Create

Set the environment variables

considering that the installer directory is the same you are running this command

export CLUSTER_NAME=$(awk '/infrastructureName: / {print $2}' manifests/cluster-infrastructure-02-config.yml)
export CLUSTER_REGION=$(awk '/region: / {print $2}' manifests/cluster-infrastructure-02-config.yml)

export DIR_CCO="${PWD}/_output"
export OIDC_BUCKET_NAME="${CLUSTER_NAME}-oidc"
export OIDC_BUCKET_CONTENT="${DIR_CCO}/bucket-content"

mkdir -p ${OIDC_BUCKET_CONTENT}

Create the Origin Access Identity

Steps to create the Origin Access Identity (OAI) to be used to access the bucket through CloudFront Distribution:

Create the OAI and set the variable OAI_CLOUDFRONT_ID:

aws cloudfront create-cloud-front-origin-access-identity \
    --cloud-front-origin-access-identity-config \
    CallerReference="${OIDC_BUCKET_NAME}",Comment="OAI-${OIDC_BUCKET_NAME}"

export OAI_CLOUDFRONT_ID=$(aws cloudfront \
    list-cloud-front-origin-access-identities \
    --query "CloudFrontOriginAccessIdentityList.Items[?Comment==\`OAI-${OIDC_BUCKET_NAME}\`].Id" \
    --output text)

Create the Bucket

Create the private Bucket

aws s3api create-bucket \
    --bucket ${OIDC_BUCKET_NAME} \
    --region ${CLUSTER_REGION} \
    --create-bucket-configuration LocationConstraint=${CLUSTER_REGION} \
    --acl private

Create the respective tags on the Bucket (Recommended if you would like to use the ccoctl to delete resources)

aws s3api put-bucket-tagging \
    --bucket ${OIDC_BUCKET_NAME} \
    --tagging "TagSet=[{Key=Name,Value=${OIDC_BUCKET_NAME}},{Key=openshift.io/cloud-credential-operator/${CLUSTER_NAME},Value=owned}]"

Download the s3 bucket policy template that restricts access to CloudFront Origin Access Identity (OAI)

wget https://raw.githubusercontent.com/openshift/cloud-credential-operator/master/docs/sts-oidc-bucket-policy.json.tpl

Create the Bucket Policy configuration (cli-input-json) allowing OAI to retrieve objects

cat sts-oidc-bucket-policy.json.tpl \
   | envsubst \
   > ${DIR_CCO}/oidc-bucket-policy.json

Apply the policy to the Bucket to block public access

aws s3api put-bucket-policy \
    --bucket ${OIDC_BUCKET_NAME} \
    --policy file://${DIR_CCO}/oidc-bucket-policy.json

aws s3api put-public-access-block \
    --bucket ${OIDC_BUCKET_NAME} \
    --public-access-block-configuration \
    BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

Create CloudFront Distribution

Download the CloudFront Distribution template

wget https://raw.githubusercontent.com/openshift/cloud-credential-operator/master/docs/sts-oidc-cloudfront.json.tpl

Create a CloudFront Distribution configuration (cli-input-json)

cat sts-oidc-cloudfront.json.tpl \
   | envsubst \
   > ${DIR_CCO}/oidc-cloudfront.json

Create the CloudFront Distribution with Tags

aws cloudfront create-distribution-with-tags \
    --distribution-config-with-tags \
    file://${DIR_CCO}/oidc-cloudfront.json

Wait until the Distribution resource has been created
Get the CloudFront's Distribution URL

export CLOUDFRONT_URI=$(aws cloudfront list-distributions \
    --query "DistributionList.Items[?Comment==\`${CLUSTER_NAME}\`].DomainName" \
    --output text)

echo ${CLOUDFRONT_URI}

Generate OIDC configuration and keys

Generate the key pair used to create the service account tokens

./ccoctl aws create-key-pair \
    --output-dir ${DIR_CCO}

Generate the OpenID configuration

./ccoctl aws create-identity-provider \
    --name=${CLUSTER_NAME} \
    --region=${CLUSTER_REGION} \
    --public-key-file=${DIR_CCO}/serviceaccount-signer.public \
    --output-dir=${DIR_CCO}/ \
    --dry-run

Update the CloudFront URI's endpoint to the CloudFront distribution's address:

A. Patch the issuer URL on the OpenID configuration file /.well-known/openid-configuration

mkdir -p ${OIDC_BUCKET_CONTENT}/.well-known
cat ${DIR_CCO}/02-openid-configuration \
    | sed "s/https:\/\/${CLUSTER_NAME}[a-z.-].*\//https:\/\/${CLOUDFRONT_URI}\//" \
    | sed "s/https:\/\/${CLUSTER_NAME}[a-z.-].*/https:\/\/${CLOUDFRONT_URI}\",/" \
    > ${OIDC_BUCKET_CONTENT}/.well-known/openid-configuration

B. Copy keys.json

cp -v ${DIR_CCO}/03-keys.json \
    ${OIDC_BUCKET_CONTENT}/keys.json

C. Patch the issuer url on Authentication CRD in cluster-authentication-02-config.yaml

sed -i "s/https:\/\/[a-z.-].*/https:\/\/${CLOUDFRONT_URI}/" \
    ${DIR_CCO}/manifests/cluster-authentication-02-config.yaml

D. Update the IdP OIDC object configuration

sed -i "s/https:\/\/[a-z.-].*/https:\/\/${CLOUDFRONT_URI}\",/" \
    ${DIR_CCO}/04-iam-identity-provider.json

Check the output of jq . ${DIR_CCO}/04-iam-identity-provider.json

Upload the bucket content

aws s3 sync ${OIDC_BUCKET_CONTENT}/ \
    s3://${OIDC_BUCKET_NAME}

Make sure you can access the content through the public URL

NOTE: CloudFront can take some time to deploy the distribution. Please be sure the distribution has been deployed and it's available before running this step (Status=Enabled and Last Modified!=Deploying). You can access the CloudFront Console to check it.

curl https://${CLOUDFRONT_URI}/keys.json
curl https://${CLOUDFRONT_URI}/.well-known/openid-configuration

Create the OpenID Connector identity provider

Create the IAM OpenID Connect identity provider

aws iam create-open-id-connect-provider \
    --cli-input-json file://${DIR_CCO}/04-iam-identity-provider.json \
    > ${DIR_CCO}/04-iam-identity-provider-object.json

Get the ARN of the IAM OpenID Connect identity provider created above

OIDC_ARN=$(jq -r .OpenIDConnectProviderArn \
    ${DIR_CCO}/04-iam-identity-provider-object.json)

echo ${OIDC_ARN}

Create IAM Roles

Extract CredentialRequests from the release image

./oc adm release extract \
    --credentials-requests \
    --cloud=aws \
    --to=${DIR_CCO}/credrequests \
    ${RELEASE_IMAGE}

Create IAM Roles for the OpenShift components

./ccoctl aws create-iam-roles \
    --name=${CLUSTER_NAME} \
    --region=${CLUSTER_REGION}\
    --credentials-requests-dir=${DIR_CCO}/credrequests \
    --identity-provider-arn=${OIDC_ARN} \
    --output-dir ${DIR_CCO}

We have now created IAM OpenID Connect identity provider and IAM roles, you can return to step 8 to continue with installation in STS mode.

Steps to Delete

These steps should be followed after you've removed the resources created by ccoctl described in the delete section.

Requirements:

You should have set the CloudFront Distribution Comment as ${CLUSTER_NAME}, as described in the section above. Otherwise, you should specify the value you've set when creating the CloudFront Distribution.
You should have set the CloudFront Origin Access Identity (OAI) with the field Comment with the value OAI-${OIDC_BUCKET_NAME}. Otherwise, you should specify the value you've set when creating the CloudFront OAI.

Remove the Bucket

If the ccoctl delete command failed due to non-empty Bucket (BucketNotEmpty), you should follow those steps to complete the Bucket removal.

Remove the Bucket objects

aws s3api delete-object \
    --bucket ${OIDC_BUCKET_NAME} \
    --key ".well-known/openid-configuration"

aws s3api delete-object \
    --bucket ${OIDC_BUCKET_NAME} \
    --key "keys.json"

Remove the Bucket using ccoctl

./ccoctl aws delete \
    --name=${CLUSTER_NAME} \
    --region=${CLUSTER_REGION}

Remove the CloudFront Distribution

CloudFront Distributions can be removed only when it is disabled. To do so, you need to get the current configuration, setting the field Enabled to false, apply the new configuration, then remove the Distribution.

Get the CloudFront Distribution ID

DISTRIBUTION_ID=$(aws cloudfront list-distributions \
    --query "DistributionList.Items[?Comment==\`${CLUSTER_NAME}\`].Id" \
    --output text)

Get the CloudFront Distribution Config ETag

ETAG=$(aws cloudfront get-distribution-config \
    --id ${DISTRIBUTION_ID} \
    | jq -r '.ETag')

Get the CloudFront Distribution Configuration, setting the Enabled field to false

aws cloudfront get-distribution-config --id ${DISTRIBUTION_ID} \
    | jq '.DistributionConfig' \
    | jq '.Enabled=false' \
    > ${DIR_CCO}/oidc-cloudfront-to-delete.json

Apply the new Distribution configuration

ETAG=$(aws cloudfront update-distribution \
    --id ${DISTRIBUTION_ID} \
    --if-match ${ETAG} \
    --distribution-config file://${DIR_CCO}/oidc-cloudfront-to-delete.json \
    | jq -r '.ETag')

Get the new ETag (it's also returned on the last command).

The last command updates the ETAG variable referencing to a new configuration (Disabled distribution). If you didn't note it you need to re-run the get-distribution-config command, as desribed above.

Delete the CloudFront Distribution

aws cloudfront delete-distribution \
    --id ${DISTRIBUTION_ID} \
    --if-match ${ETAG}

Remove the Origin Access Identity (OAI)

Get the OAI ID

OAI_CLOUDFRONT_ID=$(aws cloudfront \
    list-cloud-front-origin-access-identities \
    --query "CloudFrontOriginAccessIdentityList.Items[?Comment==\`OAI-${OIDC_BUCKET_NAME}\`].Id" \
    --output text)

Get the OAI ETag by ID

OAI_ETAG=$(aws cloudfront \
    get-cloud-front-origin-access-identity-config \
    --id ${OAI_CLOUDFRONT_ID} \
    | jq -r .ETag)

Remove the OAI by ID

aws cloudfront \
    delete-cloud-front-origin-access-identity \
    --id ${OAI_CLOUDFRONT_ID} \
    --if-match ${OAI_ETAG}

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sts-private-bucket.md

sts-private-bucket.md

Short-lived Credentials with AWS Security Token Service using AWS CloudFront and private S3 bucket

Requirements

Steps to Create

Set the environment variables

Create the Origin Access Identity

Create the Bucket

Create CloudFront Distribution

Generate OIDC configuration and keys

Create the OpenID Connector identity provider

Create IAM Roles

Steps to Delete

Remove the Bucket

Remove the CloudFront Distribution

Remove the Origin Access Identity (OAI)

Files

sts-private-bucket.md

Latest commit

History

sts-private-bucket.md

File metadata and controls

Short-lived Credentials with AWS Security Token Service using AWS CloudFront and private S3 bucket

Requirements

Steps to Create

Set the environment variables

Create the Origin Access Identity

Create the Bucket

Create CloudFront Distribution

Generate OIDC configuration and keys

Create the OpenID Connector identity provider

Create IAM Roles

Steps to Delete

Remove the Bucket

Remove the CloudFront Distribution

Remove the Origin Access Identity (OAI)