Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the cross-account example with working policy #9019

Merged
merged 3 commits into from
May 1, 2020

Conversation

olemarkus
Copy link
Member

No description provided.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Apr 28, 2020
@hakman
Copy link
Member

hakman commented Apr 28, 2020

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 28, 2020
@hakman
Copy link
Member

hakman commented Apr 28, 2020

/retest

docs/state.md Outdated Show resolved Hide resolved
Co-Authored-By: Peter Rifel <rifelpet@users.noreply.github.com>
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 30, 2020
docs/state.md Outdated Show resolved Hide resolved
]
}
}
]
}
```

Kops will then use that bucket as if it was in the remote account, including creating appropriate IAM policies that limits nodes from doing bad things.
Note that any user/role with full S3 access will be able to delete any cluster from the state store, but may not delete any instances or other things outside of S3.
Copy link
Member

@zetaab zetaab Apr 30, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are using policy like this

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowRootAndHomeListingOfCompanyBucket",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::<state bucket>",
            "Condition": {
                "StringEquals": {
                    "s3:delimiter": "/",
                    "s3:prefix": ""
                }
            }
        },
        {
            "Sid": "AllowListingOfUserFolder",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::<state bucket>",
            "Condition": {
                "StringLike": {
                    "s3:prefix": "${aws:username}/*"
                }
            }
        },
        {
            "Sid": "AllowAllS3ActionsInUserFolder",
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::<state bucket>/${aws:username}/*"
        }
    ]
}

So what we do:

  1. create new IAM group and attach that policy there (this needs to be done only once)
  2. When we need new cluster:
    • create new IAM user which matches to clustername, example: mycluster.k8s.local
    • attach IAM group to user
    • use IAM user credentials to authenticate against S3 bucket (env vars S3_ACCESS_KEY_ID and S3_SECRET_ACCESS_KEY needs to be defined)

After that each cluster has access to only own state store folder. It makes it possible to also have "root" account which has access to all clusters if needed for ops purposes, but note that all modifying kops commands should be executed using those IAM user credentials.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is in addition to the content in the PR, right? So the group is done once per account, and the IAM user is once per cluser? You share this user among everyone that can operate that particular cluster?

Co-Authored-By: John Gardiner Myers <jgmyers@proofpoint.com>
@johngmyers
Copy link
Member

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 1, 2020
@rifelpet
Copy link
Member

rifelpet commented May 1, 2020

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: olemarkus, rifelpet

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 1, 2020
@rifelpet
Copy link
Member

rifelpet commented May 1, 2020

/retest

@k8s-ci-robot k8s-ci-robot merged commit 0c96a23 into kubernetes:master May 1, 2020
@k8s-ci-robot k8s-ci-robot added this to the v1.18 milestone May 1, 2020
@olemarkus olemarkus deleted the docs-s3-cross-account branch May 10, 2020 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/documentation cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants