-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Document minimum IAM requirements #204
Comments
@NunoPinheiro I'd recommend creating a cluster with an admin account, and looking and all resources that get created and use those as a basis to define a restricted policy. Please note that we have #122. The code already accommodates for separating out IAM resources, all the resources are defined here: It should be fairly doable to implement what's being discussed in #122. |
I believe @mhausenblas was looking into this also. |
Here are the minimum AWS IAM permissions to create and delete clusters via I verified them by running CloudFormation {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "eksCtlCloudFormation",
"Effect": "Allow",
"Action": "cloudformation:*",
"Resource": "*"
}
]
} EKS {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"eks:*"
],
"Resource": "*"
}
]
} AutoScaling {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"autoscaling:CreateLaunchConfiguration",
"autoscaling:DeleteLaunchConfiguration"
],
"Resource": "arn:aws:autoscaling:*:*:launchConfiguration:*:launchConfigurationName/*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"autoscaling:UpdateAutoScalingGroup",
"autoscaling:DeleteAutoScalingGroup",
"autoscaling:CreateAutoScalingGroup"
],
"Resource": "arn:aws:autoscaling:*:*:autoScalingGroup:*:autoScalingGroupName/*"
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": [
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeLaunchConfigurations"
],
"Resource": "*"
}
]
} IAM {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"iam:CreateInstanceProfile",
"iam:DeleteInstanceProfile",
"iam:GetRole",
"iam:GetInstanceProfile",
"iam:RemoveRoleFromInstanceProfile",
"iam:CreateRole",
"iam:DeleteRole",
"iam:AttachRolePolicy",
"iam:PutRolePolicy",
"iam:ListInstanceProfiles",
"iam:AddRoleToInstanceProfile",
"iam:ListInstanceProfilesForRole",
"iam:PassRole",
"iam:DetachRolePolicy",
"iam:DeleteRolePolicy",
"iam:GetRolePolicy"
],
"Resource": [
"arn:aws:iam::<AWS Acct Id>:instance-profile/eksctl-*",
"arn:aws:iam::<AWS Acct Id>:role/eksctl-*"
]
}
]
} Networking {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EksInternetGateway",
"Effect": "Allow",
"Action": "ec2:DeleteInternetGateway",
"Resource": "arn:aws:ec2:*:*:internet-gateway/*"
},
{
"Sid": "EksNetworking",
"Effect": "Allow",
"Action": [
"ec2:AuthorizeSecurityGroupIngress",
"ec2:DeleteSubnet",
"ec2:DeleteTags",
"ec2:CreateNatGateway",
"ec2:CreateVpc",
"ec2:AttachInternetGateway",
"ec2:DescribeVpcAttribute",
"ec2:DeleteRouteTable",
"ec2:AssociateRouteTable",
"ec2:DescribeInternetGateways",
"ec2:CreateRoute",
"ec2:CreateInternetGateway",
"ec2:RevokeSecurityGroupEgress",
"ec2:CreateSecurityGroup",
"ec2:ModifyVpcAttribute",
"ec2:DeleteInternetGateway",
"ec2:DescribeRouteTables",
"ec2:ReleaseAddress",
"ec2:AuthorizeSecurityGroupEgress",
"ec2:DescribeTags",
"ec2:CreateTags",
"ec2:DeleteRoute",
"ec2:CreateRouteTable",
"ec2:DetachInternetGateway",
"ec2:DescribeNatGateways",
"ec2:DisassociateRouteTable",
"ec2:AllocateAddress",
"ec2:DescribeSecurityGroups",
"ec2:RevokeSecurityGroupIngress",
"ec2:DeleteSecurityGroup",
"ec2:DeleteNatGateway",
"ec2:DeleteVpc",
"ec2:CreateSubnet",
"ec2:DescribeSubnets"
],
"Resource": "*"
}
]
} |
@dougireton thank you so much!! |
@dougireton @mhausenblas how do you think it would be best to document this? I worry if we just add this to the readme, it will get out of date too soon. Ideally, we should be able to generate this. The code already knows of this, but it's a little indirect. |
I tried to create a custom IAM policy out of the above and after a couple of failed attempts (some actions were misssing, ie. 1. Create a custom IAM policy out of this JSONNote that you need to replace
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iam:CreateInstanceProfile",
"iam:DeleteInstanceProfile",
"iam:GetRole",
"iam:GetInstanceProfile",
"iam:RemoveRoleFromInstanceProfile",
"iam:CreateRole",
"iam:DeleteRole",
"iam:AttachRolePolicy",
"iam:PutRolePolicy",
"iam:ListInstanceProfiles",
"iam:AddRoleToInstanceProfile",
"iam:ListInstanceProfilesForRole",
"iam:PassRole",
"iam:DetachRolePolicy",
"iam:DeleteRolePolicy",
"iam:GetRolePolicy"
],
"Resource": [
"arn:aws:iam::<AWS Acct Id>:instance-profile/eksctl-*",
"arn:aws:iam::<AWS Acct Id>:role/eksctl-*"
]
},
{
"Effect": "Allow",
"Action": "cloudformation:*",
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"eks:*"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeScalingActivities",
"autoscaling:CreateLaunchConfiguration",
"autoscaling:DeleteLaunchConfiguration",
"autoscaling:UpdateAutoScalingGroup",
"autoscaling:DeleteAutoScalingGroup",
"autoscaling:CreateAutoScalingGroup"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": "ec2:DeleteInternetGateway",
"Resource": "arn:aws:ec2:*:*:internet-gateway/*"
},
{
"Effect": "Allow",
"Action": [
"ec2:AuthorizeSecurityGroupIngress",
"ec2:DeleteSubnet",
"ec2:DeleteTags",
"ec2:CreateNatGateway",
"ec2:CreateVpc",
"ec2:AttachInternetGateway",
"ec2:DescribeVpcAttribute",
"ec2:DeleteRouteTable",
"ec2:AssociateRouteTable",
"ec2:DescribeInternetGateways",
"ec2:CreateRoute",
"ec2:CreateInternetGateway",
"ec2:RevokeSecurityGroupEgress",
"ec2:CreateSecurityGroup",
"ec2:ModifyVpcAttribute",
"ec2:DeleteInternetGateway",
"ec2:DescribeRouteTables",
"ec2:ReleaseAddress",
"ec2:AuthorizeSecurityGroupEgress",
"ec2:DescribeTags",
"ec2:CreateTags",
"ec2:DeleteRoute",
"ec2:CreateRouteTable",
"ec2:DetachInternetGateway",
"ec2:DescribeNatGateways",
"ec2:DisassociateRouteTable",
"ec2:AllocateAddress",
"ec2:DescribeSecurityGroups",
"ec2:RevokeSecurityGroupIngress",
"ec2:DeleteSecurityGroup",
"ec2:DeleteNatGateway",
"ec2:DeleteVpc",
"ec2:CreateSubnet",
"ec2:DescribeSubnets"
],
"Resource": "*"
}
]
} 2. Create a new IAM service role for CloudFormation and attach the above policy3. Run
|
Thanks everyone for sharing so well-documented steps! |
Why is |
@bensussman The above is not an official list of IAM policies; I compiled it while playing with eksctl and we don't really use it in production. Feel free to improve as you want. |
💔 @VojtechVitek bummer. Is the idea that users of
|
I added |
Benjamin, I don't think there were any changes that would affect these. If
you need more help, please do connect on Slack and we can discuss in real
time. As I pointed out, I am keen to do what would need to be done to close
this issue, but some form of help would be appreciated. At present all of
my focus is on things related to cluster upgrades.
…On Tue, 5 Mar 2019, 10:21 pm Benjamin Sussman, ***@***.***> wrote:
I added ec2:DescribeAvailabilityZones and it let me move on to the next
one, looks like it's unable to find the ami-0c28139856aaf9c3b. I wonder
how any of these IAM policies were made, or has eksctl just changed a
bunch in the 2 months since they were written (i doubt that)?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#204 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAPWS6OEXk-cE7kHDp6gRTso874ZWCqiks5vTu3MgaJpZM4WjUhl>
.
|
I got it working by giving it all ec2 permissions I would advise someone get the minimum set of perms (not sure if this list above has anything extraneous, but it's certainly missing a few things like |
We should really automate this!
To begin with we can add a command that creates the policy, and use it as
part of the integration test suite. Any takers?
|
I added some missing permissions to the sample policy. Following IAM permissions should be enough to launch EKS cluster with worker nodes in a new VPC using eksctl.
|
These are missing
"iam:DeleteServiceLinkedRole",
|
And if you need to delete a stack as well:
"ec2:DeleteLaunchTemplate",
(And for decoding the authorization error message that you will get in the console without that: sts:DecodeAuthorizationMessage.) |
Lets document these complete in wiki page or somewhere first, so many people are dwelling for same stuffs. |
And probably some additional permissions are required for SSH access. Trying to enable it results in: [X] searching for SSH public key "eksctl-{cluster-name}-nodegroup-{nodegroup-name}-{fingerprint}" in EC2: UnauthorizedOperation: |
Here is the policy that worked for me. Fixes SSH import key and describe key: So here we have the full policy with @jonin contributions and mine:
|
FWIW I had to add the following Resource to the iam-statement from the policy above in order to get this to work:
[...]
"arn:aws:iam::<AWS_ACCOUNT_ID>:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling",
|
I have all of these permissions in the IAM role I'm using to try to use eksctl create cluster. I have verified using sts that the IAM role expected is the one executing the command. I still get errors saying that I am "not authorized to perform this operation" on eksctl referencing operations I have the permissions to perform in the policy I can directly see. - example:AWS::EC2::InternetGateway/InternetGateway: CREATE_FAILED – "API: ec2:CreateInternetGateway You are not authorized to perform this operation." I have run exactly the same command with exactly the same permissions in another account exactly the same way and it works. I cannot find any difference in the accounts other than the non-working one has MFA.. but it only errors out on 3 commands. |
I tried to create a cluster under the policies mentioned in #204 (comment) , but failed. It said that
Then I found the document https://docs.aws.amazon.com/autoscaling/ec2/userguide/control-access-using-iam.html, which said that
I added the following policies, and it worked (^▽^)
|
I have |
@mutabletao are you using a MFA user? If so, I believe you need to set up a session token to ensure proper connectivity between running CLI commands and your user. |
Hello, same here. I'm using the latest compiled set of permissions for the EKS service role and still not able to have the node-group created: AWS::AutoScaling::AutoScalingGroup/NodeGroup: CREATE_FAILED – "API: autoscaling:CreateAutoScalingGroup You are not authorized to use launch template: eksctl-dev-k8s-us-east-1-nodegroup-dev-k8s-us-east-1-standard" I'm launching eksctl with an admin user which is not an MFA enabled user eksctl create cluster -f cluster-config.yaml |
This is ridiculous, is not there any exact list of permissions / roles to enable to try this stuff out? |
Under IAM -> Groups -> Permissions, click Attach Policy worked for me, using the above json from @mailjunze and steps from @VojtechVitek. |
If you use
|
@uny You might want to make sure you remove your AWS Account ID from the above comment. |
@JulienDefrance |
CloudFormation template example for EKSctl manged cluster policy ref: #204 (comment)
|
The above policy didn't allow to create unmanaged Node Group and delete a cluster. Added additional permissions to the policy. Please find the updated one.. { |
Please @sreejithcts comment the difference between your and above, otherwise is hard to see what you add, thanks |
I ran "eksctl create cluster" with this policy and it works for me
|
Ran the following command: With the following policy:
Replaced account-id with my actual AWS account id to get it to run. This policy is based off of the work done above but cleaned up so it's better understood. |
I just want to thank everyone for this thread. I came here after running into several issues. With special thanks to @dougireton , @mailjunze ,@jonin , and @OscarAyoy , I was able to get it done. |
Hello folks,
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "EC2IG",
"Effect": "Allow",
"Action": "ec2:DeleteInternetGateway",
"Resource": "arn:aws:ec2:*:*:internet-gateway/*"
},
{
"Sid": "ELBIAM",
"Effect": "Allow",
"Action": "iam:CreateServiceLinkedRole",
"Resource": "*",
"Condition": {
"StringEquals": {
"iam:AWSServiceName": "elasticloadbalancing.amazonaws.com"
}
}
},
{
"Sid": "IAM",
"Effect": "Allow",
"Action": [
"iam:CreateInstanceProfile",
"iam:DeleteInstanceProfile",
"iam:GetRole",
"iam:GetInstanceProfile",
"iam:RemoveRoleFromInstanceProfile",
"iam:CreateRole",
"iam:DeleteRole",
"iam:AttachRolePolicy",
"iam:PutRolePolicy",
"iam:ListInstanceProfiles",
"iam:AddRoleToInstanceProfile",
"iam:ListInstanceProfilesForRole",
"iam:PassRole",
"iam:CreateServiceLinkedRole",
"iam:DetachRolePolicy",
"iam:DeleteRolePolicy",
"iam:DeleteServiceLinkedRole",
"iam:GetRolePolicy",
"iam:ListAttachedRolePolicies"
],
"Resource": [
"arn:aws:iam::*:instance-profile/eksctl-*",
"arn:aws:iam::*:role/eksctl-*",
"arn:aws:iam::*:role/aws-service-role/eks.amazonaws.com/*",
"arn:aws:iam::*:role/aws-service-role/eks-nodegroup.amazonaws.com/*"
]
},
{
"Sid": "IAMOIDC",
"Effect": "Allow",
"Action": "iam:GetOpenIDConnectProvider",
"Resource": "arn:aws:iam::557309985620:oidc-provider/oidc.eks.eu-west-1.amazonaws.com/*"
},
{
"Sid": "EC2",
"Effect": "Allow",
"Action": [
"ec2:AuthorizeSecurityGroupIngress",
"ec2:DescribeInstances",
"ec2:AttachInternetGateway",
"ec2:DeleteRouteTable",
"ec2:RevokeSecurityGroupEgress",
"ec2:CreateRoute",
"ec2:CreateInternetGateway",
"ec2:DescribeVolumes",
"ec2:DeleteInternetGateway",
"ec2:DescribeKeyPairs",
"ec2:ImportKeyPair",
"ec2:CreateTags",
"ec2:RunInstances",
"ec2:DisassociateRouteTable",
"ec2:CreateVolume",
"ec2:RevokeSecurityGroupIngress",
"ec2:DescribeImageAttribute",
"ec2:DeleteNatGateway",
"ec2:CreateSubnet",
"ec2:DescribeSubnets",
"ec2:AttachVolume",
"ec2:CreateNatGateway",
"ec2:CreateVpc",
"ec2:DescribeVpcAttribute",
"ec2:ModifySubnetAttribute",
"ec2:DescribeAvailabilityZones",
"ec2:ReleaseAddress",
"ec2:DeleteLaunchTemplate",
"ec2:DescribeSecurityGroups",
"ec2:CreateLaunchTemplate",
"ec2:DescribeVpcs",
"ec2:DeleteSubnet",
"ec2:DescribeVolumesModifications",
"ec2:AssociateRouteTable",
"ec2:DescribeInternetGateways",
"ec2:DeleteVolume",
"ec2:DescribeAccountAttributes",
"ec2:DescribeRouteTables",
"ec2:DetachVolume",
"ec2:ModifyVolume",
"ec2:DescribeLaunchTemplates",
"ec2:CreateRouteTable",
"ec2:DetachInternetGateway",
"ec2:DeleteVpc",
"ec2:DescribeAddresses",
"ec2:DeleteTags",
"ec2:DescribeDhcpOptions",
"ec2:DescribeNetworkInterfaces",
"ec2:CreateSecurityGroup",
"ec2:ModifyVpcAttribute",
"ec2:ModifyInstanceAttribute",
"ec2:AuthorizeSecurityGroupEgress",
"ec2:DescribeTags",
"ec2:DeleteRoute",
"ec2:DescribeLaunchTemplateVersions",
"ec2:DescribeNatGateways",
"ec2:AllocateAddress",
"ec2:DescribeImages",
"ec2:DeleteSecurityGroup"
],
"Resource": "*"
},
{
"Sid": "ELB",
"Effect": "Allow",
"Action": [
"elasticloadbalancing:ModifyListener",
"elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer",
"elasticloadbalancing:CreateTargetGroup",
"elasticloadbalancing:AddTags",
"elasticloadbalancing:DeleteLoadBalancerListeners",
"elasticloadbalancing:ModifyLoadBalancerAttributes",
"elasticloadbalancing:CreateLoadBalancerPolicy",
"elasticloadbalancing:CreateLoadBalancer",
"elasticloadbalancing:DeleteTargetGroup",
"elasticloadbalancing:SetLoadBalancerPoliciesOfListener",
"elasticloadbalancing:DescribeTargetGroups",
"elasticloadbalancing:DeleteListener",
"elasticloadbalancing:DetachLoadBalancerFromSubnets",
"elasticloadbalancing:RegisterTargets",
"elasticloadbalancing:DeleteLoadBalancer",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeLoadBalancerPolicies",
"elasticloadbalancing:ModifyTargetGroupAttributes",
"elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
"elasticloadbalancing:RegisterInstancesWithLoadBalancer",
"elasticloadbalancing:DeregisterTargets",
"elasticloadbalancing:DescribeLoadBalancerAttributes",
"elasticloadbalancing:DescribeTargetGroupAttributes",
"elasticloadbalancing:ConfigureHealthCheck",
"elasticloadbalancing:CreateListener",
"elasticloadbalancing:DescribeListeners",
"elasticloadbalancing:ApplySecurityGroupsToLoadBalancer",
"elasticloadbalancing:AttachLoadBalancerToSubnets",
"elasticloadbalancing:CreateLoadBalancerListeners",
"elasticloadbalancing:DescribeTargetHealth",
"elasticloadbalancing:ModifyTargetGroup"
],
"Resource": "*"
},
{
"Sid": "ECR",
"Effect": "Allow",
"Action": [
"ecr:GetAuthorizationToken",
"ecr:InitiateLayerUpload",
"ecr:ListImages",
"ecr:BatchCheckLayerAvailability",
"ecr:GetDownloadUrlForLayer",
"ecr:PutImage",
"ecr:BatchGetImage",
"ecr:DescribeImages",
"ecr:UploadLayerPart",
"ecr:CompleteLayerUpload",
"ecr:DescribeRepositories"
],
"Resource": "*"
},
{
"Sid": "AUTOSCALING",
"Effect": "Allow",
"Action": [
"autoscaling:DeleteAutoScalingGroup",
"autoscaling:DescribeScalingActivities",
"autoscaling:CreateLaunchConfiguration",
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:UpdateAutoScalingGroup",
"autoscaling:CreateAutoScalingGroup",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DeleteLaunchConfiguration"
],
"Resource": "*"
},
{
"Sid": "SSM",
"Effect": "Allow",
"Action": [
"ssm:GetParametersByPath",
"ssm:GetParameter",
"ssm:DeleteParameter",
"ssm:DescribeParameters",
"ssm:GetParameters",
"ssm:DeleteParameters",
"ssm:PutParameter",
"ssm:GetParameterHistory"
],
"Resource": "*"
},
{
"Sid": "Full",
"Effect": "Allow",
"Action": ["cloudformation:*", "eks:*"],
"Resource": "*"
},
{
"Sid": "KMS",
"Effect": "Allow",
"Action": ["kms:DescribeKey"],
"Resource": "*"
},
{
"Sid": "IAMGetRole",
"Effect": "Allow",
"Action": ["iam:GetRole"],
"Resource": "*"
}
]
} |
Rename verify-all file name
It would be nice to have a documentation listing the minimum IAM permissions to run eksctl.
I'm trying to set this up with a minimum service account, and now I have to add dependencies one by one.
If they were documented as a YAML that would be the best!
The text was updated successfully, but these errors were encountered: