Minimal IAM policy

[jsok]

I can't find any documentation around exactly which AWS permissions are necessary when using the amazonec2 driver. I'd prefer to create a user/role with the minimum IAM policy rather than using a full Administrator user.

[nathanleclaire]

Yeah, I'd definitely like this.

cc @bmangold did you ever figure this out?

[bmangold]

I did ... sort of:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "TheseActionsDontSupportResourceLevelPermissions",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateKeyPair",
                "ec2:DeleteKeyPair",
                "ec2:ImportKeyPair",
                "ec2:Describe*",
                "ec2:CreateTags"
            ],
            "Resource": "*"
        },
        {
            "Sid": "ThisActionsSupportResourceLevelPermissions",
            "Effect": "Allow",
            "Action": ["ec2:RunInstances"],
            "Resource": [
                "arn:aws:ec2:us-east-1::image/ami-*",
                "arn:aws:ec2:us-east-1:<<your AWS account ID>>:instance/*",
                "arn:aws:ec2:us-east-1:<<your AWS account ID>>:key-pair/*",
                "arn:aws:ec2:us-east-1:<<your AWS account ID>>:network-interface/*",
                "arn:aws:ec2:us-east-1:<<your AWS account ID>>:placement-group/*",
                "arn:aws:ec2:us-east-1:<<your AWS account ID>>:security-group/*",
                "arn:aws:ec2:us-east-1:<<your AWS account ID>>:subnet/*",
                "arn:aws:ec2:us-east-1:<<your AWS account ID>>:volume/*"
                ]
        },
        {
            "Sid": "TheseActionsSupportResourceLevelPermissions",
            "Effect": "Allow",
            "Action": [
                "ec2:TerminateInstances",
                "ec2:StopInstances",
                "ec2:StartInstances",
                "ec2:RebootInstances"
            ],
            "Resource": [
                "arn:aws:ec2:us-east-1:<<your AWS account ID>>:instance/*"
            ]
        }
    ]
}

Note: this requires security groups already created and certain ports allowed (ingress): TCP/2376 , TCP/3376 , SSH/22 ...

@jsok (or anyone else) I can explain this in more detail if you want. AWS doesn't support resource level perms on everything so there are tradeoffs here .... but the idea is that you pre-create security groups, vpc, and subnet (IIRC) and give these to your 'users' to provide in the docker-machine params. Also, the way this works means that you should restrict users to a region or use a separate AWS account, as the perms as they stand allow for terminate,stop,start,restart for all ec2 instances in a single region.

[ehazlett]

Yeah the intention was to be able to not have to require users to setup a bunch of things before using Machine. However, I think it would be good to document what is needed for users to lock it down. Thanks @bmangold for the policy. @bmangold would you mind submitting that as a PR to the EC2 Machine docs?

[drewcrawford]

I also needed (at least initially)

                "ec2:CreateSecurityGroup",
                "ec2:AuthorizeSecurityGroupIngress"

I discovered these by running docker-machine -D [blah] to diagnose 403 forbidden errors.

[nathanleclaire]

Yes, if the docker-machine security group does not exist, Machine will need these permissions to create the security group and authorize ingress e.g. on port 22.

[tmaslen]

Can anyone tell me the exact additional policy needed if I want docker-machine to create an instance with an attached iam-role? I can make it work with "IAMFullAccess" but am a bit worried this is too much access.

Thanks.

[lihouyu]

For using --amazonec2-iam-instance-profile , we need "iam:PassRole".

[lizepeng]

I added Condition to ensure that docker-machine could only destroy the instances it created.

        {
            "Sid": "TheseActionsSupportResourceLevelPermissions",
            "Effect": "Allow",
            "Action": [
                "ec2:TerminateInstances",
                "ec2:StopInstances",
                "ec2:StartInstances",
                "ec2:RebootInstances"
            ],
            "Condition": {
                "StringEquals": {
                    "ec2:ResourceTag/created-by": "gitlab-ci-runners"
                }
            },
            "Resource": [
                "arn:aws:ec2:aaa:nnn:instance/*"
            ]
        }

[sbeeker]

This is a bit old and when I try it I get a "Version" string is invalid type error. Do you have a more recent version of this policy. Kind of new to all this and just trying to get a Docker-Machine from mac to EC2.
When I change out the AWS Account I get the following error when doing the validate policy.

"This policy contains the following error: The policy must contain a valid version string For more information about the IAM policy grammar, see AWS IAM Policies."

Or is someone has a more recent tutorial on the whole process for Docker Machine creation on AWS.

[mfrister]

@lizepeng

to ensure that docker-machine could only destroy the instances it created.

Unfortunately, this doesn't work as you describe it and doesn't prevent the runner from terminating arbitrary instances: With ec2:CreateTags allowed for the instance, it can just add the created-by: gitlab-ci-runners tag to any existing instance and then terminate it.

Annoyingly, docker-machine doesn't pass tags with the RunInstances API call, so it's currently not possible to prevent it from adding tags to arbitrary instances (without CreateTags permission, it can't add any tags any more).

[vegardx]

Following in on this, the driver seems a little incomplete when we can't limit the policy at all.

[simonbuchan]

It will only use what it needs, so you can avoid giving it permissions that it won't use. It's a bit unfortunate that it doesn't give the actual missing permission, and optionally which configuration would avoid it.

Reading through the driver code will give you the exact mapping, but from my reading (where "passed x" means --amazonec2-x):

• ec2:DescribeAccountAttributes unless vpc-id is passed

• ec2:DescribeSubnets always (to validate passed subnet is in vpc-id if you pass it, otherwise to find it)

• ec2:DescribeSecurityGroups always (to check if it needs to create it)

• ec2:CreateSecurityGroup unless passed security-group (or the default docker-machine) is an existing security group in the VPC

• ec2:AuthorizeSecurityGroupIngress unless security-group already has the required ports

  • required ports are all TCP:
    • 22 for SSH,
    • open-port or default 2376 for Docker,
    • if docker swarm flags are passed, the swarm port or default 3376
  • note it only checks IpProtocol and FromPort, so you can lock down the source!

• ec2:DescribeKeyPairs always (to validate the keypair does or does not exist if you pass or don't pass it (when it will use the machine name))

• ec2:ImportKeyPair / ec2:DeleteKeyPair unless both keypair-name and ssh-keypath are passed (ec2:DeleteKeyPair on docker-machine rm)

• ec2:RequestSpotInstances / ec2:DescribeSpotInstanceRequests / ec2:CancelSpotInstanceRequests if request-spot-instance is passed

• ec2:RunInstances otherwise

• ec2:DescribeInstances always (so it can tell when it's ready, get the IP address etc)

• ec2:CreateTags always to set the Name tag and anything you pass in tags

• iam:PassRole if iam-instance-profile is passed - this one can and should use resource-level permissions to lock it down to the profile you pass

• ec2:StartInstances if you use docker-machine start

• ec2:StopInstances if you use docker-machine stop or kill

• ec2:RebootInstances if you use docker-machine restart

• ec2:TerminateInstances if you use docker-machine rm

I think that's everything? Other than iam:PassRole I don't think there is much useful you can do with Resource normally, as you are limited to you own account already.

With this, the very minimal policy, assuming you use --amazonec2-vpc-id, --amazonec2-security-group with an already configured group, --amazonec2-keypair-name and --amazonec2-ssh-keypath and only docker-machine create and ... rm, (i.e. not stop, start, kill or restart) is:

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeKeyPairs",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSubnets",
        "ec2:CreateTags",
        "ec2:RunInstances",
        "ec2:TerminateInstances"
      ],
      "Resource": "*"
    }
  ]
}

If you use --amazonec2-iam-instance-profile, add:

    {
      "Effect": "Allow",
      "Action": "iam:PassRole",
      "Resource": "arn:aws:iam::ACCOUNT_ID:role/ROLE_NAME"
    }

(Edit: added iam:PassRole when using instance profiles, example policy doc)

[Shocktrooper]

While I know this is old but still useful for those using docker+machine GitLab executor. You will need these extra permissions on the manager role for the kms key that encrypts the AMI you are trying to use otherwise the runner manager will try to start the worker instances but instantly terminate.

    {
      "Sid": "KMSAMIPermissions",
      "Effect": "Allow",
      "Action": [
        "kms:CreateGrant",
        "kms:Decrypt",
        "kms:DescribeKey",
        "kms:Encrypt",
        "kms:GenerateDataKey",
        "kms:GenerateDataKeyWithoutPlaintext",
        "kms:ListGrants",
        "kms:ReEncryptFrom",
        "kms:ReEncryptTo",
        "kms:RevokeGrant"
      ],
      "Resource": "*"
    }