Narrowing the Serverless IAM Deployment Policy

[ghost]

I’ve been spending time recently trying to remove Admin rights as a requirement for sls deployments. Still a work in progress, but so far I have this policy that I can attach to any “serverless-agent” AWS user, so that the serverless-agent user is empowered enough to deploy:

    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "cloudformation:Describe*",
                "cloudformation:List*",
                "cloudformation:Get*",
                "cloudformation:PreviewStackUpdate"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "cloudformation:CreateStack",
                "cloudformation:UpdateStack",
                "cloudformation:DeleteStack"
            ],
            "Resource": "arn:aws:cloudformation:*:*:stack/${project}*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "lambda:Get*",
                "lambda:List*",
                "lambda:CreateFunction"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "lambda:AddPermission",
                "lambda:CreateAlias",
                "lambda:DeleteFunction",
                "lambda:InvokeFunction",
                "lambda:PublishVersion",
                "lambda:RemovePermission",
                "lambda:Update*"
            ],
            "Resource": "arn:aws:lambda:*:*:function:${project}*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "lambda:*"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "apigateway:GET"
            ],
            "Resource": [
                "arn:aws:apigateway:*::/restapis"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "apigateway:GET",
                "apigateway:POST",
                "apigateway:PUT",
                "apigateway:DELETE"
            ],
            "Resource": [
                "arn:aws:apigateway:*::/restapis/*/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:PassRole"
            ],
            "Resource": "arn:aws:iam::*:role/*"
        },
        {
            "Effect": "Allow",
            "Action": "kinesis:*",
            "Resource": "arn:aws:kinesis:*:*:stream/${project}*"
        },
        {
            "Effect": "Allow",
            "Action": "iam:*",
            "Resource": "arn:aws:iam::*:role/${project}*"
        },
        {
            "Effect": "Allow",
            "Action": "sqs:*",
            "Resource": "arn:aws:sqs:*:*:${project}*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "events:Put*",
                "events:Remove*",
                "events:Delete*"
            ],
            "Resource": "arn:aws:events:*:*:rule/${project}*"
        }
    ]
}

Right now, I'm focused on a single policy that can deploy to all stages. But some enterprises may need this IAM policy to allow dev and staging deployments, but limit who can deploy to production. So, I've also been experimenting with adding "${stage}" to some of the resource ARNs, but don't have it fully worked out yet. For example:

"Resource": "arn:aws:events:*:*:rule/${project}-${stage}*"

There are still a few places where the permissions could be narrowed further. Specifically, the REST API section allows delete of ALL apis right now. And the lambda permissions are too broad. But I’ve had some annoying technical issues trying to narrow those two sections.

The API Gateway policy is still broad because you must have the 'api-id' in the ARN. But you don't know that until a deployment generates it. So on the surface, seems like a chicken/egg problem to me, but maybe there is a way to supply that api-id, instead of having AWS generate it.

And the lambda permissions are still broad because I can't see the particular Arn it is trying to manipulate to add an event mapping to a lambda, and the obvious ARNs don't work. Maybe there is a way to show the ARN being accessed in serverless, when the deployment fails so that I can add it to the policy, but no luck so far.

As I said, still a work in progress, so use with caution. Will post back any further 'narrowing' as I figure it.

[Depado]

Thanks for the great work, I'll look into that, I also need to narrow down the IAM policy.

[pgasiorowski]

Thanks for sharing. I'm going to give it a go a provide some feedback.
My first impressions it that the policy could be narrowed further by including the ${region} in appropriate Resource sections. What do you think?

[tobyhede]

This is really good, thanks

[pgasiorowski]

Here is one change that I did to make it work for me:

    {
      "Effect": "Allow",
      "Action": [
        "apigateway:GET"
      ],
      "Resource": [
        "arn:aws:apigateway:*::/restapis"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "apigateway:GET",
        "apigateway:POST",
        "apigateway:PUT",
        "apigateway:DELETE"
      ],
      "Resource": [
        "arn:aws:apigateway:*::/restapis/*/*"
      ]
    },

    {
      "Effect": "Allow",
      "Action": [
        "apigateway:GET",
        "apigateway:HEAD",
        "apigateway:OPTIONS"
        "apigateway:PATCH",
        "apigateway:POST",
        "apigateway:PUT",
        "apigateway:DELETE"
      ],
      "Resource": [
        "arn:aws:apigateway:*::/restapis",
        "arn:aws:apigateway:*::/restapis/*"
      ]
    },

2nd-level POST /api/v1/resource didn't work with your IAM but the above would handle it.
It also enables pushing missing Head, Patch and Options methods.

Going further with the restrictions, adding ${region} to all Resources with the exception of * and arn:aws:iam:* seems to work fine eg:

"arn:aws:apigateway:${region}::/restapis",
"arn:aws:apigateway:${region}::/restapis/*"
"arn:aws:lambda:${region}::function:${project}*"

And the lambda permissions are still broad because I can't see the particular Arn it is trying to manipulate to add an event mapping to a lambda, and the obvious ARNs don't work.

This one is particularly nasty but it doesn't seem to work without it.

[cknowles]

You may be able to change PassRole to just role/${project}* to restrict that part further? I found it would also be helpful if serverless populated a variable with the ID of the Gateway to use to restrict the apigateway permissions. Some duplicate effort in #588.

[ecoruh]

This is great, thanks. I got it working except one bit. We are using serverless-client-s3 plugin for deploying s3 buckets.

Anyone knows what the minimum set of permissions are for sls client deploy to work?. I tried this in my policy:

    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBuckets",
                "s3:ListObjects",
                "s3:DeleteObjects",
                "s3:CreateBucket",
                "s3:PutBucketWebsite",
                "s3:PutBucketTagging",
                "s3:PutBucketPolicy",
                "s3:PutObject"
            ],
            "Resource": "*"
        }

I scanned the plugin source code to come up with the s3 function list and added them into the policy. But I received this error:

AccessDenied: Access Denied
    at Request.extractError (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/services/s3.js:350:35)
    at Request.callListeners (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
...
From previous event:
    at ServerlessProviderAws.request (/usr/local/lib/node_modules/serverless/lib/ProviderAws.js:72:10)
    at ClientDeploy._processDeployment (/home/eric/BuildAgent/work/3a164dd9bff1550c/ucr/node_modules/serverless-client-s3/index.js:134:24)

This line seems to yield Access Denied:

      return _this.aws.request('S3', 'listBuckets', {}, _this.evt.options.stage, _this.evt.options.region)

even though I had listBuckets in my permission list. What am I missing?

note: I shall refine Resource further once I get it working..

[cknowles]

I think try s3:ListAllMyBuckets. Some of the others don't exist, it's confusing as the AWS CLI/SDK does not use the same names as the IAM actions - see these docs. s3:ListBucket with no s is the action to list the content of a bucket.

[ghost]

I think I have the serverless deployment policy nailed at this point. A bit more testing is in order, but mostly there now. In this second iteration, I've taken a slightly different approach. Instead of putting everything into 1 giant policy, I've broken it down into 5 separate put related policies:

• s-policy-base.json - Read-only permissions generally required by serverless deployments and relatively harmless via their read-only limitation
• s-policy-endpoint.json - Write permissions (I.e. update, delete actions etc.) required by sls endpoint deploy
• s-policy-event.json - Write permissions required by sls event deploy
• s-policy-function.json - Write permissions required by sls function deploy
• s-policy-resources.json - Write permissions required by sls resources deploy

FILE: s-policy-base.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "NonDestructiveCoreReaderActionsUsedThroughoutServerless",
            "Effect": "Allow",
            "Action": [
                "cloudformation:Describe*",
                "cloudformation:List*",
                "cloudformation:Get*",
                "cloudformation:PreviewStackUpdate",
                "lambda:Get*",
                "lambda:List*",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSubnets",
                "s3:GetObject",
                "s3:List*",
                "apigateway:GET",
                "iam:List*",
                "iam:Get*",
                "iam:Simulate*",
                "kinesis:Describe*",
                "kinesis:List*",
                "dynamodb:Describe*",
                "dynamodb:List*",
                "sqs:List*"
            ],
            "Resource": "*"
        }
    ]
}

FILE: s-policy-resources.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ResourceBoundCloudFormationWritersForSlsResourcesDeploy",
            "Effect": "Allow",
            "Action": "cloudformation:*",
            "Resource": "arn:aws:cloudformation:${region}:*:stack/${project}-*"
        },
        {
            "Sid": "ResourceDeniedCloudFormationWritersForSlsResourcesDeploy",
            "Effect": "Deny",
            "Action": "cloudformation:*",
            "Resource": "arn:aws:cloudformation:*:*:stack/${project}-${prod}*"
        },
        {
            "Sid": "ResourceBoundIamWritersForSlsFunctionDeploy",
            "Effect": "Allow",
            "Action": "iam:*",
            "Resource": "arn:aws:iam::*:role/${project}-*"
        },
        {
            "Sid": "ResourceDeniedIamWritersForSlsFunctionDeploy",
            "Effect": "Deny",
            "Action": "iam:*",
            "Resource": "arn:aws:iam::*:role/${project}-${prod}*"
        },
        {
            "Sid": "ResourceBoundKinesisStreamWritersForSlsResourcesDeploy",
            "Effect": "Allow",
            "Action": "kinesis:*",
            "Resource": "arn:aws:kinesis:${region}:*:stream/${project}-*"
        },
        {
            "Sid": "ResourceDeniedKinesisStreamWritersForSlsResourcesDeploy",
            "Effect": "Deny",
            "Action": "kinesis:*",
            "Resource": "arn:aws:kinesis:*:*:stream/${project}-${prod}*"
        },
        {
            "Sid": "ResourceBoundDynamoDbStreamWritersForSlsResourcesDeploy",
            "Effect": "Allow",
            "Action": "dynamodb:*",
            "Resource": "arn:aws:dynamodb:${region}:*:table/${project}-*"
        },
        {
            "Sid": "ResourceDeniedDynamoDbStreamWritersForSlsResourcesDeploy",
            "Effect": "Deny",
            "Action": "dynamodb:*",
            "Resource": "arn:aws:dynamodb:*:*:table/${project}-${prod}*"
        },
        {
            "Sid": "ResourceBoundSqsClientForSqsResourcesDeploy",
            "Effect": "Allow",
            "Action": "sqs:*",
            "Resource": "arn:aws:sqs:${region}:*:${project}-*"
        },
        {
            "Sid": "ResourceDeniedSqsClientForSqsResourcesDeploy",
            "Effect": "Deny",
            "Action": "sqs:*",
            "Resource": "arn:aws:sqs:*:*:${project}-${prod}*"
        }
    ]
}

FILE: s-policy-function.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PotentiallyDestructiveLambdaActionsRequiringAllAccessResourceSpecForSlsFunctionDeploy",
            "Effect": "Allow",
            "Action": [
                "lambda:CreateFunction"
            ],
            "Resource": "*"
        },
        {
            "Sid": "ResourceBoundLambdaExecutionRolePassingForSlsFunctionDeploy",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::*:role/${project}-*"
        },
        {
            "Sid": "ResourceDeniedLambdaExecutionRolePassingForSlsFunctionDeploy",
            "Effect": "Deny",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::*:role/${project}-${prod}*"
        },
        {
            "Sid": "ResourceBoundLambdaWritersForSlsFunctionDeploy",
            "Effect": "Allow",
            "Action": "lambda:*",
            "Resource": "arn:aws:lambda:${region}:*:function:${project}-*"
        },
        {
            "Sid": "ResourceDeniedLambdaWritersForSlsFunctionDeploy",
            "Effect": "Deny",
            "Action": "lambda:*",
            "Resource": "arn:aws:lambda:${region}:*:function:${project}-${prod}*"
        }
    ]
}

FILE: s-policy-endpoint.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ResourceBoundApiGatewayWritersForSlsEndpointDeploy",
            "Effect": "Allow",
            "Action": "apigateway:*",
            "Resource": [
                "arn:aws:apigateway:${region}::/restapis",
                "arn:aws:apigateway:${region}::/restapis/${api_id}*"
            ]
        },
        {
            "Sid": "ResourceDeniedApiGatewayWritersForSlsEndpointDeploy",
            "Effect": "Deny",
            "Action": "apigateway:*",
            "Resource": [
                "arn:aws:apigateway:*::/restapis/${prod_api_id}/*"
            ]
        }
    ]
}

FILE: s-policy-event.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PotentiallyDestructiveLambdaActionsRequiringAllAccessResourceSpecForSlsEventDeploy",
            "Effect": "Allow",
            "Action": [
                "lambda:CreateEventSourceMapping",
                "lambda:DeleteEventSourceMapping",
                "lambda:UpdateEventSourceMapping"
            ],
            "Resource": "*"
        },
        {
            "Sid": "ResourceBoundEventsInvokingLambdaForSlsEventDeploy",
            "Effect": "Allow",
            "Action": "events:*",
            "Resource": "arn:aws:events:${region}:*:rule/${project}-*"
        },
        {
            "Sid": "ResourceDeniedEventsInvokingLambdaForSlsEventDeploy",
            "Effect": "Deny",
            "Action": "events:*",
            "Resource": "arn:aws:events:*:*:rule/${project}-${prod}*"
        },
        {
            "Sid": "ResourceBoundLambdaWritersForSlsEventDeploy",
            "Effect": "Allow",
            "Action": "lambda:AddPermission",
            "Resource": "arn:aws:lambda:${region}:*:function:${project}-*"
        },
        {
            "Sid": "ResourceDeniedLambdaWritersForSlsEventDeploy",
            "Effect": "Deny",
            "Action": "lambda:AddPermission",
            "Resource": "arn:aws:lambda:*:*:function:${project}-${prod}*"
        }
    ]
}

To use, manually replace all ${key} variables for your specific project. Then copy all 5 variable-expanded policy documents above into separate AWS managed policies. Then attach all 5 polices to a user account needing serverless deployment capability. (Note: you could limit users whom you don't want to deploy resources, for example, by excluding the s-policy-resources.json policy from their user, if that is useful to you.)

First caveat, these policies enable you to deny deploying to production, while allowing deployment to other stages. You may want only a designated user (or Jenkins) to be the only party privileged enough to deploy to production. In which case, these "Deny" blocks that exclude "${prod}*" resource access become very useful. So, set ${prod} equal to whatever you call production. I.e. prod="prod" to protect production from harm. But, you ask, "what if you want to allow deployment to production?" To do that, set prod="open_season_on_prod". This will deny deployment to a stage named 'open_season_on_prod', but will allow deployment to 'prod' via the prior Allow blocks. So, set prod="prod" to generate policies that can't harm production. And set prod='open_season_on_prod' to allow production to be targeted via the policy. (Note: alternatively a template with conditional inclusion could be used to remove the Deny blocks when generating a production policy. But that assumes a serverless plugin.. more on that later.)

Second caveat, after much testing and research, I finally figured out why some of my Actions were failing for ARNs I knew I specified correctly in the policy. The reason is, there are 4 actions that Serverless uses that do not allow ARN-limited resource specifications, even though intuitively they should. They are:

       {
            "Effect": "Allow",
            "Action": [
                "lambda:CreateEventSourceMapping",
                "lambda:DeleteEventSourceMapping",
                "lambda:UpdateEventSourceMapping",
                "lambda:CreateFunction"
            ],
            "Resource": "*"
        },

These 4 lambda actions only work if you pass '*' for the resource. Trying to scope limit them to a ${project} ARN (as I was initially attempting) breaks the deployment. So, these are potentially destructive actions that these policies allow, because AWS gives us no choice but to allow them without appropriate limitation, as documented here:

http://docs.aws.amazon.com/lambda/latest/dg/lambda-api-permissions-ref.html

Third caveat, the s-policy-endpoint.json policy document has a special case: the Api Gateway "api-id" is not generated until after you deploy the API. This leaves 2 choices: 1) allow full access to all ApiGateway APIs, even APIs you shouldn't touch, or 2) generate the rest API api-id separately, and include it in the s-policy-endpoint.json. This ensures you only have write permissions to the APIs you should. You can do the latter using this command:

api_id=$(aws apigateway create-rest-api \
   --name ${project} \
   --description "${project} API" \
   --output text \
   --query 'id')
echo "> API ID is: $api_id"

Citation: http://codurance.com/2016/05/25/aws-api-gateway/

"But sls endpoint deploy is suppose to create the rest API", you say. Not necessarily. This is only provisioning an API-id, not deploying the full API. And Serverless will just reuse the api_id generated above, when the sls endpoint deploy command is first run. So, generate the api_id (as above), plug it into your endpoint-policy document and then your policy will be constrained to work only with your serverless-generated API.

As you might imagine, a Serverless "policy" plugin to manage all this would make a lot of sense. And I've actually started building a Serverless "policy" plugin to automate the variable substitution, api-gateway api-id generation, ${prod} variable setting, and aws deployment of these 5 policies. However, legal red-tape must be cut before I can share.

[pgasiorowski]

This is absolutely outstanding research and I think it makes a lot of sense to split into multiple policies for multi-user deployments. I yet need to try 1.0-alpha but do you think these policies would also be appropriate for CloudFormation-based deployments?

[ghost]

Thanks @WooDzu! I haven't tried 1.0-alpha yet either - been too heads down on this thing. I hope to get the chance in the next couple weeks to give 1.0-alpha a good tire kicking. Meanwhile, I don't really know how well these policies will translate.

[ploopaltar]

So we are using these permissions for the s3 client:

Permissions

s3:GetObject
s3:GetObjectVersion
s3:PutObject
s3:DeleteObject
s3:CreateBucket
s3:DeleteBucket
s3:ListBucket
s3:ListBucketVersions
s3:ListAllMyBuckets
s3:GetBucketVersioning
s3:PutBucketVersioning
s3:GetBucketLocation
s3:PutBucketPolicy
s3:GetBucketWebsite
s3:PutBucketWebsite
s3:DeleteBucketWebsite

[ghost]

Are these S3 policies needed for runtime, deployment time, or both? This thread is really only concerned with permissions required to deploy a serverless application. Are you finding these S3 policies are also necessary to deploy your serverless app?

[str3tch]

Hi Ron,

Yes, we need the s3 permissions to deploy our serverless application as we are using the serverless-client-s3 plugin to deploy files to s3, along with the lambda code. I think this is a quite common use case. So having these permissions helps us.

cheers,
Chris

[ghost]

Thanks @str3tch for the clarification. This has got me thinking out loud a bit, so forgive the stream of consciousness style... but I'm questioning whether it would be best to co-locate all possible permissions used by serverless, its plugins, your application code etc. That would certainly be simpler, but that sort of union problem might be hard to maintain, and would force everyone to use the union of all possible permissions which would often be too broad. There is a distinction between the permissions required by "serverless-core" versus incremental permissions required by serverless plugins and your own serverless application code. And even those incremental permissions could be further delineated according to which phase of deployment requires them -- though that might be going too far and introducing complexity. Sounding more and more like the domain of a serverless plugin and a template engine to dynamically generate and deploy the IAM policies you need based on the contents of your CloudFormation and plugin configurations. @str3tch and @ploopaltar, what are you thoughts?

[ploopaltar]

I guess one of the challenges we had and still have is that there are some permissions which we feel serverless does not need but seems to be required for the deployment process using the s3-client.

As a starting point clarity around why each permission is needed and at which part of the deployment process would help get us to the point where we can have permissions grouped according to the phase of deployments.

[str3tch]

@xtenix That's an interesting idea, but the problem I would face with getting that through here at work is that the engine would require IAM role create privileges to dynamically create new policies. And we're trying really hard here to restrict those, and not have service accounts have those elevated rights. I think the approach that you've taken is fantastic, split them up into separate policies. The plugin you mention to do the substituting would be awesome too.. how's it going on being able to share that work?

Fantastic work though, you've saved me and others countless hours of testing! 👍

[ghost]

Hi @str3tch

the engine would require IAM role create privileges to dynamically create new policies

Yes, I agree and I consider that as a challenging workflow issue for how policy changes are routed to proper authorities for review and deployment. In short, the AWS profile you are using to deploy your serverless app cannot be the same one used to deploy these policies. That would fundamentally defeat the purpose, because that AWS profile could choose to create an admin policy, and that opens up a back door.

To solve this problem, I've thought of 2 distinct but related approaches:

1. "Expand-And-Pass":

Don't automatically deploy any policies, only create and write them locally (similar to the -c option on 'resources'). The policy templates would be expanded (variable substitution), either manually or automatically via plugin. A plugin could expand the policy templates and pass these intermediate files to an AWS admin for manual deployment. This will, of course, imply some communication overhead, but it works. For example:

$ sls policy deploy -s dev -o s3://bucket/folder

2. "Expand-And-Administer":

Similar to option #1, but the expanded policy is automatically deployed via a separate and more privileged AWS profile. To afford this, introduce a "--profile" option to the plugin's "deploy" command that expects an admin key/secret. Serverless' standard (pre-existing) AWS profile would not be able to deploy a policy, but the one supplied here could. But only Operations team members would be able to invoke that option, and supply the requisite AWS admin profile. So, the workflow becomes... developers can commit policy changes to Git, but they can't deploy them - only issue a PR about them. Then the Operations team receives a PR request through Github, where they review the proposed policy changes, and if they agree, merge the branch and issue the following plugin command to actually deploy them:

`$ sls policy deploy -s dev --profile <aws_admin_profile>``

Anyway, this is what I was thinking about for the 'policy' plugin I had in mind, but these ideas are tentative and may not be completely thought through.

The plugin you mention to do the substituting would be awesome too.. how's it going on being able to share that work?

Not allowed at this time. Hopefully later this year...

Fantastic work though, you've saved me and others countless hours of testing!

Thanks, glad it has been helpful to you.

[jokeyrhyme]

I'm exploring a use case with multi-tenant requirements: resources in a single AWS account, managed by teams from different customers / companies. From the above research (thanks so much, by the way), it's looking increasingly like there's not really a safe way to offer HTTP-mapped Lambda functions in such a multi-tenant scenario.

Assuming we instead create separate AWS sub-accounts for each tenant, and use the IAM Policy templates provided further up, are there any other gotchas relating to using Serverless this way?

[jenyayel]

I cannot figure out how ${project} passed to IAM policy - do I need to define it in serverless.yml?

[ProTip]

I realize this is closed, but just a word of warning: Most IAM policies that can create and attach IAM policies are nearly indistinguishable from a super user policy. The privilege escalation path is pretty straight:

1. Create super user policy
2. Assign it to something you control
3. Be super user

[lamarrh]

@ProTip is exactly right. Has anyone figured out a good way to narrow the iam:* policy so that privilege escalation isn't possible. A potential fix would be by better utilizing the resources section similar to what Rhino did.

Is there a clever way to say iam:* but only for resources provisioned in the template?

[ProTip]

@lamarrh IAM permission boundaries slipped in under my radar but may provide a reasonable path forward: https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_boundaries.html .

By requiring a specific boundary be attached create users and roles, or be applied when attaching policies, it effectively limits the maximum permissions an entity can grant to another entity.

[uclaeagit]

What about trust relationships? That's something I just learned of that I don't yet fully understand, but seems like it could help.

[erikerikson]

At one time we defined a deployment role that was only able to create narrowly defined (i.e. named) IAM roles and only had exactly the rights a particular set of serverless projects needed to deploy and therefore could only pass those rights. The changes on that separate serverless project were minimal and targeted so the division of risk was fairly good. Only allowing the deployment pipeline to use that role was effective.

[jt-helsinki]

Thought I'd post this here. If anyone gets an error which mentions s3:SetBucketEncryption then try this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "apigateway:GET",
                "apigateway:POST",
                "apigateway:PUT",
                "apigateway:DELETE",
                "cloudformation:CreateStack",
                "cloudformation:Describe*",
                "cloudformation:ValidateTemplate",
                "cloudformation:UpdateStack",
                "cloudformation:List*",
                "iam:GetRole",
                "iam:PassRole",
                "iam:CreateRole",
                "iam:DeleteRole",
                "iam:CreateServiceLinkedRole",
                "iam:DetachRolePolicy",
                "iam:PutRolePolicy",
                "iam:AttachRolePolicy",
                "iam:DeleteRolePolicy",
                "lambda:UpdateFunctionCode",
                "lambda:Get*",
                "lambda:CreateFunction",
                "lambda:InvokeFunction",
                "lambda:UpdateFunctionConfiguration",
                "lambda:PublishVersion",
                "lambda:DeleteFunction",
                "lambda:DeleteLayerVersion",
                "lambda:List*",
                "lambda:AddPermission",
                "s3:CreateBucket",
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:GetBucketLocation",
                "s3:ListBucket",
                "s3:PutObject",
                "s3:DeleteBucket",
                "s3:GetEncryptionConfiguration",
                "s3:PutEncryptionConfiguration",
                "logs:Describe*",
                "logs:CreateLogGroup",
                "logs:DeleteLogGroup",
                "events:PutRule",
                "events:DescribeRule",
                "events:PutTargets"
            ],
            "Resource": "*"
        }
    ]
}

Note: the addition of:

"apigateway:POST",
 "s3:GetEncryptionConfiguration",
 "s3:PutEncryptionConfiguration",

I hope it helps someone.

[lamarrh]

So i thought i'd come back and share how i solved this for my use case. This is a policy we created that only allows people to create and manage their own resources with serverless.

---
Version: '2012-10-17'
Statement:
# Allow S3 deployment bucket creation and management
- Effect: Allow
  Action: s3:*
  Resource: arn:aws:s3:::<stack-name>-serverlessdeploymentbucket*
# Allow cloud formation stack creation, management, and template validation
- Effect: Allow
  Action: cloudformation:ValidateTemplate
  Resource: "*" 
- Effect: Allow
  Action: cloudformation:*
  Resource:
  - arn:aws:cloudformation:*:<AWS_AccountId>:changeset/<stack-name>/*
  - arn:aws:cloudformation:*:<AWS_AccountId>:stack/<stack-name>/*
# Allow limited IAM role creation limited to roles in the stack
- Effect: Allow
  Action:
  - iam:GetRole
  - iam:PassRole
  - iam:DeleteRolePolicy
  - iam:CreateRole
  - iam:DeleteRole
  - iam:AttachRolePolicy
  - iam:DetachRolePolicy
  - iam:PutRolePolicy
  Resource: arn:aws:iam::<AWS_AccountId>:role/<StackName>*
# Allow log creation and management
- Effect: Allow
  Action: logs:*
  Resource: arn:aws:logs:*:<AWS_AccountId>:log-group:/aws/lambda/<StackName>*
# Allow lambda creation and management
- Effect: Allow
  Action: lambda:*
  Resource: arn:aws:lambda:*:<AWS_AccountId>:function:<StackName>*
# Allow lambda layer creation and management
- Effect: Allow
  Action: lambda:*
  Resource: arn:aws:lambda:*:<AWS_AccountId>:layer:<StackName>*
# Allow secret creation and management
- Effect: Allow
  Action: secretsmanager:*
  Resource: arn:aws:secretsmanager:*:<AWS_AccountId>:secret:<SecretName>*
# Allow list secrets, necessary to be able to modify secret. Only allows listing, i.e the secret names and descriptions
- Effect: Allow
  Action: secretsmanager:ListSecrets
  Resource: "*" 
# Allow creating and deleting network interfaces for VPC
- Effect: Allow
  Action: 
  - ec2:CreateNetworkInterface
  - ec2:DescribeNetworkInterfaces
  - ec2:DeleteNetworkInterface
  Resource: "*" 

Not 100% complete, but it's easily extensible for other services.

[uclaeagit]

I keep finding new needed permissions. I realize now why servleress recommends to just give full admin access.

[erikerikson]

@uclaeagit that recommendation is meant to reduce friction in getting started, not as production ready guidance.

[uclaeagit]

Well then the path to get to production-ready is a lot of trial and error as you find out what permissions you need.

[eedwards-sk]

I'm a little surprised a project with this many commits and contributions lacks ops documentation for production hardening.

production policy support should ideally be capable of:

• running under roles pertaining to each service (cloudwatch, ec2, etc)

• scoping resources as finely as possible, e.g. using conditions that restrict operations based on tags where possible, and based on naming patterns otherwise

Allowing a service account s3:DeleteBucket with "*" is a really bad idea, for example. I suspect there are numerous users with this exact configuration in production.

[driskell]

Is there any further improvements on this?

The majority of policies seem to include things like iam:AttachRolePolicy which although restricted to the current role can be used to add the Administrators policy. Things like iam:PassRole alongside lambda:CreateFunction are also problematic as you can just pass the Administrators role to the lambda function.

I'm tempted to say it should be documented that it cannot be secured effectively when using CloudFormation - and that a dedicated Sub Organisation with reasonable limits imposed should be used to limit impact.

I guess the best approach could be for the CloudFormation template to be uploaded manually initially, and for that template to automatically include necessary permissions for UPDATE of resources only, as I would assume that can then bypass things like iam:xxx and lambda:CreateFunction since it should theoretically only need lambda:UpdateFunctionCode?

This is my reference for the escalation issues: https://github.com/RhinoSecurityLabs/AWS-IAM-Privilege-Escalation/blob/master/README.md

[erikerikson]

@driskell -it's good that you're keeping an eye out for security. You might be pleased to know that you are able to restrict the roles that can be passed when using the PassRole provision. See: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_passrole.html (if you don't want to read all the text, search for "example 1").

[enricop89]

@hybby Thanks mate! I followed your suggestions and created two main roles:

Deploy Role: https://gist.github.com/enricop89/80a36d6b73417d3ef40bfe00c09a8a35#file-serverless-deploy-role-json

CFRole: https://gist.github.com/enricop89/80a36d6b73417d3ef40bfe00c09a8a35#file-cloudformation-role-json

[linjunpop]

It would be great if some plugin can generate the policies from a serverless service.

[cmendes0101]

It would be great if some plugin can generate the policies from a serverless service.

There is this tool https://github.com/dancrumb/generator-serverless-policy

It's mentioned in a blog from Serverless.com Blog but its not perfect https://serverless.com/blog/abcs-of-iam-permissions/

[sachintha97]

We also created a tool that will help to generate AWS IAM policies: https://github.com/Open-SL/serverless-permission-generator
Please let us know your thoughts.

[joshuaquek]

Awesome, this has been working great for me @sachintha97

Direct link to the working policy generator tool: https://open-sl.github.io/serverless-permission-generator/

[driskell]

@joshuaquek @sachintha97 The generator appears to grant iam PassRole for all roles - along with CreateFunction for any function name - so it seems to offer little additional security as one can use the role to create function with Administrator permissions. Granted, it will make exploitation a little more time-consuming.

Ideally that PassRole would be limited as @erikerikson pointed out to only allow passing of the roles created by the CloudFormation. Additionally it is not limiting policy attachments so could still attach an Administrator policy. There's also no limitation on the API Gateway modifications allowing potential hijacking of other gateways.

Ideally the tool should note that this does not provide secure CloudFormation deployments. I don't think it wise to put such a tool into the public domain as it is as I worry many people will assume it secures the CloudFormation deployment in ways it does not. It does make privilege escalation harder, but by no means secured.

[BrandonE]

I successfully used the instructions on this article after making these modifications.

[driskell]

@BrandonE The modifications made are adding IAM permissions that allow creation of administrator users. So may be defeating the purpose of the original article. It looks like you'd just be able to add a new administrator to serverless.yml and the deployment would still accept it and create that user. I sometimes think it may be more secure to just use the Administrator user, as it's clear what access is given, where with many of these minimal privileges policies it hides the fact that it provides little protection from a genuine attack, so could lead into false sense of security.

[BrandonE]

@driskell The additions allow you to manage roles, not manage IAM users, no?

[driskell]

@BrandonE I guess depending on the restrictions put on place on them but I'm not sure it can be restricted sufficiently. With a "*" as Resource for sure it would let you attach the Administrator policy to any role using PutRolePolicy and the IAM user already is able to assume the deployment role. There might be a way around it with permissions boundaries etc but none of that is mentioned in the articles and I haven't personally worked it out, as I think even with boundaries on all the 3 roles, the CloudFormation still needs to create the Lambda Role and a Function to run as that and so the serverless.yml can inject anything with any permissions still.

I think closest I got was allowing only updates to existing stuff since CloudFormation won't need permissions to things that don't change but it adds a barrier to adding new functions and things.