New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Narrowing the Serverless IAM Deployment Policy #1439

Closed
ghost opened this Issue Jun 29, 2016 · 66 comments

Comments

Projects
None yet
@ghost

ghost commented Jun 29, 2016

I’ve been spending time recently trying to remove Admin rights as a requirement for sls deployments. Still a work in progress, but so far I have this policy that I can attach to any “serverless-agent” AWS user, so that the serverless-agent user is empowered enough to deploy:

    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "cloudformation:Describe*",
                "cloudformation:List*",
                "cloudformation:Get*",
                "cloudformation:PreviewStackUpdate"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "cloudformation:CreateStack",
                "cloudformation:UpdateStack",
                "cloudformation:DeleteStack"
            ],
            "Resource": "arn:aws:cloudformation:*:*:stack/${project}*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "lambda:Get*",
                "lambda:List*",
                "lambda:CreateFunction"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "lambda:AddPermission",
                "lambda:CreateAlias",
                "lambda:DeleteFunction",
                "lambda:InvokeFunction",
                "lambda:PublishVersion",
                "lambda:RemovePermission",
                "lambda:Update*"
            ],
            "Resource": "arn:aws:lambda:*:*:function:${project}*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "lambda:*"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "apigateway:GET"
            ],
            "Resource": [
                "arn:aws:apigateway:*::/restapis"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "apigateway:GET",
                "apigateway:POST",
                "apigateway:PUT",
                "apigateway:DELETE"
            ],
            "Resource": [
                "arn:aws:apigateway:*::/restapis/*/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:PassRole"
            ],
            "Resource": "arn:aws:iam::*:role/*"
        },
        {
            "Effect": "Allow",
            "Action": "kinesis:*",
            "Resource": "arn:aws:kinesis:*:*:stream/${project}*"
        },
        {
            "Effect": "Allow",
            "Action": "iam:*",
            "Resource": "arn:aws:iam::*:role/${project}*"
        },
        {
            "Effect": "Allow",
            "Action": "sqs:*",
            "Resource": "arn:aws:sqs:*:*:${project}*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "events:Put*",
                "events:Remove*",
                "events:Delete*"
            ],
            "Resource": "arn:aws:events:*:*:rule/${project}*"
        }
    ]
}

Right now, I'm focused on a single policy that can deploy to all stages. But some enterprises may need this IAM policy to allow dev and staging deployments, but limit who can deploy to production. So, I've also been experimenting with adding "${stage}" to some of the resource ARNs, but don't have it fully worked out yet. For example:

"Resource": "arn:aws:events:*:*:rule/${project}-${stage}*"

There are still a few places where the permissions could be narrowed further. Specifically, the REST API section allows delete of ALL apis right now. And the lambda permissions are too broad. But I’ve had some annoying technical issues trying to narrow those two sections.

The API Gateway policy is still broad because you must have the 'api-id' in the ARN. But you don't know that until a deployment generates it. So on the surface, seems like a chicken/egg problem to me, but maybe there is a way to supply that api-id, instead of having AWS generate it.

And the lambda permissions are still broad because I can't see the particular Arn it is trying to manipulate to add an event mapping to a lambda, and the obvious ARNs don't work. Maybe there is a way to show the ARN being accessed in serverless, when the deployment fails so that I can add it to the policy, but no luck so far.

As I said, still a work in progress, so use with caution. Will post back any further 'narrowing' as I figure it.

@ghost ghost changed the title from Serverless IAM Deployment Policy to Narrowing the Serverless IAM Deployment Policy Jun 29, 2016

@Depado

This comment has been minimized.

Depado commented Jul 4, 2016

Thanks for the great work, I'll look into that, I also need to narrow down the IAM policy.

@WooDzu

This comment has been minimized.

Contributor

WooDzu commented Jul 4, 2016

Thanks for sharing. I'm going to give it a go a provide some feedback.
My first impressions it that the policy could be narrowed further by including the ${region} in appropriate Resource sections. What do you think?

@tobyhede

This comment has been minimized.

Contributor

tobyhede commented Jul 5, 2016

This is really good, thanks

@WooDzu

This comment has been minimized.

Contributor

WooDzu commented Jul 5, 2016

Here is one change that I did to make it work for me:

    {
      "Effect": "Allow",
      "Action": [
        "apigateway:GET"
      ],
      "Resource": [
        "arn:aws:apigateway:*::/restapis"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "apigateway:GET",
        "apigateway:POST",
        "apigateway:PUT",
        "apigateway:DELETE"
      ],
      "Resource": [
        "arn:aws:apigateway:*::/restapis/*/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "apigateway:GET",
        "apigateway:HEAD",
        "apigateway:OPTIONS"
        "apigateway:PATCH",
        "apigateway:POST",
        "apigateway:PUT",
        "apigateway:DELETE"
      ],
      "Resource": [
        "arn:aws:apigateway:*::/restapis",
        "arn:aws:apigateway:*::/restapis/*"
      ]
    },

2nd-level POST /api/v1/resource didn't work with your IAM but the above would handle it.
It also enables pushing missing Head, Patch and Options methods.

Going further with the restrictions, adding ${region} to all Resources with the exception of * and arn:aws:iam:* seems to work fine eg:

"arn:aws:apigateway:${region}::/restapis",
"arn:aws:apigateway:${region}::/restapis/*"
"arn:aws:lambda:${region}::function:${project}*"

And the lambda permissions are still broad because I can't see the particular Arn it is trying to manipulate to add an event mapping to a lambda, and the obvious ARNs don't work.

This one is particularly nasty but it doesn't seem to work without it.

@c-knowles

This comment has been minimized.

c-knowles commented Jul 6, 2016

You may be able to change PassRole to just role/${project}* to restrict that part further? I found it would also be helpful if serverless populated a variable with the ID of the Gateway to use to restrict the apigateway permissions. Some duplicate effort in #588.

@ecoruh

This comment has been minimized.

ecoruh commented Jul 8, 2016

This is great, thanks. I got it working except one bit. We are using serverless-client-s3 plugin for deploying s3 buckets.

Anyone knows what the minimum set of permissions are for sls client deploy to work?. I tried this in my policy:

    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBuckets",
                "s3:ListObjects",
                "s3:DeleteObjects",
                "s3:CreateBucket",
                "s3:PutBucketWebsite",
                "s3:PutBucketTagging",
                "s3:PutBucketPolicy",
                "s3:PutObject"
            ],
            "Resource": "*"
        }

I scanned the plugin source code to come up with the s3 function list and added them into the policy. But I received this error:

AccessDenied: Access Denied
    at Request.extractError (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/services/s3.js:350:35)
    at Request.callListeners (/usr/local/lib/node_modules/serverless/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
...
From previous event:
    at ServerlessProviderAws.request (/usr/local/lib/node_modules/serverless/lib/ProviderAws.js:72:10)
    at ClientDeploy._processDeployment (/home/eric/BuildAgent/work/3a164dd9bff1550c/ucr/node_modules/serverless-client-s3/index.js:134:24)

This line seems to yield Access Denied:

      return _this.aws.request('S3', 'listBuckets', {}, _this.evt.options.stage, _this.evt.options.region)

even though I had listBuckets in my permission list. What am I missing?

note: I shall refine Resource further once I get it working..

@c-knowles

This comment has been minimized.

c-knowles commented Jul 8, 2016

I think try s3:ListAllMyBuckets. Some of the others don't exist, it's confusing as the AWS CLI/SDK does not use the same names as the IAM actions - see these docs. s3:ListBucket with no s is the action to list the content of a bucket.

@ghost

This comment has been minimized.

ghost commented Jul 8, 2016

I think I have the serverless deployment policy nailed at this point. A bit more testing is in order, but mostly there now. In this second iteration, I've taken a slightly different approach. Instead of putting everything into 1 giant policy, I've broken it down into 5 separate put related policies:

  • s-policy-base.json - Read-only permissions generally required by serverless deployments and relatively harmless via their read-only limitation
  • s-policy-endpoint.json - Write permissions (I.e. update, delete actions etc.) required by sls endpoint deploy
  • s-policy-event.json - Write permissions required by sls event deploy
  • s-policy-function.json - Write permissions required by sls function deploy
  • s-policy-resources.json - Write permissions required by sls resources deploy

FILE: s-policy-base.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "NonDestructiveCoreReaderActionsUsedThroughoutServerless",
            "Effect": "Allow",
            "Action": [
                "cloudformation:Describe*",
                "cloudformation:List*",
                "cloudformation:Get*",
                "cloudformation:PreviewStackUpdate",
                "lambda:Get*",
                "lambda:List*",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSubnets",
                "s3:GetObject",
                "s3:List*",
                "apigateway:GET",
                "iam:List*",
                "iam:Get*",
                "iam:Simulate*",
                "kinesis:Describe*",
                "kinesis:List*",
                "dynamodb:Describe*",
                "dynamodb:List*",
                "sqs:List*"
            ],
            "Resource": "*"
        }
    ]
}

FILE: s-policy-resources.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ResourceBoundCloudFormationWritersForSlsResourcesDeploy",
            "Effect": "Allow",
            "Action": "cloudformation:*",
            "Resource": "arn:aws:cloudformation:${region}:*:stack/${project}-*"
        },
        {
            "Sid": "ResourceDeniedCloudFormationWritersForSlsResourcesDeploy",
            "Effect": "Deny",
            "Action": "cloudformation:*",
            "Resource": "arn:aws:cloudformation:*:*:stack/${project}-${prod}*"
        },
        {
            "Sid": "ResourceBoundIamWritersForSlsFunctionDeploy",
            "Effect": "Allow",
            "Action": "iam:*",
            "Resource": "arn:aws:iam::*:role/${project}-*"
        },
        {
            "Sid": "ResourceDeniedIamWritersForSlsFunctionDeploy",
            "Effect": "Deny",
            "Action": "iam:*",
            "Resource": "arn:aws:iam::*:role/${project}-${prod}*"
        },
        {
            "Sid": "ResourceBoundKinesisStreamWritersForSlsResourcesDeploy",
            "Effect": "Allow",
            "Action": "kinesis:*",
            "Resource": "arn:aws:kinesis:${region}:*:stream/${project}-*"
        },
        {
            "Sid": "ResourceDeniedKinesisStreamWritersForSlsResourcesDeploy",
            "Effect": "Deny",
            "Action": "kinesis:*",
            "Resource": "arn:aws:kinesis:*:*:stream/${project}-${prod}*"
        },
        {
            "Sid": "ResourceBoundDynamoDbStreamWritersForSlsResourcesDeploy",
            "Effect": "Allow",
            "Action": "dynamodb:*",
            "Resource": "arn:aws:dynamodb:${region}:*:table/${project}-*"
        },
        {
            "Sid": "ResourceDeniedDynamoDbStreamWritersForSlsResourcesDeploy",
            "Effect": "Deny",
            "Action": "dynamodb:*",
            "Resource": "arn:aws:dynamodb:*:*:table/${project}-${prod}*"
        },
        {
            "Sid": "ResourceBoundSqsClientForSqsResourcesDeploy",
            "Effect": "Allow",
            "Action": "sqs:*",
            "Resource": "arn:aws:sqs:${region}:*:${project}-*"
        },
        {
            "Sid": "ResourceDeniedSqsClientForSqsResourcesDeploy",
            "Effect": "Deny",
            "Action": "sqs:*",
            "Resource": "arn:aws:sqs:*:*:${project}-${prod}*"
        }
    ]
}

FILE: s-policy-function.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PotentiallyDestructiveLambdaActionsRequiringAllAccessResourceSpecForSlsFunctionDeploy",
            "Effect": "Allow",
            "Action": [
                "lambda:CreateFunction"
            ],
            "Resource": "*"
        },
        {
            "Sid": "ResourceBoundLambdaExecutionRolePassingForSlsFunctionDeploy",
            "Effect": "Allow",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::*:role/${project}-*"
        },
        {
            "Sid": "ResourceDeniedLambdaExecutionRolePassingForSlsFunctionDeploy",
            "Effect": "Deny",
            "Action": "iam:PassRole",
            "Resource": "arn:aws:iam::*:role/${project}-${prod}*"
        },
        {
            "Sid": "ResourceBoundLambdaWritersForSlsFunctionDeploy",
            "Effect": "Allow",
            "Action": "lambda:*",
            "Resource": "arn:aws:lambda:${region}:*:function:${project}-*"
        },
        {
            "Sid": "ResourceDeniedLambdaWritersForSlsFunctionDeploy",
            "Effect": "Deny",
            "Action": "lambda:*",
            "Resource": "arn:aws:lambda:${region}:*:function:${project}-${prod}*"
        }
    ]
}

FILE: s-policy-endpoint.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ResourceBoundApiGatewayWritersForSlsEndpointDeploy",
            "Effect": "Allow",
            "Action": "apigateway:*",
            "Resource": [
                "arn:aws:apigateway:${region}::/restapis",
                "arn:aws:apigateway:${region}::/restapis/${api_id}*"
            ]
        },
        {
            "Sid": "ResourceDeniedApiGatewayWritersForSlsEndpointDeploy",
            "Effect": "Deny",
            "Action": "apigateway:*",
            "Resource": [
                "arn:aws:apigateway:*::/restapis/${prod_api_id}/*"
            ]
        }
    ]
}

FILE: s-policy-event.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PotentiallyDestructiveLambdaActionsRequiringAllAccessResourceSpecForSlsEventDeploy",
            "Effect": "Allow",
            "Action": [
                "lambda:CreateEventSourceMapping",
                "lambda:DeleteEventSourceMapping",
                "lambda:UpdateEventSourceMapping"
            ],
            "Resource": "*"
        },
        {
            "Sid": "ResourceBoundEventsInvokingLambdaForSlsEventDeploy",
            "Effect": "Allow",
            "Action": "events:*",
            "Resource": "arn:aws:events:${region}:*:rule/${project}-*"
        },
        {
            "Sid": "ResourceDeniedEventsInvokingLambdaForSlsEventDeploy",
            "Effect": "Deny",
            "Action": "events:*",
            "Resource": "arn:aws:events:*:*:rule/${project}-${prod}*"
        },
        {
            "Sid": "ResourceBoundLambdaWritersForSlsEventDeploy",
            "Effect": "Allow",
            "Action": "lambda:AddPermission",
            "Resource": "arn:aws:lambda:${region}:*:function:${project}-*"
        },
        {
            "Sid": "ResourceDeniedLambdaWritersForSlsEventDeploy",
            "Effect": "Deny",
            "Action": "lambda:AddPermission",
            "Resource": "arn:aws:lambda:*:*:function:${project}-${prod}*"
        }
    ]
}

To use, manually replace all ${key} variables for your specific project. Then copy all 5 variable-expanded policy documents above into separate AWS managed policies. Then attach all 5 polices to a user account needing serverless deployment capability. (Note: you could limit users whom you don't want to deploy resources, for example, by excluding the s-policy-resources.json policy from their user, if that is useful to you.)

First caveat, these policies enable you to deny deploying to production, while allowing deployment to other stages. You may want only a designated user (or Jenkins) to be the only party privileged enough to deploy to production. In which case, these "Deny" blocks that exclude "${prod}*" resource access become very useful. So, set ${prod} equal to whatever you call production. I.e. prod="prod" to protect production from harm. But, you ask, "what if you want to allow deployment to production?" To do that, set prod="open_season_on_prod". This will deny deployment to a stage named 'open_season_on_prod', but will allow deployment to 'prod' via the prior Allow blocks. So, set prod="prod" to generate policies that can't harm production. And set prod='open_season_on_prod' to allow production to be targeted via the policy. (Note: alternatively a template with conditional inclusion could be used to remove the Deny blocks when generating a production policy. But that assumes a serverless plugin.. more on that later.)

Second caveat, after much testing and research, I finally figured out why some of my Actions were failing for ARNs I knew I specified correctly in the policy. The reason is, there are 4 actions that Serverless uses that do not allow ARN-limited resource specifications, even though intuitively they should. They are:

       {
            "Effect": "Allow",
            "Action": [
                "lambda:CreateEventSourceMapping",
                "lambda:DeleteEventSourceMapping",
                "lambda:UpdateEventSourceMapping",
                "lambda:CreateFunction"
            ],
            "Resource": "*"
        },

These 4 lambda actions only work if you pass '*' for the resource. Trying to scope limit them to a ${project} ARN (as I was initially attempting) breaks the deployment. So, these are potentially destructive actions that these policies allow, because AWS gives us no choice but to allow them without appropriate limitation, as documented here:

http://docs.aws.amazon.com/lambda/latest/dg/lambda-api-permissions-ref.html

Third caveat, the s-policy-endpoint.json policy document has a special case: the Api Gateway "api-id" is not generated until after you deploy the API. This leaves 2 choices: 1) allow full access to all ApiGateway APIs, even APIs you shouldn't touch, or 2) generate the rest API api-id separately, and include it in the s-policy-endpoint.json. This ensures you only have write permissions to the APIs you should. You can do the latter using this command:

api_id=$(aws apigateway create-rest-api \
   --name ${project} \
   --description "${project} API" \
   --output text \
   --query 'id')
echo "> API ID is: $api_id"

Citation: http://codurance.com/2016/05/25/aws-api-gateway/

"But sls endpoint deploy is suppose to create the rest API", you say. Not necessarily. This is only provisioning an API-id, not deploying the full API. And Serverless will just reuse the api_id generated above, when the sls endpoint deploy command is first run. So, generate the api_id (as above), plug it into your endpoint-policy document and then your policy will be constrained to work only with your serverless-generated API.

As you might imagine, a Serverless "policy" plugin to manage all this would make a lot of sense. And I've actually started building a Serverless "policy" plugin to automate the variable substitution, api-gateway api-id generation, ${prod} variable setting, and aws deployment of these 5 policies. However, legal red-tape must be cut before I can share.

@WooDzu

This comment has been minimized.

Contributor

WooDzu commented Jul 9, 2016

This is absolutely outstanding research and I think it makes a lot of sense to split into multiple policies for multi-user deployments. I yet need to try 1.0-alpha but do you think these policies would also be appropriate for CloudFormation-based deployments?

@ghost

This comment has been minimized.

ghost commented Jul 9, 2016

Thanks @WooDzu! I haven't tried 1.0-alpha yet either - been too heads down on this thing. I hope to get the chance in the next couple weeks to give 1.0-alpha a good tire kicking. Meanwhile, I don't really know how well these policies will translate.

@ploopaltar

This comment has been minimized.

ploopaltar commented Jul 18, 2016

So we are using these permissions for the s3 client:

Permissions

s3:GetObject
s3:GetObjectVersion
s3:PutObject
s3:DeleteObject
s3:CreateBucket
s3:DeleteBucket
s3:ListBucket
s3:ListBucketVersions
s3:ListAllMyBuckets
s3:GetBucketVersioning
s3:PutBucketVersioning
s3:GetBucketLocation
s3:PutBucketPolicy
s3:GetBucketWebsite
s3:PutBucketWebsite
s3:DeleteBucketWebsite

@ghost

This comment has been minimized.

ghost commented Jul 19, 2016

Are these S3 policies needed for runtime, deployment time, or both? This thread is really only concerned with permissions required to deploy a serverless application. Are you finding these S3 policies are also necessary to deploy your serverless app?

@str3tch

This comment has been minimized.

str3tch commented Jul 19, 2016

Hi Ron,

Yes, we need the s3 permissions to deploy our serverless application as we are using the serverless-client-s3 plugin to deploy files to s3, along with the lambda code. I think this is a quite common use case. So having these permissions helps us.

cheers,
Chris

@ghost

This comment has been minimized.

ghost commented Jul 19, 2016

Thanks @str3tch for the clarification. This has got me thinking out loud a bit, so forgive the stream of consciousness style... but I'm questioning whether it would be best to co-locate all possible permissions used by serverless, its plugins, your application code etc. That would certainly be simpler, but that sort of union problem might be hard to maintain, and would force everyone to use the union of all possible permissions which would often be too broad. There is a distinction between the permissions required by "serverless-core" versus incremental permissions required by serverless plugins and your own serverless application code. And even those incremental permissions could be further delineated according to which phase of deployment requires them -- though that might be going too far and introducing complexity. Sounding more and more like the domain of a serverless plugin and a template engine to dynamically generate and deploy the IAM policies you need based on the contents of your CloudFormation and plugin configurations. @str3tch and @ploopaltar, what are you thoughts?

@ploopaltar

This comment has been minimized.

ploopaltar commented Jul 20, 2016

I guess one of the challenges we had and still have is that there are some permissions which we feel serverless does not need but seems to be required for the deployment process using the s3-client.

As a starting point clarity around why each permission is needed and at which part of the deployment process would help get us to the point where we can have permissions grouped according to the phase of deployments.

@str3tch

This comment has been minimized.

str3tch commented Jul 21, 2016

@xtenix That's an interesting idea, but the problem I would face with getting that through here at work is that the engine would require IAM role create privileges to dynamically create new policies. And we're trying really hard here to restrict those, and not have service accounts have those elevated rights. I think the approach that you've taken is fantastic, split them up into separate policies. The plugin you mention to do the substituting would be awesome too.. how's it going on being able to share that work?

Fantastic work though, you've saved me and others countless hours of testing! 👍

@ghost

This comment has been minimized.

ghost commented Jul 22, 2016

Hi @str3tch

the engine would require IAM role create privileges to dynamically create new policies

Yes, I agree and I consider that as a challenging workflow issue for how policy changes are routed to proper authorities for review and deployment. In short, the AWS profile you are using to deploy your serverless app cannot be the same one used to deploy these policies. That would fundamentally defeat the purpose, because that AWS profile could choose to create an admin policy, and that opens up a back door.

To solve this problem, I've thought of 2 distinct but related approaches:

  1. "Expand-And-Pass":

Don't automatically deploy any policies, only create and write them locally (similar to the -c option on 'resources'). The policy templates would be expanded (variable substitution), either manually or automatically via plugin. A plugin could expand the policy templates and pass these intermediate files to an AWS admin for manual deployment. This will, of course, imply some communication overhead, but it works. For example:

$ sls policy deploy -s dev -o s3://bucket/folder

  1. "Expand-And-Administer":

Similar to option #1, but the expanded policy is automatically deployed via a separate and more privileged AWS profile. To afford this, introduce a "--profile" option to the plugin's "deploy" command that expects an admin key/secret. Serverless' standard (pre-existing) AWS profile would not be able to deploy a policy, but the one supplied here could. But only Operations team members would be able to invoke that option, and supply the requisite AWS admin profile. So, the workflow becomes... developers can commit policy changes to Git, but they can't deploy them - only issue a PR about them. Then the Operations team receives a PR request through Github, where they review the proposed policy changes, and if they agree, merge the branch and issue the following plugin command to actually deploy them:

`$ sls policy deploy -s dev --profile <aws_admin_profile>``

Anyway, this is what I was thinking about for the 'policy' plugin I had in mind, but these ideas are tentative and may not be completely thought through.

The plugin you mention to do the substituting would be awesome too.. how's it going on being able to share that work?

Not allowed at this time. Hopefully later this year...

Fantastic work though, you've saved me and others countless hours of testing!

Thanks, glad it has been helpful to you.

@flomotlik flomotlik added this to the v1.0 milestone Aug 5, 2016

@flomotlik flomotlik modified the milestones: v1.0.0-beta.3, v1.0 Aug 19, 2016

@flomotlik flomotlik modified the milestones: v1.0.0-rc.1, v1.0 Sep 6, 2016

@jokeyrhyme

This comment has been minimized.

jokeyrhyme commented Sep 14, 2016

I'm exploring a use case with multi-tenant requirements: resources in a single AWS account, managed by teams from different customers / companies. From the above research (thanks so much, by the way), it's looking increasingly like there's not really a safe way to offer HTTP-mapped Lambda functions in such a multi-tenant scenario.

Assuming we instead create separate AWS sub-accounts for each tenant, and use the IAM Policy templates provided further up, are there any other gotchas relating to using Serverless this way?

@AlanTurner

This comment has been minimized.

AlanTurner commented Jun 28, 2017

Hi trying to deploy a basic templated project (nodejs) to AWS. I went from v1.11.x to v1.16.x - I now get the error

The CloudFormation template is invalid: User: arn:aws:iam::XXXX:user/myuser is not authorized to perform: cloudformation:ValidateTemplate

The permission is:
{ "Effect": "Allow", "Action": [ "cloudformation:*" ], "Resource": [ "arn:aws:cloudformation:*:*:stack/myuser-*/*” ] },

I don't see any reference to validateTemplate in the minimum requirements listed above... what am I missing? Thanks

@HyperBrain

This comment has been minimized.

Member

HyperBrain commented Jun 28, 2017

I think the cloudformation:ValidateTemplate is missing in the minimum requirements. Since one of the last versions SLS checks the template before sending it to CF. So the permission is needed now.

@ramote

This comment has been minimized.

ramote commented Jun 28, 2017

Indeed cloudformation:ValidateTemplate is now also needed since one of the recent versions.
For now I had to give cloudformation:ValidateTemplate access to '*' resources, given it access only to the serverless services wasn't enough. Is there a more restrictive way to add this?

@tenKinetic

This comment has been minimized.

tenKinetic commented Jun 28, 2017

You're missing the action cloudformation:ValidateTemplate.

As Frank said, recently a validation step has been added to deployment.

If you want a minimal approach rather than cloudformation:* you can add it like I did:

{

"Version": "2012-10-17",
"Statement": [
    {
        "Effect": "Allow",
        "Action": [
            "cloudformation:Describe*",
            "cloudformation:Get*",
            "cloudformation:PreviewStackUpdate",
            "cloudformation:List*",
            "cloudformation:ValidateTemplate"
        ],
        "Resource": "*"
    },
    {
        "Effect": "Allow",
        "Action": [
            "cloudformation:CreateStack",
            "cloudformation:UpdateStack",
            "cloudformation:DeleteStack"
        ],
        "Resource": "arn:aws:cloudformation:*:*:stack/*"
    }
]

}

Which also doesn't apply it to all resources.
I'm sure I used a predefined policy and modified it to arrive at that one.
It's not my only policy either so it isn't necessarily the whole picture for all of serverless but it does take care of the validation step.

@ramote

This comment has been minimized.

ramote commented Jun 28, 2017

You can be even more restrictive, the only thing that needs a "Resource": "*" is "cloudformation:ValidateTemplate".

We use something like this:

{
    "Version": "2012-10-17",
    "Statement": [
    {
        "Action": [
            "cloudformation:DescribeStackResources",
            "cloudformation:CreateStack",
            "cloudformation:DescribeStackEvents",
            "cloudformation:DescribeStackResource",
            "cloudformation:UpdateStack",
            "cloudformation:DescribeStacks",
            "cloudformation:DeleteStack",
            "cloudformation:ValidateTemplate"
        ],
        "Effect": "Allow",
        "Resource": [
            "arn:aws:cloudformation:us-east-1:xxxx:stack/<service_name>*",
            "arn:aws:cloudformation:us-east-1:xxxx:stack/<other_service_name>*"
        ]
    },
    {
        "Action": [
            "cloudformation:ValidateTemplate"
        ],
        "Effect": "Allow",
        "Resource": [
            "*"
        ]
    }
}```

the service name is the service name defined in your serverless.yaml. You can add all your services there.
@erikerikson

This comment has been minimized.

Member

erikerikson commented Jun 28, 2017

Thanks everyone for the update!

I've been stuck on 1.11.0 for a bit due to a bug.

@anthonylau

This comment has been minimized.

anthonylau commented Jun 29, 2017

The policy below allow deployment bucket, api gateway, cron, vpc usage

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "cloudformation:Describe*",
        "cloudformation:List*",
        "cloudformation:Get*",
        "cloudformation:PreviewStackUpdate",
        "cloudformation:CreateStack",
        "cloudformation:UpdateStack"
      ],
      "Resource": "arn:aws:cloudformation:<region>:<account_no>:stack/<service_name>*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "cloudformation:ValidateTemplate"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:Get*",
        "s3:List*"
      ],
      "Resource": [
        "arn:aws:s3:::<your_deployment_bucket>"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:*"
      ],
      "Resource": [
        "arn:aws:s3:::<your_deployment_bucket>/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "logs:DescribeLogGroups"
      ],
      "Resource": "arn:aws:logs:<region>:<account_no>:log-group::log-stream:*"
    },
    {
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:DeleteLogGroup",
        "logs:DeleteLogStream",
        "logs:DescribeLogStreams",
        "logs:FilterLogEvents"
      ],
      "Resource": "arn:aws:logs:<region>:<account_no>:log-group:/aws/lambda/<service_name>*:log-stream:*",
      "Effect": "Allow"
    },
    {
      "Effect": "Allow",
      "Action": [
        "iam:GetRole",
        "iam:PassRole",
        "iam:CreateRole",
        "iam:DeleteRole",
        "iam:DetachRolePolicy",
        "iam:PutRolePolicy",
        "iam:AttachRolePolicy",
        "iam:DeleteRolePolicy"
      ],
      "Resource": [
        "arn:aws:iam::<account_no>:role/<service_name>*-lambdaRole"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "apigateway:GET",
        "apigateway:POST",
        "apigateway:PUT",
        "apigateway:DELETE"
      ],
      "Resource": [
        "arn:aws:apigateway:<region>::/restapis"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "apigateway:GET",
        "apigateway:POST",
        "apigateway:PUT",
        "apigateway:DELETE"
      ],
      "Resource": [
        "arn:aws:apigateway:<region>::/restapis/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "lambda:GetFunction",
        "lambda:CreateFunction",
        "lambda:DeleteFunction",
        "lambda:UpdateFunctionConfiguration",
        "lambda:UpdateFunctionCode",
        "lambda:ListVersionsByFunction",
        "lambda:PublishVersion",
        "lambda:CreateAlias",
        "lambda:DeleteAlias",
        "lambda:UpdateAlias",
        "lambda:GetFunctionConfiguration",
        "lambda:AddPermission",
        "lambda:InvokeFunction"
      ],
      "Resource": [
        "arn:aws:lambda:*:<account_no>:function:<service_name>*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSubnets",
        "ec2:DescribeVpcs"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "events:Put*",
        "events:Remove*",
        "events:Delete*",
        "events:Describe*"
      ],
      "Resource": "arn:aws:events::<account_no>:rule/<service_name>*"
    }
  ]
}

Thanks @MitchellBurton for finding the issue

@MitchellBurton

This comment has been minimized.

MitchellBurton commented Jul 6, 2017

I had a slight issue with the above. I was getting:

An error occurred while provisioning your stack: TestLambdaFunction - User: arn:aws:iam::<account_number>:user/serverless-deploy-user is not authorized to perform: lambda:GetFunction on resource: arn:aws:lambda:us-east-1:<account_number>:function:test-function-dev-test.

I had to change the resource in the lambda section from:
arn:aws:lambda::<account_no>:function:<service_name>*
to:
arn:aws:lambda:*:<account_no>:function:<service_name>*

Note the extra * in the middle. Cloudformation doesn't like you leaving out the region apparently.

@NicolasRitouet

This comment has been minimized.

NicolasRitouet commented Jul 7, 2017

if anyone is interested, here's a Cloudformation stack (yaml) to automatically create a user and a couple of policies to deploy using serverless:
https://gist.github.com/NicolasRitouet/7b8367b79734e01deb36ef707dae91a4

@dinvlad

This comment has been minimized.

dinvlad commented Jul 12, 2017

@anthonylau unfortunately, iam:PutRolePolicy allows to inline any policy into associated role. So either one needs to create managed policies outside of this template first (e.g. through a "superuser" template) and then attach them to function roles, or live with the fact that the deployment role could be abused to grant unrestricted permissions to function roles.

@kenjinp

This comment has been minimized.

kenjinp commented Aug 24, 2017

@dinvlad could you elaborate on this 'superuser' strategy?

@dinvlad

This comment has been minimized.

dinvlad commented Aug 25, 2017

@kenjinp iam:PutRolePolicy allows you to attach any inline policy, including Allow * on *. Instead, we could pre-create necessary roles in a different template, and only then pass them to the serverless template. Granted, it would be more cumbersome, and would be harder to automate (as creation of the 'roles' template still needs admin privileges to create all of those roles and policies). But that may be a necessary trade-off (i.e. make creation of roles and policies a supervised step that requires human approval).

@hybby

This comment has been minimized.

hybby commented Oct 30, 2017

I didn't want to grant iam:PutRolePolicy either.

Turns out you can get away with creating a locked down role (and associated Managed Polic(y|ies)) for Serverless Lambda Functions and specifying it using the role configuration option in serverless.yml. The only IAM permission that Serverless then needs is iam:PassRole on that role.

You miss out on being able to use iamRoleStatements in serverless.yml, but if you manage IAM Roles and Policies meaningfully then you probably don't want them there anyways.

@alexbielen

This comment has been minimized.

alexbielen commented Jan 17, 2018

Any reason why cloudformation:ValidateTemplate needs access to Resource: "*"?

@anthonylau

This comment has been minimized.

anthonylau commented Jan 22, 2018

@alexbielen just lazy as it just read and for validation only

@mafrosis

This comment has been minimized.

mafrosis commented Feb 4, 2018

@hybby Can you elaborate a bit on how your setup ended up working exactly please?

The role config option provides a runtime role for the AWS lambda functions, but most of this ticket is about the IAM perms for a serverless deployment user. I'm trying to setup my serverless usage the way you describe, but it's not clear which role is which (runtime or deployment).

@hybby

This comment has been minimized.

hybby commented Feb 5, 2018

@mafrosis - I ended up with two IAM roles. One for the Lambda execution role (role), which I granted permissions that the Lambda required to operate. The second role is assigned to the AWS CloudFormation stack created by Serverless and dictates what resources the stack is allowed to manage. It's referenced in serverless.yaml as cfnRole (see the serverless.yml reference for more info).

The user of Serverless has to have permission to iam:PassRole to the cfnRole, which has to have a trust relationship to be used by the CloudFormation service.

Hope this helps.

@mafrosis

This comment has been minimized.

mafrosis commented Feb 6, 2018

@hybby Thanks for the info. This is exactly the setup I'm trying to acheive.

  • I have a role named "deployrole", with sts:AssumeRole on cloudformation.amazonaws.com.
  • Attached to "deployrole" is a policy with all relevant permissions.
  • The cfnRole field in serverless.yml is set to the role ARN of "deployrole".
  • The user running sls deploy is an AWS admin - so can PassRole by default.

Everything looks right.. And it doesn't work :) I get: Role arn:aws:iam::00000000000:role/deployrole is invalid or cannot be assumed.

Any ideas? Does this sound correct according to your working model?

@hybby

This comment has been minimized.

hybby commented Feb 6, 2018

@mafrosis - My setup has three IAM roles:

Serverless "deploy" role

A "deploy" role, assumed by Jenkins to perform the serverless deploy command. This role is permitted to be iam:AssumeRole'd from my Jenkins instance and is allowed to perform the following actions:

  • Read/Write access to my sole deployment bucket:
    • s3:GetBucketLocation
    • s3:ListBucket
    • s3:GetObject*
    • s3:PutObject
    • s3:DeleteObject
  • Read-only access to CloudFormation
    • cloudformation:Describe*
    • cloudformation:List*
    • cloudformation:Get*
    • cloudformation:PreviewStackUpdate
    • cloudformation:ValidateTemplate
  • Create-access to specifically named stacks
    • cloudformation:CreateStack
    • cloudformation:UpdateStack
    • cloudformation:DeleteStack
  • The ability to assume the Serverless CloudFormation IAM role (see below)
    • iam:PassRole
  • The ability to assume the Lambda Execution IAM role
    • iam:PassRole
  • Various Lambda permissions (this is to allow "deploy one function only" Serverless deployments):
    • lambda:Get*
    • lambda:List*
    • lambda:UpdateFunctionCode
    • lambda:UpdateFunctionConfiguration
    • lambda:InvokeFunction
  • Couple of API Gateway permissions

This role / policy is assigned to the user of the Serverless tool - in my case, mostly Jenkins.

Serverless "cloudformation" role

This role is assigned to the CloudFormation stack that's created by the user of the deployment role. It details

The main thing you need is a trust relationship set up on cloudformation.amazonaws.com to allow CloudFormation to assume the role. When you specify this role as cfnRole, the stack is effectively created with the IAM Role of this role, and subsequent updates depend on the permissions granted to it rather than the principal of the user who invoked the update.

The other thing you'll need is this role to be allowed to perform iam:PassRole of your Lambda Execution role so that Lambdas can be created with the correct IAM role. This is important because it's this CloudFormation role that needs the permissions to pass the role, not your admin user.

The body of this role's policy should be "the list of things you want Serverless to be able to create as part of one of its CloudFormation stacks". This depends on what features of Serverless you're working with, but for me it was mainly around permission to manage Lambdas, API Gateway, CloudWatch Log Groups, deletion of ENIs (to allow cleanup of Lambdas configured inside a VPC), etc... This policy will probably get pretty big and require a bit of trial and error to make sure you hit everything you want.

My advice would be to test both launching of stacks and deletion of stacks to make sure you don't end up with stacks that can't be properly cleaned down because you missed a "delete" permission.

Lambda Execution Role

This is the role that's assigned to the Lambdas created by Serverless. This has to be created with a trust relationship of lambda.amazonaws.com which allows the Lambdas to assume the role.

The body of this role's policy is "anything you want the Lambda to be able to access", so I've got things like access to specific S3 buckets, creation and deletion of ENIs (to allow configuration of a Lambda inside a VPC), log group permissions, kinesis perms, SQS perms, SNS perms... etc...

Hope this helps. It was quite a ride, but it seems to work quite neatly once you get your head around the Deployer -> CloudFormation -> Lambda flow of events. Drop me a line if I can help any more.

@ali-himindz

This comment has been minimized.

ali-himindz commented Mar 14, 2018

@hybby just wondering if you thought of creating a cloudformation template for generating this policy?
I am working on automating this whole process to auto generate these policies along with S3 bucket to place the artifact and creation of aws code build.
Just searching if someone has already worked on similar.

@bsamuel-ui

This comment has been minimized.

bsamuel-ui commented Apr 16, 2018

I thought I'd post a first cut of a cloudformation template. Thanks to @hybby for his notes, my only departures are:

  • specifying ARNs
  • add iam:GetRole for the serverless role.
  • read access to CloudFormation is limited to the given project / stage because we can.

I've only done minimal testing so far, but it will deploy an (existing) project.

(Added API gateway, CW events and CW logs permissions mostly stolen from @anthonylau )

(Added an output field if you want to use a role with cloudformation, or you can use the user directly.)

@bsamuel-ui

This comment has been minimized.

bsamuel-ui commented Apr 16, 2018

Any reason why cloudformation:ValidateTemplate needs access to Resource: "*"?

@alexbielen Sometimes AWS will document the IAM requirements of an API call, though I don't see it here. We can infer, though, that ValidateTemplate itself passed only a body is not actually touch any CF resources. So when CF is internally calling IAM to validate the call, it's passing * as the "resource" being affected, so we have to authorize *. It's designed that way because IAM only knows about named resources in AWS, not, as in this case, the parameters to an API call.

@uclaeagit

This comment has been minimized.

uclaeagit commented Nov 8, 2018

For what it's worth, this is the final list of IAM permissions I had to use to get serverless deploy to work with AWS CodeBuild for deploying node lambdas:

{
    "Sid": "NonDestructiveCoreReaderActionsUsedThroughoutServerless",
    "Effect": "Allow",
    "Action": [
        "cloudformation:CreateStack",
        "cloudformation:DescribeStacks",
        "cloudformation:DescribeStackEvents",
        "cloudformation:DescribeStackResource",
        "cloudformation:ValidateTemplate",
        "cloudformation:UpdateStack",
        "cloudformation:ListStacks",
        "iam:GetRole",
        "lambda:UpdateFunctionCode",
        "lambda:UpdateFunctionConfiguration",
        "lambda:GetFunctionConfiguration",
        "lambda:ListVersionsByFunction",
        "lambda:AddPermission",
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:ListBucket",
        "s3:PutObject"
    ],
    "Resource": "*"
},

FYI this list may not be complete. Different conditions seem to trigger new permissions needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment