Add documentation on permissions

[tdhopper]

I use an IAM User with All S3 permissions which allowed me to successfully call pywren create_config --bucket_name YOUR_S3_BUCKET_NAME. Then I tried to run create_role. I got

20:10 $ pywren create_role
config= {'s3': {'bucket': 'BUCKET', 'pywren_prefix': 'pywren.jobs'}, 'account': {'aws_account_id': ID, 'aws_region': 'us-west-2', 'aws_lambda_role': 'pywren_exec_role'}, 'runtime': {'s3_key': 'condaruntime.nomkl_sklearn.tar.gz', 's3_bucket': 'ericmjonas-public'}, 'lambda': {'memory': 1536, 'timeout': 300, 'function_name': 'pywren1'}}
Traceback (most recent call last):
  File "/Users/tdhopper/miniconda2/envs/pywren/bin/pywren", line 11, in <module>
    load_entry_point('pywren', 'console_scripts', 'pywren')()
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/tdhopper/repos/pywren/pywren/scripts/pywrencli.py", line 87, in create_role
    AssumeRolePolicyDocument=json_policy)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/boto3/resources/factory.py", line 520, in do_action
    response = action(self, *args, **kwargs)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(**params)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/botocore/client.py", line 251, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/botocore/client.py", line 537, in _make_api_call
    raise ClientError(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the CreateRole operation: User: arn:aws:iam::X:user/Y is not authorized to perform: iam:CreateRole on resource: arn:aws:iam::X:role/pywren_exec_role

I attached the IAMFullAccess policy to my user and then that command ran successfully. Then I tried to run deploy_lambda and I got

20:17 $ pywren deploy_lambda
Traceback (most recent call last):
  File "/Users/tdhopper/miniconda2/envs/pywren/bin/pywren", line 11, in <module>
    load_entry_point('pywren', 'console_scripts', 'pywren')()
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/tdhopper/repos/pywren/pywren/scripts/pywrencli.py", line 124, in deploy_lambda
    b = lambclient.list_functions()
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/botocore/client.py", line 251, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/botocore/client.py", line 537, in _make_api_call
    raise ClientError(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the ListFunctions operation: User: arn:aws:iam::X:user/Y is not authorized to perform: lambda:ListFunctions

I haven't yet figured out how to get around this. :)

Probably need to add some docs explaining how users might configurer AWS permissions.

[ericmjonas]

Can you show me the contents of your ~/.pywren_config?

[tdhopper]

    aws_account_id: 212595366334
    aws_lambda_role: pywren_exec_role
    aws_region: us-west-2

lambda:
    memory : 1536
    timeout : 300
    function_name : pywren1

s3:
    bucket: xxx-pywren-test
    pywren_prefix: pywren.jobs

runtime:
    s3_bucket: ericmjonas-public
    s3_key: condaruntime.nomkl_sklearn.tar.gz

[ericmjonas]

Are you running the commands as an IAM user with reduced permissions? Creating the role, changing the permissions of the role, and deploying is only ever done as your default boto user. The role management is all there to create the roles/permissions for the lambda process itself.

[tdhopper]

I see. I think I was running them as an IAM user. I was doing this on a computer I'd never used boto from before.

I do my best not to understand AWS permissions so I never have to be responsible for them.

[ericmjonas]

I think that is the 100% correct strategy. I find it concerning that Full Access didn't somehow just do the right thing.