New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation on permissions #27

Closed
tdhopper opened this Issue Nov 4, 2016 · 5 comments

Comments

Projects
None yet
3 participants
@tdhopper
Collaborator

tdhopper commented Nov 4, 2016

I use an IAM User with All S3 permissions which allowed me to successfully call pywren create_config --bucket_name YOUR_S3_BUCKET_NAME. Then I tried to run create_role. I got

20:10 $ pywren create_role
config= {'s3': {'bucket': 'BUCKET', 'pywren_prefix': 'pywren.jobs'}, 'account': {'aws_account_id': ID, 'aws_region': 'us-west-2', 'aws_lambda_role': 'pywren_exec_role'}, 'runtime': {'s3_key': 'condaruntime.nomkl_sklearn.tar.gz', 's3_bucket': 'ericmjonas-public'}, 'lambda': {'memory': 1536, 'timeout': 300, 'function_name': 'pywren1'}}
Traceback (most recent call last):
  File "/Users/tdhopper/miniconda2/envs/pywren/bin/pywren", line 11, in <module>
    load_entry_point('pywren', 'console_scripts', 'pywren')()
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/tdhopper/repos/pywren/pywren/scripts/pywrencli.py", line 87, in create_role
    AssumeRolePolicyDocument=json_policy)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/boto3/resources/factory.py", line 520, in do_action
    response = action(self, *args, **kwargs)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(**params)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/botocore/client.py", line 251, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/botocore/client.py", line 537, in _make_api_call
    raise ClientError(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the CreateRole operation: User: arn:aws:iam::X:user/Y is not authorized to perform: iam:CreateRole on resource: arn:aws:iam::X:role/pywren_exec_role

I attached the IAMFullAccess policy to my user and then that command ran successfully. Then I tried to run deploy_lambda and I got

20:17 $ pywren deploy_lambda
Traceback (most recent call last):
  File "/Users/tdhopper/miniconda2/envs/pywren/bin/pywren", line 11, in <module>
    load_entry_point('pywren', 'console_scripts', 'pywren')()
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/tdhopper/repos/pywren/pywren/scripts/pywrencli.py", line 124, in deploy_lambda
    b = lambclient.list_functions()
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/botocore/client.py", line 251, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/tdhopper/miniconda2/envs/pywren/lib/python2.7/site-packages/botocore/client.py", line 537, in _make_api_call
    raise ClientError(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the ListFunctions operation: User: arn:aws:iam::X:user/Y is not authorized to perform: lambda:ListFunctions

I haven't yet figured out how to get around this. :)

Probably need to add some docs explaining how users might configurer AWS permissions.

@ericmjonas

This comment has been minimized.

Show comment
Hide comment
@ericmjonas

ericmjonas Nov 4, 2016

Collaborator

Can you show me the contents of your ~/.pywren_config?

Collaborator

ericmjonas commented Nov 4, 2016

Can you show me the contents of your ~/.pywren_config?

@tdhopper

This comment has been minimized.

Show comment
Hide comment
@tdhopper

tdhopper Nov 5, 2016

Collaborator
    aws_account_id: 212595366334
    aws_lambda_role: pywren_exec_role
    aws_region: us-west-2

lambda:
    memory : 1536
    timeout : 300
    function_name : pywren1

s3:
    bucket: xxx-pywren-test
    pywren_prefix: pywren.jobs

runtime:
    s3_bucket: ericmjonas-public
    s3_key: condaruntime.nomkl_sklearn.tar.gz
Collaborator

tdhopper commented Nov 5, 2016

    aws_account_id: 212595366334
    aws_lambda_role: pywren_exec_role
    aws_region: us-west-2

lambda:
    memory : 1536
    timeout : 300
    function_name : pywren1

s3:
    bucket: xxx-pywren-test
    pywren_prefix: pywren.jobs

runtime:
    s3_bucket: ericmjonas-public
    s3_key: condaruntime.nomkl_sklearn.tar.gz
@ericmjonas

This comment has been minimized.

Show comment
Hide comment
@ericmjonas

ericmjonas Nov 5, 2016

Collaborator

Are you running the commands as an IAM user with reduced permissions? Creating the role, changing the permissions of the role, and deploying is only ever done as your default boto user. The role management is all there to create the roles/permissions for the lambda process itself.

Collaborator

ericmjonas commented Nov 5, 2016

Are you running the commands as an IAM user with reduced permissions? Creating the role, changing the permissions of the role, and deploying is only ever done as your default boto user. The role management is all there to create the roles/permissions for the lambda process itself.

@tdhopper

This comment has been minimized.

Show comment
Hide comment
@tdhopper

tdhopper Nov 5, 2016

Collaborator

I see. I think I was running them as an IAM user. I was doing this on a computer I'd never used boto from before.

I do my best not to understand AWS permissions so I never have to be responsible for them.

Collaborator

tdhopper commented Nov 5, 2016

I see. I think I was running them as an IAM user. I was doing this on a computer I'd never used boto from before.

I do my best not to understand AWS permissions so I never have to be responsible for them.

@ericmjonas

This comment has been minimized.

Show comment
Hide comment
@ericmjonas

ericmjonas Nov 5, 2016

Collaborator

I think that is the 100% correct strategy. I find it concerning that Full Access didn't somehow just do the right thing.

Collaborator

ericmjonas commented Nov 5, 2016

I think that is the 100% correct strategy. I find it concerning that Full Access didn't somehow just do the right thing.

@ericmjonas ericmjonas modified the milestone: PyWren 1.0 Jan 27, 2017

@shivaram shivaram modified the milestones: v0.2, PyWren 0.1 Mar 9, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment