New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serverless Stack Update to IAM #108

Open
jayair opened this Issue Jul 12, 2017 · 30 comments

Comments

Projects
None yet
@jayair
Member

jayair commented Jul 12, 2017

Update July 19

The guide and the sample projects have been updated. A few links for reference:


What is going on

We are updating the Serverless backend API to use IAM as an authorizer and React.js app to make signed API requests.

What is going to happen

Over the next few days we'll post the chageset for the proposed update to give people a chance to review it. And then we'll update Serverless-Stack.com and all the code for the demo project. We'll still leave the changeset up and tag the repo in case people still want to look at the older version of the tutorial.

What should I do

If you are currently working through the tutorial there will be a changeset to refer to. And use this comment thread for help if you have any questions.

Why the change

The current version of the tutorial uses the User Pool directly as an authorizer for the backend. While this works, the recommended way to do authorization in AWS is by using the IAM. All AWS API requests must be securely signed with Signature Version 4 using IAM credentials. Serverless Framework added support for IAM as an authorizer relatively recently. To use IAM as an authorizer we need to use our Cognito Identity Pool to grant our users temporary IAM credentials. This setup is also necessary in the case where you would like to add Google or Facebook as a way to authenticate your users.

Is this a big change

It is not a very big change to the code base of our demo project. But the flow is a little different and the two setups (the current one and the IAM one) cannot work together at the same time. Also, IAM as an authorizer is more secure but it can be a lot more confusing to set up and difficult to work through.


As always feel free to contact us via Twitter (@fanjiewang or @jayair) or email if you have any questions.

@jayair jayair added the Discussion label Jul 12, 2017

@dbeja

This comment has been minimized.

dbeja commented Jul 13, 2017

That's great! Will you also include an example with a Social Identity Provider (Google, Facebook,...)?

@jayair

This comment has been minimized.

Member

jayair commented Jul 13, 2017

@dbeja It's not in this update but we will work on that after we roll this out. We haven't been able to write about the social logins pretty much because we needed to do make this update first.

@mmiinnovations

This comment has been minimized.

mmiinnovations commented Jul 13, 2017

Thanks! This is great.

@jayair

This comment has been minimized.

Member

jayair commented Jul 13, 2017

Proposed Chageset (WIP)

https://github.com/AnomalyInnovations/serverless-stack-com/compare/dev

The new changes are being worked on in the above brach. Feel free to take an early peek.

@larrybek

This comment has been minimized.

larrybek commented Jul 14, 2017

Waiting for your update !!! You're doing great work.

@bharloe

This comment has been minimized.

bharloe commented Jul 14, 2017

Love this tutorial. Any idea of a specific time frame we can expect these changes to get published?

@jayair

This comment has been minimized.

Member

jayair commented Jul 14, 2017

@bharloe Yeah they are just about ready to be published. We just wanted to make sure we give folks who are going through the tutorial currently a bit of a heads up.

Ideally, it should be up early next week.

@bharloe

This comment has been minimized.

bharloe commented Jul 14, 2017

Thanks, looking forward to it!

@zamirkhan

This comment has been minimized.

zamirkhan commented Jul 14, 2017

Thanks for the heads up! For those who have already worked through most of the tutorial and made some customizations for their own specific needs, do these changes imply starting from scratch on the API side of things or is it something that can be migrated to with relative ease?

@jayair

This comment has been minimized.

Member

jayair commented Jul 14, 2017

@zamirkhan Yeah you should be able to migrate. It's just a few changes to different parts of the setup.

@muescha

This comment has been minimized.

muescha commented Jul 16, 2017

i would like to start the tutorial - if it is ready to publish, why not publish it now and make the old tutorial available via subfolder / subdomain...

@jayair

This comment has been minimized.

Member

jayair commented Jul 16, 2017

@muescha It's almost ready. Just need to wrap up the code samples for all the chapters. I'm aiming for Wednesday.

@CharithW

This comment has been minimized.

CharithW commented Jul 18, 2017

This is great.
I have struggling through the net to find proper way to integrate with apigClient sdk to my react app.
Hopefully you will be able to release this soon.
Thank you for the great work :)

@jayair

This comment has been minimized.

Member

jayair commented Jul 19, 2017

Serverless Stack has been updated to use IAM as an authorizer. A few links for reference:

@huatzhi

This comment has been minimized.

huatzhi commented Jul 26, 2017

Will this guide include how to write tests for the system in the future? (e.g. with mocha or jasmine)

@larrybek

This comment has been minimized.

larrybek commented Jul 26, 2017

@jayair wanted to ask you if you could help me with an issue I tried to upload S3 bucket nested folder with IAM example that you have provided but Im getting Network Forbidden. Do you have some ideas why it can be. I tried without nesting folder its working properly.

@jayair

This comment has been minimized.

Member

jayair commented Jul 26, 2017

@larrybek The ARN we are using for S3 arn:aws:s3:::YOUR_S3_UPLOADS_BUCKET_NAME/${cognito-identity.amazonaws.com:sub}* is looking for files inside a given bucket pre-fixed the identity user sub (or user id for our case).

What does the path of your S3 bucket (with the nested folders) look like?

@jayair

This comment has been minimized.

Member

jayair commented Jul 26, 2017

@huatzhi Yeah we are planning to. We do have a repo that we are using with tests running using Jest. If folks are interested then we can share that.

@larrybek

This comment has been minimized.

larrybek commented Jul 26, 2017

@jayair I changed the identity pool policy to access nested folders and it worked. Thanks.

@rajeget

This comment has been minimized.

rajeget commented Jul 28, 2017

@jayair the cognito is using customized authentication ? Also are you using Secure Remote Password (SRP) protocol. or planning to ?
Thanks for doing this 👍

@jayair

This comment has been minimized.

Member

jayair commented Jul 29, 2017

@rajegit We are using the User Pools as the authentication provider for now. According to the AWS docs, the Cognito JS SDK (and iOS and Android) uses SRP for it's auth flow.

@zamirkhan

This comment has been minimized.

zamirkhan commented Jul 31, 2017

@jayair I've tried to migrate the authentication changes, but am a bit stuck.

I know this is likely beyond the scope of what you're trying to provide, but I found it a bit difficult to navigate the markup diffs to try to find out what I needed to change, instead of having to go through the entire tutorial again (since it had been a while).

However, now that I thought I had made the required changes, although the "Test the API" step works for me.

I am getting 403 responses, both via the react app and my own Mocha.js tests for the API. I rigged the Mocha.js tests up to use the same authentication scheme (e.g. using your sigV4Client.js and aws helper). This is the response I get to my requests:

message: 'The request signature we calculated does not match the signature you provided.
Check your AWS Secret Access Key and signing method. Consult the service documentation for details.

The Canonical String for this request should have been
(... omitted...)

The String-to-Sign should have been
(... omitted ...)

I've tried to check over all of my config files (id's, key's etc) and can't seem to find any obvious errors. The strange thing is it works using the manual testing suggested in the guide. Any ideas?

@jayair

This comment has been minimized.

Member

jayair commented Jul 31, 2017

@zamirkhan This seems related to the way the request is being constructed. Can I see the part that makes the call to the helper to make the request and the helper function as well?

@zamirkhan

This comment has been minimized.

zamirkhan commented Aug 1, 2017

@jayair Here is a gist of the code involved. If I missed anything you need, let me know. I'm using Chakram.js (which is based on Mocha.js and uses the request library) for testing.

https://gist.github.com/zamirkhan/e5e6284ceb522edf073b0439b5350a30

It seems that the "Test the API step" directly tests that lambda through API gateway, but not by making an HTTP request, is that right? I'd prefer to have these tests do the latter to test the full request pathway, in the same way that the React client will. If there's a better way to do that then what I am that would avoid these issues or make this simpler, let me know.

@jayair

This comment has been minimized.

Member

jayair commented Aug 2, 2017

@zamirkhan It does make a normal HTTP request. It just makes it easier by calculating the security headers as a part of the process.

Just glancing at the gist, it looks okay but I guess the flow is different from the one we use in the React client (or in the Test APIs chapter). Any reason why you are using adminInitiateAuth?

@maletor

This comment has been minimized.

maletor commented Aug 18, 2017

This has got to be a regression, right?

Using AWS SDK to communicate with S3 directly? Sure, this makes enough sense, but ultimately, you probably want to do this through API Gateway for rate limiting features, etc.

Using AWS SDK to hit resources on API Gateway with Signature Version 4? I don't think so. Just use the Authorization header. What is your source for creating this huge internal library?

@jayair

This comment has been minimized.

Member

jayair commented Aug 18, 2017

@maletor The AWS SDK is used to generate temporary IAM credentials. This is exactly as it was before.

Using the user token vs IAM as an authorizer depends entirely on your setup. If you are only going to talk to API Gateway then stick with the user token. But if you are going to use multiple AWS resources then IAM gives you better control over those resources. In this tutorial we want to cover the minimal setup but still allow people to build on it (something a lot of our readers do). Hence, the decision to cover the IAM setup.

@forrest-akin

This comment has been minimized.

forrest-akin commented Sep 8, 2017

You may have just mentioned it (better control over access to AWS resources), but could elaborate (or provide a resource) on why IAM should be preferred over Cognito User Pools for auth in AWS?

@jayair

This comment has been minimized.

Member

jayair commented Sep 8, 2017

@forrest-akin We'll be writing more on this soon but here is the gist of it. IAM auth is what AWS uses for all their resources. You see this in the tutorial with S3 where we need to get the IAM credentials. You also need the IAM auth if you are rolling out your own auth system using Federated Identities.

The special case here is if you are using only API Gateway and all your users are stored in the User Pool. For this case, AWS added the user token method of authentication (there are a few reasons why I think AWS has done this). If this applies to your setup, you don't need to worry about using IAM.

The part of IAM (in conjunction with the Cognito Identity Pool) that gives more control is evident in this chapter (https://serverless-stack.com/chapters/create-a-cognito-identity-pool.html) where we set the policy for a user based on their cognito-identity.amazonaws.com:sub. So we can decide what resources a user get's access to based on their Cognito Identity (regardless of if they are a User Pool, FB, Google, or custom auth user).

For these reasons we decided to show people how to use the IAM sigV4 way of authentication. In the most recent update (https://github.com/AnomalyInnovations/serverless-stack-com/releases/tag/v1.1) we refactored the code so that we authenticate using the User Pool and get the IAM credentials at the same time. This allows folks to use this setup and not be limited to the AWS resources they use or to the User Pool.

Finally, if you are only going to use API Gateway with the User Pool you can pass in the user token here (https://github.com/AnomalyInnovations/serverless-stack-demo-client/blob/v1.1/src/libs/awsLib.js#L36) instead of using sigV4. So the rest of your client code does not depend on the system you are using.

Hope that clears things up. There are a few other things here but we are going to be writing about that soon.

@sean9keenan

This comment has been minimized.

sean9keenan commented Oct 10, 2017

Regarding this change, the size of the bundle was driving me a bit nuts - so I went and figured out how to minimize the bundled size of the aws-sdk. It should shave off over half of the size of the application bundle.

AnomalyInnovations/serverless-stack-demo-client#15

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment