Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API Development Part 1 (OpenID Connect Infrastructure) #6182

Open
KentShikama opened this issue Jul 9, 2015 · 35 comments

Comments

Projects
None yet
6 participants
@KentShikama
Copy link
Contributor

commented Jul 9, 2015

If you are planning to join in right now, please see the part 2 issue for the API

The original PR is now getting kind of long and hard to follow if you are just starting. Hence, I'm creating this issue to inform everyone where we are at in terms of the API development and to breakdown what needs to be done next so it will be easier for anyone interested to hop in.

Resource Owner Password Credentials Grant

Note this flow is NO LONGER supported (Revert ee7f188 to re-enable). In case you still want to test it out after you revert the referenced commit, here is the complete flow using cURL. See tests for more "documentation".

  1. Register your client

curl http://localhost:3000/api/openid_connect/clients -d "redirect_uris[]=http://localhost:3000/&client_name=<client_name_here>"

Note that the redirect uri isn't actually used in this flow. It should return the following:

{
 "o_auth_application":
  {
    "id":3,
    "client_id":<your_client_id>",
    "client_secret":"<your_client_secret>",
    "name":null,
    "redirect_uris":"[\"http://localhost:3000/\"]",
    "created_at":"2015-07-07T17:30:08.248Z",
    "updated_at":"2015-07-07T17:30:08.259Z"
  }
}
  1. Retrieve an access token from the token endpoint

curl -d "grant_type=password&username=<your_username>&password=<your_password>&client_id=<your_client_id_from_above>&client_secret=<your_client_secret_from_above>" http://localhost:3000/api/openid_connect/access_tokens

It should return the following:

{
  "access_token":"<your_access_token>",
  "token_type":"bearer",
  "expires_in":86399
}
  1. Retrieve user info using your access token

Via Authorization Request Header Field
curl -H "Authorization: Bearer <your_access_token>" http://localhost:3000/api/v0/user/

Via Form-encoded Body Parameter
curl -d "access_token=<your_access_token>" http://localhost:3000/api/v0/user/

Via URI Query Parameter
curl http://localhost:3000/api/v0/user/?access_token=<your_access_token>

It should return the following:

{
  "name":"<your_username>@localhost:3000",
  "email":"<your_email>",
  "language":"<your_lang>",
  "username":"<your_username>"
}

Remaining tasks

  • Add support for refresh tokens (See 8e55da3 and dc925f4 by @AugierLe42e )
  • Add protection/tests "against brute force attacks (e.g., using rate-limitation or generating alerts)" as specified (No longer needed as this flow is disabled)

Dynamic client registeration

We are almost done here. See commit [8d2b5cd].

Implicit Flow

Here is the current flow using cURL. Note I will assume your pod URL is http://localhost:3000/. @AugierLe42e has recently started Griaspora, a client that plans to use this flow.

  1. Register your client

curl http://localhost:3000/api/openid_connect/clients -d "redirect_uris[]=http://localhost:3000/&client_name=<your_client_name>"

It should return the following:

{
 "o_auth_application":
  {
    "id":3,
    "client_id":<your_client_id>",
    "client_secret":"<your_client_secret>",
    "name":"<your_client_name>",
    "redirect_uris":"[\"http://localhost\"]",
    "created_at":"2015-07-07T17:30:08.248Z",
    "updated_at":"2015-07-07T17:30:08.259Z"
  }
}
  1. Run bundle exec rake db:seed to load the scopes

  2. Send an Authentication Request to start off the Implicit flow

curl -L http://localhost:3000/api/openid_connect/authorizations/new?client_id=<your_client_id>\&redirect_uri=http://localhost:3000/\&response_type=id_token%20token\&scope=openid%20read\&nonce=hi\&state=hi --cookie-jar cookie

It should return HTML code with the following snippet. The authenticity_token value is needed in the next step to pass the CSRF filter.

...

<form class="block-form" id="new_user" action="/users/sign_in" accept-charset="UTF-8" method="post"><input name="utf8" type="hidden" value="&#x2713;" /><input type="hidden" name="authenticity_token" value="7aKgpujMvPwmjBrzcGFIIRcsCzg+JY9u3MqaqS8D4mvBxfL0klexwgUa63WhSrMtq7DvKYARTexi8rGe3TFzmw==" />  <fieldset>

...more HTML
  1. Login to Diaspora (with cURL)

curl -L -H "X-CSRF-TOKEN: <your_authenticity_token_value_from_above>" -d "user[username]=<your_username>&user[password]=<your_password>&user[remember_me]=1" http://localhost:3000/users/sign_in --cookie-jar cookie --cookie cookie

It should return HTML code with the following snippet.

...

<form class="approve" action="/openid_connect/authorizations" accept-charset="UTF-8" method="post"><input name="utf8" type="hidden" value="&#x2713;" /><input type="hidden" name="authenticity_token" value="a2lqmSwa4Bhov8SbKCVvt/PYUTDX0y/Efnfq1vj/1ei6J6awPrqi944gMcupSxZriLAZrgPZAFSAEQ0KokwO5Q==" />
      <input type="hidden" name="client_id" id="client_id" value="4bb87c09fd99949d74a0ad0af20278bf" />

...more HTML

5a) If the user has not yet approved the application, approve (or deny) the request

To approve

curl -H "X-CSRF-TOKEN: <your_authenticity_token_value_from_above>" -d "approve=true" http://localhost:3000/api/openid_connect/authorizations --cookie-jar cookie --cookie cookie

To deny

curl -H "X-CSRF-TOKEN: <your_authenticity_token_value_from_above>" -d "approve=false" http://localhost:3000/api/openid_connect/authorizations --cookie-jar cookie --cookie cookie

If you approve, it should return a ID token and access token in a fragment as follows. The ID token should contain two periods.

<html><body>You are being <a href="http://localhost:3000/#access_token=<your_access_token_value>&amp;expires_in=86399&amp;id_token=<your_id_token_value>&amp;token_type=bearer">redirected</a>.</body></html>

5b) If the user has already approved the application once

It should return a ID token and access token in a fragment as follows:

<html><body>You are being <a href="http://localhost:3000/#access_token=<your_access_token_value>&amp;expires_in=86399&amp;id_token=<your_id_token_value>&amp;token_type=bearer">redirected</a>.</body></html>
  1. Verify the ID token

See spec/controllers/openid_connect/id_tokens_controller_spec.rb and spec/controllers/openid_connect/authorizations_controller_spec.rb#create.

  1. Verify the Access Token using the ID token's at_hash value

See spec/controllers/api/openid_connect/authorizations_controller_spec.rb#create (specifically the test "it should return a valid access token in a fragment").

  1. Retrieve user info using your access token

Same as password flow step 3.

Remaining tasks

  • Add support for token response type (and id_token token)
  • Implement a real ID token. This involves creating the model, replacing the current random hex stub with the token generation, and adding tests.
  • Implement "read" and "write" scopes.
  • Create/verify a script that validates ID tokens (See id_tokens_controller_spec.rb)

Authorization Code Flow

  1. Run bundle exec rake db:seed to load the scopes

  2. Register your client

curl http://localhost:3000/api/openid_connect/clients -d "redirect_uris[]=http://localhost:3000/&client_name=<your_client_name>"

It should return the following:

  {
    "id":3,
    "client_id":<your_client_id>",
    "client_secret":"<your_client_secret>",
    "name":"<your_client_name>",
    "redirect_uris":"[\"http://localhost:3000/\"]",
    "created_at":"2015-07-07T17:30:08.248Z",
    "updated_at":"2015-07-07T17:30:08.259Z"
  }
  1. Send an Authentication Request to start off the Authorization Code flow

curl -L http://localhost:3000/api/openid_connect/authorizations/new?client_id=<your_client_id>\&redirect_uri=http://localhost:3000/\&response_type=code\&scope=openid%20read%20write\&nonce=hi\&state=hi --cookie-jar cookie

It should return HTML code with the following snippet. The authenticity_token value is needed in the next step to pass the CSRF filter.

...

<form class="block-form" id="new_user" action="/users/sign_in" accept-charset="UTF-8" method="post"><input name="utf8" type="hidden" value="&#x2713;" /><input type="hidden" name="authenticity_token" value="7aKgpujMvPwmjBrzcGFIIRcsCzg+JY9u3MqaqS8D4mvBxfL0klexwgUa63WhSrMtq7DvKYARTexi8rGe3TFzmw==" />  <fieldset>

...more HTML
  1. Login to Diaspora (with cURL)

curl -L -H "X-CSRF-TOKEN: <your_authenticity_token_value_from_above>" -d "user[username]=<your_username>&user[password]=<your_password>&user[remember_me]=1" http://localhost:3000/users/sign_in --cookie-jar cookie --cookie cookie

It should return HTML code with the following snippet.

...

<form class="approve" action="/openid_connect/authorizations" accept-charset="UTF-8" method="post"><input name="utf8" type="hidden" value="&#x2713;" /><input type="hidden" name="authenticity_token" value="a2lqmSwa4Bhov8SbKCVvt/PYUTDX0y/Efnfq1vj/1ei6J6awPrqi944gMcupSxZriLAZrgPZAFSAEQ0KokwO5Q==" />
      <input type="hidden" name="client_id" id="client_id" value="4bb87c09fd99949d74a0ad0af20278bf" />

...more HTML

5a) If the user has not yet approved the application, approve (or deny) the request

To approve

curl -H "X-CSRF-TOKEN: <your_authenticity_token_value_from_above>" -d "approve=true" http://localhost:3000/api/openid_connect/authorizations --cookie-jar cookie --cookie cookie

To deny

curl -H "X-CSRF-TOKEN: <your_authenticity_token_value_from_above>" -d "approve=false" http://localhost:3000/api/openid_connect/authorizations --cookie-jar cookie --cookie cookie

If you approve, it should return your auth code as a parameter as follows.

<html><body>You are being <a href="http://localhost:3000/?code=<your_auth_code_value>">redirected</a>.</body></html>

5b) If the user has already approved the application once

It should return an auth code as follows:

<html><body>You are being <a href="http://localhost:3000/?code=<your_auth_code_value>">redirected</a>.</body></html>
  1. Send a token request to the token endpoint

curl -d "grant_type=authorization_code&client_id=<your_client_id>&client_secret=<your_client_secret>&redirect_uri=http://localhost:3000/&code=<your_auth_code_value>" http://localhost:3000/api/openid_connect/access_tokens --cookie-jar cookie --cookie cookie

It should return an ID token and access token as follows:

{
    "access_token":"<access_token_value>",
    "token_type":"bearer",
    "expires_in":86399,
    "id_token":"<id_token_value>"
}
  1. Verify the ID token

See spec/controllers/openid_connect/id_tokens_controller_spec.rb and spec/controllers/openid_connect/authorizations_controller_spec.rb#create.

  1. Verify the Access Token using the ID token's at_hash value

See spec/controllers/api/openid_connect/authorizations_controller_spec.rb#create (specifically the test "it should return a valid access token in a fragment").

  1. Access protected resources using access token

Same as password flow step 3.

Remaining tasks

  • Add support for code response type
  • Add necessary code to the token endpoint

Hybrid Flow

See remaining tasks at the bottom of this post.

OpenID Connect Discovery

Here we need to further adjust the discovery controller.

Remaining tasks

  • Add route, controller, and tests.

Remaining Tasks for Initial API Release

Remaining Tasks for Future API Release

Or at least I will finish all the required API routes before coming back to these.

All of the below are optional based on the OpenID Connect Specification

  • Background task for removing ID tokens
  • Add support for code token, code id_token (Hybrid flows)
  • Implement "extended" profile scope. The basic profile will just be the required "openid" scope. See loomio discussion.
  • Implement signed request objects
  • Make user info access logs available to user
  • Add mobile support for user end consent form
  • Properly implement user hint
  • Properly support ui_locales
  • Support key rotation
  • Add support for display=popup
  • Discuss whether we should allow Devise authorization in API endpoints
  • Polish error pages
  • Fix fallback for when logo_uri is pointing to a broken URL to not use the default avatar but the default application logo.
  • Add rate-limiting for dynamic client registration


Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

@ghost

This comment has been minimized.

Copy link

commented Jul 9, 2015

Thank you for taking time to do this ;)

@ghost

This comment has been minimized.

Copy link

commented Jul 10, 2015

Implement a real ID token. This involves creating the model, replacing the current random hex stub with the token generation, and adding tests.

By what do you think we may replace the current random hex sting?

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Jul 10, 2015

By what do you think we may replace the current random hex sting?

Do you mean when? Or how? If you mean when, I was planning to implement this next. I'l be going to sleep now, so if you want to go ahead and implement it, feel free to do so. Or just inform me of what you plan to work on, and I'll start on something that doesn't conflict tomorrow. If you mean how, it might just be faster to read the spec and Nov's sample provider code than me explaining here (unless you have a specific question aspect about the ID token in mind?).

@ghost

This comment has been minimized.

Copy link

commented Jul 10, 2015

Now, I meant what is the problem with the current random hex? Why do you want to replace it, and by what. This is how Nov's sample works. I took this solution from it.

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Jul 11, 2015

Hmm I believe you are talking about the access token? I'm talking about the ID token.

Here is an example ID token (quoted from the OIDC spec itself)

{
   "iss": "https://server.example.com",
   "sub": "24400320",
   "aud": "s6BhdRkqt3",
   "nonce": "n-0S6_WzA2Mj",
   "exp": 1311281970,
   "iat": 1311280970,
   "auth_time": 1311280969,
   "acr": "urn:mace:incommon:iap:silver"
  }
@ghost

This comment has been minimized.

Copy link

commented Jul 11, 2015

Ah! Ok, my bad, I misunderstood :)

@ghost ghost referenced this issue Jul 13, 2015

Merged

[WIP] OpenID Connect #6095

@ghost

This comment has been minimized.

Copy link

commented Jul 14, 2015

c13a923 adds support for authorizations by default. When an authorization doesn't already exists it is created by default on password flow.

@goobertron

This comment has been minimized.

Copy link

commented Jul 14, 2015

Just quickly: thank you both so much for all your work on this. Wishing you success!

@ghost

This comment has been minimized.

Copy link

commented Jul 15, 2015

Thank you very much @goobertron ;)

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Jul 15, 2015

Thanks for the support @goobertron .

I added in the real ID token with the revised Authorization model. See Implicit flow 5b) above.

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Jul 16, 2015

I added in support for access tokens in the implicit flow. See implicit flow steps 7) and 8) above. I will now work on adding support for request objects.

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Jul 16, 2015

Actually since http://openid.net/specs/openid-connect-core-1_0.html#ClaimsParameter

"Support for the claims parameter is OPTIONAL. Should an OP not support this parameter and an RP uses it, the OP SHOULD return a set of Claims to the RP that it believes would be useful to the RP and the End-User using whatever heuristics it believes are appropriate."

I think we can just leave this be for now? @AugierLe42e I'm going to work on scopes.

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Jul 16, 2015

Pardon me. Claims and requests objects are two different things. I will work on scopes first and then I will work on request objects. I will ignore claims until later.

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Jul 16, 2015

Hmm never mind. Pardon me for all my changes. Claims are a large part of request objects because most other attributes that are in the request object itself are duplicated in the request, most notably, the client id and the scope. So I suppose I will be working with both after I finish with scopes.

@ghost

This comment has been minimized.

Copy link

commented Jul 16, 2015

Don't know. I haven't explored this part of the spec.

@ghost

This comment has been minimized.

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Jul 21, 2015

Nice 👍 I'll take a look soon.

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Jul 30, 2015

I've updated this issue with the latest changes.

@jaywink

This comment has been minimized.

Copy link
Contributor

commented Jul 30, 2015

Awesome work guys <3

@ghost

This comment has been minimized.

Copy link

commented Jul 30, 2015

@theworldbright : I have reset your last changes and just neutralized the code ;)

@jaywink

This comment has been minimized.

Copy link
Contributor

commented Jul 30, 2015

@AugierLe42e I doubt commented out code will be accepted to be merged in... please, just don't do it :)

@ghost

This comment has been minimized.

Copy link

commented Jul 30, 2015

Nope, it's just temporary. We'll see if we want to reactivate it when we start to work on OAuth for the API.

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Jul 31, 2015

We do want to merge in this PR in before we start the API routes though right? @AugierLe42e @jhass

I'm fine with keeping it commented out for now, although since we're on our last stretch I might just continue to work off my current branch (without the neutralization commit) and just push --force when I'm done (unless @AugierLe42e are you planning have a coding session this weekend?)

Also I've updated the original post (at the bottom) with things that I believe we should do before we start the API routes and things that we can leave until after.

@jhass

This comment has been minimized.

Copy link
Member

commented Jul 31, 2015

Yes, I wouldn't mind having this PR in as early as possible. Then maybe a second one to setup a proper authorization infrastructure and then we can design and spread the API routes across many.

@ghost

This comment has been minimized.

Copy link

commented Jul 31, 2015

Ok, so I'll keep the code in a separate branch on my repo, you can force push @theworldbright.
I'll be free for coding this week-end, so we can keep in touch to finish this early. Maybe this week-end?
Then, I'll work on Griaspora and make an android mock to test it.
We can also plan to develop a "Connect with diaspora*" widget?

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Jul 31, 2015

@AugierLe42e Yeah I'll definitely continue working this weekend. I just did the force push. Perhaps you can choose some task that is under the "Remaining Tasks for Initial API Release" that I haven't marked as "will do over the weekend"? (If you are interested in doing one that I have marked, just message me). Lets try to get the PR merged by the end of the weekend.

Yeah a "Connect with diaspora*" widget sounds great. Perhaps even create some branding guidelines for it. Maybe @pablocubico would be interested? I suppose we should create a new issue for this though. I'm fairly interested in having an android app that I can handle posts and chats through.

@jhass Good to know that you don't mind having the PR in soon. I'm really excited at the prospect of having the PR merged :p I'll create a "API Part 2" issue as a design proposal for how the API routes may look like soon.

@pablocubico

This comment has been minimized.

Copy link
Contributor

commented Jul 31, 2015

@theworldbright Of course, I've been doing some typographic experiments for my pod (and some styling), I'll be happy to try and design a widget for that. I think that feature is key.

As for guidelines, I feel like I can only give recommendations, as there is no proper "design signoff" flow in the diaspora community, and design is still operating in an "I like / I don't like" basis, which is natural, but I like to support my design on data, not personal taste.

Anyone wanting to sketch a "Connect with diaspora" widget at least on pen and paper? Just to check some possible layouts.

@ghost

This comment has been minimized.

Copy link

commented Jul 31, 2015

@theworldbright: I can do the front-end part (forms, user consent, etc.) if you are ok with it.
I'll also start the "connect with" button during the week-end. I think it will be quick. I'll make some basic button and leave the styling to @pablocubico if you are ok with it.

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Jul 31, 2015

@pablocubico Glad you're up for designing the widget :D

@AugierLe42e Yeah I'll leave the front-end part to you: styling the user consent form and then creating another page where you can manage OAuthApplications (basically the github equivalent of https://github.com/settings/applications ).

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Aug 1, 2015

I'm going to add "Make user info access logs available to end user" to "Remaining Tasks for Initial API Release".

Edit: I'm moving this to the "Remaining Tasks for Future API Release"

@ghost

This comment has been minimized.

Copy link

commented Aug 4, 2015

@theworldbright, you can check Create views where user can reject/un-reject clients that have once been approved BTW ;)

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Aug 4, 2015

Yeah I feel like we might have to adjust it based on the whether we are going to add more scopes.

@ghost

This comment has been minimized.

Copy link

commented Aug 4, 2015

From a design point of view, you mean? Otherwise, you would just need to add a new item in the scopes section in the translations files.

@KentShikama

This comment has been minimized.

Copy link
Contributor Author

commented Aug 4, 2015

Maybe some indented lists (if we support scopes that include other scopes) or something else that I haven't thought of that may come up in this discussion: #6289

But yeah I doubt there will be that radical of a design change. I'll check it off for now 👍 It is now really only the prompts parameter. I'm working on it!

@jaywink jaywink referenced this issue Nov 19, 2015

Closed

Diaspora api #4554

@cmrd-senya

This comment has been minimized.

Copy link
Member

commented Mar 30, 2016

I have implemented "Authorization Code Flow" support in my gem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.