Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OIDC Configuration Parsing Error and Key Mismatch in iRODS HTTP API #227

Closed
bl000m opened this issue Jan 23, 2024 · 26 comments
Closed

OIDC Configuration Parsing Error and Key Mismatch in iRODS HTTP API #227

bl000m opened this issue Jan 23, 2024 · 26 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@bl000m
Copy link

bl000m commented Jan 23, 2024

Feature

  • Please describe the feature you are requesting and your use case.

OIDC for Single Sign-On with Indigo IAM OIDC Provider

Both myself and @sigau are encountering issues while configuring the OIDC settings. The config.json has recently changed, and if we retain the previous version:

 "http_server": {
        "host": "0.0.0.0",
        "port": 9000,
        "log_level": "trace",
        "authentication": {
            "eviction_check_interval_in_seconds": 60,
            "basic": {
                "timeout_in_seconds": 3600
            },
            "oidc": {
                "config_host": "fits-indigo-iam-test.in2p3.fr",
                "port": 8080,
                "well_known_uri": "https://fits-indigo-iam-test.in2p3.fr/.well-known/openid-configuration",
                "client_id": "***",
                "redirect_uri": "http://***:5599/irods-http-api/0.1.0/authenticate",
                "state_timeout_in_seconds": 600
            }
        },

Executing docker run --rm --name irods_http_api -v ./config.json:/config.json:ro -p 9000:9000 irods/irods_http_api results in the following output, where the HTML of the OIDC provider login page is parsed as if it were the expected JSON:

[2024-01-23 13:26:22.860] [P:1] [info] [T:1] Initializing server.
[2024-01-23 13:26:22.860] [P:1] [trace] [T:1] Verifing OIDC endpoint configuration
[2024-01-23 13:26:22.878] [P:1] [debug] [T:1] Got the following back: 

<!DOCTYPE html>
<html lang="en_US">

<head>

... (rest of the HTML content)
</html>

Error: [json.exception.parse_error.101] parse error at line 11, column 1: syntax error while parsing value - invalid literal; last read: '<U+000A><U+000A><U+000A><U+000A><U+000A><U+000A><U+000A><U+000A><U+000A><U+000A><'
  • Even with the new config.json configuration, replacing oidc with openid_connect:
"openid_connect": {
                "provider_url": "https://fits-indigo-iam-test.in2p3.fr:8080/.well-known/openid-configuration",
		"client_id": "***",
                "redirect_uri": "http://***:5599/irods-http/0.1.0/authenticate",
                "state_timeout_in_seconds": 600
            }

We encounter the following issue:

Initializing server.
[2024-01-23 13:28:59.834] [P:1] [trace] [T:1] Verifing OIDC endpoint configuration
[2024-01-23 13:28:59.834] [P:1] [trace] [T:1] Invalid OIDC configuration, ignoring. Reason: [json.exception.out_of_range.403] key 'oidc' not found

As if the oidc key in config.json wasn't been updated to openid_connect

May you please help understanding where we are wrong
Thanks

iRODS Version, OS and Version

iRODS: latest,
OS: Ubuntu 22.04

@bl000m
Copy link
Author

bl000m commented Jan 23, 2024

We've fixed it. The issue was related to the well_known_uri path. Apparently, there's no need to specify it because it is retrieved directly from the config_host path. I haven't checked your code, but you are probably parsing the OIDC well-known endpoint by default on the OIDC provider hostname. I've noticed that you updated the OIDC config keys, and in version 0.2.0, it will be clearer for sure. However, at the moment:

  • The string value of the key well_known_uri should be empty in our case for the API to work.
  • If we remove the key-value well_known_uri line, the program skips the OIDC config because it doesn't find the well_known_uri key.

I have another question: since iRODS user mapping with the OIDC provider user is not yet available in the config.json, how can it be set?"

@MartinFlores751
Copy link
Contributor

since iRODS user mapping with the OIDC provider user is not yet available in the config.json, how can it be set?

Currently, the server looks for an irods_username claim in the OpenID token received. To map an OpenID user to an iRODS user, you have to add an irods_username claim, and set the value of the claim to the desired iRODS user.

I haven't checked your code, but you are probably parsing the OIDC well-known endpoint by default on the OIDC provider hostname.

That is correct, we automatically add /.well-known/openid-configuration to well_known_uri.

@bl000m
Copy link
Author

bl000m commented Jan 24, 2024

Currently, the server looks for an irods_username claim in the OpenID token received. To map an OpenID user to an iRODS user, you have to add an irods_username claim, and set the value of the claim to the desired iRODS user.

Since Indigo IAM is not allowing to set custom claim would it be possible to map the irods_username in iRODS to the nickname claim in Indigo IAM ?

@trel
Copy link
Member

trel commented Jan 24, 2024

In the upcoming 0.2.0 (and tip of main), you can specify this value in the configuration...

See #163

irods_user_claim

the diff is here... 17a3abf

@korydraughn
Copy link
Contributor

@bl000m A couple questions ...

Q. By iRODS: latest, are you talking about iRODS 4.3.1?
Q. Are you using HTTP API 0.1.0 or a custom build of the HTTP API?

@sigau
Copy link

sigau commented Jan 25, 2024

Hello
I'd like to take the liberty of replying for @bl000m
we are using irods server version 4.3.1 as well as version 0.1.0 of the HTTP api that we have built from the irods docker repo.

@trel
Copy link
Member

trel commented Jan 26, 2024

0.2.0 is now released. https://github.com/irods/irods_client_http_api/releases/tag/0.2.0

Please let us know if it works for the nickname claim from the Indigo IAM OIDC Provider.

@korydraughn korydraughn added this to the 0.3.0 milestone Jan 26, 2024
@bl000m
Copy link
Author

bl000m commented Feb 6, 2024

Hello,
when fetching the collections endpoint, the iRODS http api instance is rejecting the request for a bearer AT matching problem

resolve_client_identity: Authorization value: [Bearer eyJra***]
[2024-02-06 16:00:44.873] [P:1] [debug] [T:1] resolve_client_identity: Bearer token: [eyJra***]
[2024-02-06 16:00:44.873] [P:1] [error] [T:1] resolve_client_identity: Could not find bearer token matching [eyJra***].

Here is the decoded AT

{
  "sub": "***",
  "iss": "https://fits-indigo-iam-test.in2p3.fr",
  "groups": [],
  "preferred_username": "***",
  "organisation_name": "indigo-IAM-instance-auto",
  "client_id": "<irods_api_client_id>",
  "nbf": 1707235244,
  "scope": "openid profile",
  "name": "...",
  "exp": 1707238844,
  "iat": 1707235244,
  "jti": "***"
}

here below the iRODS http api instance oidc config:

"openid_connect": {
    "timeout_in_seconds": 3600,
    "provider_url": "https://fits-indigo-iam-test.in2p3.fr",
    "client_id": "<irods http api client_id in Indigo IAM>",
    "redirect_uri": "https://<fits client>/iam/irods/callback",
    "irods_user_claim": "preferred_username",
    "tls_certificates_directory": "/etc/ssl/certs",
    "state_timeout_in_seconds": 600
}

To give you some context:

  • I registered 2 clients in Indigo IAM instance: FITS (the portal) and iRods HTTP API, with exactly the same configuration
  • If the user on the FITS client clicks on "storage," a route controller executes a GET request to the iRODS HTTP API authentication endpoint (http://***:9000/irods-http-api/0.2.0/authenticate).
  • when the FITS client makes a GET request to the authentication endpoint, the iRODS HTTP api client returns the location key with its value corresponding to the Indigo IAM path page for the user to authorize iRODS.
  • The FITS client intercepts the location value, add the profilescope to obtain the preferred_username claim in the access token (doc Indigo) and redirects the user to it.
  • After authorization, we return to the redirect URI set in the iRODS API conf https:///iam/irods/callback, where the FITS client intercepts the authorization grant generated by Indigo IAM and uses it to request the access token to fetch the iRODS HTTP API collections endpoint.

NB I know that you suggest to set the redirect uri as to the HTTP API authentication point as it is specified here

the OIDC redirect_uri parameter must be set to the HTTP API's authentication endpoint. This is required, as the Authorization Code Grant needs to be redirected back to the HTTP API, to complete HTTP API token generation.

but in our case it doesn't work that way. So we handle the authorization grant exchange for the access token on the backend of the portal FITS, that we would like to be the entrypoint for the user's iRODS via the HTTP API. Please let me know of you want me to share the php code handling that, if could help

Please tell me if you need more details.
What am I doing wrong ?

Thanks for your help

@MartinFlores751
Copy link
Contributor

when fetching the collections endpoint, the iRODS http api instance is rejecting the request for a bearer AT matching problem

This is because the iRODS HTTP API does not act as an OAuth protected resource at this moment.
It is being worked on in #155. Once it's done, you should be able to use the HTTP API as you are trying to here.

Currently, the HTTP API needs the authorization code, since we assume we get it back after the user authenticates.
From there, we extract the irods_user_claim, and then give back an 'iRODS HTTP API' token, which is in the format of a UUID.
All further requests to the other iRODS HTTP API endpoints, like collections, require this UUID token at the moment.

@bl000m
Copy link
Author

bl000m commented Feb 6, 2024

Currently, the HTTP API needs the authorization code, since we assume we get it back after the user authenticates.
From there, we extract the irods_user_claim, and then give back an 'iRODS HTTP API' token, which is in the format of a UUID.

May you be more precise about this flow? How can we make the engine work at the moment ?

From what I understand, in order to have the oidc provider claim mapped to the irods_user_claim you need an AT containing that claim. The authorization code sent back after authorization by the oidc provider doesn't contain it directly. And in the case of Indigo IAM you get this kind of AT if you add the scope "profile" when creating the url based on oidc provider url for user authorisation. Once the user authorise irods, the authorization code sent back to the redirect uri has to be sent back to the oidc provider for it to generate the token. I don't see how the iRODS http api could generate the token itself without interacting with oidc provider.
Please help me understand better

@MartinFlores751
Copy link
Contributor

From what I understand, in order to have the oidc provider claim mapped to the irods_user_claim you need an AT containing that claim.

The irods_user_claim is what we look for in the ID Token. The following section gives a more complete example.

The authorization code sent back after authorization by the oidc provider doesn't contain it directly.

That is correct. We take the authorization code and exchange it at the OpenID Provider's token endpoint for something like the following:

{
   "access_token": "SlAV32hkKG",
   "token_type": "Bearer",
   "refresh_token": "8xLOxBtZp8",
   "expires_in": 3600,
   "id_token": "very.long.jwt"
}

The above JSON example is taken from the OpenID Specification, with a change to the id_token: https://openid.net/specs/openid-connect-core-1_0.html#TokenResponse

From there, we extract the id_token, and look for the string set in irods_user_claim in id_token.
If it exists, we create an 'iRODS HTTP Token' (which is currently just a UUID) and give it back to the user for subsequent endpoint authorization.
Otherwise, we log an error.

Once the user authorise irods, the authorization code sent back to the redirect uri has to be sent back to the oidc provider for it to generate the token. I don't see how the iRODS http api could generate the token itself without interacting with oidc provider.

That is correct. The iRODS HTTP API does exchange the authorization code at the OpenID Provider's token endpoint.

The iRODS HTTP API does not accept Access Tokens for endpoint authorization at this moment, only 'iRODS HTTP Tokens'.
Once #155 is completed, the iRODS HTTP API will be able to accept Access Tokens for endpoint authorization, since the iRODS HTTP API may run as an OAuth Protected Resource.

@bl000m
Copy link
Author

bl000m commented Feb 8, 2024

The irods_user_claim is what we look for in the ID Token.

and

From there, we extract the id_token, and look for the string set in irods_user_claim in id_token. If it exists, we create an 'iRODS HTTP Token' (which is currently just a UUID) and give it back to the user for subsequent endpoint authorization. Otherwise, we log an error.

Not sure to understand. Based on what is specified in your repo README (below), the value of irods_user_claim is the OIDC user claim to be mapped onto irods_user_claim, which makes sense. In other words, you are searching for the string set in irods_user_claim that identifies the key in the token corresponding to the OIDC user claim, and the value of this key needs to be the same value as irods_user_claim for the mapping to occur.
Can you confirm that?

From the README:

Currently, the server looks for the custom claim in the ID Token, which is specified in the irods_user_claim parameter. This serves as the mapping mechanism for an OIDC User to an iRODS User.
A user who authenticates but does not have the claim specified in irods_user_claim mapped in their account will not have access to the API. A HTTP 400 Bad Request status code will be returned if the claim specified in irods_user_claim is not found.

@trel
Copy link
Member

trel commented Feb 8, 2024

The following entry in the iRODS HTTP API config.json file will tell the iRODS HTTP API to look for a custom claim in the id_token returned by the OIDC provider with a key named irods_username.

                // The name of the OIDC claim which provides the mapping of an
                // OIDC user to an iRODS user account
                "irods_user_claim": "irods_username",

The value associated with this irods_username key in the id_token will be used by the iRODS HTTP API when interacting with the iRODS server as that iRODS user.

@bl000m
Copy link
Author

bl000m commented Feb 8, 2024

The following entry in the iRODS HTTP API config.json file will tell the iRODS HTTP API to look for a custom claim in the id_token returned by the OIDC provider with a key named irods_username.

                // The name of the OIDC claim which provides the mapping of an
                // OIDC user to an iRODS user account
                "irods_user_claim": "irods_username",

The value associated with this irods_username key in the id_token will be used by the iRODS HTTP API when interacting with the iRODS server as that iRODS user.

Okay, thanks for clarifying. So, a custom OIDC claim is needed for the mapping to occur. I had interpreted it differently. Unfortunately, the Indigo IAM OIDC provider doesn't currently allow the creation of custom claims such as irods_username, and I was attempting to map the preferred_username claim to the irods_username. Its value effectively matches the irods_username value. Do you think that mapping with a claim that better fits the OIDC provider's constraints could be a possible goal for the next milestone?

@trel
Copy link
Member

trel commented Feb 8, 2024

If I'm reading your statement correctly.... and you have a value coming from the OIDC provider that you want to use as the iRODS username... then I think this is what you put in your config.json:

                "irods_user_claim": "preferred_username",

This will look for a key/claim named preferred_username in the id_token and use its value as the iRODS username when talking to the iRODS server.

@bl000m
Copy link
Author

bl000m commented Feb 8, 2024

If I'm reading your statement correctly.... and you have a value coming from the OIDC provider that you want to use as the iRODS username... then I think this is what you put in your config.json:


                "irods_user_claim": "preferred_username",

This will look for a key/claim named preferred_username in the id_token and use its value as the iRODS username when talking to the iRODS server.

Yes it's exact. Thanks for confirming that

@trel
Copy link
Member

trel commented Feb 8, 2024

Ah, very good.

Please let us know if this works as you expected/wanted.

@trel trel modified the milestones: 0.3.0, 0.4.0 Mar 15, 2024
@korydraughn
Copy link
Contributor

@bl000m We've just released HTTP API 0.3.0.

Please give it a try and let us know if it resolves your issue.

@bl000m
Copy link
Author

bl000m commented Mar 18, 2024

@korydraughn thank you for letting us know.
Before proceeding with testing, I have a question regarding the OIDC settings in the configuration file.
The values for sub and email in the user_attribute_mapping section are hardcoded:

"user_attribute_mapping": {
                    "irods_username": {
                        "sub": "123-abc-456-xyz",
                        "email": "rods_user@example.org"
                    }
                },

Shouldn't these values be dynamically extracted from the OIDC provider?
Could we configure them as follows instead?

"sub": "sub", // where 'sub' represents the OIDC provider's sub claim
"email": "email" // where 'email' represents the OIDC provider's email claim

@korydraughn
Copy link
Contributor

The hard-coded values are for demonstration purposes.

You can change/remove those mappings to what you like. That must be done for each user. For example:

"user_attribute_mapping": {
    "alice": {
        // "alice" is identified as the user if and only if these
        // key-value pairs exist in the OIDC response.
        "oidc_prop_1": "value_1",
        "oidc_prop_N": "value_N",
        "email": "alice@example.com"
    },
    "bob": {
        // "bob" is identified as the user if and only if these
        // key-value pairs exist in the OIDC response. Notice how
        // "bob" requires one additional key-value pair.
        "oidc_prop_1": "value_1",
        "oidc_prop_N": "value_N",
        "special_prop": "value"
    },
    "rods": {
        // Same thing is true for "rods".
        "sub": "sub_value"
    }
},

So the answer to your question is yes.

@bl000m
Copy link
Author

bl000m commented Mar 18, 2024

You can change/remove those mappings to what you like. That must be done for each user.

If I understand this correctly, then the answer is no rather than yes: the values must be hardcoded for each user and are not automatically extracted from the OIDC provider access token claims.

If this assumption is correct, and considering that we will need to map thousands of users in the long term, manually doing so for each user is not practical. In this case, if I refer to the documentation in the "iRODS as an OAuth Protected Resource" section:

The token must be able to be mapped to a user, using either irods_user_claim or user_attribute_mapping.

If I am understanding correctly, we can proceed at this point with just using irods_user_claim and remove the user_attribute_mapping section. Right?

@korydraughn
Copy link
Contributor

Correct.

@MartinFlores751 Thoughts? The conversation starts at #227 (comment).

@MartinFlores751
Copy link
Contributor

@korydraughn Your explanation of user_attribute_mapping is correct.

If this assumption is correct, and considering that we will need to map thousands of users in the long term, manually doing so for each user is not practical. In this case, if I refer to the documentation in the "iRODS as an OAuth Protected Resource" section:

The token must be able to be mapped to a user, using either irods_user_claim or user_attribute_mapping.

If I am understanding correctly, we can proceed at this point with just using irods_user_claim and remove the user_attribute_mapping section. Right?

Yes, you should be able to use irods_user_claim instead of user_attribute_mapping.

Shouldn't these values be dynamically extracted from the OIDC provider? Could we configure them as follows instead?

"sub": "sub", // where 'sub' represents the OIDC provider's sub claim
"email": "email" // where 'email' represents the OIDC provider's email claim

I'm curious about the use case you had in mind for this. Did you want to extract some values and do something with them in the iRODS HTTP API?

You can change/remove those mappings to what you like. That must be done for each user.

If I understand this correctly, then the answer is no rather than yes: the values must be hardcoded for each user and are not automatically extracted from the OIDC provider access token claims.

That is correct. In order to map a user using user_attribute_mapping, all specified attributes must match. When we recieve the response from the OpenID Provider, we compare the user_attribute_mapping values to the values in the token. If they match, they map to the specified user.

@trel trel added the enhancement New feature or request label Mar 18, 2024
@trel
Copy link
Member

trel commented Mar 18, 2024

This work was implemented in 3b1bb67 and addressed #108, #155, and #235.

@bl000m
Copy link
Author

bl000m commented Mar 18, 2024

Thanks for the clarification

@trel
Copy link
Member

trel commented Mar 18, 2024

Will close if there is no other points of discussion.

Of course, @bl000m please continue trying your use cases against 0.3.0 and we'll fix/add anything for 0.4.0.

Thanks everyone.

@trel trel closed this as completed Mar 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants