Mapping external IdP claims to local claims

[sadilchamishka]

Summary

Propose a single, IdP-level claim mapping mechanism that translates an external identity provider's claims into ThunderID's local claims. The same mapping is reused by both paths that bring external identities into ThunderID: RFC 8693 token exchange and federated login.

Problem

External IdPs emit claims under their own names and shapes (e.g. given_name, org_name, URI-style names, or structured/nested objects). ThunderID and its applications expect local claims. Today external claims flow through verbatim, so issued tokens carry the right claims only if the external names happen to match. We need a way to normalize external claims to local claims.

Proposed model

Attach the mapping to the identity provider as configuration — a mapping from the IdP's external claims to ThunderID's local claims. It is part of the trust relationship with that IdP, defined once and reused everywhere ThunderID consumes that IdP's identity.

The mapping is represented as a native JSON structure on the IdP (the configuration store is already JSONB), and supports both flat and complex/nested claims — i.e. selecting a nested field out of a structured external claim, or mapping into a nested local attribute — not only flat top-level name-to-name.

Where it applies

Two entry points share the same mapping:

1. Token exchange — an external subject token's claims are mapped to local claims before becoming attributes on the issued token.
2. Federated login — claims received from the IdP during an auth flow are mapped before they become the authenticated user's attributes (and subsequently flow into the issued access/ID tokens).

Key design decisions

• Flat and complex mappings — supports renaming top-level claims as well as mapping nested/structured claims, so providers with structured profiles are first-class.
• Direction is external → local, applied early — before downstream claim filtering (the application's allowed-attributes config) and before federated-identity consistency checks — so the rest of the system only ever sees local claims.
• Non-destructive defaults — unmapped claims pass through unchanged; on a collision the mapped value wins.
• Protected names — standard/reserved token claims (sub, iss, scope, …) cannot be mapping targets, so a mapping can't alter token identity or authorization semantics.
• Subject identity is never remapped — the external subject used to link/look up the user is left as-is.

Open questions

• We can implement a transformation engine which accept kind of contract defined in the claim mapping. Which will be reusable for other transformation layers.
• Is a global/default mapping useful in addition to per-IdP?

[sahandilshan]

Strong +1, Thunder needs this, and putting the mapping on the IdP (the trust relationship) instead of scattering mappers across apps/scopes is the right call. Sharing one mapping across token exchange and federated login keeps both paths from drifting. Great initiative.

Three things I'd want to nail down:

1. Don't make pass-through the silent default. "Unmapped claims pass through, mapped wins" is convenient but a footgun, an external claim whose name happens to collide with a local one silently becomes the local value with no mapping ever declared. I'd lean toward requiring an explicit mapping for every claim we forward (allowlist posture); drop or namespace the rest. Pass-through can stay as an opt-in dev convenience.

2. Define precedence, not just renaming. "Mapped value wins" only covers two external claims hitting the same target. The harder collision is mapped external value vs. the attribute already on the local user, which is live for us since we have JIT provisioning + account linking (provisioningExecutor / federated auth resolver). On re-login of a linked user, do mapped values overwrite the stored profile or not? Some claims the IdP should own (email_verified, groups → refresh every login), some the local profile should own (user-edited → don't clobber). So we need an explicit per-IdP (maybe per-claim) sync/precedence policy. Token exchange is simpler here since there's often no persistent local user, worth noting the two entry points may need slightly different precedence even with a shared mapping.

3. Shape transforms, not just names. IdPs differ in value shape, roles as space-delimited string vs. array, "true" vs. boolean, epoch vs. ISO. The string↔array case will hit us immediately. Keep v1 to flat + single-level nested paths, but design the contract as { from, to, transform? } so a value transform can slot in later without a rewrite.

On the open question, I'd skip the global/default mapping; per-IdP is the right granularity, and a default+override layer just creates precedence confusion.

[sadilchamishka]

1. The pass-through will happen only if there exist attribute define for the corresponding user type.
2. Yes for the JIT provisioning, the attribute syncing approach should be considered. Haven't decided yet. The existing approach of the ThunderID needs to be checked and propose an approach with another proposal.
3. This is great catch. I will change the attribute mapping schema accordingly.

[ThaminduDilshan]

1. We need to do this, but we should decide between two implementation choices.

   • Store claim mappings as another IDP property
   • Store claim mappings in top level outside properties

2. We may also need to make subject claim configurable along with this effort

3. What's the advantage of having a global/ default mapping? Is it to improve the configuration experience?

[sadilchamishka]

1 - As the existing properties are changeable based the type of IDP, and IDP claim mapping is common concept irrespective of the IDP type, we can consider to have claim mappings as first class attribute. +1 for the suggesion.

2 - This need to be discussed. Having this capability give greater flexibility, but allowing to map an standard/reserved token claims should be done with some care.

3. If OIDC claims can be mapped to local claims and keep as an template to be used as starting point or default mapping can be considered as possible improvement as each claim mapping has to be done for each IDP (This may not be a common use case hence may not required to be considered at this point)

[ayshsandu]

It would be good to externalize the mappings from the IdP and template it. At the IdP level, it should be able to define which collection of mappings to use.
So the mappings can be not only used at the federated login, but at trusted token issuers, as well as other applicable transformations.

[sadilchamishka]

Thats a valid point. The current integration plan is to let define integrations like google, facebook where we don't allow to create multiple integrations of same type. As the attribute mappings are integration specific, the templating may not give the advantage.

But we can group the integrations as OIDC like wise so attribute mappings can be templated to be re-used across IDPs (integrations). But to use a template, the attribute mappings should be common. Due to those reasons, we can further check how we can get benefit from the templates.

The proposed model apply the same mapping for federation, token exchange when the same integration is used for both.

[thiva-k]

Hi @sadilchamishka ,

How are we planning to represent the internally used claims?
Ideally we should have an internal representation, to which the IdP's external claims can be mapped to. Currently in flow executors, we have a set of hardcoded user attribute constants which are used internally for all flows. We should make them first-class or at least document such claims right?
If external IDPs define mapping to other format of these claims(e.g. email_address, user_email, email) our executors won't pick it up and flows may fail, which is an issue currently.

We may also need mapping for user stores, etc. too in the future, so better to make it as a reusable transformation layer as you suggested.

And have we considered whether mapping from internal representation -> external formats is needed too with this effort? e.g. mapping our internal claims to OIDC/SCIM dialects, etc.

Here is a related issue created sometime back #557

[sadilchamishka]

The IDP claim mapping has to be done for a particular attribute set under a user type. I will share the updated the design.
Need to discuss further about the current pain points. Thanks for sharing the information.

[sadilchamishka]

Refined model

attributeConfiguration is a dedicated top-level field on the identity provider (its own JSONB column, separate from the type-specific properties). It describes how an external identity becomes, and stays, a local identity — reused by both token exchange and federated login. Rather than one flat "mapping", it is organized so each concern sits on its natural axis.

Model

{
    "name": "Asgardeo",
    "type": "OIDC",
    "properties": [
        {
            "name": "issuer",
            "value": "https://api.asgardeo.io/t/sadiltestchoreo/oauth2/token"
        },
        {
            "name": "jwks_endpoint",
            "value": "https://api.asgardeo.io/t/sadiltestchoreo/oauth2/jwks"
        },
        {
            "name": "token_exchange_enabled",
            "value": "true"
        },
        {
            "name": "client_id",
            "value": "YN5R_Sw1rCX9spEHeNfz4SPPM6wa",
            "isSecret": false
        }
    ],
    "attributeConfiguration": {
        "userTypeResolution": {
            "externalAttribute": "user_type",
            "valueMapping": {
                "employee": "Employee",
                "customer": "Person"
            },
            "default": "Person"
        },
        "transforms": [
            {
                "externalAttribute": "given_name",
                "transform": {
                    "type": "toUpperCase"
                }
            },
            {
                "externalAttribute": "org_name",
                "valueMapping": {
                    "Acme Corp": "ACME CORPORATION"
                }
            }
        ],
        "sync": {
            "default": "override",
            "attributes": [
                {
                    "externalAttribute": "given_name",
                    "policy": "preserve-local"
                }
            ]
        },
        "accountLinking": {
            "attributes": [
                "email"
            ]
        },
        "userTypeAttributeMappings": [
            {
                "userType": "Person",
                "attributes": [
                    {
                        "externalAttribute": "given_name",
                        "localAttribute": "firstName"
                    },
                    {
                        "externalAttribute": "address.email",
                        "localAttribute": "email"
                    },
                    {
                        "externalAttribute": "org_name",
                        "localAttribute": "orgName"
                    }
                ]
            }
        ]
    }
}

Structure

Section	Purpose
userTypeResolution	Resolves which local user type the incoming identity becomes: reads externalAttribute, maps via valueMapping, and falls back to default.
transforms	Value normalization applied to the raw external value before mapping (e.g., case normalization, value normalization, type coercion). This operates per external attribute and is source-side, so shared across all user types.
sync	On re-login of a linked user, controls whether the incoming value overwrites the stored local value. Includes a default policy with per-attribute exceptions.
accountLinking	Local attributes used to match an incoming identity to an existing local user before provisioning.
userTypeAttributeMappings	Per-user-type externalAttribute → localAttribute mappings. The only section scoped by user type.

Processing pipeline

1. Resolve user type — userTypeResolution picks the local user type.
2. Transform — each raw external value is normalized via transforms (source-side, before mapping).
3. Map — userTypeAttributeMappings for the resolved type maps externalAttribute → localAttribute.
4. Link / provision — accountLinking attributes are used to find an existing local user; if none, JIT-provision as the resolved user type.
5. Sync — on subsequent logins, sync decides per attribute whether the IdP value overwrites the stored local value.

Design principle

Each concern lives where it actually varies:

• Transforms and sync act on the source attribute, independent of destination → defined once at the IdP level, keyed by external attribute. They are "normally common", as expected.
• Mapping is the only inherently per-user-type step → lives in userTypeAttributeMappings.

Phasing

Phase 1 (shipped) is userTypeResolution.default + userTypeAttributeMappings with plain externalAttribute → localAttribute. transforms, sync, accountLinking, and claim-driven userTypeResolution are later.

[rajithacharith]

While I don't have a specific use case in mind at the moment, it may be worth considering support for mapping a single external claim to multiple local claims. This could provide additional flexibility for future integration scenarios where one external attribute needs to populate several local attributes.

[sadilchamishka]

I have updated the model but the question still valid.
Let's assume IDP send the sub claim as an email. We need to map the sub claim for both email and username as we have kind of email as username setup. I hope you meant such a case.

It should be achievable as follows. The validation layer should make sure it only map to one localAttribute. I assume previous comment might implied it wrongly.

"userTypeAttributeMappings": [
      {
          "userType": "Person",
          "attributes": [
              {
                  "externalAttribute": "sub",
                  "localAttribute": "username"
              },
              {
                  "externalAttribute": "sub",
                  "localAttribute": "email"
              }
          ]
      }
  ]

If you mean for local attributes of multiple userTypes, that also possible with the proposed model as we have attribute mappings per user type.

[ThaminduDilshan]

Reserved/standard token claims (sub, iss, aud, exp, nbf, iat, jti, scope, client_id, act, grant_type) cannot be mapping targets — so a mapping can't alter token identity or authorization semantics.

Why do we need to prevent mapping these claims? Well we can keep some claims such as iss, exp, nbf, etc reserved. But some claims we might still need to allow mappings. Ex: sub to userId

[sadilchamishka]

Yes the restriction for external claim of reserved claim mapping to local claim is unnecessary. Though those claims are not practically useable except sub claim.

As per my reply for the above comment, as you mentioned we should be able to map the sub extrenalAttribute to any localAttribute. But I think we shouldn't allow for userID (One reason is userId is not an attribute, and it is generated id). But I assume you imply an userID attribute defined under the user type, hence mapping should be allowed.

[ThaminduDilshan]

But I think we shouldn't allow for userID (One reason is userId is not an attribute, and it is generated id)

Agree. Maybe we shouldn't allow this.

Use case I'm thinking is related to this requirement #3032. Here the requirement is users can signup with both their google and github accounts. In the current implementation, these will be created as two distinct users in thunderid. Ideally what they need is account linking, but we're trying to see whether we can cater that requirement with attribute mapping. I.e. google return their unique id as sub (or id). Github return it as id. I'm thinking of creating two attributes in the user type like google_fed_id and github_fed_id, so that both these signups can be linked to the same internal user.

[sadilchamishka]

The Entity holds two types of attributes:

• ATTRIBUTES — attributes defined by the user type schema.
• SYSTEM_ATTRIBUTES — attributes reserved for internal system operations.

For associating an IDP subject with a user, SYSTEM_ATTRIBUTES is the right place rather than extending the user type schema with IDP-specific fields like google_fed_id. This keeps the user type schema clean and avoids polluting it with federation concerns that are fundamentally infrastructure-level metadata.
We could store federated identities as a structured list under SYSTEM_ATTRIBUTES:

{
  "federated_ids": [
    { "idp": "<google-idp-id>", "sub": "<google_subject>" },
    { "idp": "<github-idp-id>", "sub": "<github_subject>" }
  ]
}

This way, when a user signs up via Google and later via GitHub, both federation records can be attached to the same internal user entry — which directly addresses the account linking requirement in #3032 without needing dedicated per-IDP fields on the user type.

Also the user can fetch the linked accounts based on this information we have stored under system attributes. The system attributes are not exposed as normal attributes, but we can let edit via different APIs which is not related to attribute update.
Ex: Association removal API. Explicit association creation API.

[ThaminduDilshan]

Configurable subject claim (@ThaminduDilshan's point 2) — separate effort.

Can't we take this also to the same effort? It's just special casing userid mapping right? Maybe on a seperate thought, rather than special casing it, we may able to map it to the userid claim as well in the claim mappings. Is there any other use case to have subject claim separately?

[sadilchamishka]

The sub claim of the application issued token will be decided by the application. So I assume here we discussing about the sub claim of the exchanged token.
As we are doing attribute mapping for userType defined attributes, we can think of which attribute should be the sub claim seperatly.

May be as an addition to the above model.

"attributeConfiguration": {
    "userTypeResolution" : {},
     "userTypeAttributeMappings" : [],
     "accountLinking": {},
     "sync": {},
     "subjectAttribute":  "email"
}

[ThaminduDilshan]

No. What I meant is allowing to use a external id as the thunderid user id. With subject claim, they can specify which attributes to be used as internal user id. Maybe this is a extended usecase and may not be required for now

[sadilchamishka]

Implementation update — what shipped vs. the refined model

The implementation for this effort landed in #3127. A couple of things changed from the refined model above, based on review feedback:

1. "user type" → "entity type" in the model.
The config now uses entityTypeResolution / entityTypeAttributeMappings / entityType instead of the userType* names. Rationale: the backend already speaks "entity type" (the entitytype package, EntityTypeServiceInterface, TypeCategory*), entity types cover users today and agents later. This iteration still resolves against the user category; agent support slots in later without a breaking change.

"attributeConfiguration": {
  "entityTypeResolution": { "default": "Person" },
  "entityTypeAttributeMappings": [
    {
      "entityType": "Person",
      "attributes": [
        { "externalAttribute": "given_name",    "localAttribute": "firstName" },
        { "externalAttribute": "address.email", "localAttribute": "email" }
      ]
    }
  ]
}

2. Mapping moved to the authn service layer (not the flow executor).
Per @ThumulaPerera / @ThaminduDilshan — for atomic auth APIs the transformation belongs in one place and the executor should stay lightweight. Mapping is now applied inside the per-IDP-type authn service's Authenticate() (OAuth/OIDC; GitHub and Google inherit it by wrapping those services), and in the token-exchange validator. The flow executors are unchanged from main.

3. Entity-type resolution no longer drives JIT provisioning.
Per the feedback that we shouldn't rely on IDP claims to pick the provisioning type, entityTypeResolution.default now only (a) selects which mapping entry applies and (b) validates mapping targets against that entity type's schema at write time. JIT provisioning resolves the entity type through the existing flow logic (app-allowed + self-registration).

What's intentionally deferred (kept as additive roadmap, not implemented):
transform / valueMapping, sync / useForAccountLinking, claim-driven entityTypeResolution.externalAttribute, and multiple entity types per IDP — each is a new optional key on the existing shape, so no breaking change when added.

Re: reserved claims / subject mapping (@ThaminduDilshan's points) — the reserved set still blocks mapping sub/iss/exp/… as targets in this iteration; making the subject claim mappable (e.g. sub → user id) is a good follow-up and the shape doesn't preclude it. Happy to scope that separately.

[ayshsandu]

Is the attribute configuration defined at the Federated IDP level?
Why not externalize it and define it in a common place, and then keep a reference from the IdP configurations?

How would it work when you define a trusted issuer for token exchange?

Will the claim transformation work for the JWT bearer grant - https://datatracker.ietf.org/doc/html/rfc7523?

[sadilchamishka]

The answer is related to the same comment #3157 (reply in thread)
The IDP (integration/connection) can be used for federated login, trusted token issuer, (may be outbound provision).
For all the cases, the attribute mappings defined at the IDP level is used.

From IDP to IDP the attribute mapping can be changed as here IDP refers to an external system, templating it might not provide the advantage. But we can further discuss whether we have such use cases and implement the templating approach.