Langfuse SSO "No email found in the user profile" when idToken enabled, or "id_token detected in the response" when disabled

[vemonet]

Describe your question

Hi, I am trying to setup SSO for self-hosted Langfuse (docker compose) to an OIDC keycloak endpoint.

When I set AUTH_KEYCLOAK_ID_TOKEN=true : I am getting No email found in the user profile

According to this issue: #4310 (comment) nextauth/langfuse is trying to get the email directly from the auth callbak object (which contains the access_token) instead of calling the userinfo endpoint, and to make nextauth check for the userinfo endpoint we should disable ID_TOKEN

Then when I set AUTH_KEYCLOAK_ID_TOKEN=false I am getting:

[next-auth][error][OAUTH_CALLBACK_ERROR] 
https://next-auth.js.org/errors#oauth_callback_error id_token detected in the response, you must use client.callback() instead of client.oauthCallback() {
  error: Error [OAuthCallbackError]: id_token detected in the response, you must use client.callback() instead of client.oauthCallback()
      at async k (.next/server/pages/api/auth/[...nextauth].js:1:1123) {
    code: undefined
  },
  providerId: 'keycloak',
  message: 'id_token detected in the response, you must use client.callback() instead of client.oauthCallback()'
}

Which seems to indicate now nextauth complains that there is an ID token in the response so it can't do its job (which is weird because it could still just do its job with the ACCESS_TOKEN, it does not need to fail early like this), but we need to set ID_TOKEN=false to fix the issue with nextauth not calling the userinfo endpoint...

I have also manually implemented the auth flow, so I can tell you that our OIDC endpoint returns this object to the callback:

{
    "access_token": "###",
    "expires_in": 3600,
    "refresh_expires_in": 0,
    "token_type": "Bearer",
    "id_token": "###",
    "not-before-policy": 0,
    "session_state": "###-###-###-###-###",
    "scope": "openid profile email"
}

Note that when calling the userinfo endpoint I properly get an email returned as part of the user object.

Is there a solution to this issue? Thanks a lot

Langfuse Cloud or Self-Hosted?

Self-Hosted

If Self-Hosted

latest (3)

If Langfuse Cloud

No response

SDK and integration versions

No response

Pre-Submission Checklist

• I have checked for existing issues/discussions and consulted Langfuse AI.

[inkeep-ai-assistant[bot]]

For Keycloak SSO with Langfuse, there are specific configuration options and known data handling requirements you should be aware of.

Keycloak Configuration

For Keycloak, Langfuse supports the following environment variables(1):

• AUTH_KEYCLOAK_CLIENT_ID
• AUTH_KEYCLOAK_CLIENT_SECRET
• AUTH_KEYCLOAK_ISSUER
• AUTH_KEYCLOAK_ALLOW_ACCOUNT_LINKING=true (optional)
• AUTH_KEYCLOAK_ID_TOKEN (Defaults to true, set to false if you want to make a request to the userinfo endpoint instead of extracting user information from the id_token claims)(1)
• AUTH_KEYCLOAK_SCOPE (defaults to "openid email profile")(1)

The OAuth Redirect URL should be /api/auth/callback/keycloak(1).

Known Keycloak Data Handling

The Langfuse authentication code includes specific handling for Keycloak responses. When linking an account, the code removes incompatible fields returned by Keycloak(2):

// Keycloak returns incompatible data with the nextjs-auth schema
// (refresh_expires_in and not-before-policy in).
// So, we need to remove this data from the payload before linking an account.
// https://github.com/nextauthjs/next-auth/issues/7655
if (data.provider.endsWith("keycloak")) {
  // endsWith required as the multi-tenant cloud SSO providers are in the "domain.provider" format
  delete data["refresh_expires_in"];
  delete data["not-before-policy"];
}

(2)

Additional Configuration Options

If you're still experiencing issues with specific fields in the response, you can use the AUTH_IGNORE_ACCOUNT_FIELDS environment variable to remove fields that cause problems(1):

• AUTH_IGNORE_ACCOUNT_FIELDS: Comma-separated list of fields to ignore from the SSO IDP account when creating an account. Use this to correct errors with custom IDP providers(1).

You can also configure additional authentication checks using AUTH_KEYCLOAK_CHECKS(1).

Troubleshooting

The documentation notes that Langfuse authentication relies on email addresses, and when using SSO, you must ensure that the user's email address is included in their IDP profile(1). If authentication with your provider fails, try different applicable combinations of the CLIENT_AUTH_METHOD and CHECKS as outlined in the documentation(1).

Based on your description, setting AUTH_KEYCLOAK_ID_TOKEN=false should make Langfuse call the userinfo endpoint. The error you're encountering appears to be a NextAuth.js behavior issue rather than a Langfuse configuration issue. You might want to try the AUTH_IGNORE_ACCOUNT_FIELDS option to filter out problematic fields from the Keycloak response, or adjust the AUTH_KEYCLOAK_SCOPE if needed.

📚 Sources:

1. Authentication and SSO (self-hosted)
2. langfuse/langfuse/web/src/server/auth.ts

---

Have another question? Just tag @inkeep.

[vemonet]

Setting AUTH_IGNORE_ACCOUNT_FIELDS=id_token does not change anything

The issue is that if we enable AUTH_KEYCLOAK_ID_TOKEN then nextauth will not request the userinfo endpoint, and langfuse will try to get the email directly from the decoded id_token

If we disable AUTH_KEYCLOAK_ID_TOKEN, then nextauth will fail because there is an id_token in the response, so it will never try to get the email from userinfo

That means if the auth endpoint returns an id_token, then there is no other choice than including the email in this id_token, and we cannot make it so langfuse get the email from the userinfo endpoint

[Steffen911]

Hey @vemonet,
Are you aware of any options to do this automatically using nextauth? We're mainly relying on their auth behaviour and the available settings.

[vemonet]

There is this issue raising a similar problem in nextauth github repo: nextauthjs/next-auth#7750

But the person closed with this comment:

However, this doesn't change the fact that setting idToken: false breaks the app...

It does not seems really possible out of the box with nextauth unfortunately...

• If an id_token exists nextauth assumes it contains the profile claims
• Only if no id_token exists nextauth calls userinfo

The solution that seems to be recommended by nextauth would be to override the profile() function of the nextauth provider: https://next-auth.js.org/v3/configuration/providers#using-a-custom-provider

  async profile(profile, tokens) {
    // You can use the tokens, in case you want to fetch more profile information
    // For example several OAuth providers do not return email by default.
    // Depending on your provider, will have tokens like `access_token`, `id_token` and or `refresh_token`

In our case we could force the call to userinfo even when the profile has been retrieved from id_token (because this one does not have the email).

For example something along these lines might work:

if (
  env.AUTH_KEYCLOAK_CLIENT_ID &&
  env.AUTH_KEYCLOAK_CLIENT_SECRET &&
  env.AUTH_KEYCLOAK_ISSUER
)
  staticProviders.push(
    KeycloakProvider({
      clientId: env.AUTH_KEYCLOAK_CLIENT_ID,
      clientSecret: env.AUTH_KEYCLOAK_CLIENT_SECRET,
      issuer: env.AUTH_KEYCLOAK_ISSUER,
      idToken: env.AUTH_KEYCLOAK_ID_TOKEN === "true",
      allowDangerousEmailAccountLinking:
        env.AUTH_KEYCLOAK_ALLOW_ACCOUNT_LINKING === "true",
      authorization: {
        params: { scope: env.AUTH_KEYCLOAK_SCOPE ?? "openid email profile" },
      },
      client: {
        token_endpoint_auth_method: env.AUTH_KEYCLOAK_CLIENT_AUTH_METHOD,
      },
      ...(env.AUTH_KEYCLOAK_CHECKS ? { checks: env.AUTH_KEYCLOAK_CHECKS } : {}),
      

      // Add profile override when enabled
      ...(env.AUTH_KEYCLOAK_PROFILE_FROM_USERINFO === "true"
        ? {
            async profile(tokens) {
              const userinfo = await fetch(`${env.AUTH_KEYCLOAK_ISSUER}/protocol/openid-connect/userinfo`, {
                headers: {Authorization: `Bearer ${tokens.access_token}`},
              }).then((res) => res.json());
              return {
                id: userinfo.sub,
                email: userinfo.email,
                name: userinfo.name ?? userinfo.preferred_username ?? userinfo.email
              };
            },
          }
        : {}),
    }),
  );

[Steffen911]

@vemonet I think that has the potential to break existing flows that users have created around Keycloak, right? In that case, I would recommend to build your own image and apply this as a patch during the build process.
If you find a way to bring this upstream in a backwards-compatible way, we'd be happy to accept a PR around this.