Skip to content

Misleading error message on local evaluation 401: always blames personal_api_key even when project_api_key is the problem #526

@haacked

Description

@haacked

Summary

When local flag evaluation fails with a 401, the SDK logs a hardcoded message that always attributes the failure to `personal_api_key`, even when the actual cause is an invalid or rotated `project_api_key`.

Affected code

`posthog/client.py:1352–1356` — the 401 handler in `_fetch_feature_flags_from_api()`:

```python
except APIError as e:
if e.status == 401:
self.log.error(
"[FEATURE FLAGS] Error loading feature flags: To use feature flags, "
"please set a valid personal_api_key. More information: https://posthog.com/docs/api/overview"
)
```

Why this is wrong

The local evaluation endpoint requires two keys:

```
GET /api/feature_flag/local_evaluation/?token={project_api_key}&send_cohorts
Authorization: Bearer {personal_api_key}
```

(`request.py:344` for the Bearer header; `client.py:1295` for the URL with `token=`)

If either key is invalid, the server returns 401. The SDK unconditionally tells the user to fix `personal_api_key`.

Real-world impact

A customer (Zendesk #54814) rotated their `project_api_key` via the PostHog UI. Their local evaluation immediately started failing with 401. The error message told them their `personal_api_key` was invalid, so they regenerated their personal key and their feature flags secure API key — neither of which helped. The real fix was deploying the new `project_api_key`, which they eventually discovered independently. The misleading error cost significant debugging time.

The server already distinguishes between the two cases

The backend returns different `detail` fields depending on which key failed:

  • Invalid `?token=` (project API key): `routing.py:462` raises `AuthenticationFailed()` with no message → DRF default: `"Incorrect authentication credentials."`
  • Invalid personal API key (Bearer header): `auth.py:260` raises `AuthenticationFailed("Personal API key found in request Authorization header is invalid.")`

The SDK already captures this in `_process_response()` (`request.py:269`):

```python
raise APIError(res.status_code, payload["detail"], retry_after=retry_after)
```

So `e.message` in the `except APIError` block already contains the server's distinguishing message. The current fix ignores it entirely.

Suggested fix

Surface `e.message` from the server response rather than replacing it with a hardcoded string:

```python
if e.status == 401:
self.log.error(
f"[FEATURE FLAGS] Error loading feature flags: authentication failed (HTTP 401): {e.message}. "
"Check that both your project API key and your personal API key "
"(or Feature Flags secure API key) are valid. "
"More information: https://posthog.com/docs/api/overview"
)
```

This surfaces the server's specific message (which already identifies which key failed) while keeping the fallback guidance that covers both keys.

Cross-SDK status

All server-side SDKs that support local evaluation use the same two-key pattern. Error messaging quality varies:

SDK 401 message Issue
posthog-python "please set a valid personal_api_key" Worst — hardcoded, unambiguously wrong
posthog-node "Your project key or personal API key is invalid" Ambiguous but at least mentions both keys
posthog-ruby Debug-level log only: `"Failed to load feature flags: #{res}"` Silent — no user-visible error at all
posthog-go "Unable to fetch feature flags, status: 401" Generic — no mention of which key
posthog-php Passes server's `detail` field, or silent Depends on server response format
posthog-dotnet Generic "Unauthorized" Least specific
posthog-elixir N/A — no local evaluation support

Python is the most egregious (actively wrong), but most SDKs could be clearer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions