LLM provider auth errors are sometimes surfaced as 500 API errors

[anders-swanson]

Checklist

• I have searched the existing issues for similar issues.
• I added a very descriptive title to this issue.
• I have provided sufficient information below to help reproduce this issue.

Summary

If I have an expired or invalid billing plan for an LLM provider (like OpenAI), the following error appears in the exporter log when attempting to use that LLM provider:

2025-Oct-27 16:50:27 (v0.0.0) - INFO     - (client.utils.api_call): POST Request: {'url': 'http://localhost:8000/v1/embed', 'headers': {'Authorization': '*****', 'Client': 'cf9476ab-caad-418d-8db5-c487a6be1952'}, 'timeout': 7200, 'params': {'client': 'cf9476ab-caad-418d-8db5-c487a6be1952', 'rate_limit': None}, 'json': {'vector_store': 'MYTESTDB_OPENAI_TEXT_EMBEDDING_3_SMALL_1536_308_COSINE_HNSW', 'alias': 'mytestdb', 'model': 'openai/text-embedding-3-small', 'chunk_size': 1536, 'chunk_overlap': 308, 'distance_metric': 'COSINE', 'index_type': 'HNSW'}}
2025-Oct-27 16:50:34 (v0.0.0) - INFO     - (client.utils.api_call): POST Response: <Response [500]>
2025-Oct-27 16:50:34 (v0.0.0) - ERROR    - (client.utils.api_call): 500 Server Error: Internal Server Error for url: http://localhost:8000/v1/embed/?client=cf9476ab-caad-418d-8db5-c487a6be1952

In the UI, a generic error message is shown (from embeddings page): "Failed to create table due to an unexpected error."

Inspecting the API server log, we see this is actually a billing error with the LLM provider:

[oracleai@5720e59d8584 ~]$ tail apiserver_8000.log
  File "/opt/.venv/lib64/python3.11/site-packages/openai/resources/embeddings.py", line 132, in create
    return self._post(
           ^^^^^^^^^^^
  File "/opt/.venv/lib64/python3.11/site-packages/openai/_base_client.py", line 1259, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/.venv/lib64/python3.11/site-packages/openai/_base_client.py", line 1047, in request
    raise self._make_status_error_from_response(err.response) from None
openai.RateLimitError: Error code: 429 - {'error': {'message': 'You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.', 'type': 'insufficient_quota', 'param': None, 'code': 'insufficient_quota'}}
2025-Oct-27 16:50:34 (v0.0.0) - INFO     - (uvicorn.access): 127.0.0.1:44890 - "POST /v1/embed/?client=cf9476ab-caad-418d-8db5-c487a6be1952 HTTP/1.1" 500

Suggested fix

Surface errors appropriately in the optimizer, so it is clear it is a 4xx error and not a 5xx error.

[gotsysdba]

Closed by #344