[Bug]: gpt-5-nano chat breaks with "context window exceeded" on a simple weather query (input grows to 372K tokens)

[FireTable]

What happened?

A simple chat query (今天天气如何, "what's the weather today") using the gpt-5-nano model fails with:

"Your input exceeds the context window of this model. Please adjust your input and try again."

Observability metrics for the run:

• LLM CALLS: 5
• TOOL CALLS: 4
• HITL: 1
• FAILED: 5 / 5 — every LLM call in the run failed
• INPUT: 372K tokens
• OUTPUT: 155 tokens
• TOTAL: 372K tokens
• DURATION: 28.08s

gpt-5-nano's context window is on the order of ~128K tokens (and likely smaller on some versions). 372K is several times over, so every call in the run collapses. The user then waits 28 seconds before seeing the error.

Notably, the user input itself is one short Chinese sentence (今天天气如何). The 372K tokens almost certainly come from accumulated memory system blocks and/or prior tool / turn results that the chat graph is re-injecting on every LLM call without any token-aware truncation.

What did you expect to happen?

• A simple, fresh-feeling query like "what's the weather today" should not blow the context window, even on a small model.
• The chat agent should either:
  • Cap / summarize memory and prior-turn content before each LLM call so the assembled prompt fits the chosen model's window, or
  • Fail fast (before the first LLM call) with a clear error if the assembled prompt exceeds the window, instead of burning 28s + 5 retries before showing the error.

Steps to reproduce

1. Sign in, open https://ai.firetable.tech/chat.
2. Use a session / thread where memory has accumulated over time (so the memory system block has grown).
3. With OPENAI_MODEL=gpt-5-nano (or whatever selector picks the nano model), send 今天天气如何 (or any short, fresh query).
4. Observe: 5 LLM calls, 5 failed, 28s wasted, final response is the context-window-exceeded error.

Severity

S1 — feature broken, no workaround from the user side. The user can't fix this by retrying; the only escape is to switch to a larger-context model or wipe memory manually.

Environment

• Site: https://ai.firetable.tech/chat
• Model: gpt-5-nano
• Deployed commit: 6730007 (refactor(env): replace NEXT_PUBLIC_* with window.__CONFIG__ injection)
• Input size at failure: 372K tokens (vs ~128K context window for the nano model)

Investigation hints

Likely root cause (for whoever picks this up):

• The chat graph likely prepends the memory system block and/or prior thread messages on every LLM call without checking the assembled token count against the chosen model's limit.
• Suggested mitigations:
  • Token-aware prompt assembly — count input tokens → summarize / truncate memory if it exceeds ~80% of the model's window.
  • Pre-call validation — reject before invoking the model when the assembled prompt is over the window, so the user gets a fast, clear error.
  • Smaller-window models should auto-summarize memory instead of letting raw accumulated memory flow through unchanged.

Relevant logs / screenshots

Self-checks

• I reproduced on main (current deployed commit 6730007).
• I searched existing issues and didn't find a duplicate.
• No secrets in this report.

[FireTable]

Root cause (confirmed)

Not token-unaware trimming as first suspected — it's the avatar. Better Auth UI's ChangeAvatar uploaded the picked image, and with no avatar.upload configured it fell back to fileToBase64(...), writing a base64 data URL into user.image.

getAuthInfo() reads user.image → mergeMemory() overlays it into the memory doc → the <memory> system block carries it on every LLM call. MEMORY_PROFILE_MAX_BYTES (8 KB) only caps the stored doc; the auth overlay is uncapped, so a ~280 KB base64 image became ~370 KB of prompt on a one-line query, and every call on gpt-5-nano overflowed.

Fix (branch fix/avatar-r2-not-base64)

1. New POST /api/avatar/presign — presigns a PUT to u/<userId>/avatar/<id>-<name>, returns the public URL only, no DB row; 503 when R2 unset.
2. ChangeAvatar uploads via presign and stores only the URL — base64 fallback removed. Upload is disabled when R2 isn't configured (gated on the existing ATTACHMENTS_ENABLED flag — reused, no new key).
3. Memory auth-overlay key renamed image → avatar.

Tests: presign route (401/400×3/201/503) + updated overlay tests. Full suite green (740).

Follow-up (not in this branch)

Existing users who already saved a base64 avatar still have it in user.image, so their prompts stay bloated until they re-upload or delete. A one-off cleanup (null out user.image values starting with data:) would fully clear the S1 for them — say the word if you want it.