Bug: `x-manifest-tier: [custom]` header causes `o3-deep-research` requests to fail or return empty output

[izzoa]

First and foremost, great job on this! Absolutely LOVE this router, use it daily, and have recommended it to myriad people. Keep up the great work!

Summary

When the x-manifest-tier: research (from the custom option) header is sent to a local Manifest endpoint along with model: openai/o3-deep-research and a high max_completion_tokens value (≥ 4000), the request either:

1. Returns HTTP 500 after ~180s with "Manifest encountered an internal error. Try again shortly." (when max_completion_tokens ≥ 8000), or
2. Returns 200 with choices[0].finish_reason = "length" and choices[0].message.content = null / "" (when max_completion_tokens is in the 4000–8000 range) — i.e. the tier handler consumed the entire token budget on routing logic but never produced visible output.

Removing the x-manifest-tier: research header and hitting the same endpoint with the same model, auth, and body returns a valid 200 in ~15–20s with real content.

Environment

• Endpoint: http://<MANIFEST_HOST>:3001 (local Manifest instance, OpenAI-compatible /v1/chat/completions)
• Tier config: The dashboard "Research" tier is configured with OpenAI: o3 Deep Research as the primary model.
• Auth: Standard Manifest API key (mnfst_*), validated against the same endpoint.
• Client: Python urllib.request through a local reverse proxy that injects x-manifest-tier: research based on URL path.
• Manifest version: unknown — /v1/models returns only {"id": "auto", "object": "model", "type": "model", "display_name": "Manifest Auto"}, so no version endpoint was found.

Reproduction

All tests hit http://<MANIFEST_HOST>:3001/v1/chat/completions directly.

Test	x-manifest-tier	max_completion_tokens	Result
1	(none)	16384	✅ 13.3s, finish_reason=stop, content_len=404
2	research	16384	❌ HTTP 500 after 180.0s — {"error":{"message":"Manifest encountered an internal error. Try again shortly.","type":"server_error"}}
3	research	4000	⚠️ 73.2s, finish_reason=length, content_len=0
4	(none)	16384	✅ 11.7s, content_len=484 (reproduction confirmed)

Reproduction script (Python)

import json, urllib.request, time, yaml

# Load Manifest API key
with open('/path/to/config.yaml') as f:
    cfg = yaml.safe_load(f)
key = next(p['api_key'] for p in cfg['custom_providers']
           if p.get('name') == 'ManifestResearch')

url = "http://<MANIFEST_HOST>:3001/v1/chat/completions"
body = json.dumps({
    "model": "openai/o3-deep-research",
    "messages": [{"role": "user",
                  "content": "List 3 Phase II GLP-1 agonists in development. One sentence each."}],
    "max_completion_tokens": 16384,
}).encode()

# Failing call: WITH tier header
req = urllib.request.Request(url, data=body, method="POST", headers={
    "Authorization": f"Bearer {key}",
    "Content-Type": "application/json",
    "User-Agent": "HermesAgent/1.0",
    "x-manifest-tier": "research",   # <-- this header triggers the bug
})
t0 = time.time()
try:
    with urllib.request.urlopen(req, timeout=300) as r:
        print(json.loads(r.read()))
except urllib.error.HTTPError as e:
    print(f"HTTP {e.code} after {time.time()-t0:.1f}s — {e.read()[:300]}")

Expected: 200 in 15–30s with a substantive completion (~400+ chars of visible content).

Actual (with x-manifest-tier: research): HTTP 500 after 180s, or content: null / content_len: 0 with finish_reason: length at lower token budgets.

Hypothesis

The research tier handler appears to have a bug specifically when routing openai/o3-deep-research. The tier wrapping adds overhead such that:

• High token budgets: the upstream OpenAI call exceeds an internal Manifest timeout (likely ~100s), surfacing as a generic 500.
• Low token budgets: the tier's own processing consumes the entire max_completion_tokens budget before delegating to OpenAI, leaving no budget for actual model output — consistent with the tier handler doing token-consuming work (routing, transformation, or a pre-LLM call) that the client isn't aware of.

The bug does not reproduce when:

• The x-manifest-tier header is omitted entirely (untiered path works perfectly).
• Lower-tier models are used with the research tier header (e.g. gpt-5.5 and other smaller models via the tier path return valid completions).

Impact

This makes openai/o3-deep-research unusable through the research tier — the exact model the tier is configured to use as primary. Clients that need deep-research behavior must either:

1. Strip the x-manifest-tier: research header (losing all tier-specific routing/observability), or
2. Use a different model entirely (losing deep-research capability).

Further Questions

I haven't investigated other research-esque long-running models (Perplexity deep-research, etc), but I'm wondering if they'd have the same issue. Anyway, something to consider. Thank you for your attention to this!