Unknown model_tokens silently falls back to 8192 and truncates — warning goes to stderr, once per process, and never reaches the result

[SilverBulletNeo]

Summary

When a model is not in the token table, the graph warns once to stderr and then proceeds with an 8192-token window. For a model with a 1M-token context this silently truncates every non-trivial page, changing the answer without failing, without appearing in the returned object, and without any indication in execution_info.

Max input tokens for model google_genai/gemini-2.5-flash not found, please specify
the model_tokens parameter in the llm section of the graph configuration.
Using default token size: 8192

Why the default is the problem

A silent 8192 window is a plausible answer generator. The run succeeds, the JSON validates, the fields look reasonable — and the model simply never saw the part of the page that mattered. On a long privacy policy or a services page, the extracted answer flips based on where the truncation landed.

Two properties make it easy to miss:

• the warning is on stderr, so it vanishes in any batch/worker/async context — I hit this inside a 15-agent pipeline where nothing surfaced it
• it is emitted once per process, not once per run, so a long job warns on the first URL and stays silent for the remaining hundreds

What I'd suggest, in preference order

1. Raise when model_tokens is absent for an unknown model, with the message telling the user to set it. Loud beats quietly wrong for something that changes results.
2. If a hard raise is too breaking, keep the fallback but put it in the result — e.g. execution_info["truncated"] = True plus the effective window — so downstream code can detect it.
3. Failing both, make the default large rather than small. An over-long prompt fails visibly at the provider; an over-short one fails invisibly.
4. Independently: gemini-2.5-flash / gemini-flash-latest are worth adding to the token table — Google's current flash line is 1M input.

Related

#1099 / #1002 cover model_tokens being rejected on some provider paths. This is the opposite case: it is accepted, absent, and silently defaulted.

Environment

scrapegraphai fresh from PyPI, Python 3.13, macOS, langchain-google-genai (note: not bundled — installing it is a required extra step for Google backends), Playwright chromium headless.

Happy to open a PR for (1) or (2) if you indicate which you'd accept.

[dosubot]

Hi @SilverBulletNeo! I'm Dosu and I'm helping the ScrapeGraphAI, Inc team.

Your analysis is accurate. The fallback lives in abstract_graph.py lines 207–222 — when the model/provider pair isn't in the models_tokens registry, a logger.warning is emitted and self.model_token is hard-coded to 8192 [1]. That token budget is then used directly as the chunk_size for document splitting [2], so for a 1M-context model this silently over-chunks and the model never sees the full page — exactly the failure mode you describe.

On gemini-2.5-flash: confirmed missing from the registry. The google_genai section currently has gemini-2.0-flash-latest (1M) and gemini-2.0-pro-exp (2M), but nothing for the 2.5 line or gemini-flash-latest [3]. No open PR adds them.

On the 8192 default: your option (1) — raising on an unknown model — would be the cleanest fix. The current behavior is a plausible-but-wrong silent degradation, which is worse than a loud failure. Your observation about logger.warning vanishing in batch/async contexts and firing only once per process (Python's logging dedup) makes this especially dangerous at scale.

Workaround for now: you can set model_tokens explicitly in your config to bypass the lookup entirely:

graph_config = {
    "llm": {
        "model": "google_genai/gemini-2.5-flash",
        "api_key": "...",
        "model_tokens": 1000000,
    },
}

This takes the else branch at line 221 and skips the registry entirely. Note that on v2.1.4 model_tokens leaked into the LLM client call [4], but that was fixed in v2.1.5 via PR #1100 [5], so make sure you're on ≥ 2.1.5.

A PR for (1) or (4) — or both — would be a solid contribution.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.