fix: pop model_tokens so it is not forwarded to the model client

[dylanpfirn]

Description

Fixes #1099.

On the plain llm-config path, AbstractGraph._create_llm() reads model_tokens into self.model_token but never removes it from llm_params. llm_params is then splatted into init_chat_model(**llm_params), so model_tokens is forwarded to the model client. With a ChatOpenAI client (any OpenAI-compatible base_url) this surfaces as:

TypeError: Completions.create() got an unexpected keyword argument 'model_tokens'. Did you mean 'max_tokens'?

This one-line change pops model_tokens after it is consumed, so it can't leak to the client. It mirrors the existing model_instance path, which already returns before the key can reach a client (and is why that path works today).

Repro (before this patch)

from scrapegraphai.graphs import SmartScraperGraph

graph_config = {
    "llm": {
        "api_key": "<key>",
        "model": "openai/gpt-4o-mini",
        "base_url": "https://<any-openai-compatible-endpoint>/v1",
        "model_tokens": 128000,
    },
}
SmartScraperGraph(prompt="Summarize", source="https://example.com", config=graph_config).run()

Change

else:
    self.model_token = llm_params["model_tokens"]

# Consumed by ScrapeGraphAI; must not be forwarded to the model client.
llm_params.pop("model_tokens", None)

Type of change

• Bug fix (non-breaking change which fixes an issue)

Notes

Verified locally against an OpenAI-compatible endpoint (Cloudflare AI Gateway /compat): with the patch, passing model_tokens via plain config no longer raises and the value is correctly used for chunk sizing.

[github-actions]

🎉 This PR is included in version 2.1.5 🎉

The release is available on:

• v2.1.5
• GitHub release

Your semantic-release bot 📦🚀