Skip to content

fix: disable thinking on Moonshot calls so kimi-k2.6 populates content#623

Closed
jasonm4130 wants to merge 1 commit into
Graphify-Labs:v5from
jasonm4130:fix/kimi-disable-thinking
Closed

fix: disable thinking on Moonshot calls so kimi-k2.6 populates content#623
jasonm4130 wants to merge 1 commit into
Graphify-Labs:v5from
jasonm4130:fix/kimi-disable-thinking

Conversation

@jasonm4130

@jasonm4130 jasonm4130 commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Disable Moonshot's thinking mode for kimi-k2.6 by passing extra_body={"thinking": {"type": "disabled"}} on the OpenAI-compatible call
  • Gated on "moonshot" in base_url so other OpenAI-compatible providers (real OpenAI, Together, Groq) aren't affected — they'd 400 on the flag
  • Add 2 mock-based tests covering the moonshot path and the non-moonshot path

Why

kimi-k2.6 is a reasoning model. Its OpenAI-compatible response splits across two fields:

resp.choices[0].message.content            # the structured answer
resp.choices[0].message.reasoning_content  # chain-of-thought

With max_completion_tokens=8192, large chunks (the default chunk_size=20 regularly produces 50k+ token user messages) cause reasoning to consume the entire output budget. content comes back empty, _parse_llm_json("{}") returns {"nodes":[],...}, and the chunk silently contributes nothing.

Reproduction

162-file repo (~125k words, mixed code/docs/images), default settings:

input tokens output tokens nodes edges
Before 453,282 71,361 2 1
After 446,890 40,523 138 160

The 71k output tokens before were almost entirely reasoning_content that graphify ignores. Empty content, empty graph, full bill.

Test plan

  • test_kimi_call_disables_thinking — verifies the flag is sent on a moonshot URL
  • test_non_moonshot_call_does_not_disable_thinking — verifies the flag is not sent on api.openai.com
  • Full existing test suite (443 tests) passes with zero regressions

@jasonm4130
jasonm4130 force-pushed the fix/kimi-disable-thinking branch from 7835fec to b97ba3a Compare April 30, 2026 11:40
@jasonm4130 jasonm4130 changed the title fix: disable thinking for kimi-k2.6 so content is populated (PR #1 of 2) disable thinking on Moonshot calls so kimi-k2.6 populates content Apr 30, 2026
kimi-k2.6 is a reasoning model that writes its chain-of-thought to
`reasoning_content` and the answer to `content`. With graphify's default
`max_completion_tokens=8192`, large chunks (the default 20-file batches
frequently produce 50k+ token user messages) cause reasoning to consume
the entire output budget — `content` comes back empty, graphify parses
zero nodes, and the user pays for input + reasoning tokens without
getting a graph.

Reproduction on a 162-file repo (~125k words):
- Before: 9 chunks × 8192 tokens = ~74k completion tokens → 2 nodes total
- After:  same chunks → 138 nodes (chunk 1 alone: 42 nodes / 75 edges)

Moonshot's API exposes `extra_body={"thinking": {"type": "disabled"}}`.
For structured graph extraction we don't need chain-of-thought, so disable
it for Moonshot endpoints only — other OpenAI-compatible providers (real
OpenAI, Together, Groq) would 400 on the flag.
@jasonm4130
jasonm4130 force-pushed the fix/kimi-disable-thinking branch from b97ba3a to 528a3d2 Compare April 30, 2026 12:07
@jasonm4130 jasonm4130 changed the title disable thinking on Moonshot calls so kimi-k2.6 populates content fix: disable thinking on Moonshot calls so kimi-k2.6 populates content Apr 30, 2026
@Qodo-Free-For-OSS

Copy link
Copy Markdown

Hi, tests/test_llm.py patches openai.OpenAI, but CI does not install the optional openai dependency, so the patch target import will raise ModuleNotFoundError and fail the test suite.

Severity: action required | Category: reliability

How to fix: Install or skip openai tests

Agent prompt to fix - you can give this to your LLM of choice:

Issue description

CI does not install the optional openai dependency, but tests/test_llm.py patches openai.OpenAI, which requires importing openai. This causes ModuleNotFoundError and fails CI.

Issue Context

openai is only declared in the optional kimi extra, while CI installs .[mcp,pdf,watch].

Fix Focus Areas

Choose one:

  • Install openai in CI by adding the kimi extra (or all) to the editable install.
  • Or, make tests/test_llm.py skip when openai is not installed (e.g., pytest.importorskip("openai") or conditional skip using importlib.util.find_spec).

References:

  • .github/workflows/ci.yml[24-32]
  • pyproject.toml[44-55]
  • tests/test_llm.py[1-57]

We noticed a couple of other issues in this PR as well - happy to share if helpful.


Found by Qodo code review

@safishamsi safishamsi closed this in 3fdae8f May 1, 2026
@jasonm4130
jasonm4130 deleted the fix/kimi-disable-thinking branch May 2, 2026 03:24
matzls pushed a commit to matzls/graphify that referenced this pull request May 10, 2026
…bs#638 Graphify-Labs#589 Graphify-Labs#586 Graphify-Labs#593: kimi thinking, manifest, inline comments, query boost, cache race, markdownify, content hash

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants