fix(litellm): make temperature=0 override opt-in via explicit REFLEXIO_LLM_SEED

[yilu331]

Summary

The seed-injection block in litellm_client.py was silently overriding any caller-configured temperature with 0.0 on every non-restricted model. The fallback seed_env = os.environ.get("REFLEXIO_LLM_SEED") or "42" meant the override always fired — LiteLLMConfig(temperature=0.3) and friends never reached the provider.

Behavior change

• seed=42 is still injected by default. Cheap, harmless on providers that ignore it, helpful on providers that honor it.
• The companion temperature=0.0 override now fires only when the operator explicitly opts in via REFLEXIO_LLM_SEED=<int>. Caller-configured temperature flows through unchanged otherwise.

To restore the prior "always deterministic" behavior in a deployment, set REFLEXIO_LLM_SEED=42 in the environment.

Tests

• Renamed test_non_restricted_model_defaults_to_deterministic_temperature → test_non_restricted_model_default_injects_seed_only; asserts seed is injected but configured temperature flows through.
• Added test_explicit_seed_env_forces_temperature_zero to cover the opt-in path with monkeypatch.setenv("REFLEXIO_LLM_SEED", "7").
• Patched _cheap_should_run_reject in test_should_run_before_extraction_combines_all_extractor_criteria so the new cheap pre-filter short-circuit doesn't pre-empt the mocked LLM call.

Test plan

• uv run pytest tests/server/llm/test_litellm_client_unit.py::TestTemperatureRestriction — all 13 pass.
• uv run pytest tests/server/services/test_profile_generation_service.py::test_should_run_before_extraction_combines_all_extractor_criteria — passes.
• Verified the pre-existing 44 unit-tier failures on main are unchanged by this PR (orthogonal — same count, same names, with or without the diff applied).

Summary by CodeRabbit

• Bug Fixes

  • Improved deterministic behavior for LLM requests: client now always supplies an integer seed (defaulting to 42 when unset or invalid) and only forces temperature to zero when an explicit seed is present and the model allows it.

• Tests

  • Updated and expanded unit tests to cover seed defaults, explicit seed handling, invalid seed fallback, and related prompt/response behavior.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: a43a76c4-0b97-497e-805b-34b691254986

📥 Commits

Reviewing files that changed from the base of the PR and between b16ce09 and d332c1b.

📒 Files selected for processing (2)

• reflexio/server/llm/litellm_client.py
• tests/server/llm/test_litellm_client_unit.py

🚧 Files skipped from review as they are similar to previous changes (2)

• reflexio/server/llm/litellm_client.py
• tests/server/llm/test_litellm_client_unit.py

---

📝 Walkthrough

Walkthrough

The LiteLLM client now always injects an integer seed (from REFLEXIO_LLM_SEED or default 42) and only forces temperature=0.0 when that env var is explicitly set; unit tests and a profile-generation test scaffold were updated to reflect these rules.

Changes

Seed and Temperature Determinism Updates

Layer / File(s)	Summary
Seed and temperature determinism logic reflexio/server/llm/litellm_client.py, tests/server/llm/test_litellm_client_unit.py	_build_completion_params always injects an integer seed (from REFLEXIO_LLM_SEED or default 42, warning + fallback for non-integers). temperature=0.0 is forced only when REFLEXIO_LLM_SEED is explicitly present and the model isn't temperature-restricted. Tests added/updated for absent, present, and invalid env values.
Profile generation service test support tests/server/services/test_profile_generation_service.py	test_should_run_before_extraction_combines_all_extractor_criteria now patches _cheap_should_run_reject to return None within the existing test context.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 I tuck a seed in every run tonight,

Forty-two by default, when none alight,
If you set the var, I hush the temp to zero,
If you set it wrong, I warn but keep the hero,
Tests hop along, keeping outputs tight.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title directly captures the main behavior change: making the temperature=0 override conditional on explicit REFLEXIO_LLM_SEED environment variable instead of applying it unconditionally.
Docstring Coverage	✅ Passed	Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/seed-injection-respects-explicit-env

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

• Generate code and open pull requests
• Plan features and break down work
• Investigate incidents and troubleshoot customer tickets together
• Automate recurring tasks and respond to alerts with triggers
• Summarize progress and report instantly

Built for teams:

• Shared memory across your entire org—no repeating context
• Per-thread sandboxes to safely plan and execute work
• Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@reflexio/server/llm/litellm_client.py`:
- Around line 773-780: The code currently skips setting params["seed"] when
REFLEXIO_LLM_SEED is present but not an int; update the try/except in the block
that reads seed_explicit/seed_raw so that on ValueError you set params["seed"]
to the default (int("42") or the same default used when REFLEXIO_LLM_SEED is
absent) and log a warning that the value was invalid and the default is being
used; reference the variables seed_explicit, seed_raw, params, and self.logger
so you locate and modify the seed parsing/except branch to always inject a seed.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 402224de-712c-406a-9ff4-0a7afef386c4

📥 Commits

Reviewing files that changed from the base of the PR and between 6cfdb8e and b16ce09.

📒 Files selected for processing (3)

• reflexio/server/llm/litellm_client.py
• tests/server/llm/test_litellm_client_unit.py
• tests/server/services/test_profile_generation_service.py