test/fix: update fixtures for NEW-N/governance validators + storage decomposition (audit findings)

[guangyu-reflexio]

Update test fixtures for hardened validators + storage decomposition (found by the test-skill audit)

A live audit run surfaced test-vs-code drift where fixtures weren't updated after model/validation refactors. All are test/fixture fixes (+ lint/format); no product-logic change.

• Consolidation eval (21 failing tests) — builder helpers + fixtures defaulted new_id='n1', which the _coerce_new_id NEW-N validator now rejects. Updated to NEW-1/NEW-N (helpers, _case(), illustrative_cases.json, test_real_judge_smoke).
• Playbook optimizer (3 failing tests) — set_source_windows_for_agent_playbook gained a governance existence-check; tests now create the UserPlaybook rows first.
• Benchmark — test_retrieval_latency_smoke needs real embeddings; added @skip_low_priority so it isn't run in the standard tier.
• Lint — # noqa: S101 on a pyright-invariant assert; SIM117/UP031 cleanups; ruff format normalization.

Test plan

• uv run pytest tests/eval/consolidation/test_consolidation_eval.py tests/server/services/playbook_optimizer/test_playbook_optimizer.py -o 'addopts=' → 57 passed, 2 skipped.
• ruff check + ruff format --check clean.

Summary by CodeRabbit

• Bug Fixes

  • Improved billing event coverage by adding a negative check to ensure no event is recorded when nothing is surfaced.
  • Adjusted one benchmark smoke test to skip in more low-priority scenarios.

• Tests

  • Updated several test cases and fixtures to use new identifier values and dynamic IDs.
  • Refined multiple assertions and mock setups for clearer, more consistent test behavior.

• Style

  • Cleaned up formatting across code and tests for readability, with no functional changes.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 77a3bcc1-dbf5-40ab-b954-dac50ac1153e

📥 Commits

Reviewing files that changed from the base of the PR and between d3d5a49 and c41f062.

📒 Files selected for processing (42)

• reflexio/server/llm/_litellm_text_generation.py
• reflexio/server/llm/litellm_client.py
• reflexio/server/services/extraction/resume_scheduler.py
• tests/benchmarks/test_retrieval_latency_smoke.py
• tests/cli/test_env_loader_user_env_file.py
• tests/conftest.py
• tests/e2e_tests/test_complete_workflows.py
• tests/e2e_tests/test_failure_path_polarity_e2e.py
• tests/eval/consolidation/fixtures/illustrative_cases.json
• tests/eval/consolidation/test_consolidation_eval.py
• tests/lib/test_config_unit.py
• tests/lib/test_generation_unit.py
• tests/lib/test_search_unit.py
• tests/models/test_lineage_models.py
• tests/server/api_endpoints/test_applied_learnings_metering.py
• tests/server/api_endpoints/test_evaluations_regenerate_api.py
• tests/server/api_endpoints/test_grade_on_demand_integration.py
• tests/server/cache/test_reflexio_cache.py
• tests/server/llm/test_openclaw_provider.py
• tests/server/llm/test_openclaw_provider_integration.py
• tests/server/services/agent_success_evaluation/test_evaluation_only_scheduler_integration.py
• tests/server/services/lineage/test_gc_scheduler_multitenant_integration.py
• tests/server/services/lineage/test_profile_expiry_reclamation_integration.py
• tests/server/services/lineage/test_reclamation_class_b_integration.py
• tests/server/services/playbook_optimizer/test_playbook_optimizer.py
• tests/server/services/profile/test_dedup_always_soft_integration.py
• tests/server/services/profile/test_profile_module_contract.py
• tests/server/services/storage/sqlite_storage/test_agent_run_characterization.py
• tests/server/services/storage/sqlite_storage/test_pending_tool_call_expiry_integration.py
• tests/server/services/storage/sqlite_storage/test_playbook_optimization_candidate_metadata_migration.py
• tests/server/services/storage/sqlite_storage/test_share_link_expiry_integration.py
• tests/server/services/storage/test_hard_delete_events_integration.py
• tests/server/services/storage/test_lineage_b1_statuschange_integration.py
• tests/server/services/storage/test_lineage_b3b_reconstruct_playbook_changelog_integration.py
• tests/server/services/storage/test_shadow_verdicts_contract_integration.py
• tests/server/services/storage/test_sqlite_lineage_columns_integration.py
• tests/server/services/storage/test_sqlite_merge_records_integration.py
• tests/server/services/storage/test_sqlite_share_links.py
• tests/server/services/tagging/test_module_contract.py
• tests/server/services/test_profile_source_filtering.py
• tests/server/test_billing_meter.py
• tests/server/test_logging_timezone.py

---

📝 Walkthrough

Walkthrough

This PR is primarily a large-scale code formatting pass across production and test files (line-wrapping, single/multi-line conversions, import reordering), combined with small targeted changes: renaming test fixture identifiers to uppercase, dynamic ID usage in playbook optimizer tests, a new test skip decorator, and an added assertion in a billing test.

Changes

Formatting cleanup and minor fixes

Layer / File(s)	Summary
Production code lint/formatting tweaks reflexio/server/llm/_litellm_text_generation.py, reflexio/server/llm/litellm_client.py, reflexio/server/services/extraction/resume_scheduler.py	Adds # noqa: S101 to an assert, inserts a blank line, and reformats a variable assignment to single-line; no behavior change.
Consolidation eval new_id renaming tests/eval/consolidation/fixtures/illustrative_cases.json, tests/eval/consolidation/test_consolidation_eval.py	Candidate new_id values changed from cand-*/n1 patterns to uppercase NEW-* identifiers in fixtures and test helper defaults.
Playbook optimizer dynamic ID tests tests/server/services/playbook_optimizer/test_playbook_optimizer.py	Tests updated to use dynamically generated UserPlaybook IDs instead of hardcoded IDs for source mapping, scenario resolution, and successor snapshot preservation checks.
Test skip conditions and billing assertions tests/benchmarks/test_retrieval_latency_smoke.py, tests/server/test_billing_meter.py	Adds skip_low_priority decorator to a smoke test and adds a missing assert_not_called() check in a billing meter no-op test, alongside formatting changes.
Broad test formatting-only changes tests/**/*.py (many files)	Line-wrapping, import reordering, and single/multi-line conversions applied across dozens of test files with no logic changes.

Estimated code review effort: 2 (Simple) | ~15 minutes

Possibly related PRs

• ReflexioAI/reflexio#222: Reformats the same test_added_then_superseded_still_in_added lineage changelog test originally introduced in that PR.
• ReflexioAI/reflexio#280: Both PRs touch _completion_with_hard_timeout in _litellm_text_generation.py, with this PR only adding a lint suppression to the assert added/modified there.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title is specific and matches the PR’s main test-fixture, validator, and storage-refactor updates.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/test-fixture-drift-2026-07

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}