feat(hooks): inject meta-rules into LLM context at session start#45
feat(hooks): inject meta-rules into LLM context at session start#45
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Gradata has reached the 50-review limit for trial accounts. To continue receiving code reviews, upgrade your plan.
|
Warning Rate limit exceeded
Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 0 minutes and 41 seconds. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughAdds optional meta-rule injection to the inject_brain_rules hook: if Changes
Sequence DiagramsequenceDiagram
participant Hook as inject_brain_rules.main()
participant FS as FileSystem
participant Loader as load_meta_rules()
participant Formatter as format_meta_rules_for_prompt()
participant Builder as Prompt Builder
Hook->>FS: check existence of brain_dir/system.db
alt system.db exists and helpers importable
Hook->>Loader: load_meta_rules(db_path)
Loader-->>Hook: meta_rules or error
Hook->>Hook: filter metas by INJECTABLE_META_SOURCES
Hook->>Hook: rank/sort and truncate to MAX_META_RULES
Hook->>Formatter: format_meta_rules_for_prompt(top_metas, context, limit=MAX_META_RULES)
Formatter-->>Hook: formatted_meta_block
alt formatted_meta_block non-empty
Hook->>Builder: append <brain-rules> + <brain-meta-rules>
else
Hook->>Builder: append <brain-rules> only
end
else
Hook->>Builder: append <brain-rules> only
end
Builder-->>Hook: return prompt
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested labels
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Deploying gradata-dashboard with
|
| Latest commit: |
56db167
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://b5dbeeb1.gradata-dashboard.pages.dev |
| Branch Preview URL: | https://feat-inject-meta-rules.gradata-dashboard.pages.dev |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/gradata/hooks/inject_brain_rules.py`:
- Around line 154-158: The current code slices metas by raw confidence into
metas_sorted before calling format_meta_rules_for_prompt, preventing
context-aware re-ranking from promoting lower-confidence but more relevant
rules; fix by removing the [:MAX_META_RULES] cap here (pass the full
sorted-by-confidence list or the original metas into
format_meta_rules_for_prompt) and apply the MAX_META_RULES limit only after
format_meta_rules_for_prompt's context-aware ranking, or alternatively add a
limit parameter to format_meta_rules_for_prompt (e.g., limit=MAX_META_RULES) and
have that function perform the context ranking first then truncate; update calls
accordingly (symbols: metas, metas_sorted, format_meta_rules_for_prompt,
MAX_META_RULES, context).
In `@tests/test_hooks_learning.py`:
- Around line 76-124: Add a parametrized test that verifies the MAX_META_RULES
cap and context-aware ordering by creating more than MAX_META_RULES MetaRule
instances via MetaRule and save_meta_rules, then calling inject_main with a
context that should boost a lower-confidence rule into the top-N result; assert
that the output contains exactly MAX_META_RULES meta-rules, includes the
promoted rule text, and preserves ordering by relevance. Locate test usage
around test_inject_also_emits_meta_rules_block_when_db_has_them /
test_inject_tolerates_missing_meta_rules_db and reuse tmp_path,
patch.dict(os.environ, {"GRADATA_BRAIN_DIR": str(tmp_path)}), ensuring the new
test references MAX_META_RULES constant and inject_main to exercise the
context-aware ranking path. Ensure the test is parametrized (e.g., varying
number of meta rules and context strings) to cover the boundary condition.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 1505dc64-f478-4140-9e18-4c4090e75c6c
📒 Files selected for processing (2)
src/gradata/hooks/inject_brain_rules.pytests/test_hooks_learning.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
- GitHub Check: Test (Python 3.12)
- GitHub Check: Test (Python 3.13)
- GitHub Check: Test (Python 3.11)
- GitHub Check: test (3.13)
- GitHub Check: Python 3.12
- GitHub Check: Python 3.11
- GitHub Check: test (3.11)
- GitHub Check: test (3.12)
- GitHub Check: Cloudflare Pages
🧰 Additional context used
📓 Path-based instructions (3)
tests/**
⚙️ CodeRabbit configuration file
tests/**: Test files. Verify: no hardcoded paths, assertions check specific values not just truthiness,
parametrized tests preferred for boundary conditions, floating point comparisons use pytest.approx.
Files:
tests/test_hooks_learning.py
src/gradata/**/*.py
⚙️ CodeRabbit configuration file
src/gradata/**/*.py: This is the core SDK. Check for: type safety (from future import annotations required), no print()
statements (use logging), all functions accepting BrainContext where DB access occurs, no hardcoded paths. Severity
scoring must clamp to [0,1]. Confidence values must be in [0.0, 1.0].
Files:
src/gradata/hooks/inject_brain_rules.py
src/gradata/hooks/**
⚙️ CodeRabbit configuration file
src/gradata/hooks/**: JavaScript hooks for Claude Code integration. Check for: no shell injection (no execSync with user
input), temp files must use per-user subdirectory, HTTP calls must have timeouts, errors must be silent (never block
the tool chain).
Files:
src/gradata/hooks/inject_brain_rules.py
Structural gap: meta-rules (tier-1 compound principles from 3+ graduated rules) were being created, stored, and loaded by the SDK — but never injected into the LLM prompt. The LLM saw <brain-rules> but never <brain-meta-rules>, which meant the entire compounding layer was dormant at the inference surface. Fix: inject_brain_rules.py now ALSO loads meta-rules from system.db via meta_rules_storage.load_meta_rules, caps them at MAX_META_RULES=5, context-ranks via format_meta_rules_for_prompt, and appends a <brain-meta-rules> block to the existing <brain-rules> block. Defensive: import is try/except so sites without the meta_rules module don't break. DB load is try/except so a corrupt/missing system.db degrades to "rules only" rather than failing the session start hook. Tests: 2 new — meta-rules appear in injection block when DB has them, and injection tolerates missing system.db without raising. Co-Authored-By: Gradata <noreply@gradata.ai>
…c principles
2026-04-14 ablation (432 trials, 3 LLMs, judged blind) showed deterministic
auto-generated meta-rule principles regress correctness:
Sonnet 4.6: -1.1% (full vs base)
DeepSeek V3: -1.4% (full vs base)
qwen2.5-coder:14b: drops correctness gain from +8.1% to +2.9%
The OSS clusterer produces principles like "Code: Avoid: foo. Prefer: bar"
or self-contradictory token frequencies. These are not safe to inject.
Changes:
- MetaRule gains a `source: str = "deterministic"` field. Default is the
safe option — pre-existing rows are auto-generated and stay excluded.
- INJECTABLE_META_SOURCES = {"llm_synth", "human_curated"}. Anything else
is excluded from prompt injection.
- meta_rules_storage: new ALTER TABLE migration adds the `source` column
with default 'deterministic'. save/load read+write the field.
- inject_brain_rules hook filters by source before formatting. If metas
exist but none are injectable, logs a debug line and skips silently.
- Tests: positive case uses source='llm_synth'; new negative case asserts
high-confidence deterministic metas are NOT injected.
When LLM synthesis ships (cloud-side discover_meta_rules), it must set
source='llm_synth' to flow through. Hand-curated brains can use
source='human_curated'.
Co-Authored-By: Gradata <noreply@gradata.ai>
Gradata
force-pushed
the
feat/inject-meta-rules
branch
from
0b46097 to
9dab67e
Compare
There was a problem hiding this comment.
Gradata has reached the 50-review limit for trial accounts. To continue receiving code reviews, upgrade your plan.
Replaces placeholder with output of export_ab_proof.py against .tmp/rule-ablation-v2 (Sonnet/DeepSeek/qwen14b × base/rules/full × 16 tasks × 3 iters, judged blind by Haiku 4.5). Headline numbers (with_full_mean — rules + meta-rules): - correctness: 0.833 → 0.832 (-0.1 pp) - preference_adherence: 0.732 → 0.755 (+2.3 pp) - quality: 0.793 → 0.799 (+0.6 pp) Note: with_rules_mean is higher than with_full_mean across all dimensions. The deterministic meta-rule overlay regresses results; this is the empirical evidence that drove the source-aware injection filter (PR #45). Co-Authored-By: Gradata <noreply@gradata.ai>
|
Amended with empirically-grounded source filter (9dab67e + force-push for clean rebase on main):
Empirical justification: 2026-04-14 ablation (432 trials, 3 LLMs, blind judge) showed deterministic meta-rules regress correctness on Sonnet (-1.1%), DeepSeek (-1.4%), and halve qwen14b's correctness gain from +8.1% to +2.9%. Cloud LLM synthesis (when shipped) sets source='llm_synth' to flow through. Hand-curated brains use source='human_curated'. |
…ort (#44) * feat(cloud): honest A/B proof — /public/proof endpoint backed by ablation Replaces the ABProofPanel's fabricated marketing copy ("200 blind expert evaluators", "3,000 comparisons", "70% win rate") with real ablation-backed numbers served by a new public endpoint. New pieces: - cloud/app/routes/proof.py: GET /public/proof, reads cloud/data/proof_results.json, returns {available, source, subjects, judge, trials, dimensions, per_model}. Graceful empty state if file missing or corrupt — never fabricates. - cloud/scripts/export_ab_proof.py: aggregates an ablation run's JSONL judgments into proof_results.json. Computes per-dimension means, 95% CIs, deltas, and per-model breakdown. Run: python cloud/scripts/export_ab_proof.py - cloud/data/proof_results.json: placeholder (overwritten by export script). - cloud/dashboard/src/components/brain/ABProofPanel.tsx: fetches /public/proof, shows real trials/subjects/judge when live, falls back to demo fixture with a visible "demo data" label when empty. Delta color flips on regressions (honest: we show -pp in red rather than only shipping positive results). - cloud/tests/test_proof.py: 5 new tests (missing file, present file, corrupt file, unauthenticated public access, export helper loadable). Pipeline: run ablation → run export script → redeploy cloud → dashboard lights up with honest data automatically. No marketing claims the data doesn't support. Co-Authored-By: Gradata <noreply@gradata.ai> * fix(cloud): address CR on #44 — wrong-shape JSON, with_full_mean, regression test - proof.py: validate parsed JSON is a dict before calling setdefault; degrade gracefully when someone drops a list/string into the results file (test coverage added). - ABProofPanel: use with_full_mean (rules + meta) for the 'rules+meta vs baseline' comparison cell, with fallback to best_mean, then to 0. Matches panel copy. - test_proof: regression test for valid-but-wrong-shape JSON (`[]`). Co-Authored-By: Gradata <noreply@gradata.ai> * data: commit real ablation results to proof_results.json (427 trials) Replaces placeholder with output of export_ab_proof.py against .tmp/rule-ablation-v2 (Sonnet/DeepSeek/qwen14b × base/rules/full × 16 tasks × 3 iters, judged blind by Haiku 4.5). Headline numbers (with_full_mean — rules + meta-rules): - correctness: 0.833 → 0.832 (-0.1 pp) - preference_adherence: 0.732 → 0.755 (+2.3 pp) - quality: 0.793 → 0.799 (+0.6 pp) Note: with_rules_mean is higher than with_full_mean across all dimensions. The deterministic meta-rule overlay regresses results; this is the empirical evidence that drove the source-aware injection filter (PR #45). Co-Authored-By: Gradata <noreply@gradata.ai> --------- Co-authored-by: Gradata <noreply@gradata.ai>
There was a problem hiding this comment.
Gradata has reached the 50-review limit for trial accounts. To continue receiving code reviews, upgrade your plan.
|
Addressed CR feedback (commit d1ebc51):
Full suite: 2074 passed, 23 skipped. |
|
@coderabbitai review — fixes pushed, please re-verify. |
|
Re-verify both fix batches from commits ✅ Actions performedFull review triggered. |
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/gradata/enhancements/meta_rules.py (1)
377-395:⚠️ Potential issue | 🟡 MinorHonor the
""contract whenlimitremoves every rule.
limit=0currently leavesmetasempty after ranking/slicing, but this function still returns the section header instead of"". That would leak an empty<brain-meta-rules>block and breaks the documented return contract. The same branch also behaves oddly for negative limits.Suggested fix
- # Re-rank by context weight when a context is provided. Pass `limit` - # through as `max_rules` so ranking + capping happens atomically; - # otherwise apply the cap after the fact (no ranking case). - if context: + if limit is not None and limit <= 0: + return "" + + # Re-rank by context weight when a context is provided. Pass `limit` + # through as `max_rules` so ranking + capping happens atomically; + # otherwise apply the cap after the fact (no ranking case). + if context: metas = rank_meta_rules_by_context( metas, context, max_rules=limit if limit is not None else len(metas), ) elif limit is not None: metas = metas[:limit] + + if not metas: + return "" lines = ["## Brain Meta-Rules (compound principles)"]🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/gradata/enhancements/meta_rules.py` around lines 377 - 395, After ranking/slicing, if the resulting metas list is empty the function must return an empty string to honor the "" contract; update the logic in the block using metas/rank_meta_rules_by_context so that after applying rank_meta_rules_by_context or metas = metas[:limit] you check if not metas and return "" immediately. Also normalize negative limit values (e.g., treat limit <= 0 as empty result or clamp negative values to None) so negative limits don't produce the header; adjust the use of limit when calling rank_meta_rules_by_context (pass max_rules=max(0, limit) or handle limit<=0 early) and then proceed to build the lines only when metas is non-empty.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/gradata/hooks/inject_brain_rules.py`:
- Around line 155-180: Wrap the entire meta-rule processing pipeline (not just
load_meta_rules) in a try/except so any exception during loading, filtering, or
formatting is caught and the code falls back to silent degradation (i.e., treat
metas as empty and skip injection). Specifically, surround the block that calls
load_meta_rules(db_path), filters into injectable, and calls
format_meta_rules_for_prompt(injectable, context=context, limit=MAX_META_RULES)
with a single try/except; on exception log at debug level and set
metas/injectable/formatted to empty values so meta_block is never set and
SessionStart continues normally.
In `@tests/test_hooks_learning.py`:
- Around line 224-237: Add a sibling test to
test_inject_tolerates_missing_meta_rules_db that writes a malformed-but-readable
system.db file into tmp_path (while still creating lessons.md) and then calls
inject_main({}) under the same with patch.dict(os.environ, {"GRADATA_BRAIN_DIR":
str(tmp_path)}): context; the malformed system.db should contain a deliberately
corrupted payload (e.g. truncated/invalid JSON or non-deserializable bytes) but
be present so inject_main sees a file, and the test should assert inject_main()
returns non-None, the result contains "<brain-rules>" and does NOT contain
"<brain-meta-rules>", and that no exception is raised; reuse the same test
scaffolding and names (e.g. test_inject_tolerates_missing_meta_rules_db -> new
test_corrupt_meta_rules_db_degrades_to_rules_only) and reference inject_main and
GRADATA_BRAIN_DIR to locate behavior to validate.
---
Outside diff comments:
In `@src/gradata/enhancements/meta_rules.py`:
- Around line 377-395: After ranking/slicing, if the resulting metas list is
empty the function must return an empty string to honor the "" contract; update
the logic in the block using metas/rank_meta_rules_by_context so that after
applying rank_meta_rules_by_context or metas = metas[:limit] you check if not
metas and return "" immediately. Also normalize negative limit values (e.g.,
treat limit <= 0 as empty result or clamp negative values to None) so negative
limits don't produce the header; adjust the use of limit when calling
rank_meta_rules_by_context (pass max_rules=max(0, limit) or handle limit<=0
early) and then proceed to build the lines only when metas is non-empty.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: acec069c-ff40-4300-a0d2-9fab209dc0d2
📒 Files selected for processing (3)
src/gradata/enhancements/meta_rules.pysrc/gradata/hooks/inject_brain_rules.pytests/test_hooks_learning.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Cloudflare Pages
🧰 Additional context used
📓 Path-based instructions (3)
tests/**
⚙️ CodeRabbit configuration file
tests/**: Test files. Verify: no hardcoded paths, assertions check specific values not just truthiness,
parametrized tests preferred for boundary conditions, floating point comparisons use pytest.approx.
Files:
tests/test_hooks_learning.py
src/gradata/**/*.py
⚙️ CodeRabbit configuration file
src/gradata/**/*.py: This is the core SDK. Check for: type safety (from future import annotations required), no print()
statements (use logging), all functions accepting BrainContext where DB access occurs, no hardcoded paths. Severity
scoring must clamp to [0,1]. Confidence values must be in [0.0, 1.0].
Files:
src/gradata/hooks/inject_brain_rules.pysrc/gradata/enhancements/meta_rules.py
src/gradata/hooks/**
⚙️ CodeRabbit configuration file
src/gradata/hooks/**: JavaScript hooks for Claude Code integration. Check for: no shell injection (no execSync with user
input), temp files must use per-user subdirectory, HTTP calls must have timeouts, errors must be silent (never block
the tool chain).
Files:
src/gradata/hooks/inject_brain_rules.py
Round-2 CR: a partially corrupt system.db can deserialize successfully and then blow up inside format_meta_rules_for_prompt (e.g. JSON null in source_lesson_ids becomes None and len() raises). The previous try/except only guarded load_meta_rules, so failures in the filter/format pipeline still aborted SessionStart. - Wrap entire load -> filter -> format pipeline in one try/except so any downstream failure degrades to rules-only injection. - Preserve context-aware cap: full injectable set still passed to formatter with limit=MAX_META_RULES so context ranking happens before slicing. - Add test_corrupt_meta_rules_db_degrades_to_rules_only as sibling to the missing-DB case; writes garbage bytes to system.db and asserts the hook returns only <brain-rules> without raising. Co-Authored-By: Gradata <noreply@gradata.ai>
|
Addressed round-2 CR (commit 58fa161):
Tests: 2074 pass, 23 skipped (one pre-existing flake in |
There was a problem hiding this comment.
Gradata has reached the 50-review limit for trial accounts. To continue receiving code reviews, upgrade your plan.
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
|
@coderabbitai review — CR Pro active, please re-verify. |
|
Re-verify all three fix batches from commits ✅ Actions performedFull review triggered. |
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/gradata/enhancements/meta_rules_storage.py (1)
160-167:⚠️ Potential issue | 🟠 MajorLegacy
meta_rulestables won't load until something writes the migration.
load_meta_rules()now selectssourceunconditionally, but read-only callers such assrc/gradata/hooks/inject_brain_rules.pynever invokeensure_table(). On an upgraded brain with an oldermeta_rulesschema, this raisesno such column: source, so meta-rules silently disappear until a later save happens. Either migrate before reading or probe the table schema and synthesize"deterministic"when the column is absent.Suggested fix
rows = conn.execute( - """SELECT id, principle, source_categories, source_lesson_ids, - confidence, created_session, last_validated_session, - scope, examples, context_weights, applies_when, never_when, - transfer_scope, source - FROM meta_rules - ORDER BY confidence DESC""" + f"""SELECT id, principle, source_categories, source_lesson_ids, + confidence, created_session, last_validated_session, + scope, examples, context_weights, applies_when, never_when, + transfer_scope, + {"source" if "source" in {r[1] for r in conn.execute("PRAGMA table_info(meta_rules)")} else "'deterministic'"} AS source + FROM meta_rules + ORDER BY confidence DESC""" ).fetchall()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/gradata/enhancements/meta_rules_storage.py` around lines 160 - 167, load_meta_rules() unconditionally selects the new "source" column causing "no such column: source" on older read-only DBs; change load_meta_rules() to first probe the meta_rules table schema (e.g., PRAGMA table_info or SELECT with zero rows) or catch the specific sqlite error and, if "source" is missing, synthesize source="deterministic" for each row instead of failing; alternatively, call/trigger ensure_table() migration defensively before reading, but prefer detecting missing column in load_meta_rules() and populating the returned dicts with source="deterministic" so callers like inject_brain_rules.py work without forcing a write migration.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/gradata/hooks/inject_brain_rules.py`:
- Around line 163-175: The current call to
format_meta_rules_for_prompt(injectable, context=context, limit=...) in
inject_brain_rules.py ignores rule gating based on session payload, causing
applies_when/never_when to be bypassed; update the call inside the injectable
handling block to pass the hook payload (or a normalized subset) via the
condition_context parameter (e.g., condition_context=hook_payload or a sanitized
payload dict) so format_meta_rules_for_prompt can evaluate
applies_when/never_when correctly during SessionStart; ensure you normalize/trim
sensitive fields before passing the payload.
---
Outside diff comments:
In `@src/gradata/enhancements/meta_rules_storage.py`:
- Around line 160-167: load_meta_rules() unconditionally selects the new
"source" column causing "no such column: source" on older read-only DBs; change
load_meta_rules() to first probe the meta_rules table schema (e.g., PRAGMA
table_info or SELECT with zero rows) or catch the specific sqlite error and, if
"source" is missing, synthesize source="deterministic" for each row instead of
failing; alternatively, call/trigger ensure_table() migration defensively before
reading, but prefer detecting missing column in load_meta_rules() and populating
the returned dicts with source="deterministic" so callers like
inject_brain_rules.py work without forcing a write migration.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 2aa04d93-d3c4-445f-885e-1a34f61649ad
📒 Files selected for processing (4)
src/gradata/enhancements/meta_rules.pysrc/gradata/enhancements/meta_rules_storage.pysrc/gradata/hooks/inject_brain_rules.pytests/test_hooks_learning.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Cloudflare Pages
🧰 Additional context used
📓 Path-based instructions (3)
src/gradata/**/*.py
⚙️ CodeRabbit configuration file
src/gradata/**/*.py: This is the core SDK. Check for: type safety (from future import annotations required), no print()
statements (use logging), all functions accepting BrainContext where DB access occurs, no hardcoded paths. Severity
scoring must clamp to [0,1]. Confidence values must be in [0.0, 1.0].
Files:
src/gradata/enhancements/meta_rules_storage.pysrc/gradata/enhancements/meta_rules.pysrc/gradata/hooks/inject_brain_rules.py
src/gradata/hooks/**
⚙️ CodeRabbit configuration file
src/gradata/hooks/**: JavaScript hooks for Claude Code integration. Check for: no shell injection (no execSync with user
input), temp files must use per-user subdirectory, HTTP calls must have timeouts, errors must be silent (never block
the tool chain).
Files:
src/gradata/hooks/inject_brain_rules.py
tests/**
⚙️ CodeRabbit configuration file
tests/**: Test files. Verify: no hardcoded paths, assertions check specific values not just truthiness,
parametrized tests preferred for boundary conditions, floating point comparisons use pytest.approx.
Files:
tests/test_hooks_learning.py
CodeRabbit round-3 feedback: 1. load_meta_rules() probes pragma for `source` column and synthesizes 'deterministic' when absent, so brains upgraded from a pre-source schema no longer raise `no such column: source` on read-only paths (inject_brain_rules never calls ensure_table()). 2. inject_brain_rules now forwards a sanitized hook payload as `condition_context`, so applies_when / never_when are honored during SessionStart instead of being silently bypassed. Only string/int/ float/bool fields from session_type/task_type/source/cwd are forwarded — no transcripts, file contents, or secrets. Regression tests: - test_load_meta_rules_legacy_schema_without_source_column: creates a legacy meta_rules table, confirms load returns rows with source='deterministic' instead of crashing. - test_inject_respects_applies_when_never_when: an llm_synth rule with never_when=session_type=code is excluded on code sessions and injected on prose sessions. All 2077 tests pass, 23 skipped.
There was a problem hiding this comment.
Gradata has reached the 50-review limit for trial accounts. To continue receiving code reviews, upgrade your plan.
|
Rebased on origin/main + round-3 CR addressed:
Full suite: 2077 passed, 23 skipped. |
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
There was a problem hiding this comment.
Gradata has reached the 50-review limit for trial accounts. To continue receiving code reviews, upgrade your plan.
|
@coderabbitai review — rebased on latest main, all prior fixes applied |
|
Re-verify the full patch after rebase on ✅ Actions performedFull review triggered. |
SessionStart injection picks top-N rules once for the whole session. JIT flips the axis: on every user prompt, score each graduated rule against the draft with a cheap Jaccard on word-unigrams and inject only the top-k that match THIS draft. Saves context budget, tightens per-prompt relevance. Gated by env var GRADATA_JIT_ENABLED (default false) so existing SessionStart behavior is unchanged for anyone who hasn't opted in. When both are active they complement: SessionStart = broad priors, JIT = tight per-prompt overlay. Emits JIT_INJECTION events to brain/events.jsonl for telemetry. Rule must clear both confidence floor (0.60 default) and similarity floor (0.05 default); rather inject zero rules than noise, same philosophy as the PR #45 source-filter gate. Hook wired as an additional UserPromptSubmit entry alongside context_inject so the two complement rather than collide. Co-Authored-By: Gradata <noreply@gradata.ai>
…68) * feat(jit): per-call rule injection hook with draft-relevance ranking SessionStart injection picks top-N rules once for the whole session. JIT flips the axis: on every user prompt, score each graduated rule against the draft with a cheap Jaccard on word-unigrams and inject only the top-k that match THIS draft. Saves context budget, tightens per-prompt relevance. Gated by env var GRADATA_JIT_ENABLED (default false) so existing SessionStart behavior is unchanged for anyone who hasn't opted in. When both are active they complement: SessionStart = broad priors, JIT = tight per-prompt overlay. Emits JIT_INJECTION events to brain/events.jsonl for telemetry. Rule must clear both confidence floor (0.60 default) and similarity floor (0.05 default); rather inject zero rules than noise, same philosophy as the PR #45 source-filter gate. Hook wired as an additional UserPromptSubmit entry alongside context_inject so the two complement rather than collide. Co-Authored-By: Gradata <noreply@gradata.ai> * refactor(jit): drop redundant zero-intersection branch in _jaccard Division by len(a|b) already yields 0.0 when intersection is empty. The early return was dead code. Co-Authored-By: Gradata <noreply@gradata.ai> --------- Co-authored-by: Gradata <noreply@gradata.ai>
1 participant
Summary
Closes a structural gap: meta-rules were being created, stored, and loadable — but never injected into the LLM prompt. The inject_brain_rules hook only emitted ``, so the entire compounding layer sat dormant at the inference surface.
Changes
Why it matters
Without this, the whole meta-rule subsystem was observable (via dashboard) but not operational (the LLM never saw the principles). Any session-to-session improvement from meta-rule emergence was silent. This PR makes the compounding layer actually influence the model.
Test plan