Implement normalization into canonical Formula with semantic checks (4.1.5) - #104
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
Core Formula modellingIntroduces a canonical formula AST in crates/sempai-core/src/formula.rs (exported via pub mod formula). Formula unifies legacy pattern* and v2 match syntaxes with variants: Atom (Pattern / Regex / TreeSitterQuery), unary operators (Not, Inside, Anywhere) and boolean composition (And, Or). Any node can be wrapped in Decorated carrying metadata (where_clauses, as_name, fix, span). Constraints are modelled as a Constraint enum (MetavariableRegex, MetavariablePattern, Other(String)). The new execplan docs/execplans/4-1-5-normalization-into-canonical-formula-model.md documents the design and acceptance criteria and is referenced by code and docs. Normalization pipelineAdds normalize_search_principal in crates/sempai/src/normalize.rs to lower parsed SearchQueryPrincipal variants (legacy pattern* forms, v2 match forms, and the r2c-internal-project-depends-on principal) into Decorated. Behaviour:
Semantic validationAdds validate_formula in crates/sempai/src/semantic_check.rs with:
Engine integrationImplements Engine::compile_yaml in crates/sempai/src/engine.rs:
Tracing helpers (crates/sempai/src/normalize_trace.rs) add discriminants and pattern/branch introspection to improve observability; compile_yaml records rule counts and principal discriminants. TestsLarge test additions and refactors to exercise normalization, validation and engine wiring:
Documentation & migrationUpdates and new docs:
DependenciesAdds serde_json and tracing to crates/sempai/Cargo.toml; adds insta and proptest to dev-dependencies for snapshot and property testing. Reviewer feedback applied / tracked follow-ups
Acceptance / validationPer tests and docs: normalization parity between legacy and v2 fixtures, schema-invalid diagnostics for malformed where-clauses, semantic diagnostics for the two enforced invariants, end-to-end Engine::compile_yaml producing QueryPlan(s) with Decorated, extensive unit/integration/property/snapshot tests, and documentation/migration guidance are present. The new execplan (docs/execplans/4-1-5-normalization-into-canonical-formula-model.md) is included and referenced by code and docs changes. WalkthroughSummarise the addition of a canonical Formula AST with Decorated metadata; implement normalization from legacy/v2 principals and structural parsing of where-clauses; add semantic validation and depth/shape checks; wire normalization into Engine::compile_yaml to emit per-language QueryPlans sharing Arc<Decorated>; add tests and docs. ChangesCanonical Formula Normalisation and Semantic Validation
Sequence Diagram(s)sequenceDiagram
participant Client as Client
participant Engine as Engine
participant Parser as Parser
participant Normaliser as Normaliser
participant Validator as Validator
participant Compiler as Compiler
participant QueryPlan as QueryPlan
Client->>Engine: compile_yaml(yaml_str)
Engine->>Parser: Parse YAML rules
Parser-->>Engine: Parsed rules (SearchQueryPrincipal list)
Engine->>Engine: Filter to Search principals
Engine->>Normaliser: normalize_search_principal(principal)
Normaliser-->>Engine: Decorated<Formula>
Engine->>Validator: validate_formula(formula)
alt Validation passes
Validator-->>Engine: Ok
Engine->>Compiler: compile_rule_plans(rule, Arc<formula>)
loop For each language
Compiler->>QueryPlan: QueryPlan::new(rule_id, language, Arc<formula>)
QueryPlan-->>Compiler: QueryPlan
end
Compiler-->>Engine: Vec<QueryPlan>
Engine-->>Client: Ok(Vec<QueryPlan>)
else Validation fails
Validator-->>Engine: Err(DiagnosticReport)
Engine-->>Client: Err(DiagnosticReport)
end
Possibly related issues
Possibly related PRs
Poem
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
|
![codescene-delta-analysis[bot]](https://avatars.githubusercontent.com/in/53921?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Reviewer's GuideImplements a canonical Formula model in sempai-core and wires a full normalization + semantic validation pipeline from YAML search rules through to Engine::compile_yaml, replacing the NOT_IMPLEMENTED placeholder with real QueryPlan construction and updating tests and docs accordingly. Sequence diagram for Engine::compile_yaml normalization and planningsequenceDiagram
actor User
participant Engine
participant YamlParser as sempai_yaml_parse_rule_file
participant ModeValidator as validate_supported_modes
participant Normalizer as normalize_search_principal
participant Semantic as validate_formula
participant Planner as compile_rule_plans
User->>Engine: compile_yaml(yaml)
Engine->>YamlParser: parse_rule_file(yaml, None)
YamlParser-->>Engine: RuleFile or DiagnosticReport
alt parse_error
Engine-->>User: Err(DiagnosticReport)
else parse_ok
Engine->>ModeValidator: validate_supported_modes(file)
ModeValidator-->>Engine: Ok or DiagnosticReport
alt mode_error
Engine-->>User: Err(DiagnosticReport)
else modes_ok
loop for each Rule in file.rules()
Engine->>Engine: principal = rule.principal()
alt principal_is_search
Engine->>Normalizer: normalize_search_principal(principal, rule.rule_span())
Normalizer-->>Engine: Decorated_Formula or DiagnosticReport
alt normalization_error
Engine-->>User: Err(DiagnosticReport)
else normalized
Engine->>Semantic: validate_formula(&formula)
Semantic-->>Engine: Ok or DiagnosticReport
alt semantic_error
Engine-->>User: Err(DiagnosticReport)
else validated
Engine->>Planner: compile_rule_plans(rule, &formula)
Planner-->>Engine: List~QueryPlan~ or DiagnosticReport
alt plan_error
Engine-->>User: Err(DiagnosticReport)
else plans_ok
Engine->>Engine: append plans to result
end
end
end
else non_search_principal
Engine->>Engine: skip or unsupported mode handled earlier
end
end
Engine-->>User: Ok(List~QueryPlan~)
end
end
Class diagram for canonical Formula model and decoratorsclassDiagram
class Formula {
+Atom atom
+Not not
+Inside inside
+Anywhere anywhere
+And and
+Or or
}
class Atom {
+Pattern pattern
+Regex regex
+TreeSitterQuery tree_sitter_query
}
class PatternAtom {
+String text
}
class RegexAtom {
+String pattern
}
class TreeSitterQueryAtom {
+String query
}
class Decorated_Formula {
+Formula node
+List~WhereClause~ where_clauses
+Option~String~ as_name
+Option~String~ fix
+Option~SourceSpan~ span
}
class WhereClause {
+Value raw
}
class SourceSpan
Formula --> Atom : uses
Atom --> PatternAtom : has
Atom --> RegexAtom : has
Atom --> TreeSitterQueryAtom : has
Decorated_Formula --> Formula : wraps
Decorated_Formula --> WhereClause : contains
Decorated_Formula --> SourceSpan : optional
Class diagram for Engine normalization and QueryPlanclassDiagram
class Engine {
-EngineConfig config
+compile_yaml(yaml : &str) Result~List~QueryPlan~~, DiagnosticReport~
}
class QueryPlan {
-String rule_id
-Language language
-Decorated_Formula formula
+new(rule_id : String, language : Language, formula : Decorated_Formula) QueryPlan
+rule_id() String
+language() Language
+formula() &Decorated_Formula
}
class Decorated_Formula {
+Formula node
+List~WhereClause~ where_clauses
+Option~String~ as_name
+Option~String~ fix
+Option~SourceSpan~ span
}
class Formula
class WhereClause
class SourceSpan
class EngineConfig
class DiagnosticReport
class Language
class Rule {
+id() &str
+languages() &List~String~
+rule_span() Option~SourceSpan~
+principal() &RulePrincipal
}
class RulePrincipal {
+Search search
+Other other
}
class SearchQueryPrincipal
class NormalizeModule {
+normalize_search_principal(principal : &SearchQueryPrincipal, rule_span : Option~&SourceSpan~) Result~Decorated_Formula, DiagnosticReport~
}
class SemanticCheckModule {
+validate_formula(formula : &Decorated_Formula) Result~(), DiagnosticReport~
}
Engine --> EngineConfig : has
Engine ..> QueryPlan : creates
Engine ..> Rule : uses
Engine ..> NormalizeModule : calls
Engine ..> SemanticCheckModule : calls
Engine ..> DiagnosticReport : returns
QueryPlan --> Language : has
QueryPlan --> Decorated_Formula : has
Rule --> RulePrincipal : has
RulePrincipal --> SearchQueryPrincipal : may_wrap
NormalizeModule ..> Decorated_Formula : returns
NormalizeModule ..> DiagnosticReport : error
SemanticCheckModule ..> DiagnosticReport : error
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
leynos
changed the title
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix: crates/sempai/src/tests/normalization_tests.rs Comment on file fn legacy_pattern_normalizes_to_atom() {
let legacy = LegacyFormula::Pattern(String::from("foo($X)"));
let principal = SearchQueryPrincipal::Legacy(legacy);
let result = normalize_search_principal(&principal, None).expect("should normalize");
assert_eq!(
result.node,
Formula::Atom(Atom::Pattern(PatternAtom {
text: String::from("foo($X)")
}))
);
}❌ New issue: Code Duplication |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix: crates/sempai/src/normalize.rs Comment on file fn normalize_match(
formula: &MatchFormula,
fallback_span: Option<SourceSpan>,
) -> Decorated<Formula> {
match formula {
MatchFormula::Pattern(text) | MatchFormula::PatternObject(text) => Decorated {
node: Formula::Atom(Atom::Pattern(PatternAtom { text: text.clone() })),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
},
MatchFormula::Regex(pattern) => Decorated {
node: Formula::Atom(Atom::Regex(RegexAtom {
pattern: pattern.clone(),
})),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
},
MatchFormula::All(branches) => {
let normalized_branches = branches
.iter()
.map(|branch| normalize_match(branch, fallback_span.clone()))
.collect();
Decorated {
node: Formula::And(normalized_branches),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
}
}
MatchFormula::Any(branches) => {
let normalized_branches = branches
.iter()
.map(|branch| normalize_match(branch, fallback_span.clone()))
.collect();
Decorated {
node: Formula::Or(normalized_branches),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
}
}
MatchFormula::Not(inner) => {
let normalized_inner = normalize_match(inner, fallback_span.clone());
Decorated {
node: Formula::Not(Box::new(normalized_inner)),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
}
}
MatchFormula::Inside(inner) => {
let normalized_inner = normalize_match(inner, fallback_span.clone());
Decorated {
node: Formula::Inside(Box::new(normalized_inner)),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
}
}
MatchFormula::Anywhere(inner) => {
let normalized_inner = normalize_match(inner, fallback_span.clone());
Decorated {
node: Formula::Anywhere(Box::new(normalized_inner)),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
}
}
MatchFormula::Decorated {
formula: inner_formula,
where_clauses: raw_where,
as_name,
fix,
} => {
let mut normalized = normalize_match(inner_formula, fallback_span);
normalized.where_clauses = raw_where
.iter()
.map(|raw| WhereClause { raw: raw.clone() })
.collect();
normalized.as_name.clone_from(as_name);
normalized.fix.clone_from(fix);
normalized
}
}
}❌ New issue: Large Method |
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix: crates/sempai/src/normalize.rs Comment on file fn normalize_legacy(
formula: &LegacyFormula,
fallback_span: Option<SourceSpan>,
) -> Decorated<Formula> {
match formula {
LegacyFormula::Pattern(text) => Decorated {
node: Formula::Atom(Atom::Pattern(PatternAtom { text: text.clone() })),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
},
LegacyFormula::PatternRegex(pattern) => Decorated {
node: Formula::Atom(Atom::Regex(RegexAtom {
pattern: pattern.clone(),
})),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
},
LegacyFormula::Patterns(clauses) => normalize_legacy_patterns(clauses, fallback_span),
LegacyFormula::PatternEither(branches) => {
let normalized_branches = branches
.iter()
.map(|branch| normalize_legacy(branch, fallback_span.clone()))
.collect();
Decorated {
node: Formula::Or(normalized_branches),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
}
}
LegacyFormula::PatternNot(value) => {
let inner = normalize_legacy_value(value, fallback_span.clone());
Decorated {
node: Formula::Not(Box::new(inner)),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
}
}
LegacyFormula::PatternInside(value) => {
let inner = normalize_legacy_value(value, fallback_span.clone());
Decorated {
node: Formula::Inside(Box::new(inner)),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
}
}
LegacyFormula::PatternNotInside(value) => {
let inner = normalize_legacy_value(value, fallback_span.clone());
let inside = Decorated {
node: Formula::Inside(Box::new(inner)),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span.clone(),
};
Decorated {
node: Formula::Not(Box::new(inside)),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
}
}
LegacyFormula::PatternNotRegex(pattern) => {
let regex_atom = Decorated {
node: Formula::Atom(Atom::Regex(RegexAtom {
pattern: pattern.clone(),
})),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span.clone(),
};
Decorated {
node: Formula::Not(Box::new(regex_atom)),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
}
}
LegacyFormula::Anywhere(value) => {
let inner = normalize_legacy_value(value, fallback_span.clone());
Decorated {
node: Formula::Anywhere(Box::new(inner)),
where_clauses: vec![],
as_name: None,
fix: None,
span: fallback_span,
}
}
}
}❌ New issue: Large Method |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
leynos
changed the title
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (1)
crates/sempai/src/normalize.rs (1)
252-258:⚠️ Potential issue | 🟠 MajorReplace the matchable
(ERROR)dependency placeholder.Fix Line 257. Stop emitting a query that can match parser recovery nodes in malformed source. Encode a guaranteed non-match placeholder path instead of a live query shape.
Safer placeholder strategy (non-match by contradiction)
- Formula::Atom(Atom::TreeSitterQuery(TreeSitterQueryAtom { - query: String::from("(ERROR) `@_dependency_check`"), - })), + Formula::Atom(Atom::TreeSitterQuery(TreeSitterQueryAtom { + // Keep this syntactically valid while making it semantically impossible to match. + query: String::from( + "(ERROR) `@_dependency_check` (`#eq`? `@_dependency_check` \"__sempai_never__\") (`#eq`? `@_dependency_check` \"__sempai_also_never__\")", + ), + })),In Tree-sitter query semantics, does `(ERROR) `@cap`` match parse-recovery ERROR nodes in malformed syntax trees? Are multiple predicates on the same capture conjunctive, such that contradictory `#eq?` predicates force a no-match result?🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@crates/sempai/src/normalize.rs` around lines 252 - 258, Replace the live Tree-sitter query string "(ERROR) `@_dependency_check`" with a guaranteed non-matching placeholder so the rule cannot accidentally match parser-recovery nodes; update the TreeSitterQueryAtom creation in the bare(Formula::Atom(Atom::TreeSitterQuery(TreeSitterQueryAtom { query: ... }))) expression to use a non-existent node name or an explicitly contradictory predicate (for example a node type like "__never_node__" or a capture with a contradiction) instead of "(ERROR) `@_dependency_check`" so the query is impossible to satisfy at parse time.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@crates/sempai/src/engine.rs`:
- Around line 17-39: The code currently clones Decorated<Formula> for each
language in compile_rule_plans; change QueryPlan to hold Arc<Decorated<Formula>>
and update compile_rule_plans to wrap the incoming formula in Arc::new(...) once
and clone the Arc for each QueryPlan instead of deep-cloning the formula. Update
other call sites mentioned (the blocks at the other two places referenced) to
construct QueryPlan with Arc<Decorated<Formula>> as well, and adjust
signature/usages of QueryPlan::new and any fields to accept
Arc<Decorated<Formula>> so the formula is shared across per-language plans.
In `@crates/sempai/src/normalize.rs`:
- Line 25: Update the malformed module doc bullet that currently reads //! -
`regex: "...` → `Formula::Atom(Atom::Regex(...))` by closing the inline code
literal (change it to //! - `regex: "..."` → `Formula::Atom(Atom::Regex(...))`)
so the backtick pair is balanced; locate the module-level doc comment containing
that snippet in normalize.rs and fix the punctuation to keep generated docs
syntactically valid.
In `@crates/sempai/src/tests/normalization_tests.rs`:
- Around line 260-267: Add a new unit test that constructs a
SearchQueryPrincipal::ProjectDependsOn variant and calls
normalize_search_principal with a non‑None SourceSpan to ensure the
ProjectDependsOn branch is exercised; assert the returned normalized principal
has the canonical shape you expect for ProjectDependsOn (match the same
structural form used elsewhere in tests) and that normalized.span == Some(span)
to lock metadata propagation for this branch. Locate the normalization
entrypoint normalize_search_principal and mirror the style of the existing
span_propagates_from_search_principal_to_decorated test: create a SourceSpan,
build a ProjectDependsOn principal value, call
normalize_search_principal(&principal, Some(&span)), then add explicit
assertions for both the normalized principal variant/contents and the span
propagation.
- Around line 37-154: Multiple small tests repeat the same
normalize_legacy/normalize_v2 assertions; replace these duplicated single-case
tests (e.g., legacy_pattern_normalizes_to_atom,
legacy_pattern_regex_normalizes_to_regex_atom,
v2_pattern_shorthand_normalizes_to_atom, v2_regex_normalizes_to_regex_atom,
v2_all_normalizes_to_and, v2_any_normalizes_to_or, v2_not_normalizes_to_not,
v2_inside_normalizes_to_inside, v2_anywhere_normalizes_to_anywhere) with
parameterized rstest table cases that iterate inputs
(LegacyFormula::Pattern/PatternRegex/... and
MatchFormula::Pattern/Regex/All/Any/Not/Inside/Anywhere) and expected normalized
predicate types from normalize_legacy and normalize_v2; keep the
nested-structure tests (like legacy_pattern_not_regex_normalizes_to_not_regex
and legacy_pattern_not_inside_normalizes_to_not_inside) as bespoke tests. Use
#[rstest] with case(...) entries and assert matches!(result, ...) or equality as
appropriate, referencing normalize_legacy and normalize_v2 in the fixture.
---
Duplicate comments:
In `@crates/sempai/src/normalize.rs`:
- Around line 252-258: Replace the live Tree-sitter query string "(ERROR)
`@_dependency_check`" with a guaranteed non-matching placeholder so the rule
cannot accidentally match parser-recovery nodes; update the TreeSitterQueryAtom
creation in the bare(Formula::Atom(Atom::TreeSitterQuery(TreeSitterQueryAtom {
query: ... }))) expression to use a non-existent node name or an explicitly
contradictory predicate (for example a node type like "__never_node__" or a
capture with a contradiction) instead of "(ERROR) `@_dependency_check`" so the
query is impossible to satisfy at parse time.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: c1d318bc-36dd-49fe-aafc-3c221f1544a6
📒 Files selected for processing (3)
crates/sempai/src/engine.rscrates/sempai/src/normalize.rscrates/sempai/src/tests/normalization_tests.rs
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. Code Duplicationcrates/sempai/src/tests/engine_tests.rs: What lead to degradation?The module contains 2 functions with similar structure: compile_yaml_returns_semantic_error_for_invalid_not_in_or,compile_yaml_returns_semantic_error_for_missing_positive_term_in_and Why does this problem occur?Duplicated code often leads to code that's harder to change since the same logical change has to be done in multiple functions. More duplication gives lower code health. How to fix it?A certain degree of duplicated code might be acceptable. The problems start when it is the same behavior that is duplicated across the functions in the module, ie. a violation of the Don't Repeat Yourself (DRY) principle. DRY violations lead to code that is changed together in predictable patterns, which is both expensive and risky. DRY violations can be identified using CodeScene's X-Ray analysis to detect clusters of change coupled functions with high code similarity. Read More |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. crates/sempai/src/tests/engine_tests.rs Comment on lines +121 to +134 fn compile_yaml_returns_semantic_error_for_missing_positive_term_in_and() {
let engine = default_engine();
let result = engine.compile_yaml(concat!(
"rules:\n",
" - id: demo.missing.positive.term.in.and\n",
" message: missing positive term in and\n",
" languages: [rust]\n",
" severity: ERROR\n",
" patterns:\n",
" - pattern-not: foo($X)\n",
" - pattern-inside: bar($Y)\n",
));
let (code, _diag) = first_diagnostic_of_err(result);
assert_eq!(code, DiagnosticCode::ESempaiMissingPositiveTermInAnd);❌ New issue: Code Duplication |
This comment was marked as resolved.
This comment was marked as resolved.
leynos
changed the title
Clone the first diagnostic before snapshotting and assert its stable code and message fragment outside the redacted JSON payload. Keep snapshots focused on the redacted report shape while guarding the semantic fields directly.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Extend Sempai engine coverage for decorated query plan metadata and span propagation, and add malformed metavariable-pattern coverage for the fallible constraint parser. Add normalization metadata tests for decorated All, Any, Not, Inside, and Anywhere wrappers so metadata stays on the outer node while child spans and empty child metadata are preserved.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. crates/sempai/src/tests/normalization_metadata_tests.rs Comment on lines +88 to +108 fn v2_decorated_over_all_preserves_metadata_and_spans() {
let span = SourceSpan::new(12, 99, Some(String::from("file:///rule.yaml")));
let principal = decorated_match_formula(MatchFormula::All(vec![
MatchFormula::Pattern(String::from("a")),
MatchFormula::Pattern(String::from("b")),
]));
let decorated =
normalize_search_principal(&principal, Some(&span)).expect("formula should normalize");
assert_decorated_metadata(&decorated, &span);
match &decorated.node {
Formula::And(children) => {
assert_two_pattern_branches(children, "a", "b");
for child in children {
assert_empty_metadata_with_span(child, &span);
}
}
other => panic!("expected And formula, got {other:?}"),
}
}❌ New issue: Code Duplication |
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. Ensure that this is validated against the current version of the codegraph. If further refinement to address this finding would be deleterious, please supply a clear explanatory one to two paragraph markdown message I can paste into the CodeScene web ui's diagnostic suppression function so this diagnostic can be silenced. crates/sempai/src/tests/normalization_constraint_tests.rs Comment on lines +67 to +88 fn legacy_patterns_with_only_constraints_produces_and_with_no_children_and_where_clauses() {
let constraint = json!({"metavariable-regex": {"metavariable": "$X", "regex": "foo.*"}});
let legacy = LegacyFormula::Patterns(vec![LegacyClause::Constraint(constraint)]);
let decorated = normalize_legacy_decorated(legacy);
assert!(matches!(&decorated.node, Formula::And(children) if children.is_empty()));
assert_eq!(
decorated.where_clauses.first().map(|c| &c.constraint),
Some(&Constraint::MetavariableRegex {
metavariable: String::from("$X"),
regex: String::from("foo.*"),
})
);
let err = validate_formula(&decorated).expect_err("constraint-only And should fail");
let first = err.diagnostics().first().expect("expected diagnostic");
assert_eq!(
first.code(),
DiagnosticCode::ESempaiMissingPositiveTermInAnd
);
}❌ New issue: Code Duplication |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
Extract shared legacy constraint helpers and collapse malformed known constraint scenarios into a parameterized rstest. Keep the named constraint-only validation tests intact while sharing their diagnostic tail.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
♻️ Duplicate comments (1)
crates/sempai/src/tests/normalization_constraint_tests.rs (1)
183-245: 🧹 Nitpick | 🔵 Trivial | ⚡ Quick winCollapse these two YAML checks into one
#[rstest]table.The setup, execution, and diagnostic assertions are duplicated across both cases; parameterise the YAML snippet and expected message so this stays easy to extend.
♻️ Proposed refactor
-#[test] -fn compile_yaml_reports_schema_invalid_for_malformed_where_clause() { - let yaml = concat!( - "rules:\n", - " - id: demo.invalid.where\n", - " message: invalid where\n", - " languages: [rust]\n", - " severity: ERROR\n", - " patterns:\n", - " - pattern: foo($X)\n", - " - metavariable-regex:\n", - " metavariable: $X\n", - ); - - let report = Engine::new(EngineConfig::default()) - .compile_yaml(yaml) - .expect_err("malformed known constraint should fail"); - - assert_eq!( - first_diagnostic_code(&report), - DiagnosticCode::ESempaiSchemaInvalid - ); - assert!( - report - .diagnostics() - .first() - .expect("expected diagnostic") - .message() - .contains("invalid where-clause") - ); -} - -#[test] -fn compile_yaml_reports_schema_invalid_for_malformed_metavariable_pattern_where_clause() { - let yaml = concat!( - "rules:\n", - " - id: demo.invalid.pattern.where\n", - " message: invalid metavariable pattern\n", - " languages: [rust]\n", - " severity: ERROR\n", - " patterns:\n", - " - pattern: foo($X)\n", - " - metavariable-pattern:\n", - " pattern: x\n", - ); - - let report = Engine::new(EngineConfig::default()) - .compile_yaml(yaml) - .expect_err("malformed known constraint should fail"); - - assert_eq!( - first_diagnostic_code(&report), - DiagnosticCode::ESempaiSchemaInvalid - ); - assert!( - report - .diagnostics() - .first() - .expect("expected diagnostic") - .message() - .contains("expected {metavariable, pattern} string fields") - ); -} +#[rstest] +#[case::metavariable_regex( + concat!( + "rules:\n", + " - id: demo.invalid.where\n", + " message: invalid where\n", + " languages: [rust]\n", + " severity: ERROR\n", + " patterns:\n", + " - pattern: foo($X)\n", + " - metavariable-regex:\n", + " metavariable: $X\n", + ), + "invalid where-clause", +)] +#[case::metavariable_pattern( + concat!( + "rules:\n", + " - id: demo.invalid.pattern.where\n", + " message: invalid metavariable pattern\n", + " languages: [rust]\n", + " severity: ERROR\n", + " patterns:\n", + " - pattern: foo($X)\n", + " - metavariable-pattern:\n", + " pattern: x\n", + ), + "expected {metavariable, pattern} string fields", +)] +fn compile_yaml_reports_schema_invalid_for_malformed_constraint( + #[case] yaml: &str, + #[case] expected_message: &str, +) { + let report = Engine::new(EngineConfig::default()) + .compile_yaml(yaml) + .expect_err("malformed known constraint should fail"); + + assert_eq!( + first_diagnostic_code(&report), + DiagnosticCode::ESempaiSchemaInvalid + ); + assert!( + report + .diagnostics() + .first() + .expect("expected diagnostic") + .message() + .contains(expected_message) + ); +}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@crates/sempai/src/tests/normalization_constraint_tests.rs` around lines 183 - 245, Collapse the two tests compile_yaml_reports_schema_invalid_for_malformed_where_clause and compile_yaml_reports_schema_invalid_for_malformed_metavariable_pattern_where_clause into a single parameterised rstest: create a new function compile_yaml_reports_schema_invalid_for_malformed_constraint annotated with #[rstest] and two #[case] entries (e.g. metavariable_regex and metavariable_pattern) that supply the YAML string and expected_message; keep the shared test body that calls Engine::new(...).compile_yaml(yaml).expect_err(...), asserts first_diagnostic_code(&report) == DiagnosticCode::ESempaiSchemaInvalid, and checks report.diagnostics().first().unwrap().message().contains(expected_message); remove the duplicate #[test] functions and ensure you reference the same helper symbols first_diagnostic_code, Engine, compile_yaml and DiagnosticCode::ESempaiSchemaInvalid.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@crates/sempai/src/tests/normalization_constraint_tests.rs`:
- Around line 183-245: Collapse the two tests
compile_yaml_reports_schema_invalid_for_malformed_where_clause and
compile_yaml_reports_schema_invalid_for_malformed_metavariable_pattern_where_clause
into a single parameterised rstest: create a new function
compile_yaml_reports_schema_invalid_for_malformed_constraint annotated with
#[rstest] and two #[case] entries (e.g. metavariable_regex and
metavariable_pattern) that supply the YAML string and expected_message; keep the
shared test body that calls Engine::new(...).compile_yaml(yaml).expect_err(...),
asserts first_diagnostic_code(&report) == DiagnosticCode::ESempaiSchemaInvalid,
and checks
report.diagnostics().first().unwrap().message().contains(expected_message);
remove the duplicate #[test] functions and ensure you reference the same helper
symbols first_diagnostic_code, Engine, compile_yaml and
DiagnosticCode::ESempaiSchemaInvalid.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 7dd30e3e-b2ad-4026-ae6d-8ba99fbe9dbc
📒 Files selected for processing (1)
crates/sempai/src/tests/normalization_constraint_tests.rs
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. Ensure that this is validated against the current version of the codegraph. If further refinement to address this finding would be deleterious, please supply a clear explanatory one to two paragraph markdown message I can paste into the CodeScene web ui's diagnostic suppression function so this diagnostic can be silenced. crates/sempai/src/tests/normalization_constraint_tests.rs Comment on lines +109 to +125 fn legacy_patterns_with_only_constraints_produces_and_with_no_children_and_where_clauses() {
let constraint = json!({"metavariable-regex": {"metavariable": "$X", "regex": "foo.*"}});
let legacy = make_legacy_patterns_with_constraints([constraint]);
let decorated = normalize_legacy_decorated(legacy);
assert!(matches!(&decorated.node, Formula::And(children) if children.is_empty()));
assert_eq!(
decorated.where_clauses.first().map(|c| &c.constraint),
Some(&Constraint::MetavariableRegex {
metavariable: String::from("$X"),
regex: String::from("foo.*"),
})
);
assert_missing_positive_term_in_and_for_decorated(&decorated);
}❌ New issue: Code Duplication |
Collapse malformed where-clause engine scenarios into a parameterized rstest and share their schema-invalid assertion helper. Preserve the normalization scenario names and expected diagnostics while reducing repeated assertion code in the constraint test module.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Replace repeated decorated All, Any, Not, Inside, and Anywhere span tests with named rstest cases. Keep the same metadata and child-span assertions while sharing normalization and extraction helpers.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@crates/sempai/src/tests/normalization_metadata_tests.rs`:
- Around line 185-224: The test duplicates decorated-match construction and
manual branch extraction; replace the inline
SearchQueryPrincipal::Match(MatchFormula::Decorated { .. }) setup with the
existing helper decorated_match_formula(MatchFormula::All(...)) and use
extract_and_branches(&decorated.node) instead of the manual match block, then
keep the assertions via assert_two_pattern_branches(children, "a", "b") and
assert_empty_metadata for each child; update
v2_decorated_over_all_wraps_preserves_metadata to call decorated_match_formula,
call normalize_search_principal as before, get children via
extract_and_branches, and run the existing branch assertions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: cbb25ba5-41db-46bd-b179-7e674fb4f909
📒 Files selected for processing (2)
crates/sempai/src/tests/normalization_constraint_tests.rscrates/sempai/src/tests/normalization_metadata_tests.rs
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. Ensure that this is validated against the current version of the codegraph. If further refinement to address this finding would be deleterious, please supply a clear explanatory one to two paragraph markdown message I can paste into the CodeScene web ui's diagnostic suppression function so this diagnostic can be silenced. Code Duplicationcrates/sempai/src/tests/semantic_validation_tests.rs: What lead to degradation?The module contains 3 functions with similar structure: missing_positive_term_in_and_prefers_node_span,missing_positive_term_in_and_uses_first_available_child_span,missing_positive_term_in_and_uses_first_child_span_when_node_span_none Why does this problem occur?Duplicated code often leads to code that's harder to change since the same logical change has to be done in multiple functions. More duplication gives lower code health. How to fix it?A certain degree of duplicated code might be acceptable. The problems start when it is the same behavior that is duplicated across the functions in the module, ie. a violation of the Don't Repeat Yourself (DRY) principle. DRY violations lead to code that is changed together in predictable patterns, which is both expensive and risky. DRY violations can be identified using CodeScene's X-Ray analysis to detect clusters of change coupled functions with high code similarity. Read More |
|
@coderabbitai Please suggest a fix for this issue and supply a prompt for an AI coding agent to enable it to apply the fix. Include the file and symbol names indicated in the issue at the head of your response. Ensure that this is validated against the current version of the codegraph. If further refinement to address this finding would be deleterious, please supply a clear explanatory one to two paragraph markdown message I can paste into the CodeScene web ui's diagnostic suppression function so this diagnostic can be silenced. Code Duplicationcrates/sempai/src/tests/normalization_constraint_tests.rs: What lead to degradation?The module contains 2 functions with similar structure: legacy_patterns_with_only_constraints_produces_and_with_no_children_and_where_clauses,legacy_patterns_with_only_metavariable_pattern_constraint_fails_validation Why does this problem occur?Duplicated code often leads to code that's harder to change since the same logical change has to be done in multiple functions. More duplication gives lower code health. How to fix it?A certain degree of duplicated code might be acceptable. The problems start when it is the same behavior that is duplicated across the functions in the module, ie. a violation of the Don't Repeat Yourself (DRY) principle. DRY violations lead to code that is changed together in predictable patterns, which is both expensive and risky. DRY violations can be identified using CodeScene's X-Ray analysis to detect clusters of change coupled functions with high code similarity. Read More |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
Fold legacy constraint-only pattern tests into one rstest, reuse decorated match helpers for v2 All metadata coverage, and parameterise missing-positive primary span cases with shared builders. Take `make_legacy_patterns_with_constraints` as a const-sized `Value` array to match the call sites without changing behaviour. Co-authored-by: Cursor <cursoragent@cursor.com>
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@crates/sempai/src/tests/normalization_constraint_tests.rs`:
- Around line 110-116: Add an explicit cardinality assertion before inspecting
the first clause: assert_eq!(decorated.where_clauses.len(), 1); then keep the
existing assertion that compares decorated.where_clauses.first().map(|c|
&c.constraint) to Some(&Constraint::MetavariableRegex { metavariable:
String::from("$X"), regex: String::from("foo.*"), }); do the same change for the
other test locations that assert decorated.where_clauses.first() (the
occurrences around the other noted ranges) so duplicate or leaked clauses will
fail loudly.
In `@crates/sempai/src/tests/normalization_metadata_tests.rs`:
- Around line 191-209: The test should explicitly assert that no spans are
present when calling normalize_search_principal(&principal, None): add
assertions that decorated.span is None (or decorated.span.as_ref().is_none())
immediately after normalization and for each child returned by
extract_and_branches(&decorated.node) assert that child.span is None (or
child.span.as_ref().is_none()) in addition to the existing assert_empty_metadata
checks so the root node and all pattern-branch children are verified to carry no
span in the no-fallback path; locate symbols normalize_search_principal,
decorated, decorated.span, extract_and_branches, children and add the None-span
assertions near where assert_two_pattern_branches and assert_empty_metadata are
invoked.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: daa250af-6185-4b0b-9c19-5adb934aa887
📒 Files selected for processing (3)
crates/sempai/src/tests/normalization_constraint_tests.rscrates/sempai/src/tests/normalization_metadata_tests.rscrates/sempai/src/tests/semantic_validation_tests.rs
Require exactly one `where_clause` before reading `.first()` in legacy and v2 normalisation tests so duplicate constraints fail clearly. Assert no spans on the root and And-branch children when normalising without a rule span in `v2_decorated_over_all_wraps_preserves_metadata`. Co-authored-by: Cursor <cursoragent@cursor.com>
1 participant
Summary
pattern*syntax and v2matchsyntax into a single representation in sempai_core.Changes
Core modelling
Formulaenum with variants:Atom,Not,Inside,Anywhere,And,Or.Atomenum:Pattern,Regex,TreeSitterQuery.PatternAtom,RegexAtom,TreeSitterQueryAtom.Decorated<T>wrapper carryingwhere_clauses,as_name,fix, andspan.WhereClauseas an opaque container (initially storing a raw JSON value).pub mod formula;and re-export fromsempai-corelib).Normalization layer
Decorated<Formula>:pattern,pattern-regex,patterns,pattern-either,pattern-inside, etc. to canonicalFormulastructures.matchforms (all,any,not,inside,anywhere, etc.) to canonical shapes.WhereClauses to the enclosingDecorated<Formula>for constraints carried over from legacy forms.Decoratedmetadata (where,as,fix) during normalization.Semantic validation
InvalidNotInOr: reject negated branches inside disjunctions.MissingPositiveTermInAnd: require at least one positive term in anAnd.E_SEMPAI_INVALID_NOT_IN_OR,E_SEMPAI_MISSING_POSITIVE_TERM_IN_AND).Engine integration
Engine::compile_yaml:Decorated<Formula>.QueryPlancontaining the normalized formula and rule metadata.QueryPlanto store theDecorated<Formula>instead of the prior placeholder.Testing
sempai-corefor construction and equality ofFormulavariants.sempaifor normalization and semantic validation scenarios.Documentation
docs/sempai-query-language-design.md(note canonicalFormulamodel andWhereClausehandling).docs/users-guide.md(update to reflectcompile_yamlreturning compiled plans for valid rules).docs/roadmap.md(mark 4.1.5 as done).Validation and acceptance criteria
Engine::compile_yamlreturnsQueryPlans with normalizedDecorated<Formula>for valid rules (no moreNOT_IMPLEMENTED).Formulavalues.How to test locally
Notes on compatibility and scope
sempai_yaml::parse_rule_filesignature remains unchanged.📎 Task: https://www.devboxer.com/task/795de59f-09d8-463e-9ef9-350b92fa57be