Skip to content

PDX-501: scope comparisonType enum sets by step type#198

Merged
mrdailey99 merged 1 commit into
developfrom
fix/PDX-501-comparisontype-sets
Jun 2, 2026
Merged

PDX-501: scope comparisonType enum sets by step type#198
mrdailey99 merged 1 commit into
developfrom
fix/PDX-501-comparisontype-sets

Conversation

@mrdailey99
Copy link
Copy Markdown
Collaborator

Summary

  • Corrects the comparisonType guidance to present two step-scoped subsets of the single com.provar.core.model.base.java.ComparisonType enum, instead of one flat/incorrect list. A value used outside its step's subset is load-blocking (IllegalArgumentException: No enum constant …ComparisonType.<value>).
    • AssertValues (assertValuesComparison): 16 values — EqualTo, NotEqualTo, GreaterThan, GreaterThanOrEqualTo, LessThan, LessThanOrEqualTo, IsPresent, IsEmpty, Matches, NotMatches, Contains, NotContains, StartsWith, NotStartsWith, EndsWith, NotEndsWith.
    • UI Assert (uiAttributeAssertion): 6 values — EqualTo, Contains, StartsWith, EndsWith, Matches, None.
  • Root cause of the dogfooding failure: NotEqualTo is valid in AssertValues but not in a UI Assert; the old docs/rules conflated the two contexts.
  • Adds schema-aware field-type notes to the step reference (encrypted fields read as null; rich-text/textarea values arrive wrapped in <p>…</p> so use Contains; Contains direction is expectedValue contains actualValue).

Jira

https://provartesting.atlassian.net/browse/PDX-501

Test plan

  • yarn compile passes
  • yarn test:only passes (1315 passing)
  • mcp-smoke.cjs passes (57/57)
  • yarn lint passes

Changes

  • docs/PROVAR_TEST_STEP_REFERENCE.md: UI Assert section now lists the 6-value subset with a cross-context explainer; AssertValues section gains the 16-value subset and the comparison/field-type semantics notes.
  • src/mcp/rules/provar_best_practices_rules.json: ASSERT-COMPARISON-001 and ASSERT-VALUES-COMPARISON-001 rule strings scoped to the AssertValues subset, with a note about the narrower UI Assert subset.
  • docs/mcp.md: AssertValues section gains a step-scoped comparisonType table.

Note for Provar team

External/customer-facing docs (docs/provar-mcp-public-docs.md, docs/university-of-provar-mcp-course.md) are maintained separately — if they document comparisonType, they should adopt the same two step-scoped sets.

RCA: comparisonType is one Provar enum but each step type accepts only a subset; the step-reference, both best-practice rule strings, and docs/mcp.md listed a single flat/incorrect set, so agents emitted load-blocking values such as NotEqualTo on a UI Assert.
Fix: scope the guidance into the AssertValues 16-value subset and the UI Assert 6-value subset matching Provar core ComparisonType exactly, in PROVAR_TEST_STEP_REFERENCE.md, both rule strings, and docs/mcp.md, plus schema-aware field-type notes (encrypted->null, rich-text wrapped, Contains direction).
Copilot AI review requested due to automatic review settings June 2, 2026 17:45
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 2, 2026

Quality Orchestrator

🟢 LOW · 0 / 100 · All changed files have mapped tests.


No test files mapped.


⚡ quality-orchestrator  ·  /qo stub <file>  ·  qo analyze-local

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates ProvarDX CLI documentation and best-practice rule messaging to correctly describe comparisonType as step-scoped subsets of com.provar.core.model.base.java.ComparisonType, preventing users from selecting enum values that are load-blocking in the wrong step context (notably NotEqualTo in UI Assert).

Changes:

  • Documented the two valid comparisonType subsets separately for UI Assert (uiAttributeAssertion) vs AssertValues (assertValuesComparison).
  • Updated best-practice rule text to reflect the AssertValues subset and explicitly call out the narrower UI Assert subset.
  • Added a step-scoped comparisonType table to docs/mcp.md and referenced the step-reference resource for full semantics.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
src/mcp/rules/provar_best_practices_rules.json Updates rule recommendation/description strings to scope comparisonType guidance to AssertValues and note the UI Assert subset.
docs/PROVAR_TEST_STEP_REFERENCE.md Corrects and expands step reference docs to list step-specific comparisonType subsets and adds field-type/semantics notes for AssertValues.
docs/mcp.md Adds a concise table describing valid comparisonType subsets per step type and explains the load-blocking failure mode.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@mrdailey99 mrdailey99 merged commit 8a946bd into develop Jun 2, 2026
5 checks passed
@mrdailey99 mrdailey99 deleted the fix/PDX-501-comparisontype-sets branch June 2, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants