Skip to content

validation: scan early lines for Feature: in .feature test files#249

Merged
hdamker merged 1 commit intocamaraproject:validation-frameworkfrom
hdamker:fix/feature-line-scan
Apr 30, 2026
Merged

validation: scan early lines for Feature: in .feature test files#249
hdamker merged 1 commit intocamaraproject:validation-frameworkfrom
hdamker:fix/feature-line-scan

Conversation

@hdamker
Copy link
Copy Markdown
Contributor

@hdamker hdamker commented Apr 30, 2026

What type of PR is this?

bug

What this PR does / why we need it:

P-007 / P-024 (validation/engines/python_checks/test_checks.py, _extract_feature_version) read only the first physical line of each .feature file. When line 1 is a Gherkin comment or a feature-level tag, the version regex finds no match and P-024 fires as a hard error.

Both styles are valid Gherkin and consistent with documentation/API-Testing-Guidelines.md in camaraproject/Commonalities. A sweep across the 49 upstream API repositories with a code/Test_definitions/ directory shows 25 files in 9 repositories with this layout (DeviceIdentifier, DeviceStatus, DeviceReachabilityStatus, DeviceRoamingStatus, ConnectedNetworkType, ApplicationEndpointDiscovery, ApplicationEndpointRegistration, ApplicationProfiles, OptimalEdgeDiscovery).

The release-time mechanical transformer is unaffected — _apply_regex runs re.subn on full file content, so it correctly transforms these files regardless of Feature-line position. The validation rule was the only thing flagging them.

This PR:

  • Rewrites _extract_feature_version() to scan the first 50 lines for a line beginning with Feature: (skipping preceding comment, tag, and blank lines) and to return the actual Feature-line number alongside the version.
  • Reports P-007 / P-024 findings at the actual Feature-line number rather than a hardcoded line=1.
  • Splits P-024 into two sub-cases with distinct messages: (1) no Feature: line in the scan window, (2) Feature line present but carrying no wip / vwip / v{version} token.
  • Drops implementation-coupling "snapshot transformation T1b" phrasing from user-facing messages.
  • Revises the P-024 rule comment block in validation/rules/python-rules.yaml.

No release-automation changes. No regression-fixture recapture (the regression/r4.2-broken-spec-test-files regression-expected.yaml has no P-007/P-024 entries).

Which issue(s) this PR fixes:

Fixes #248

Special notes for reviewers:

Verified locally:

  • 36/36 unit tests pass (python3 -m pytest validation/tests/test_python_checks_test.py -v), including 8 new cases covering comment/tag/blank-line preambles, the actual line number reported in findings, the two distinct P-024 sub-cases, and the 50-line scan cap.
  • Direct check on the three real DeviceIdentifier feature files now resolves ('vwip', 2) for each — the original false positive is gone.
  • Sweep across all 9 affected upstream API repositories returns the expected version + line number for every one of the 25 files.

Tag-move scope: rule-metadata tier (validation/engines/python_checks/**, validation/rules/python-rules.yaml, validation/tests/**). Validation Regression canary green is sufficient — no full ReleaseTest E2E required.

Changelog input

 release-note
P-007 / P-024 now correctly handle `.feature` files where the `Feature:` line is preceded by Gherkin comments, feature-level tags, or blank lines. Findings report the actual Feature-line number instead of always line 1.

Additional documentation

This section can be blank.

docs

P-007 / P-024 read only line 1 and fired P-024 as error when the
Feature line was preceded by a comment or feature-level tag (valid
Gherkin in 9 of 49 upstream API repos). Pure validation false
positive — release-time transformation is unaffected by Feature-line
position.

Scan the first 50 lines for `Feature:`, skip preceding comment, tag,
and blank lines, and report findings at the actual Feature-line
number. Split P-024 into two sub-cases with distinct messages: no
`Feature:` line in the scan window, vs. Feature line present without
a version token. Drop the implementation-coupling "T1b" phrasing
from user-facing messages; revise the P-024 rule comment.

Fixes camaraproject#248
@hdamker hdamker requested review from Kevsy and rartych as code owners April 30, 2026 14:07
Copy link
Copy Markdown
Contributor

@Kevsy Kevsy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hdamker hdamker merged commit e77ec74 into camaraproject:validation-framework Apr 30, 2026
1 check passed
@hdamker hdamker deleted the fix/feature-line-scan branch April 30, 2026 17:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants