Skip to content

qa/1337: harden MSSQL PMDA test for mssql-tools18 and improve value parsing#2594

Merged
kurik merged 2 commits into
performancecopilot:mainfrom
kurik:1337
May 19, 2026
Merged

qa/1337: harden MSSQL PMDA test for mssql-tools18 and improve value parsing#2594
kurik merged 2 commits into
performancecopilot:mainfrom
kurik:1337

Conversation

@kurik
Copy link
Copy Markdown
Contributor

@kurik kurik commented May 17, 2026

Support mssql-tools18 path with -C flag, fall back to legacy mssql-tools.
Use _wait_for_port 1433 when sqlservr binary is absent.
SQL queries now select only cntr_value, and new helper functions (_sql_scalar_from_sqlcmd_out, _pm_value_from_pmprobe_out) robustly extract numeric values for comparison.
Cumulative metrics (Logins/sec, Logouts/sec) use a wider 20% tolerance.

Summary by CodeRabbit

  • Chores
    • Improved QA tooling reliability: better detection/installation of SQL client and waiting for database readiness.
    • More robust and clearer metric extraction and validation, with explicit failures for invalid results.
    • Added configurable tolerance for metric comparisons (default 10%), with logins/logouts checks using a 20% tolerance.

Review Change Stack

…arsing

Support mssql-tools18 path with -C flag, fall back to legacy mssql-tools,
and use _wait_for_port 1433 when sqlservr binary is absent. SQL queries
now select only cntr_value, and new helper functions (_sql_scalar_from_sqlcmd_out,
_pm_value_from_pmprobe_out) robustly extract numeric values for comparison.
Cumulative metrics (Logins/sec, Logouts/sec) use a wider 20% tolerance.
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 17, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 065ab56a-2a87-4617-a442-6714372b624d

📥 Commits

Reviewing files that changed from the base of the PR and between a2c58a6 and 138a6af.

📒 Files selected for processing (1)
  • qa/1337
🚧 Files skipped from review as they are similar to previous changes (1)
  • qa/1337

📝 Walkthrough

Walkthrough

The QA 1337 script improves SQLCMD discovery and TLS detection, waits for SQL Server readiness on port 1433, refactors SQL execution/output parsing into dedicated helpers that validate scalar and pmprobe outputs, and updates metric queries to fetch cntr_value with configurable tolerances (default 10%, 20% for logins/logouts).

Changes

SQL Server QA Script Refactor

Layer / File(s) Summary
SQL Server Readiness and SQLCMD Discovery
qa/1337
SQLCMD discovery prefers /opt/mssql-tools18/bin/sqlcmd with fallback to /opt/mssql-tools/bin/sqlcmd, detects TLS-capable sqlcmd (-C) when present, and waits for port 1433 to ensure SQL Server is reachable before running queries.
SQL Execution and Output Parsing Helpers
qa/1337
_query standardizes sqlcmd output (trimmed, headerless). Added _sql_scalar_from_sqlcmd_out to extract the first numeric scalar and _pm_value_from_pmprobe_out to parse pmprobe output treating negative values as errors. _compare_values now accepts an optional tolerance (default 10%) and fails explicitly on parse/pmprobe errors.
Metric Validation Queries with Tolerances
qa/1337
Metric checks now select cntr_value for specific counters (Active Temp Tables, Logins/sec, Logouts/sec). Comparison calls use 20% tolerance for Logins/sec and Logouts/sec; other checks use the default 10% tolerance.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

🐰 A rabbit checks the server's door, waits on port one-four-three-three,

Parses numbers, flags the bad—no negatives allowed for me.
Tolerances set, comparisons sing, metrics tidy, outputs neat,
The QA script hops along the run, and lands the tests complete.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main changes: hardening the MSSQL PMDA test for mssql-tools18 compatibility and improving value parsing robustness.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@qa/1337`:
- Around line 135-136: The failure message hard-codes "10%" while the script
uses the caller-supplied $tolerance; update the call to _fail in the block that
calls _within_tolerance so it embeds the $tolerance variable (e.g. use
"$tolerance%") instead of "10%" and keep the same context variables ($metric,
$pcpvalue, $sqlvalue) so the message reads like: "$metric outside ${tolerance}%
tolerance (PCP $pcpvalue vs SQL $sqlvalue)".
- Around line 62-80: _sql_scalar_from_sqlcmd_out currently treats the first
numeric line as a scalar and exits immediately, which hides multi-row results;
update the function to scan all non-empty, non-header lines, collect numeric
matches instead of exiting on the first, and then: if exactly one numeric value
was found print it and exit 0, otherwise do not print a value and exit non‑zero
to signal an error (so callers detect multi-row results). Ensure changes are
made inside the _sql_scalar_from_sqlcmd_out awk program block referenced in the
diff so it counts numeric lines and enforces the single-row requirement.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e1d34f71-004d-4724-94d6-12c7c37d9234

📥 Commits

Reviewing files that changed from the base of the PR and between 509a6bb and a2c58a6.

📒 Files selected for processing (1)
  • qa/1337

Comment thread qa/1337
Comment thread qa/1337 Outdated
@kurik kurik merged commit 1f4bcc3 into performancecopilot:main May 19, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant