test: improve randomness and distribution tests#1010
Conversation
|
CodeAnt AI is reviewing your PR. |
📝 WalkthroughWalkthroughTests were tightened for randomness and charset correctness: Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
codeant-ai
Bot
added
the
size:M
|
CodeAnt AI finished reviewing your PR. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/auto/util/tst_util.cpp`:
- Around line 597-609: Replace the current per-digit ±50% tolerance check with a
chi-square goodness-of-fit test: compute expected = iterations / 10 and chi2 =
sum over i of (counts[i] - expected)^2 / expected, then assert chi2 is below the
critical value for 9 degrees of freedom (e.g. ~16.92 for alpha=0.05) using
QVERIFY2; reference the existing variables counts, iterations, expectedPerDigit
(or expected), count and the QVERIFY2 assertion so the message reports the
computed chi2 and the critical threshold on failure.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro
Run ID: 6d145c82-c057-475b-b1e7-1d5a7bd0e1c0
📒 Files selected for processing (1)
tests/auto/util/tst_util.cpp
|
Coverage is 21.015% — fix/test-improvements into main. No base build found for |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1010 +/- ##
=======================================
Coverage 27.11% 27.11%
=======================================
Files 39 39
Lines 3334 3334
=======================================
Hits 904 904
Misses 2430 2430
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
nogeenharrie
force-pushed
the
fix/test-improvements
branch
from
4012d11 to
edf13db
Compare
|
CodeAnt AI is running Incremental review |
codeant-ai
Bot
added
size:M
|
CodeAnt AI Incremental review completed. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/auto/util/tst_util.cpp`:
- Around line 605-609: The chi-square threshold constant chi2Critical (used in
the QVERIFY2 assertion) is too tight and yields ~5% false failures; increase the
threshold to a more lenient value (e.g., use a higher critical value or compute
it from a safe alpha) so the test is stable in CI, and update the QVERIFY2
message accordingly to reference the new chi2Critical; locate the constant named
chi2Critical and the QVERIFY2 invocation in tst_util.cpp and change the constant
or replace it with a computed critical value before the assertion.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro
Run ID: 7c9b4bd0-a5f1-4fc9-b92d-2ce816c36dad
📒 Files selected for processing (1)
tests/auto/util/tst_util.cpp
- Add randomness sanity check to generatePassword test - Add chi-square goodness-of-fit test to boundedRandom test - Use lenient chi-square threshold (25.0) for CI stability
nogeenharrie
force-pushed
the
fix/test-improvements
branch
from
edf13db to
990ae0a
Compare
There was a problem hiding this comment.
♻️ Duplicate comments (1)
tests/auto/util/tst_util.cpp (1)
605-610:⚠️ Potential issue | 🟡 MinorChi-square cutoff is still a bit tight for long-term CI stability.
Line 605 uses
25.0; this can still cause intermittent false failures on unbiased RNG output. Consider a more conservative cutoff (e.g.,27.88for df=9, α≈0.001).Suggested adjustment
- const double chi2Critical = 25.0; + // df=9; alpha≈0.001 to reduce random CI false failures. + const double chi2Critical = 27.88;🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/auto/util/tst_util.cpp` around lines 605 - 610, The chi-square cutoff stored in chi2Critical is too tight (currently 25.0) and can cause intermittent CI failures; update the value used in the QVERIFY2 assertion to a more conservative cutoff (e.g., 27.88 for df=9, α≈0.001) so the check that compares chi2 against chi2Critical in the QVERIFY2 call is less likely to fail; modify the chi2Critical declaration near the QVERIFY2(qPrintable(QStringLiteral(...).arg(chi2).arg(chi2Critical))) line accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Duplicate comments:
In `@tests/auto/util/tst_util.cpp`:
- Around line 605-610: The chi-square cutoff stored in chi2Critical is too tight
(currently 25.0) and can cause intermittent CI failures; update the value used
in the QVERIFY2 assertion to a more conservative cutoff (e.g., 27.88 for df=9,
α≈0.001) so the check that compares chi2 against chi2Critical in the QVERIFY2
call is less likely to fail; modify the chi2Critical declaration near the
QVERIFY2(qPrintable(QStringLiteral(...).arg(chi2).arg(chi2Critical))) line
accordingly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro
Run ID: a9dfda44-ce21-4229-a6fb-533c881e1e8c
📒 Files selected for processing (1)
tests/auto/util/tst_util.cpp
3 participants
User description
Summary
Testing
All tests pass:
CodeAnt-AI Description
Strengthen password randomness checks
What Changed
Impact
✅ Fewer false passes in randomness tests✅ Earlier detection of weak password generation✅ More reliable test coverage for random output🔄 Retrigger CodeAnt AI Review
Details
💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.
Summary by CodeRabbit