Skip to content

test: migrate stats/base/dists/negative-binomial/pmf to ULP-based assertions - #14101

Draft
kgryte wants to merge 1 commit into
developfrom
kgryte/ulp-negative-binomial-pmf
Draft

test: migrate stats/base/dists/negative-binomial/pmf to ULP-based assertions#14101
kgryte wants to merge 1 commit into
developfrom
kgryte/ulp-negative-binomial-pmf

Conversation

@kgryte

@kgryte kgryte commented Aug 9, 2026

Copy link
Copy Markdown
Member

Resolves a part of #11352.

Description

What is the purpose of this pull request?

This pull request:

  • migrates the tests for stats/base/dists/negative-binomial/pmf from computed relative tolerance testing (delta = abs( y - expected[i] ), tol = k * EPS * abs( expected[i] ), t.strictEqual( delta <= tol, ... )) to ULP difference testing using @stdlib/assert/is-almost-same-value.
  • updates test/test.pmf.js and test/test.factory.js, each of which contains four fixture loops (one per Julia fixture file). test/test.js contains no tolerance math and is unchanged. The package has no test/test.native.js.
  • removes the now unused @stdlib/math/base/special/abs and @stdlib/constants/float64/eps requires from both files (neither is used elsewhere in these files).

ULP bounds (tightened to the measured minimum over the full fixture set):

Fixture file Test case ULP bound Measured maximum ULP difference
fixtures/julia/high_high.json large r and p 655 655
fixtures/julia/high_small.json large r, small p 688 688
fixtures/julia/small_high.json small r, large p 143 143
fixtures/julia/small_small.json small r and p 110 110

The main export and the factory-created function produce bit-identical results on every fixture value, so test.pmf.js and test.factory.js use the same bound per fixture set.

Notes on how the bounds were determined:

  • Each bound is the minimum non-negative integer for which every fixture value in that set passes. It was measured by computing, for each of the 1000 values in each of the four fixture sets, the ULP difference between the returned value and the Julia reference value, and taking the maximum over the set. Starting from a bound of 64 and adjusting, the suite passes at the tabulated bounds and fails at each bound minus one (exactly four failing assertions, one per fixture loop), confirming the bounds are tight rather than merely sufficient.
  • Worst cases per fixture set:
    • high_high: pmf( 1, 64, 0.7906898438417092 ) returns 3.975250904822743e-6 against a reference of 3.975250904823298e-6.
    • high_small: pmf( 8, 88, 0.02637417550853125 ) returns 1.1366780042760316e-128 against a reference of 1.1366780042759434e-128.
    • small_high: pmf( 14, 3, 0.9980475521581332 ) returns 1.3955555222417722e-36 against a reference of 1.3955555222417961e-36.
    • small_small: pmf( 9, 13, 0.0038654584213952074 ) returns 1.2210093592394756e-26 against a reference of 1.2210093592394914e-26.
  • The bounds are consistent with the tolerances they replace. The previous multipliers were 650.0 * EPS, 450.0 * EPS, 80.0 * EPS, and 80.0 * EPS; since a relative tolerance of k * EPS corresponds to roughly 2k ULP, the measured bounds sit within the accuracy budget the tests already allowed. The high_small set is the one case where the measured error exceeds the naive k-to-ULP reading of the old multiplier, because the old comparison was relative to abs( expected[i] ) rather than to the representable gap at that magnitude.
  • The magnitude of the bounds reflects the implementation, which evaluates the pmf via binomcoefln/gammaln-based log terms followed by exponentiation, so argument reduction amplifies the error of the underlying kernels for large r and for small p.
  • The full suite was run twice at the final bounds with identical results (4020 assertions for test.pmf.js, 4026 for test.factory.js, and 3 for test.js, all passing, per run), so the bounds are not sensitive to FMA/contraction differences on this machine.

Related Issues

Does this pull request have any related issues?

This pull request has the following related issues:

Questions

Any questions for reviewers of this pull request?

One point:

  • The four fixture sets need materially different bounds (110 through 688), so this PR keeps a per-fixture-loop bound rather than collapsing to a single package-wide maximum. That matches the per-loop tolerances the tests previously carried, but if reviewers would prefer one bound per file, 688 would cover all four sets at the cost of loosening the three tighter loops.

Other

Any other information relevant to this pull request? This may include screenshots, references, and/or implementation notes.

  • Only the two test files are modified; no source, documentation, benchmark, or fixture files are touched.
  • Verified with make test TESTS_FILTER=".*/stats/base/dists/negative-binomial/pmf/.*". Linting is clean via make lint-javascript-tests TESTS_FILTER=".*/stats/base/dists/negative-binomial/pmf/.*", which uses etc/eslint/.eslintrc.tests.js.
  • The editorconfig pre-commit hook could not run in this environment, as it downloads its binary from a host this session cannot reach. The two files were instead checked against .editorconfig (LF endings, tab indentation, final newline, UTF-8, no trailing whitespace); the diff introduces no new violations.
  • The idiom follows previously merged conversions, in particular stats/base/dists/hypergeometric/pmf (test: migrate stats/base/dists/hypergeometric/pmf to ULP-based assertions #14091) and stats/base/dists/halfnormal/stdev (test: migrate stats/base/dists/halfnormal/stdev to ULP-based assertions #14084), which use an inline integer ULP argument per fixture loop and the 'returns expected value' assertion message.

Checklist

Please ensure the following tasks are completed before submitting this pull request.

AI Assistance

When authoring the changes proposed in this PR, did you use any kind of AI assistance?

  • Yes
  • No

If you answered "yes" above, how did you use AI assistance?

  • Code generation (e.g., when writing an implementation or fixing a bug)
  • Test/benchmark generation
  • Documentation (including examples)
  • Research and understanding

Disclosure

If you answered "yes" to using AI assistance, please provide a short disclosure indicating how you used AI assistance. This helps reviewers determine how much scrutiny to apply when reviewing your contribution. Example disclosures: "This PR was written primarily by Claude Code." or "I consulted ChatGPT to understand the codebase, but the proposed changes were fully authored manually by myself.".

This PR was authored by Claude Code, running as an unattended scheduled task. The test migration follows the idiom established by previously merged conversions, and the ULP bounds were measured empirically against the fixture data rather than guessed.


@stdlib-js/reviewers

@stdlib-bot stdlib-bot added Good First PR A pull request resolving a Good First Issue. Statistics Issue or pull request related to statistical functionality. labels Aug 9, 2026
@stdlib-bot

Copy link
Copy Markdown
Contributor

Coverage Report

Package Statements Branches Functions Lines
stats/base/dists/negative-binomial/pmf $\\color{red}532/562$
$\\color{green}+94.66\\%$
$\\color{red}49/57$
$\\color{green}+85.96\\%$
$\\color{green}5/5$
$\\color{green}+100.00\\%$
$\\color{red}532/562$
$\\color{green}+94.66\\%$

The above coverage report was generated for the changes in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Good First PR A pull request resolving a Good First Issue. Statistics Issue or pull request related to statistical functionality.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants