-
Notifications
You must be signed in to change notification settings - Fork 0
Statistical Functions Reference
This page enumerates the statistical tools Statistikles exposes to the language
model. These are the registered, LLM-facing tools defined by get_tools() in
src/tools/definitions.jl — the exact set the neural layer is allowed to call.
Each tool is dispatched to verified Julia code in src/tools/executor.jl; the LLM
never computes a result itself (see The-No-Fabrication-Guarantee).
There are 29 registered tools. The executor_router_test.jl suite parses
get_tools() at run time and asserts every registered tool is reachable through
execute_tool with a clean (non-error) Dict result, so this list cannot
silently drift from the code.
Reference-validated (✓) marks a tool with at least one code path checked against ground truth — a value hand-derived from the textbook formula or cross-checked with a second, independent implementation (R 4.5.0, SciPy 1.15.3, or Python +
fractions.Fraction), never copied from the library's own output. These live intest/reference_validation_test.jl,test/reference_validation_advanced_test.jlandtest/chi_square_validation_test.jl. Tools without the mark are still covered by the broader suite (property-based, degenerate-input and router tests) but do not yet have hand-derived ground-truth assertions.
| Tool | Sub-types (type/method) |
Description | Validated |
|---|---|---|---|
descriptive_statistics |
— | Mean, median, mode, SD, skewness, kurtosis, quartiles, confidence intervals and normality hints for a numeric vector. | ✓ |
frequency_analysis |
— | Frequency table for categorical (string) data with relative and cumulative frequencies. | ✓ |
descriptive_statistics is validated against exact moments (e.g. mean 5.0,
variance 32/7 on the reference vector); frequency_analysis against a
hand-counted table (a×3, b×2, c×1) including mode and cumulative columns.
| Tool | Sub-types | Description | Validated |
|---|---|---|---|
t_test |
independent (Welch's), paired, one_sample
|
t-tests with Cohen's d effect size. | ✓ (independent / Welch) |
anova |
— | One-way ANOVA with eta-squared and omega-squared effect sizes. | ✓ |
chi_square |
independence, goodness_of_fit
|
Chi-square test with Cramér's V; goodness-of-fit accepts optional expected_proportions. |
✓ (both) |
test_assumptions |
normality (Jarque-Bera, KS), levene
|
Test normality of a vector, or homogeneity of variance across groups. | ✓ (levene) |
Ground-truth coverage: the Welch t-test (t-statistic, Satterthwaite df,
p-value via the regularized incomplete beta, and pooled-SD Cohen's d); one-way
ANOVA (F = 3.0, η² = 0.5, closed-form p = 0.125); chi-square for both
independence and goodness-of-fit, cross-checked against R's chisq.test and
SciPy's chi2_contingency/chisquare; and Levene's test via one_way_anova.
The chi-square path was hardened during a dedicated correctness review (
chi_square_validation_test.jl): the upper-tail p-value now usesccdf(the old1 - cdfunderflowed to0.0for large statistics),Chisq(0)andmin(r,c)-1division-by-zero cases are guarded asArgumentErrors, a fully zero row/column returnsnothing+ a"note"(never aNaNdressed up as a result), and expected cell counts below 5 raise a non-fatal"warning".
| Tool | Sub-types | Description | Validated |
|---|---|---|---|
correlation |
pearson, spearman
|
Correlation with significance test and confidence intervals. | ✓ (pearson) |
regression |
simple / multiple (auto-selected by the shape of x) |
Linear regression with ANOVA table, VIF and diagnostics; optional var_names. |
✓ (both) |
The regression tool inspects x: a vector of vectors triggers
multiple_regression, a plain vector triggers simple_linear_regression. Both
are ground-truth validated — simple regression against exact coefficients
(slope 0.6, intercept 2.2), and multiple regression against an orthogonal
2-predictor design whose XᵀX is diagonal, giving hand-computable coefficients,
standard errors, t-statistics and closed-form df=3 p-values. Pearson correlation
is validated against r = 12/√148 and its exact t-statistic 6√3.
| Tool | Sub-types | Description | Validated |
|---|---|---|---|
nonparametric_test |
mann_whitney, wilcoxon, kruskal_wallis
|
Rank-based two-sample and k-sample tests. | ✓ (mann_whitney, kruskal_wallis) |
permanova |
— | Permutational MANOVA on a distance matrix with permutation-based significance; non-parametric (no multivariate-normality assumption). Accepts n_permutations (default 999, clamped ≤ 100000) and alpha. |
Mann-Whitney U is validated on complete separation (U = 0, z = 1.7457…) and
Kruskal-Wallis against the exact H = 32/7 on a no-ties design.
Advertised vs. dispatched sub-types: the
nonparametric_testschema advertisesmann_whitney,wilcoxonandkruskal_wallisto the LLM, butexecute_toolalso dispatchesfriedmanandcochrans_qfor callers that pass them directly. Only the three advertised sub-types are part of the LLM-facing contract.
| Tool | Sub-types | Description | Validated |
|---|---|---|---|
effect_size_calculator |
— (converts among whichever of cohens_d, r, eta_squared, odds_ratio you supply; optional n1, n2) |
Convert between effect-size metrics, incl. Hedges' g. | |
power_analysis |
— | Required sample size or achieved power for t-tests, from effect_size (+ optional n, alpha, power). |
|
sample_size_calculator |
means, proportions, correlation, regression, anova
|
Required sample size for the chosen design. |
| Tool | Sub-types | Description | Validated |
|---|---|---|---|
bayesian_analysis |
— | Bayesian updating of a prior with a likelihood matrix at an observed data_index. |
|
bayes_factor |
— | Bayes Factor (BIC approximation) comparing two nested models. | |
credible_intervals |
— | Bayesian credible intervals (ETI and HDI) from posterior samples; optional level (default 0.95). |
|
dempster_shafer |
— | Combine two mass functions using Dempster-Shafer evidence theory. | |
fuzzy_logic_analysis |
operation: membership, and, or
|
Fuzzy-set membership calculations for a value given a set center and width. |
| Tool | Sub-types | Description | Validated |
|---|---|---|---|
granger_causality |
— | Test whether series_x Granger-causes series_y at a given lag. |
|
james_stein |
— | James-Stein shrinkage estimator for improved multi-parameter estimation; optional grand_mean. |
| Tool | Sub-types | Description | Validated |
|---|---|---|---|
diagnostic_metrics |
— | Sensitivity, specificity, PPV, NPV and accuracy from a 2×2 count (TP/FN/TN/FP). |
|
reliability_analysis |
— | Cronbach's Alpha from an item-response matrix. | |
measurement_analysis |
omega, sem, item_analysis
|
McDonald's Omega, standard error of measurement, or item analysis. | |
validity_assessment |
— | Content Validity Ratio (CVR) from n_essential / n_total panelists. |
|
criterion_validity_test |
validity_type: concurrent, predictive
|
Criterion validity of a predictor against a criterion. |
|
inter_rater_reliability |
cohens_kappa, fleiss_kappa, icc
|
Inter-rater agreement measures. |
| Tool | Sub-types | Description | Validated |
|---|---|---|---|
qualitative_analysis |
— | Thematic-saturation estimation from new-themes-per-interview counts; optional moving-average window. |
|
calculate_pre |
— | Proportional Reduction in Error (PRE) and R-squared from observed/predicted (+ optional baseline). |
|
sampling_design |
design_effect, margin_of_error
|
Design effects, or margin of error with optional finite-population correction. |
Every tool is declared with the _tool helper, which wraps it in the
function-calling schema the LLM consumes:
_tool("t_test",
"Perform t-tests: independent (Welch's), paired, or one-sample. Includes Cohen's d effect size.",
Dict("type" => Dict("type"=>"string", "enum"=>["independent","paired","one_sample"], "description"=>"Type of t-test"),
"group1" => Dict("type"=>"array", "description"=>"First group / data"),
"group2" => Dict("type"=>"array", "description"=>"Second group (independent/paired)"),
"mu0" => Dict("type"=>"number", "description"=>"Hypothesized mean (one-sample)"),
"alpha" => Dict("type"=>"number", "description"=>"Significance level (default 0.05)")),
["type", "group1"])The last argument lists the required parameters. Sub-types are expressed as an
enum on a type (or method/operation/design/validity_type) field, and
execute_tool returns an "Unknown type '…'" error for any value outside the
supported set.
src/tools/executor.jl contains dispatch branches for a number of additional tool
names that are not currently advertised by get_tools() — so the LLM will not
call them, but they are reachable if execute_tool is invoked directly (e.g. from
tests or a future release). Being honest about this gap: the LLM-facing contract is
the 29 tools above; the following are implemented-but-unadvertised.
logistic_regression, mle_fit, complexity_analysis, p_value_adjustment,
path_analysis, pca, bootstrap, time_series (moving_average/acf/dtw),
information_theory (entropy/kl_divergence), survival_analysis
(kaplan_meier/log_rank), meta_analysis, robust_stats
(mahalanobis/huber), causal_inference (iv/did/rdd), spatial_stats
(morans_i), machine_learning (spline/rf_proxy), nlp_symbolic
(sentiment/topic_modeling), advanced_modeling
(mixed_effects/ordinal_logistic), signal_processing (ica), bayesian_em,
functional_data, algebraic_stats (mcnemar/padic), representation_stats
(clr/interval_overlap), non_classical_prob (tropical_dot/bell_test),
structured_dynamic (centrality/fractal/hurst), unconventional_frameworks
(rough_set), and pre_suite.
Several of these carry ground-truth reference tests even though they are not yet in
the advertised registry — notably kruskal_wallis, multiple_regression,
logistic_regression (saturated binary-predictor exact MLE, β = [-ln 3, 2·ln 3]), kaplan_meier (product-limit survival with a censored tie), and
meta_analysis (fixed-effects and DerSimonian-Laird random-effects, verified
with exact rational arithmetic).
These branches also enforce safety clamps: bootstrap (n_reps ≤ 100000),
permanova (n_permutations ≤ 100000), and component counts k ≤ 20 for
bayesian_em, signal_processing/ica and nlp_symbolic/topic_modeling.
| File | Role |
|---|---|
src/tools/definitions.jl |
get_tools() — the 29 registered LLM-facing tool schemas |
src/tools/executor.jl |
execute_tool(name, args) — string→Julia dispatch and argument coercion |
src/stats/*.jl |
The 40+ verified symbolic modules the tools call |
test/reference_validation_test.jl |
Ground-truth for descriptive, t-test, correlation, regression, ANOVA, Mann-Whitney, Levene, frequency, chi-square |
test/reference_validation_advanced_test.jl |
Ground-truth for Kruskal-Wallis, multiple/logistic regression, Kaplan-Meier, meta-analysis |
test/chi_square_validation_test.jl |
Chi-square correctness review vs. R / SciPy |
test/executor_router_test.jl |
Asserts every registered tool is reachable and error-free through execute_tool
|
Overview
Using
Development
Project