Skip to content

FAQ

Jump to bottom
Jonathan D.A. Jewell edited this page Jul 11, 2026 · 1 revision

FAQ

Common questions about Statistikles — the neurosymbolic statistics assistant where Julia does all the computing and the LLM only routes and explains. Answers are grounded in the source; where a claim is experimental it is labeled as such.

See also the Development-Guide, Security-and-Threat-Model, and Roadmap-and-Production-Readiness.

1. Does the LLM ever compute numbers?

No — and this is now enforced in code, not just requested in a prompt.

Statistikles is a Kautz Type 1 neurosymbolic system. The LLM (neural) understands your question and picks a tool; Julia (symbolic) performs every calculation. The boundary is src/tools/executor.jl:execute_tool() — no statistical value crosses it upward without having been computed by a verified Julia function.

The enforcement lives in src/tools/guardrail.jl. After a turn, validate_numeric_provenance(text, tool_results, user_numbers) harvests every number that actually came out of a tool call and checks every numeric literal in the assistant's prose against it. A number that matches nothing is an orphan — a likely fabrication. The guardrail flags; it never silently rewrites the text. See Security-and-Threat-Model for the full mechanism.

2. What is a "mollock"?

A mollock is a plausible-sounding but fabricated answer — an invented p-value, a hallucinated effect size, a misremembered formula, a confident-but-wrong test recommendation. LLMs are known to produce them in statistical contexts. Eliminating mollocks is the reason Statistikles exists: if you see a number in a response, it came from Julia.

3. What happens if no LLM is running?

The symbolic core still works — the LLM is only needed for the conversational interface. You can:

  • Run the offline examples: julia --project=. -e 'using Statistikles; run_examples()'
  • Call the statistical functions directly from Julia: julia --project=. -e 'using Statistikles; println(descriptive_stats([1.0,2.0,3.0]))'

The chat interface (main() / just run) needs a function-calling LLM endpoint — LM Studio on localhost:1234 by default, or any OpenAI-compatible endpoint. Without one, the chat loop cannot route, but none of the mathematics depends on it.

4. Are the formal proofs real?

They are real Agda proofs, but they are experimental and they do not (yet) prove the statistical theorems. From proofs/README.adoc:

  • Every lemma type-checks under agda --safe with no postulates and no holes, and CI (.github/workflows/agda.yml) enforces this on every module.
  • But all ten lemmas are stated over the natural numbers (ℕ), while the runtime computes over Float64 (IEEE-754 doubles). Each proof is a discrete proxy for — not a proof of — its statistical target.

For example, the "Bonferroni" entry proves x ≤ x + Σxs over ℕ; the real P(⋃Aᵢ) ≤ ΣP(Aᵢ) over ℝ is tracked as an open target. Statistikles is not "formally verified" for statistics. Restating these over the reals is deferred future work. See Experimental-Surfaces for the full catalogue.

5. Is the Zig FFI usable?

Experimental. ffi/zig/ compiles and its memory-safety tests pass under CI (zig build test, Zig 0.13.0), so it can't silently rot — but the exported entry points are placeholders not yet backed by the Julia core, and the Idris2 ABI layer described in ABI-FFI-README.md is design-only (src/abi/ does not exist). Do not treat the C-ABI as a way to reach the statistics engine. More detail on Experimental-Surfaces.

6. Why Julia?

Because the symbolic half needs deterministic, exact, well-tested numerics and a deep statistics ecosystem. Julia gives Statistikles Distributions, StatsBase, DataFrames, and friends, with results that are reproducible run-to-run (the test suite asserts determinism explicitly). Every number the user sees is produced by ordinary, auditable Julia code — which is exactly what the guarantee requires.

7. How many statistical functions are there?

Roughly 40 modules (41 files) under src/stats/ — covering descriptive, inferential, correlation/regression, non-parametric, effect sizes, power analysis, Bayesian, fuzzy logic, Dempster-Shafer, causality, reliability, validity, measurement, qualitative, survival, spatial, robust, time-series, meta-analysis, NLP, and non-classical (tropical, quantum) statistics.

The subset exposed to the LLM is 30 tools, registered in src/tools/definitions.jl (e.g. descriptive_statistics, t_test, anova, chi_square, correlation, regression, nonparametric_test, permanova, power_analysis, bayesian_analysis, reliability_analysis, …). The full list is on the Statistical-Functions-Reference page.

8. How do I add a statistical test?

Four steps (and a fifth the CI will insist on):

  1. Write the function in the appropriate src/stats/*.jl module. By convention every statistical function returns Dict{String,Any}, and guards degenerate inputs (return nothing + a "note" rather than leaking NaN/Inf; throw ArgumentError for invalid arguments — see the helpers in src/stats/validation.jl).
  2. Export it from src/Statistikles.jl.
  3. Register a tool in src/tools/definitions.jl using _tool: 
    _tool("my_new_test",
      "One-line description the LLM reads to decide when to route here.",
      Dict("x" => Dict("type"=>"array", "description"=>"…")),
      ["x"])   # required parameters
  4. Dispatch it in src/tools/executor.jl — add an arm to execute_tool, coercing arguments and returning the function's Dict. Give every inner sub-type dispatch a trailing else return Dict("error"=>…).
  5. Add tests. test/executor_router_test.jl enumerates every tool in definitions.jl and fails if your new tool has no minimal-argument table entry (or an explicit skip reason) — so add that entry. For a new statistic, also add a ground-truth assertion (see test/reference_validation_test.jl / reference_validation_advanced_test.jl) derived from a textbook or a second implementation. Full guidance is in the Development-Guide.

9. What does "Kautz Type 1" neurosymbolic mean?

It is Henry Kautz's taxonomy category for systems where the neural and symbolic components operate side-by-side with a defined, auditable interface boundary — as opposed to neural nets that internalize symbolic reasoning. In Statistikles, everything above execute_tool() is neural (language understanding and explanation) and everything below it is symbolic (Julia computation).

10. How is the "no LLM numbers" guarantee actually enforced?

By validate_numeric_provenance in src/tools/guardrail.jl, tested in test/guardrail_test.jl. A prose number passes if it approximately matches (within rtol=1e-6, or after display-rounding) a number harvested from a tool result, or its ÷100/×100 percent variant, or a number the user supplied, or it is a small structural integer in 0..12 (degrees of freedom, group counts, indices). Anything else is flagged as an orphan, and the runtime retries once then warns — it never fabricates or silently edits. Details in Security-and-Threat-Model.

11. What LLM / model do I need?

Any model that supports function (tool) calling, served over an OpenAI-compatible API. The default is LM Studio on localhost:1234. The chat client currently targets that endpoint; abstracting the backend behind a generic OpenAI-compatible interface with a fallback is a v0.2.0 roadmap item (see Roadmap-and-Production-Readiness).

12. Can the assistant be tricked by malicious data (prompt injection)?

It is defended in depth. The LLM has no side-effecting tools — every tool call dispatches to a pure Julia statistics function, and your data is parsed as data (CSV / JSON3), never executed. Even if injected text steered the model, the numeric-provenance guardrail would flag any fabricated statistic. Input-side delimiting of untrusted data (wrapping it in labeled segments) is an additional defense-in-depth item still in progress (W2-7 in the production-readiness program).

13. Does it send my data anywhere?

Computation is local Julia. The only outbound call is to whatever LLM endpoint you configure — by default LM Studio running on your own machine, so nothing leaves it. If you point the client at a remote OpenAI-compatible endpoint, your prompts (and any data you include) go there; that is your choice of backend, not a built-in behavior.

14. Is Statistikles production-ready?

The symbolic core is real, tested, and reference-validated against hand-derived and second-implementation ground truth (e.g. chi-square, ANOVA, Welch's t, regressions, Kaplan-Meier, meta-analysis — see Testing-and-Validation). The experimental surfaces (FFI and proofs) are not production and are labeled as such throughout. A ten-dimension production-readiness audit drove Wave 1 (the enforced guarantee, robustness, coverage, install, supply-chain pinning) and most of Wave 2; observability and prompt-injection delimiting remain. See Roadmap-and-Production-Readiness for the honest, per-item status.

15. What license is it under, and how do I report a security issue?

Statistikles is licensed MPL-2.0 (documentation is CC-BY-SA-4.0). See the LICENSE file.

For security issues, use a GitHub Security Advisory or email j.d.a.jewell@open.ac.uk — never a public issue. Full policy, timelines, and safe-harbour terms are on the Security-and-Threat-Model page.

See also

Statistikles

Overview

Using

Development

Project

Clone this wiki locally