Skip to content

Development Guide

Jump to bottom
Jonathan D.A. Jewell edited this page Jul 11, 2026 · 1 revision

Development Guide

This page walks a contributor from a clean machine to a green pull request: the toolchain, cloning and branching, signed commits, the merge gate, and how to run the governance checks locally before you push.

It is grounded in CONTRIBUTING.md, QUICKSTART-DEV.adoc, the Justfile, and the workflows under .github/workflows/. For the security-reporting side see Security-and-Threat-Model; for where the project is heading see Roadmap-and-Production-Readiness; for common questions see the FAQ.

Toolchain

The compute core is pure Julia and needs nothing exotic to build or test. The two experimental surfaces (Zig FFI, Agda proofs) need their own toolchains but are not required to build, test, or run Statistikles.

Tool Version Required for Notes
Julia 1.10 (pinned 1.10.11 in .tool-versions) Everything All statistical computation and the CLI
just 1.36.0 (.tool-versions) Task runner Wraps every command below
Zig 0.13.0 ffi/zig/ only (experimental) The version build.zig and .github/workflows/zig.yml target
Agda 2.6.4+ with agda-stdlib 2.x proofs/ only (experimental) Wired via ~/.agda/libraries + ~/.agda/defaults

The reference development environment is WSL Debian, with Julia installed through juliaup (Julia is only on PATH in a login shell — use bash -lc '…'). Nothing about the project is Windows- or WSL-specific, though; any Linux/macOS host with Julia 1.10 works.

Julia dependencies are declared in Project.toml and pinned by the committed Manifest.toml:

CSV, DataFrames, Dates, Distributions, HTTP, JSON3,
LinearAlgebra, Printf, Random, Statistics, StatsBase, UUIDs

The committed Manifest.toml is the supply-chain control for the numeric layer (Julia has no Dependabot ecosystem, so the pinned manifest resolved by CI is the compensating control — see Security-and-Threat-Model).

Clone and set up

git clone https://github.com/hyperpolymath/statistikles.git
cd statistikles
just setup

just setup runs julia --project=. -e 'using Pkg; Pkg.instantiate()' to resolve and install the pinned dependencies. To precompile the environment:

just build   # julia --project=. -e 'using Pkg; Pkg.precompile()'

Run the assistant (needs a function-calling LLM endpoint — LM Studio on localhost:1234 by default):

just run     # julia --project=. -e 'using Statistikles; main()'

You do not need an LLM to develop or test the symbolic core — see the FAQ.

Key recipes

The Justfile is the single entry point. The most useful recipes for contributors:

Recipe What it does
just setup Install Julia package dependencies (Pkg.instantiate())
just build Precompile the project environment (Pkg.precompile())
just test Run the full suite via julia --project=. test/runtests.jl
just test-verbose / just test-smoke Verbose / smoke variants
just run Launch the assistant (using Statistikles; main())
just lint / just fmt / just fmt-check Lint and format Julia sources
just quality Aggregate quality checks
just doctor Self-diagnostic of the local environment
just validate / just validate-rsr / just validate-state Governance / RSR structure validation
just sbom Generate an SPDX SBOM
just guix-build / just guix-shell / just nix-shell Reproducible-build shells
just tour Guided tour of the codebase
just help-me Open an issue-reporting helper

Run just (or just default) with no arguments to list everything.

Branching

Branch from an up-to-date origin/main. CONTRIBUTING.md uses type-prefixed branch names:

docs/short-description        # documentation
test/what-added               # test additions
feat/short-description        # new features
fix/issue-number-description  # bug fixes
refactor/what-changed         # code improvements
security/what-fixed           # security fixes

Keep diffs surgical and match the surrounding style. Python is banned in repository code (governance-enforced); deriving test constants with Python/R locally is fine, but only the resulting constants plus a derivation note may enter the repo.

Commits and signing

Commit messages follow Conventional Commits:

<type>(<scope>): <description>

[optional body]

[optional footer]

SECURITY.md (Security Best Practices → For Contributors) asks contributors to sign their commits and never commit secrets:

git config commit.gpgsign true

The maintainer flow signs every commit (SSH signing is used in practice) and pushes over SSH. Whatever signing method you use, keep signing on for every commit.

The merge gate

Branch protection on main is active. From the production-readiness decisions of record:

The Base ruleset requires the "E2E — Julia Test Suite" status check. A PR cannot merge red.

That check is the e2e-julia job in .github/workflows/e2e.yml (named E2E — Julia Test Suite), which runs:

- name: Run Julia tests (with coverage)
  run: julia --project=. -e 'using Pkg; Pkg.test(coverage=true)'

Coverage is reported to the job summary but is informational — not gated. Only the test suite passing gates the merge.

Other rules the CI enforces on PRs:

  • SHA-pin all GitHub Actions with a version comment (@<full-sha> # v7.0.0); never @latest.
  • Step-level conditionals only — never a job-level hashFiles()/secrets conditional (it produces a silent startup_failure).
  • Do not open GitHub issues as part of automated work, and never merge, delete, or force-push branches for someone else.

What CI runs

The repository has a heavy governance CI surface. The workflows most relevant to a code change:

Workflow Job(s) Purpose
e2e.yml E2E — Julia Test Suite, Aspect — Safety + SPDX The gate: full Julia suite + coverage; safety/SPDX aspect checks (see Testing-and-Validation)
governance.yml reusable governance-reusable.yml (from hyperpolymath/standards) Org-wide governance policy
dogfood-gate.yml A2ML, K9, empty-linter, Groove, eclexiaiser, scorecard Format/tooling compliance ("dogfooding")
agda.yml agda --safe (proofs/) Type-checks the experimental Agda proofs
zig.yml Zig FFI — build + test Builds + tests the experimental Zig FFI (Zig 0.13.0)
codeql.yml, scorecard.yml, secret-scanner.yml static analysis, OpenSSF Scorecard, TruffleHog Security scanning
install-smoke.yml install smoke test just setup + using Statistikles on a clean runner
container-build.yml container build Builds the Containerfile (no push)
release.yml, TagBot.yml release + tag Julia General registry release path (see Roadmap-and-Production-Readiness)

The Aspect safety check

The Aspect — Safety + SPDX job (in e2e.yml) fails the build if it finds dangerous escape hatches or unconstructive proofs:

# No dangerous patterns anywhere under src/
grep -rn 'believe_me\|assert_total\|sorry\|Admitted' src/
# The experimental Agda proofs must stay constructive — no postulates
grep -rnw --include='*.agda' 'postulate' proofs/

It also samples src/**/*.jl for SPDX headers. Keep the # SPDX-License-Identifier: MPL-2.0 header on every source file.

The Agda proof gate

agda.yml type-checks every module under proofs/ with agda --safe, rejects any postulate, and fails if a new .agda file is added without being registered in the workflow's EXPECTED_SORTED list. If you add a proof module, register it there.

The executor-router test

test/executor_router_test.jl enumerates every tool registered in src/tools/definitions.jl and drives it through execute_tool. It fails if you register a new tool without adding a minimal-argument table entry or an explicit skip reason — so adding a tool means adding its router-test entry too (see the FAQ on adding a test).

Running the governance checks locally

Before pushing, reproduce the important gates locally:

# 1. Format + lint
just fmt-check
just lint

# 2. Full test suite (the merge gate). In WSL, serialize Julia runs under a lock
#    and use a login shell so julia is on PATH:
flock /tmp/statistikles-julia.lock -c \
  'cd <repo> && julia --project=. -e "using Pkg; Pkg.test()"'

# 3. Governance / structure validation
just validate
just validate-rsr

# 4. The Aspect safety greps (what CI runs)
grep -rn 'believe_me\|assert_total\|sorry\|Admitted' src/    # must be empty
grep -rnw --include='*.agda' 'postulate' proofs/             # must be empty

For the experimental surfaces (only if you touched them — see Experimental-Surfaces):

# Zig FFI (Zig 0.13.0)
cd ffi/zig && zig build test --summary all

# Agda proofs (Agda 2.6.4+, agda-stdlib 2.x)
cd proofs
agda --safe Statistikles/TropicalSemiring.agda
agda --safe Statistikles/RankIdentities.agda
agda --safe Statistikles/Inequalities.agda

Opening the pull request

  • Target main with a Conventional-Commits title.
  • Body: what changed and why (with file references), the verification you ran and its result, and anything you deliberately skipped.
  • Make sure the E2E — Julia Test Suite check is green — it is the only hard gate, and a red PR cannot merge.

Governance follows a BDFL model (see GOVERNANCE.md): significant changes should be discussed before implementation, and architectural decisions are recorded as ADRs under docs/decisions/.

See also

Statistikles

Overview

Using

Development

Project

Clone this wiki locally