v1.4.0 - Sandbox hardening, upload DoS fix, and audit fixes

Sandbox hardening (a new non-LLM static pre-check), a decompression-bomb upload
fix, and correctness/perf fixes from a full code audit. No breaking changes.

Security

• Added a non-LLM static pre-check (statlee/codecheck.py) that runs before
  every sandbox execution on all three code paths (/chat, /run, /wrangle).
  It is a deterministic AST denylist (Python) / regex denylist (R) that rejects
  network and process imports, os/environment/file-exfiltration primitives,
  dynamic code execution (eval/exec/compile/__import__), the classic
  __subclasses__/__globals__ sandbox-escape idioms, and file opens outside
  the run directory. It is defense in depth added to the LLM moderation gates,
  so in the default subprocess mode the safety boundary is no longer solely
  LLM-dependent. A blocked script is refused before execution and its credit
  (when billing is on) is refunded.
• The static pre-check resolves module aliases (import os as o; o.system(...))
  and blocks direct from-imports of os/sys/shutil primitives
  (from os import system), closing two one-line evasions of the attribute
  rules.
• Docker Compose no longer bind-mounts a readable .env secret into the
  untrusted-execution container, and the subprocess-isolation warning now fires
  in development too, not only in production.

Fixed

• /export and /interpret now use the script that produced the last run
  instead of the most-recently-approved script. A /wrangle performed after an
  analysis no longer makes the project export bundle the one-line wrangle
  transform as script.py, or feed it to the auto-debugger, in place of the
  analysis script.
• Bounded upload parsing against a decompression bomb. Excel/Stata/SPSS uploads
  are size/row/cell-capped before full materialization (zip central-directory
  metadata for .xlsx, reader metadata for .sav/.dta, with a post-read cell
  backstop) and rejected with a 413, so a small compressed file can no longer
  expand to gigabytes and pin a worker.

Changed

• /data_page caches the filtered frame per filter set and gets an explicit
  tighter rate limit, removing the repeated full-frame rescan on the paging hot
  path.
• The container image now defaults WEB_CONCURRENCY to 1 so the in-memory
  rate-limit store and the monthly priority ceiling hold at their configured
  numbers by default; running more than one worker is documented to require a
  shared RATELIMIT_STORAGE_URI.

Docs

• README and docs/ARCHITECTURE.md now qualify the "network-less" and
  "secret-free" isolation claims by mode (kernel-enforced in docker mode; the
  static pre-check plus moderation gates in the default subprocess mode) and
  document the static pre-check as the non-LLM boundary.
• Documented the model-role mapping (MODEL_PRO is the default/draft tier;
  MODEL_PRO_MAX is the Pro-mode model) in config.py, .env.example, and the
  README to prevent misconfiguration.

Internal

• Test suite grew from 334 to 400 passing (plus 4 skip-marked Docker/POSIX tests
  that run in CI); the README badge reflects the new count.

v1.3.2 - Security, DB hygiene, and dependency refresh

Security

• /chat now re-moderates the model's own generated script through the run-guard before approving it, closing a gap where the primary code-generation path could reach execution without a code-level safety check. /wrangle and the edited-/run guard already did this; /chat now upholds the same "every executed generated script is moderated" invariant. A blocked script is not approved or run, and its credit (when billing is on) is refunded.

Fixed

• Running the test suite no longer poisons the instance database. Testing with no DATABASE_URL resolves to in-memory SQLite, so python -m pytest followed by an APP_ENV=development boot no longer crashes with a duplicate-column error from a half-migrated instance/statlee.db.

Changed

• Dependency refresh: google-genai 1.56 to 2.12, redis 5.2 to 8.0, anthropic 0.111 to 0.117, openai 2.43 to 2.45, matplotlib 3.10 to 3.11, filelock 3.25 to 3.30. The google-genai 2.x major bump changes only its Interactions API (unused here); the generate_content surface the app relies on is unchanged, confirmed by an API-surface check and a live Gemini round-trip on the upgraded SDK.

Docs

• Re-recorded the README demo GIF and workspace screenshot against the current v1.3.x interface, from a real analysis run on the bundled sample dataset.

---

334 tests passing; ruff, byte-compile, and pip-audit clean. No breaking changes.

v1.3.1 — Ship-Final Cycle Complete

Release v1.3.1

Status: Production-Ready ✅

This release marks the completion of the comprehensive final.md ship checklist cycle. All 85 items across 9 phases have been verified and passed.

Verification Summary

Phase	Items	Status
Secrets & Safety	9	✅
Code Quality	10	✅
Security	9	✅
Organization	7	✅
Setup / DX	12	✅
Dependency Health	4	✅
GitHub Presentation	20	✅
Web / UI	4	✅
Final Gate	7	✅

Key Assurances

✅ Zero secrets in repository or history
✅ 331 tests pass, 4 skipped (hermetic, deterministic)
✅ Build & lint clean (0 findings)
✅ Security audit passed (no injection/bypass/telemetry issues)
✅ GitHub presentation complete (README, CI, media, sole authorship)
✅ UI polished (0 console errors, mobile-responsive, WCAG AA a11y)
✅ Fresh-clone reproduction verified

Technical Details

• Python 3.12 (3.11+)
• 26 direct dependencies, all stable releases
• Multi-provider LLM support (Gemini/Claude/OpenAI)
• Elastic License 2.0 (source-available)
• Docker-ready sandbox isolation
• 100% tracked, sole authorship (yib7)

Ready for production deployment and community use.

v1.3.0

Third audit-pass cycle: 21 findings closed (7 P1, 14 P2) across billing
ordering, prompt-material caps, database migrations, account recovery, the
execution sandbox, storage integrity, and browser-side hardening. No breaking
changes for end users. The test suite grew from 229 to 311 passing (plus 4
skip-marked Docker/POSIX tests that run in CI).

Security

• All client-supplied prompt material is now length-capped server-side before
  it reaches a model call (chat history, codebook, code, converse context,
  interpret output and plots, report fields), closing an unmetered
  input-token amplification vector on the operator's key.
• /generate_report now moderates client text and, when billing is enabled,
  debits a credit before the expensive first pass and refunds it if the stream
  fails. It was previously unbilled, unmoderated, and uncapped.
• /interpret grounds on the server-recorded last-run output, plots, and
  executed script instead of trusting the client's spoofable copies, and
  moderates the client fallback when no server run exists.
• Login and registration have a dedicated rate limit (RATE_LIMIT_AUTH,
  default 10 per minute).
• Responses carry a strict Content-Security-Policy (script-src 'self'),
  X-Content-Type-Options: nosniff, and X-Frame-Options: DENY. The inline
  boot script moved to a static file fed by a JSON data island, and inline
  onclick handlers were replaced with a delegated listener.
• CSRF tokens are compared in constant time, and malformed tokens are
  rejected cleanly.
• Data-page filters treat the search term literally instead of as a regular
  expression, removing a catastrophic-backtracking DoS and fixing searches
  that contain ordinary punctuation.
• Uploads are capped per identity (file count and total bytes) to prevent
  disk-fill abuse, and oversized .txt uploads are rejected before the slow
  PDF conversion starts.
• Email verification tokens expire after 48 hours.

Added

• Database migrations (Flask-Migrate/Alembic). Fresh installs migrate to head
  at boot; legacy create_all databases are stamped at the v1.2.0 baseline
  and upgraded, so future schema changes no longer break existing
  deployments. gunicorn now preloads the app so migration runs once, not once
  per worker.
• Password reset flow: /request_password_reset (no account enumeration,
  rate-limited) and /reset_password with single-use one-hour tokens, built
  on the existing verification-email plumbing.
• flask grant-credits CLI and an optional MONTHLY_FREE_CREDITS lazy
  monthly top-up. The out-of-credits message now only promises a refresh when
  one is actually configured.
• SANDBOX_WORK_ROOT so the Docker sandbox works when the app itself runs in
  a container with a socket-mounted host daemon (sibling-container bind
  mounts), with the run directory made readable by the non-root runner.

Fixed

• /chat validates the dataset before debiting a credit, so a stale or
  invalid filename no longer costs a credit or a unit of the operator's
  monthly ceiling, and a denied debit returns the ceiling unit it took.
• All three LLM backends detect token-ceiling truncation and raise a clear
  error instead of returning silently cut-off code.
• Wrangling no longer clobbers the approved analysis script (approved scripts
  are kept per content hash, newest five), and concurrent wrangles no longer
  lose version-history entries (cross-worker file lock on the manifest).
• Uploads whose filename ends in a reserved version suffix such as __v2 are
  rejected instead of silently overwriting wrangle history.
• On POSIX, a timed-out subprocess-mode run now kills the whole process
  group, so grandchild processes no longer outlive the timeout.
• Saved history is capped at the newest 200 rows per user.

v1.2.0

Follow-up correctness and hardening cycle. Closes a second audit pass with one P0 sandbox-safety fix and a set of P1/P2 fixes across the wrangle, billing, config, auth, and chat layers. No breaking changes for end users. The test suite grew from 216 to 229.

Security

• The conversational data-wrangling path now run-guards the LLM-generated transform code through the moderation gate before it executes, closing a gap where wrangle transforms bypassed the check that analysis runs already had.

Fixed

• /chat refunds the debited credit when a code-generation stream fails, so a failed request no longer charges the account (applies when billing is enabled).
• /verify_email is now rate-limited, matching the other token-consuming auth endpoints.
• Config emits a startup warning when TRUST_PROXY_HOPS=0 in production, where the app is typically fronted by a proxy and rate limiting would otherwise key on the proxy IP.
• The chat pipeline surfaces the feature-selection fallback as an explicit SSE phase event instead of failing silently on wide datasets.

Changed

• Gemini stream usage extraction is now explicit rather than relying on implicit attribute access, and the case-insensitive moderation-verdict parsing is documented as intentional.

Tests

• Added skip-marked real-Docker sandbox integration tests (run when a Docker daemon and the statlee-runner image are available). Suite: 229 passing, 2 skipped without Docker.

v1.1.0 — Correctness & hardening

A full-codebase audit closed 16 issues (2 P0, 6 P1, 8 P2) across rate limiting, the LLM and sandbox layer, storage integrity, input validation, and billing. No breaking changes for end users. Test suite grew from 178 to 216; ruff, byte-compile, and pip-audit all clean.

Highlights

• Rate limiting: RATE_LIMIT_DEFAULT is now actually enforced — endpoints without an explicit limit (including /login and /register) were previously unthrottled.
• .xls uploads now work (added the xlrd engine; the format was advertised but failed).
• LLM robustness: the Gemini client now has an HTTP timeout, so a stalled call can't pin a worker thread.
• Sandbox: a timed-out Docker run is now killed instead of orphaned with its full resource allowance.
• Storage integrity: version-history/metadata/approved-script files are written atomically — a crash mid-write can no longer silently reset a dataset's cleaning history. TTL cleanup now runs on ordinary traffic and prunes empty dirs.
• Input validation: malformed JSON on several endpoints returns a clear 400 instead of a 500.
• Billing (when enabled): credit debiting is a single atomic operation and runs only after moderation — no double-spend, no charges for blocked requests. Constant-time master-password compare.
• Accounts: duplicate registration returns 409; a failed verification email reports a distinct message.

Removed

• The unused write-only Dataset table (never read back). On a pre-existing database the now-unreferenced datasets table remains but is harmless — it is simply no longer created or written.

Full details in CHANGELOG.md.

v1.0.1 - demo, sample data, CI bump

Maintenance and presentation release. The application's behavior is unchanged from v1.0.0.

Added

• Animated end-to-end demo GIF in the README (upload, plain-English request, generated code, chart, and written report).
• docs/examples/sample_survey.csv, a synthetic sample dataset so a new user gets a first result without supplying their own data.

Changed

• CI bumped to actions/checkout@v5 and actions/setup-python@v6, clearing the GitHub Node 20 runtime deprecation. Test matrix (3.11 / 3.12) and steps are unchanged.

178 tests pass; ruff clean; pip-audit clean. Full notes: CHANGELOG.md

v1.0.0 — first public release

First public release of STATlee: AI-assisted statistical analysis for social scientists. Describe an analysis in plain English over an uploaded dataset and STATlee generates, moderates, sandboxes, runs, and explains real Python/R.

Highlights:

• Pluggable LLM provider (Gemini default; Anthropic/OpenAI optional) behind one role-based service; Pro mode toggle for a larger code model.
• Intelligent codebook (nominal/ordinal/continuous), multi-format ingestion (CSV/TSV/Excel/Stata/SPSS), conversational data wrangling with undo/redo.
• Hardened execution sandbox (secret-free env, throwaway dir, POSIX rlimits; optional network-less Docker isolation) plus a run-guard that re-moderates edited scripts.
• CSRF, per-identity rate limiting, optional accounts, billing seam with money-safety guardrails.
• 178 tests; CI on Python 3.11 and 3.12.

Full notes in CHANGELOG.md.