v1.2.0

v1.2.0

A checkpoint-sized batch of corpus expansion, two new language categories, and
the tooling to keep doing it.

data/ grew from 152 repo-folders / 2,344 files to 218 / 3,306. Every added
folder is real, unmodified source copied from a pinned commit of a clone in the
gitgalaxy/data pool, recorded at exact confidence in
tools/independent_data_auditor.py and the per-category SOURCES.md tables — no
files were generated or hand-written. Two categories hit the pool's real-source
ceiling and now say so in their SOURCES.md rather than being padded.

New categories

embedded_python (1 folder / 14 files) — see the python / embedded_python
rows in the table below.

sqlite — 9 folders / 82 files (data/sqlite/SOURCES.md)

GitGalaxy's sqlite language standard is distinct from generic sql (its own
extensions, .sqliterc, sqlite3 shebang, ~50 rules). Real .sql exercising
that dialect: WITH RECURSIVE, STRICT / WITHOUT ROWID, ->> / json_extract
/ json_each, RETURNING, ON CONFLICT DO UPDATE, GENERATED ALWAYS AS, window
functions, FTS5 bm25() / snippet() / MATCH, R-Tree, PRAGMA journal_mode=WAL, sqlite_master, CREATE TRIGGER, ATTACH DATABASE, and the
.-prefixed CLI dot-commands. From sqlite (Public Domain), mediawiki (GPL-2.0),
sqitch (MIT), prisma (Apache-2.0), yii2 / flask (BSD-3), Dancer2.

The pool holds ~80 real SQLite .sql files total — the category is capped there.

_{* v1.1.0 root SOURCES.md said python was 487 files; see the python row.}

Expanded categories

Category	v1.1.0	v1.2.0	What's new
assembly (SOURCES)	3 / 15	16 / 253	NASM preprocessor torture (%macro/%rep, every SSE/AVX/AVX-512/AMX extension), GAS/AT&T (cosmopolitan, xv6, FreeBSD libc + kernel, x86-bare-metal), AArch64 + Darwin ABI + NEON (HelloSilicon), 6502 (cpm65), ARM bare-metal (RaspberryPi — unknown, no upstream license), 16-bit real-mode boot (os-tutorial, linux-1.0), machine-generated perlasm (blst).
groovy (SOURCES)	2 / 9	16 / 329	Was .java-only (Spock's AST transform, Gradle's Java API) — zero files the standard actually classifies. Now real Groovy across its dialects: Gradle build DSL (closure-as-config), @CompileStatic plugin authoring, Spock specs (given:/when:/then:/where:, power asserts, data tables), the Jenkins Groovy view builder-DSL, Jenkinsfile.
cobol (SOURCES)	11 / 308	15 / 589	~2×. The NIST COBOL-85 CCVS positive-conformance suite (150 programs, round-robin across every language module), the che-che4z LSP's own parser fixtures (XML/JSON PARSE, EXEC SQL/EXEC CICS, dialect keywords), Open Mainframe Project's cobol-check, JCICS interop samples, GnuCOBOL library copybooks.
tcl (SOURCES)	1 / 13	9 / 156	The SQLite .test suite — TCL harness (do_test name { db eval { …SQL… } } {…}) wrapping SQL string blocks, the polyglot boundary the standard's "SQLite mega-sensor" targets — across core DML / query planner / CTE-window / JSON-triggers / FTS5 / R-Tree; plus real application Tcl from macports-base (port1.0 DSL, registry2.0).
html (SOURCES)	7 / 19	15 / 67	GLSL-in-<script> polyglot (three.js), deep <section> state trees (reveal.js), framework microsyntax that isn't valid HTML (Angular @if/{{ }}/[x]/(y)), CSP <meta> + nonce templates (VS Code webviews), deliberately-broken DOM fixtures (Playwright), single-file inline-everything apps (Cesium), h5bp source-vs-build.
css (SOURCES)	3 / 9	11 / 49	Tailwind v4 @theme/@layer/@utility custom at-rule DSL, Gutenberg CSS Modules (composes:, :global()), minified single-line Bootstrap, machine-generated RTL sheets, reveal.js compiled themes, vendored third-party CSS, legacy MediaWiki module stylesheets.
python (SOURCES)	6 / 281*	6 / 281	No files changed. v1.1.0's 487 was a bookkeeping error — the issue-#4 audit recorded fastapi at 446 files but that folder has only ever held 240 (git ls-tree v1.1.0 confirms). The v1.2.0 regen made the count honest.
embedded_python (SOURCES)	—	1 / 14	New category. meow_turtle — original RP2040/RP2350 MicroPython firmware: boot.py, import machine, and PIO assembly embedded in Python via @rp2.asm_pio. GitGalaxy's embedded_python standard (.py/.mpy, micropython shebang, machine/UART/viper markers) classifies this separately from CPython. unknown by design.
jcl (SOURCES)	5 / 191	6 / 193	Effectively flat. This category is at the pool's real-licensed-source ceiling — the entire pool holds ~191 real licensed .jcl and the corpus already contained them; the rest is GITGALAXY-forged or unlicensed. data/jcl/SOURCES.md now documents this and what unblocks it (new licensed mainframe repos in gitgalaxy/data).

Tooling

.claude/skills/expand-language-coverage/ — a Claude Code skill packaging the
repeatable process these expansions used: read the target language's detection
standard, survey the pool for dialect-correct material (tools/survey_pool.py),
design a folder set and get it approved, stage each folder with provenance
drafting (tools/stage_folder.py), regenerate PROVENANCE.json, update every
SOURCES.md, open a PR. Hard rule: real source only, never synthesized; cap a
category and say so when the pool can't support the target size.

Provenance

data/PROVENANCE.json regenerated deterministically. exact-confidence repo-folders: 38 → 102; unknown 27 → 29
(embedded_python/meow_turtle, assembly/raspberrypi_baremetal). Full per-folder attribution (upstream URL, commit,
license, the specific parsing boundary each folder tests) is in the per-category
SOURCES.md files, indexed from the root SOURCES.md.

For GitGalaxy maintainers

Per RELEASING.md: regenerate the golden-master
fixtures and the tri-comparison / tree-sitter-accuracy artifacts against this tag
(docs/self_scan/BUMPING_THE_CRUCIBLE_PIN.md), then bump LANGUAGE_CRUCIBLE_REF

• tests/_crucible_pin.py's PINNED_TAG to v1.2.0.

Full diff: v1.1.0...v1.2.0

v1.1.0 — Mainframe, Glue-Script & Embedded-Scripting Coverage

Highlights

This release substantially deepens six language categories with real, licensed, unmodified upstream source — and closes out a full provenance audit (issue #4) across every language directory in the corpus.

Every file added is real production code from a named upstream repository, with a recorded license, and (for these six categories) an exact commit hash. See the root SOURCES.md for the full attribution index, and each data/<language>/SOURCES.md for per-repo detail.

Languages enhanced

COBOL — 53 → 308 files (5 → 11 repos)

Deepened for CICS/mainframe interpretation hardening. IBM's CICS TS GenApp and the CICS Banking Sample Application (CBSA) expanded to their complete program sets (COBOL + copybooks). Added: CICS-Cobol (GPL-2.0 language-feature demos), cobol-programming-course (Open Mainframe Project training labs), zopeneditor-sample (IBM), cash-account-cobol (IBM Stock Trader), the AWS CardDemo mainframe-modernization reference app, and expanded cobrix / gnucobol.

JCL — 5 → 191 files (1 → 5 repos)

Previously the weakest-covered category relative to its importance for CICS/mainframe work. Now full CICS resource definitions, DB2 bind/create/drop, VSAM IDCAMS DEFINE, and RACF setup JCL from cics-genapp, CBSA, cobol-programming-course, zopeneditor-sample, and cash-account-cobol.

Shell — 10 → 282 files (1 → 13 repos)

Previously almost entirely mislabeled — the sole folder was mostly Ruby source under a brew directory. Replaced with real POSIX/Bourne/ksh scripts spanning four decades of Unix-lineage kernel build tooling: FreeBSD, illumos, Linux, Darwin XNU, Haiku, SerenityOS, ReactOS — plus Kubernetes, curl, Ansible, Moby, SQLite.

PowerShell — 7 → 129 files (1 → 5 repos)

Expanded PowerShell/PowerShell core, and added PowerToys, Roslyn, the .NET runtime, and NVDA build/CI/packaging tooling.

Lua — 0 real files → 119 files (5 repos)

The prior lua/redis folder held zero actual Lua — only Redis's C-side embedding of the Lua VM. Replaced with two dialect eras of the official Lua reference test suite (5.1 vendored in Redis, 5.4 vendored in Cosmopolitan), Pandoc's Lua document-filter engine, Redbean web-server demo scripts, and FreeBSD's Lua-scripted boot loader (lualoader, running pre-kernel).

LiveCode — 2 real files → 99 files

The prior livecode/core folder mixed 2 genuine LiveCode-language files in with unrelated engine C++ source and build scripts. Replaced with a properly-scoped folder covering both real LiveCode dialects: classic .livecodescript and the newer typed .lcb ("LiveCode Builder").

Provenance audit (issue #4)

• New root SOURCES.md — a per-category attribution index with a documented confidence-level methodology (exact / pool-reference / unknown), created from scratch since the file the issue referenced was never actually committed.
• New data/PROVENANCE.json — a machine-readable manifest (repo, path, upstream URL, commit, license, confidence) covering all ~150 repo-folders across the corpus, generated by a new --provenance mode in tools/independent_data_auditor.py.
• data/<language>/SOURCES.md added for every one of the 56 populated language directories.
• Caught and hand-corrected two automated mismatches (go/core and zig/zig had matched the wrong same-named repos) by inspecting file contents before trusting them.

Process changes

• .github/PULL_REQUEST_TEMPLATE.md now requires a Source & License section for new data/ contributions.
• New RELEASING.md documents how this tag is consumed downstream by GitGalaxy's CI, and the full bump checklist.

Downstream

GitGalaxy's CI pin has been updated to this release, and consolidated from what turned out to be eleven separately-hardcoded references down to two sources of truth (a LANGUAGE_CRUCIBLE_REF repo variable and a tests/_crucible_pin.py constant) — see squid-protocol/gitgalaxy#2398.

Note on versioning

This release was briefly tagged v1.10 for a few hours before being corrected to proper three-part semver as v1.1.0. The v1.10 tag was deleted before any release was published against it.

v1.0.0

The ultimate deterministic test suite for SAST engines, AST parsers, and DevSecOps pipelines.

Language Crucible v1.0.0 is a meticulously curated, hyper-dense repository containing pure logic and edge cases across 50+ programming languages and frameworks. It is designed specifically to act as a "Golden Master" for testing the speed, resilience, and accuracy of codebase analysis engines.

Originally forged as the baseline integration test for the GitGalaxy heuristic engine, this repository guarantees that CI/CD pipelines can detect architectural drift, regex backtracking (ReDoS) vulnerabilities, and parser failures with mathematical precision.
🌌 The Crucible Payload
This release permanently locks in the directory structure and file hashes for over 900 artifacts, spanning the entire history of software engineering.

Highlights from the Data Matrix:

Deep Legacy Systems: Mainframe COBOL (CICS Banking), IBM JCL, Apollo 11 AGC Assembly, and Fortran.

Modern Systems & Web3: Rust (Tokio, Serde), Zig (TigerBeetle, Bun), Go (Kubernetes), and Solidity (OpenZeppelin).

Enterprise Monoliths: Java (Spring Boot), C# (Roslyn), PHP (Magento2, Symfony), and Ruby on Rails.

Scripting & Data Science: Python (FastAPI, NumPy), Perl (SpamAssassin), Matlab, and R.

Complex Build Environments: Dockerfiles, Nix, CMake, Makefiles, M4, and Shell.

UI & Frontend Ecosystems: TypeScript (Playwright, Zod), JavaScript (React, Three.js), Dart (Flutter), and CSS/HTML combinations.

⚙️ Why Use Language Crucible?
Zero-Drift CI/CD: Clone this tag in your GitHub Actions to baseline your static analysis tools. If your parser drops support for a language, the output hash will immediately flag the regression.

ReDoS Torture Testing: The files contained within are highly compressed, nested, or minified to intentionally trigger catastrophic backtracking in poorly written regex parsers.

AST Bottleneck Benchmarking: Test how fast your compiler or AST generator can switch contexts between 50 different syntaxes.