Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions .backlog/CHORE-UNTRACK-BUILT-ENGINE/PRD.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# CHORE-UNTRACK-BUILT-ENGINE — stop committing a build artifact

## Why

`CTLD.lua` is generated by `merge_CTLD.ps1` and committed anyway. `.gitignore` line 5 says it is
deliberate — *"intentionally tracked — available at repo root for DCS missions"* — a decision taken
at the repository bootstrap (`99b3a26`, 2026-07-07).

That reason no longer holds. Nothing points at the file in the repository: no link in the README or
`docs/`, and **VMCT, the one consumer anyone assumed depended on it, does not**. Its `vendored.yaml`
pins CTLD at `2.0.0-rc3` with `manual_steps: "(verbatim) re-download the CTLD.lua asset from the
matching VEAF/CTLD release"` and watches `github-release` — it takes the release asset, not the
repository file. Zip confirmed nothing else consumes it directly.

Meanwhile the cost is paid on every code change: **28 merges touched `src/` over 30 days, and 26 of
them carried the regenerated `CTLD.lua`** — a one-megabyte generated diff in almost every code PR,
which nobody reviews and which puts two parallel PRs in conflict on a file neither of them wrote.

What made the removal possible only now: `FEAT-DEV-BUILD-CHANNEL`. A release attaches `CTLD.lua`,
and so does the floating `dev` pre-release, so both a stable and a work-in-progress engine remain
downloadable. This lot must land **after** it.

## The trap this lot exists to avoid

`python-quality` runs on ubuntu: checkout, poetry, `pytest`. It **never builds the engine**, so the
tests guarded by `skipif(not (REPO / "CTLD.lua").is_file())` only run thanks to the committed file.
Measured by removing it and running the suite:

| | Result |
|---|---|
| With `CTLD.lua` | **262 passed** |
| Without | 234 passed, **27 skipped, 1 failed** |

Deleting the file without touching CI would leave the pipeline green while the whole `.miz`
installation suite quietly stopped running. So the build comes first, the deletion second.

The failure is a separate defect the measurement exposed: `test_web_app.py::test_inject_into_miz`
raises `KeyError: 'injected'` instead of skipping — it lacks the guard its neighbours have, so a
contributor who clones and runs `pytest` before building gets an error rather than "not built yet".

## Scope

Build the engine in the job that needs it, fix the missing guard, then untrack the file and say so
in the documentation.

## Out of scope

- **Rewriting history.** The 471 past blobs weigh 2.8 MiB packed out of a 93 MiB `.git` — they cost
nothing and rewriting a shared history would.
- **Changing what a release or a dev build publishes.** Both keep attaching `CTLD.lua`; that is
precisely what makes this lot safe.
- **VMCT's `rc3` pin.** Their repository, their upgrade.
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# 01 — Build the engine in the job that tests it

**Status:** done
**Lot:** CHORE-UNTRACK-BUILT-ENGINE

## Problem

`python-quality` (`.github/workflows/python-quality.yml`) runs on ubuntu and never builds
`CTLD.lua`; 27 tests currently run only because the file is committed. This must be fixed **before**
the file goes away, or the deletion turns a green pipeline into a lie.

`merge_CTLD.ps1` is the single source of truth for the build and must stay so — no second, simpler
"build for tests" path that could drift from it.

## Change

Make the script run on the ubuntu image and call it before `pytest`. Two path expressions are the
only Windows-only things in it:

- line 25 — `Join-Path $scriptDir "..\.."`
- line 33 — `Join-Path $repoRoot "tools\ctld-tools"`

`Join-Path` composes with the platform separator; a literal `\` inside the argument does not. Both
become `Join-Path` chains (or `/`, which Windows accepts too).

If `pwsh` turns out to be absent from the ubuntu image, the fallback decided during the grilling is
to move the job to `windows-latest` — same steps, about two minutes more, free on a public
repository. Do not invent a third build path.

## What was done

Three literal Windows paths, not two: `generate_i18n_dicts.ps1:36` carries the same `"..\.."`, and
`merge_CTLD.ps1` calls it, so it had to go too. All three are now composed one segment at a time.

`python-quality` gained a `pwsh` step running `merge_CTLD.ps1` before `pytest`, and its `paths:`
filter now includes `src/**` and `tools/build/**` — the job builds the engine, so a change to
either can break it and must trigger it.

## Acceptance

- [x] `python-quality` builds `CTLD.lua` before running `pytest`.
- [x] A local Windows build still works unchanged — same command, and `git diff` on the rebuilt
`CTLD.lua` reports no changed line. Suite still at **262 passed**.
- [x] The suite reports **262 passed, 0 skipped** on the ubuntu runner (PR #110 CI), and `pwsh` is
present — the log shows `shell: /usr/bin/pwsh` then `Merged : 32 file(s)`. The
`windows-latest` fallback is not needed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# 02 — A missing engine skips the test, it does not crash it

**Status:** done
**Lot:** CHORE-UNTRACK-BUILT-ENGINE

## Problem

Found by removing `CTLD.lua` and running the suite: `test_web_app.py::test_inject_into_miz` fails
with `KeyError: 'injected'` instead of skipping. Its neighbours in `test_install.py` and
`test_resources.py` all carry `skipif(not (REPO / "CTLD.lua").is_file())`; this one does not.

Today the file is always there, so nobody sees it. Once it is generated rather than committed, a
contributor who clones and runs `pytest` before building gets an error that says nothing about the
actual cause — and the error is a `KeyError` deep in a response payload, not a message about a
missing engine.

## Change

Give it the same guard as its neighbours. If the endpoint should instead report the missing engine
to the caller, that is a different ticket — the install path already raises a message saying how to
build it (`resources.read_engine`), and this test is not the place to redesign that.

## Acceptance

- [x] With no `CTLD.lua`: **234 passed, 28 skipped, 0 failed** (it was 27 skipped / 1 failed).
- [x] With `CTLD.lua`: **262 passed**, the test running and asserting exactly what it did before.
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# 03 — Untrack the engine and say so

**Status:** done
**Lot:** CHORE-UNTRACK-BUILT-ENGINE

## Problem

With tickets 01 and 02 in, nothing depends on the committed `CTLD.lua` any more. It remains as a
one-megabyte generated diff in almost every code PR (26 of 28 merges over 30 days), a guaranteed
conflict between parallel branches, and a file that can silently disagree with `src/` if someone
forgets to rebuild.

## Change

- `git rm --cached CTLD.lua`, and `.gitignore` gains the file — replacing the line that currently
explains why it is *not* ignored. The new comment states the real reason it can go: releases and
the `dev` pre-release both attach it (`FEAT-DEV-BUILD-CHANNEL`).
- **`CLAUDE.md`** — "rebuild after any `src/` change" becomes explicit that the rebuild is for
testing and is not committed.
- **`docs/developer/building-and-testing.{md,fr.md}`** — where the built engine comes from for a
contributor (build it), for a Mission Maker (the tool, or a release asset), and for a tester
(the `dev` pre-release).

No history rewriting: the 471 past blobs weigh 2.8 MiB packed and harm nobody.

## Acceptance

- [x] The file is untracked (`git ls-files CTLD.lua` → nothing) and ignored
(`git check-ignore` → `/CTLD.lua`), so a fresh clone has none and a local build leaves
`git status` clean.
- [x] EN and FR documentation in step: `building-and-testing.{md,fr.md}` gain a table saying where
to get an engine per role, and `CLAUDE.md` now says *never commit* as well as *never
hand-edit*.
- [x] A release still attaches `CTLD.lua`, and so does the `dev` pre-release — observed on the
first dev build: assets `ctld-tools.exe` (22.4 MB) and `CTLD.lua` (1.17 MB), the latter
declaring `ctld.VERSION = "2.0.0-rc6-182ec25"`.
- [x] CI green with the file absent from the repository: all eight checks pass on PR #110, and
`python-quality` reports **262 passed, 0 skipped** after building the engine itself.
3 changes: 2 additions & 1 deletion .backlog/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@ authored **per lot, when the lot is started** (not in batch).

| Lot | Status | Description | Branch |
|-----|--------|-------------|--------|
| `CHORE-UNTRACK-BUILT-ENGINE` | in progress | `CTLD.lua` is generated and committed anyway. `.gitignore` line 5 calls it deliberate — *"available at repo root for DCS missions"* — a bootstrap-era reason that no longer holds: nothing points at it, and **VMCT, the assumed consumer, does not** (its `vendored.yaml` pins `2.0.0-rc3` with *"re-download the CTLD.lua asset from the matching release"* and watches `github-release`). The cost is paid every time: **26 of the 28 merges touching `src/` over 30 days carried the regenerated file** — a one-megabyte generated diff nobody reviews, and a guaranteed conflict between parallel PRs. The trap, measured rather than assumed: `python-quality` runs on ubuntu and never builds the engine, so deleting the file alone would drop the suite from **262 passed** to **234 passed / 27 skipped / 1 failed** while CI stayed green. So: build the engine in that job first (`merge_CTLD.ps1` made portable — two `\` paths), fix `test_inject_into_miz` (it crashes instead of skipping), then untrack. Depends on `FEAT-DEV-BUILD-CHANNEL`, which is what keeps the engine downloadable. No history rewriting (471 blobs = 2.8 MiB packed). | `chore/untrack-built-engine` |
| `FEAT-CUSTOM-BEACON-SOUNDS` | planned | A beacon sound the Mission Maker chooses, instead of a text box naming a file the tool never installs. Grilled with Zip on 2026-08-08: custom is **derived** from `radioSound` (no second key that could disagree with the engine); a chosen file enters the mission under a **reserved name** (**ADR 0012**) because a Mission Maker whose own file is called `beacon.ogg` would otherwise see it silently overwritten; the original name survives as a schema-only label (`FIX-TOOL-I18N-LANG`'s lesson — a catalogue key would make every pre-lot configuration report a missing setting at mission start); the bytes are read at selection and live in the session, so reopening a `.miz` reinstalls them **on another machine with the original file gone**. `OggS` checked, no size cap, nothing deleted from the archive. | `feature/custom-beacon-sounds` |
| `FEAT-DEV-BUILD-CHANNEL` | in progress | An exe to hand a tester between two releases. Zip's first idea — the exe grafting an arbitrary `CTLD.lua` into a copy of itself — **works** (verified: rc6 + 1.17 MB appended still runs) and was dropped anyway: it pairs a new engine with the exe's older schema and interface, an unsigned exe altered after the build reads as tampered, and `--version` would keep lying. The `build-exe` job already produces a complete exe from a commit in **2 min 06 s** on free public-repo runners; it only lacked a trigger. Built on every merge into `develop`, published as an artifact **and** a floating `dev` pre-release (an artifact answers `401` to an anonymous download), versioned `<ctld version>-<commit hash>`. | `feature/dev-build-channel` |
| `FEAT-DEV-BUILD-CHANNEL` | merged (PR #109) | An exe to hand a tester between two releases. Zip's first idea — the exe grafting an arbitrary `CTLD.lua` into a copy of itself — **works** (verified: rc6 + 1.17 MB appended still runs) and was dropped anyway: it pairs a new engine with the exe's older schema and interface, an unsigned exe altered after the build reads as tampered, and `--version` would keep lying. The `build-exe` job already produces a complete exe from a commit in **2 min 06 s** on free public-repo runners; it only lacked a trigger. Built on every merge into `develop`, published as an artifact **and** a floating `dev` pre-release (an artifact answers `401` to an anonymous download), versioned `<ctld version>-<commit hash>`. | `feature/dev-build-channel` |
| [`FIX-MENU-DOUBLE-MULTICREW`](FIX-MENU-DOUBLE-MULTICREW/PRD.md) | merged (PR #106) | F10 menu duplication on multi-crew aircraft (CH-47 pilot + copilot); menu loss when one crew member leaves a shared group. | — |
| `DOCS-RELEASE-LIFECYCLE` | merged (PR #107) | Question from **FullGas**: does the exe download the latest build after a merge, or does a release have to be published? The answer is only in the code — `FEAT-ONE-CLICK-INSTALL` chose to **bundle** the engine (`--add-data "../../CTLD.lua;ctld_data"`, `release.yml` triggered on `published-v*` only), so an exe installs the engine of its own release, offline, and never updates itself. Nothing user-facing says it, so the natural assumption is the opposite. Documented in the README and the mission-maker guide (EN + FR), with the pre-release detail that goes with it: every rc publishes as a pre-release, so **no release carries the *Latest* badge** today and `releases/latest` redirects to the Releases index. No mechanism change. | `docs/release-lifecycle` |
| `FIX-PARACHUTE-GROUP-NAME-COLLISION` | merged (PR #103) | `parachuteTroops` spawns the DCS group under the raw config `templateName` — two groups loaded from the same troop template collide on that name, and DCS destroys the first when the second (same template) lands. Fix: suffix group/unit names with `ctld.utils.getNextUniqId()`, as `CTLDObjectRegistry.spawnObject` already does everywhere else. | `fix/parachute-group-name-collision` |
Expand Down
15 changes: 13 additions & 2 deletions .github/workflows/python-quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,15 @@ on:
branches: [develop, master]
paths:
- "tools/ctld-tools/**"
- "src/CTLD_config.yaml"
- "src/**"
- "tools/build/**"
- "tests/ci/data/config_defaults.json"
- ".github/workflows/python-quality.yml"
pull_request:
paths:
- "tools/ctld-tools/**"
- "src/CTLD_config.yaml"
- "src/**"
- "tools/build/**"
- "tests/ci/data/config_defaults.json"
- ".github/workflows/python-quality.yml"

Expand Down Expand Up @@ -44,6 +46,15 @@ jobs:
- name: Type check (mypy)
run: poetry run mypy ctld_tools

# 27 tests install the real engine into a .miz and skip themselves when it is missing, so a
# job that does not build it stays green while that whole suite quietly stops running
# (CHORE-UNTRACK-BUILT-ENGINE measured 262 passed → 234 passed / 27 skipped / 1 failed).
# merge_CTLD.ps1 is the single build path — a lookalike built for the tests would drift.
- name: Build CTLD.lua (the tests install it)
working-directory: ${{ github.workspace }}
shell: pwsh
run: ./tools/build/merge_CTLD.ps1

# Tests include the oracle drift guard (committed tests/ci/data/config_defaults.json
# == fresh emit from src/CTLD_config.yaml) and the embed/wrap round-trip.
- name: Tests (pytest + coverage)
Expand Down
10 changes: 8 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,14 @@
.idea/encodings.xml
*.xml

# CTLD.lua is intentionally tracked — available at repo root for DCS missions.
# Do NOT add it to .gitignore.
# ── The built engine (CHORE-UNTRACK-BUILT-ENGINE) ────────────────────────────
# Generated by tools/build/merge_CTLD.ps1, like /src/CTLD_config_default_yaml.lua below.
# It used to be tracked so the repo root would hold a copy for DCS missions; it is now
# published where people actually take it — attached to every release, and to the floating
# `dev` pre-release built from each merge into develop. Committing it put a one-megabyte
# generated diff in almost every code PR and made two parallel branches conflict on a file
# neither had written.
/CTLD.lua

# Build outputs (the dev-time companion is built by CI / release, not committed).
dist/
Expand Down
6 changes: 4 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,10 @@
Enforced by CI `lua-lint` (`luac5.1 -p`).
- **luacheck** `--config .luacheckrc src/` must be clean (rely on CI if not installed locally).
- **Build**: `powershell -ExecutionPolicy Bypass -File tools\build\merge_CTLD.ps1` → `CTLD.lua`
in UTF-8 without BOM. `CTLD.lua` is **generated — never hand-edit**; rebuild after any
`src/` change. The build also auto-syncs i18n dicts (`generate_i18n_dicts.ps1 -Apply`).
in UTF-8 without BOM. `CTLD.lua` is **generated and git-ignored — never hand-edit, never
commit**: rebuild after any `src/` change to test locally, and let CI build it for everything
else (it is attached to each release and to the floating `dev` pre-release). The build also
auto-syncs i18n dicts (`generate_i18n_dicts.ps1 -Apply`).
- **Git hooks**: after clone, run `git config core.hooksPath .githooks` to activate the
`pre-push` hook (blocks push when i18n keys are missing from the dictionaries).
- **TDD**: write a failing busted test first, make it pass, refactor. New/changed logic ships with
Expand Down
Loading