Skip to content

Development

special-place-administrator edited this page Oct 11, 2026 · 1 revision

Development

Use this page for

building SymForge from source, running the same checks CI runs, and understanding how a change reaches a release. Audience: human contributors.

Toolchain

The toolchain is pinned by rust-toolchain.toml: Rust 1.96.0 with rustfmt and clippy (as of 11.7.0). rustup resolves the pin automatically on first cargo use. The crate is edition 2024. The npm wrapper in npm/ needs Node.js; CI runs its tests on Node 20.

A cold build compiles 19 tree-sitter C grammars, bundled SQLite, and a vendored libgit2 (built with CMake), so the first build is long. Later builds are incremental.

Build

cargo build --release          # target/release/symforge

Cargo features:

Feature Meaning
server (default) The MCP server, daemon, sidecar, watcher, and CLI
embed The engine-only build for library embedders: parsing, live index, query, and git, with no daemon, sidecar, server, or CLI. See Embedding-the-Engine
cbm-spike Internal falsifier spike (the resolver under src/parsing/resolver/). Not a supported configuration
__test-internals Repository-internal test door. Not a supported configuration

Test

Run these from the repository root. They are the commands CI runs on every pull request.

cargo fmt --check
cargo clippy --all-targets -- -D warnings
cargo test --lib --bins --tests -- --test-threads=1
cargo bench --bench observed_refresh_gate_v1 -- --test
cargo build --release

cd npm && npm test

Two points that are easy to get wrong:

  • Use --test-threads=1. The suite is run serially on purpose.
  • Do not use cargo test --all-targets with --test-threads=1. The trailing libtest flag is forwarded to every selected binary, and the criterion bench target (observed_refresh_gate_v1, harness = false) rejects it. Select --lib --bins --tests for the suite and smoke the bench separately with -- --test, as above.

The golden-replay integration tests read corpora that are not checked in. CI fetches them with bash execution/clone_phase0_corpora.sh; without them those tests skip.

Embed builds

The engine-only build is a separate feature set. Run it in its own pass, never interleaved with the default-feature suite in one target/ directory: interleaving feature sets can corrupt build artifacts (E0786, "invalid metadata files").

cargo build --no-default-features --features embed
cargo clippy --no-default-features --features embed,__test-internals --lib -- -D warnings
cargo test --no-default-features --features embed --lib -- --test-threads=1
cargo test --no-default-features --features embed --test embed_bound_index -- --test-threads=1

Why the embed cell is its own gate: with server off, every module behind #[cfg(feature = "server")] is absent, so a #[cfg(test)] helper whose only caller lives in one of those modules becomes an unused import and fails -D warnings there while every default-feature check passes. Gate such a helper with #[cfg(all(test, feature = "server"))].

CI also cross-compiles the embed build to x86_64-unknown-linux-musl, because integrating platforms compile the engine inside musl guests:

rustup target add x86_64-unknown-linux-musl
sudo apt-get install -y musl-tools
CC_x86_64_unknown_linux_musl=musl-gcc cargo build --no-default-features --features embed --target x86_64-unknown-linux-musl

Long builds

A killed build can leave a corrupted target/, and the symptom looks like a code failure (E0786, a rustc internal compiler error, or can't find crate). Do not interrupt a cold build. If you hit one of those errors, delete target/debug/incremental, then try cargo clean -p symforge, then a full cargo clean, before suspecting your change. On Windows, a build that dies with 0xc000012d has exhausted the virtual-memory commit limit; cap build parallelism with -j 8 (or lower).

CI gates on pull requests

ci.yml runs once per pull request. Every job waits for the first two gates.

Job What it checks
conventional-commits The PR title is a conventional-commit subject: build, chore, ci, docs, feat, fix, perf, refactor, revert, style, or test
release-version Version-sync tests (python -m unittest discover -s execution -p 'test_*.py'), every text blob in the index is LF, the V11 lifecycle checks, and python execution/version_sync.py check (release metadata stays aligned)
rust cargo fmt --check, a single rmcp major in the dependency graph, cargo clippy --all-targets -- -D warnings, the suite and bench smoke from the Test section, cargo build --release, and the tool-correctness harness (node scripts/verify-tools.cjs --bin target/release/symforge, run against both fixtures; needs ripgrep)
embed-build The embed build, clippy, lib tests, and the embed_bound_index contract from the Embed builds section
embed-musl The embed build and clippy cross-compiled to musl
darwin-serve-port cargo test --test serve_port -- --test-threads=1 on macOS
npm npm test in npm/
windows-native cargo fmt --check, clippy, cargo check of the embed build, and cargo nextest run --no-fail-fast (process-per-test) on Windows

Scheduled (weekly) and manual runs add performance-smoke: the ignored 1000-file load test and the portable coupling-calibration smoke, both in --release. Full real-repository coupling calibration is operator-triggered and never part of standard CI.

Two files are pinned by tests, so editing them is a code change:

  • README.md and AGENTS.md: tests/conformance.rs asserts that seven exact phrases about checkpoint and snapshot recovery appear across the pair. Grep tests/ before restructuring either file.
  • .github/workflows/ci.yml and release.yml: tests/preventive_runtime_dark_v11.rs holds a whole-file fingerprint of each. A workflow edit must update its fingerprint in the same commit.

Releases

Releases are driven by Release Please on main, so commit and PR-title subjects decide the next version. Configuration is in .github/release-please-config.json and .github/.release-please-manifest.json; the release workflow is .github/workflows/release.yml.

  1. A push to main runs the release workflow. It re-verifies the commit (version sync, the Rust suite, the embed build, npm tests, and the tool-correctness harness) before anything is released.
  2. Release Please opens a release PR that bumps the version and writes CHANGELOG.md. The workflow merges that PR, then creates the GitHub release at the gated commit.
  3. Binaries are built for x86_64-pc-windows-msvc, x86_64-unknown-linux-gnu, aarch64-apple-darwin, and x86_64-apple-darwin, and attached to the release with the npm tarballs.
  4. The crate is published to crates.io and the npm packages (root plus one per platform) to npm, after the build and asset upload succeed.

Version fields in Cargo.toml, npm/package.json, and the four npm/platforms/*/package.json files are updated by Release Please; do not bump them by hand.

Because the version comes from commit subjects, a malformed squash message can make a change invisible to Release Please: no changelog entry and no bump, with no error. For maintainers merging with gh, squash-merge and pass an explicit short body:

gh pr merge <N> --squash --delete-branch \
  --subject "<conventional PR title> (#<N>)" \
  --body "<one short paragraph, no parentheses, no colon-bearing prose lines>"

The default squash body concatenates every inner commit message, and a line shaped like word: text inside it can break the parser for the whole commit.

Related: Embedding-the-Engine, Architecture-and-How-It-Works.

Clone this wiki locally