Migrate decomp source to a submodule (Milestone 1)#129
Merged
Conversation
Migrates the SSB64 decomp source out of the in-tree src/ and include/
directories into a real git submodule at decomp/, on the local
port-patches branch carrying our #ifdef PORT-guarded modifications.
Submodule:
- Path: decomp/
- URL: git@github.com:JRickey/ssb-decomp-re.git (JRickey fork of upstream)
- Branch: port-patches (contains port modifications wrapped in #ifdef PORT)
- Pinned SHA: f74646d59 (one commit ahead of upstream main 977cf4780)
- Submodule's .git/config remote.origin.url points at fork; remote
upstream points at VetriTheRetri/ssb-decomp-re for tracking.
CMakeLists.txt changes:
- src/<module>/ → decomp/src/<module>/ for all 14 game-code GLOBs
- src/libultra/{gu,n_audio,audio}/ → decomp/src/libultra/...
- src/credits/ encoder working dir + generated outputs → decomp/src/credits/
- ssb64_game include path now: include/ + decomp/include/ + decomp/src/ + port/
- Executable include path: decomp/src/ + port/ ONLY — decomp/include/ is
intentionally excluded so port/*.cpp don't pick up the C decomp shim
headers (stdlib.h, stddef.h) over libc++. Only the C decomp library
consumes the shim headers.
Hand fixes (decomp/ submodule, in port-patches branch):
- src/it/item.h: rewrote macro definitions to wrap each #define in
#ifdef PORT/#else/#endif rather than wrapping inside the macro body.
upstream_reconcile.py's hunk-wrap broke on macros where a continuation
line (\) was followed by a preprocessor directive.
- src/sc/sc1pmode/sc1pstageclear.c: changed
#include "../../../port/bridge/framebuffer_capture.h" to
#include <bridge/framebuffer_capture.h> — the relative path stopped
working when the file moved into decomp/.
scripts/new-worktree.sh:
- Added decomp to the per-submodule clone loop, with a guard that skips
submodules not yet configured in the main tree (so this branch can
spawn worktrees from main even before the submodule lands).
Verified: ssb64_game compiles (180 objects), BattleShip executable
links (20 MB), no errors.
Vendored src/ and include/ are still in the worktree and will be
deleted in a follow-up commit (Phase 4), once Phase 3 (reloc tooling
migration) lands.
Phase 3 + Phase 4 of the decomp submodule migration: the port now consumes
all decomp source from the decomp/ submodule. Vendored copies are removed.
Phase 3 (reloc tooling):
- tools/generate_reloc_stubs.py: SRC_DIR now scans decomp/src/ when present
(falls back to legacy src/ during the migration window). The vendored
reloc_data_symbols.us.txt stays — it uses a port-curated symbol-table
format incompatible with upstream's decomp/tools/relocFileDescriptions.us.txt
(which encodes the same data as -NNN: Name + bracketed [N] sections).
Migrating the format is a follow-up; the build doesn't depend on it.
- tools/relocData.py: deleted. The vendored copy was unused (no port script
imports it) and was an older snapshot of upstream's tools/relocData.py
which has since been refactored for JP/US.
Phase 4 (delete vendored src/ + include/):
- src/: 1014 files removed. All decomp game code now sourced from
decomp/src/ via the CMake glob retarget in the previous commit.
- include/: 17 files removed. include/reloc_data.h is kept — it's a
port-side auto-generated header (regenerated by generate_reloc_stubs.py
on each build), and the build's include path resolves it from
${CMAKE_CURRENT_SOURCE_DIR}/include before falling through to
decomp/include/ for the rest of the C decomp shim headers.
Net diff: 1034 files changed, +1773 / -301108. That delta is the size of
the vendored decomp source — its replacement (the submodule pointer) takes
two lines in .gitmodules.
Verified end-to-end:
- Clean rebuild: 511/511 build steps green, 20MB BattleShip executable
- Codegen idempotency: generate_reloc_stubs.py + generate_yamls.py +
generate_reloc_table.py all produce expected outputs
- Runtime smoke: BattleShip launches, loads BattleShip.o2r + f3d.o2r +
ssb64.o2r, parses all audio assets (sounds1, sounds2, music, FGM),
initialises controllers, no crashes during 3-second eval window.
Milestone 1 of the decomp submodule refactor is complete. Milestone 2
(compile-from-source for relocData) starts next.
Pulls in the cleanup commit on JRickey/ssb-decomp-re port-patches (0583cc8df) which drops 342 files (~275k lines) of upstream matching infrastructure, splat configs, debug subsystems, and asset-extraction scripts the port doesn't use. Submodule SHA goes from f74646d5 to 0583cc8d. Net effect on a fresh clone: - decomp/ shrinks from ~150 MB to ~70 MB (most of which is src/relocData/ kept for Milestone 2/3 compile-from-source work) - decomp/ no longer has its own .gitmodules → no nested-submodule init step required when bootstrapping Verified: full clean rebuild green from main tree, 243/243 steps, 20MB BattleShip executable.
…teardown 7a0da1e74 on JRickey/ssb-decomp-re port-patches re-applies two hand-fixes from Milestone 1 that never made it into a submodule commit (they were uncommitted in the agent/decomp-submodule worktree which got torn down): - src/it/item.h: PORT-wrap moved from inside macro bodies to whole-#define blocks so the preprocessor doesn't choke on directives in continuation lines. - src/sc/sc1pmode/sc1pstageclear.c: relative include changed to angle bracket form so it resolves from decomp/src/. Without these, fresh clones of the outer branch + submodule init would hit the original item.h preprocessor errors and a missing-header error in sc1pstageclear.c. Now self-contained.
Two real breaks and three doc updates that the M1 commits skipped:
scripts/new-worktree.sh: line 133 still cd'd into $WT_DIR/src/credits
which doesn't exist after Phase 4 (the credits .txt sources moved into
decomp/src/credits/). A fresh worktree would fail at the credits-encode
step. Update to decomp/src/credits/. Also update the usage-comment
mentioning src/ as an editable area.
scripts/package-windows.ps1: same break — Push-Location on src/credits
during the credits-encode step before the cmake build. Windows release
script would fail. Update to decomp/src/credits/.
docs/architecture.md: the source-organization tree was the pre-refactor
layout. Replace with the decomp/{include,src,tools} tree and the new
port-target structure (port/, include/reloc_data.h, libultraship/, torch/,
decomp/). Update the "What NOT to Include in Port" section to describe
what was removed from port-patches (db/.c, ovl8/, audio/, particles/,
libultra/{io,libc,os,sp,vimodes}/) and what was kept.
docs/build_and_tooling.md: the LP64 audit grep example pointed at the
old in-tree include/ src/ paths. Now decomp/include/ decomp/src/ port/.
CLAUDE.md: refresh the worktree-workflow section to reference all three
submodules (decomp, libultraship, torch) plus the per-submodule push
target (decomp → port-patches on JRickey/ssb-decomp-re).
These were one-off scripts I built during Milestone 1 (decomp submodule
migration) and lived in a gitignored .claude/ scratch dir. Promoting them
to tools/ because each handles a workflow we'll need again whenever
upstream advances or someone refactors decomp consumption:
tools/upstream_reconcile.py
Mechanically PORT-guards port modifications against an upstream
baseline. For each modified file in a git checkout: revert mode-only
and whitespace-only diffs, drop auto-generated artifacts (e.g.
include/reloc_data.h, src/credits/*.encoded), and wrap each
substantive diff hunk in #ifdef PORT/#else/#endif via difflib's
SequenceMatcher. Files that already contain ANY #if(def)? PORT marker
are left alone (assumed hand-curated).
Caveats: hunks landing inside macro continuation lines (`\` followed
by a preprocessor directive) produce invalid C. Run
tools/compare_obj_text.sh + tools/audit_text_diffs.sh after every
reconcile to catch any structural breakage; we hit this once with
src/it/item.h during M1 and had to hand-fix it.
tools/compare_obj_text.sh
Compares per-TU __TEXT,__text section bytes between two
ssb64_game builds (e.g. main branch vs a refactor branch). Hashes
each .o's text section after stripping the file-path header line
and address column from `otool -t`. Flags any TU whose compiled
instruction stream has changed.
tools/audit_text_diffs.sh
Classifies the diffs flagged by compare_obj_text.sh:
- LINE_ONLY : every changed line is `mov w2, #<imm>` — debug
line-number drift via __LINE__ baked into PORT_RESOLVE call
sites (portRelocResolvePointerDebug(token, FILE, LINE)).
Semantics-preserving and expected.
- STRUCTURAL : at least one diff is something other than a w2
immediate move. Real instruction-level change. Investigate.
Together these proved Milestone 1 was provably semantics-preserving:
446/475 TUs byte-identical, 29 LINE_ONLY, 0 STRUCTURAL.
All three are AArch64/macOS-tested. compare_obj_text.sh and
audit_text_diffs.sh use otool, so Linux users would need to swap the
otool calls for `objdump --section=.text -d`. The LINE_ONLY pattern
in audit_text_diffs.sh is AArch64-specific (mov w2, #imm); on x86
it'd be `mov esi, <imm>` or similar.
4 tasks
JRickey
added a commit
that referenced
this pull request
May 4, 2026
Bump decomp submodule to JRickey/ssb-decomp-re#agent/msvc-build-fixes (commit 62ba3fd23) which contains two pre-existing MSVC build breaks that have been blocking Windows RelWithDebInfo builds since the src/ → decomp/ submodule migration (#129) landed: (1) decomp/include/stdarg.h re-`#define`s `va_list __builtin_va_list` on MSVC, where that intrinsic doesn't exist (C2061). Fixed by guarding the GCC-builtin branch with `#ifndef _MSC_VER`; MSVC falls through the existing `#ifdef _MSC_VER` block at top via <vadefs.h> + __crt_va_*. (2) decomp/src/mn/mnmaps/mnmaps.c (line 1073) puts `#ifdef PORT` directives inside the `lbRelocGetFileData(...)` macro argument list. C standard says `#` in function-like macro args is undefined behavior; clang/gcc accept leniently, MSVC rejects with C2059. Fixed by hoisting the conditional values out into local variables before the macro call. A tree-wide scan turned up 30 candidate sites in 11 files but 29 of them sit inside regular function calls (accepted), so this was the only one that needed restructuring. Verified: clean RelWithDebInfo Windows build (MSVC 19.44.35225) succeeds end-to-end and produces BattleShip.exe. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
JRickey
added a commit
that referenced
this pull request
May 4, 2026
…omp submodule migration The Linux + macOS package scripts still cd into $ROOT/src/credits, which no longer exists after PR #129 moved decomp source into the decomp/ submodule. Both jobs of the v0.7.4-beta Release workflow failed at the "Encoding credits text" step before cmake even started. The Windows .ps1 script was already updated; the two shell scripts were missed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
src/andinclude/directories into a new submodule atdecomp/(fork:JRickey/ssb-decomp-re, branchport-patches, base: upstream977cf4780).decomp/src/anddecomp/include/. Vendored copies (1031 files) deleted; onlyinclude/reloc_data.h(port-generated) remains.port-patchesbranch to only the files the port actually consumes (drops 342 files / ~275k lines of upstream matching infra, splat configs, debug subsystems, and asset-extraction scripts).tools/for future upstream sync + semantic-equivalence auditing.What this unlocks
This is Milestone 1 of the broader refactor described in
.claude/plans/piped-squishing-popcorn.md. The next milestone (compile-from-source fordecomp/src/relocData/) is what enables modding and custom characters; that work consumes upstream's typed reloc data via this submodule rather than ROM-extracting blobs through Torch.Provable semantic equivalence
The build wasn't enough validation — header-include order changes, macro redefinitions, or hunk-wrap producing different-but-valid C could all silently change behaviour. So I ran the per-TU
__textsection comparison againstmain's build:mov w2, #<imm>immediates — the__LINE__constants baked intoPORT_RESOLVE→portRelocResolvePointerDebug(token, FILE, LINE). Verified bytools/audit_text_diffs.sh.The two scripts that produce this evidence (
tools/compare_obj_text.sh,tools/audit_text_diffs.sh) are committed alongside the migration so future reconciles can re-prove the same thing.Files changed at a glance
+decomp/submodule (fork onport-patches)+tools/{upstream_reconcile,compare_obj_text,audit_text_diffs}.{py,sh}~CMakeLists.txt—src/GLOBs →decomp/src/, include paths split between port-tree (include/reloc_data.h) and submodule (decomp/include/)~scripts/new-worktree.sh— third submodule + credits-encoder cwd~scripts/package-windows.ps1— credits-encoder cwd~tools/generate_reloc_stubs.py—SRC_DIRprefersdecomp/src/~CLAUDE.md,docs/architecture.md,docs/build_and_tooling.md— reflect new layout−src/(1014 files),include/(16 files),tools/relocData.pyTest plan
cmake --build build --target ssb64 -j 4→ 243/243 steps green, 20MB BattleShip executable__textaudit vs main baseline: 446 identical / 29 LINE_ONLY / 0 STRUCTURALBattleShip.o2r+f3d.o2r+ssb64.o2r, parses all audio assets, no crashes during eval windowgit submodule update --init --recursivefrom a fresh clone resolves all three submodules cleanlyHeads-up for in-flight branches
The 7 currently-open
agent/*branches (android-port, crash-investigation, defensive-debug, issue-103-heap-token-fixes, issue-58-locate-noexcept, issue-77-mk-crash, save-editor-test) all forked offmainbefore this refactor and editsrc/foo.c/include/foo.hdirectly. After this PR lands, each will need either:mainfirst (preferred — simpler), ordecomp/src/and committed inside the submodule onport-patches(then bumping the outer pointer)🤖 Generated with Claude Code