Rust + KID pivot — M1 Part 2: skse-rs game interop#41
Merged
Conversation
Completes M1 (combined with plan 2). Ports minimum surface of CommonLibSSE-NG to Rust: address library v2 parser, relocation resolver, partial TESForm/BSTHashMap/BSReadWriteLock/TESDataHandler/ BGSKeyword/BGSKeywordForm/MemoryManager layouts, re-implemented TESForm::LookupByID and BGSKeywordForm::AddKeyword. Upgrades smoke test to look up Iron Sword + WeapMaterialIron and add the keyword. Flips CI gate to run self-hosted Skyrim integration on any case with a rust-ready.marker. 17 tasks across 8 phases. Task 14 has an explicit Step 0 to verify the BGSKeywordForm sub-object offset within TESObjectWEAP before writing to that memory.
Adds partial layouts for TESForm, BSReadWriteLock, BSTHashMap, TESDataHandler, BGSKeyword, BGSKeywordForm, plus the Address Library v2 bin format specification and MemoryManager function IDs. Each layout is marked as minimum-viable; full surface lands when real consumers need it.
Pure-Rust port of the format documented in CommonLibSSE-NG REL/ID.h. Decodes the delta-encoded (id, offset) stream, resorts defensively in case modes 0/6/7 break monotonicity, exposes id -> offset via binary search. No I/O dependency on Skyrim — AddressLibrary::load reads any path, AddressLibrary::parse takes a byte slice.
Five tests exercising the delta-decode state machine: happy path (3-pair fixture with id-modes 7/1/4 and offset-modes 0/2-scaled/7), missing-id error, format-version mismatch, truncated input, wrong pointer_size. Also derives Debug on AddressLibrary (required by the Result<_, E> formatting in the test panic arms).
Skips silently when the bin isn't present (CI case). On a developer box with Skyrim installed at the standard Steam path, loads the bin and confirms id 400269 (TESDataHandler::Singleton) resolves — a stronger parser correctness signal than synthetic fixtures alone. Path overridable via MORA_SKYRIM_DATA env var.
Relocation::id(u64) -> Result<Relocation> looks up the id in the lazily-loaded Address Library, then adds the game image base (GetModuleHandleW(NULL) on Windows, 0 on non-Windows for tests). Exposes addr/as_ptr/as_mut_ptr/as_fn helpers. Initialization is separate (set_library / load_library_from_path) so on_load controls when the bin is parsed.
On Linux with image base = 0, a relocation resolves to its rva directly. Two tests: (a) pre-init error semantics, (b) happy-path resolve after set_library.
Layout + 4 AE Address Library IDs (LockForRead/UnlockForRead bound, LockForWrite/UnlockForWrite reserved). RAII ReadGuard for scoped read-lock acquisition. Remaining game/*.rs module files stubbed to keep the module tree buildable — each populated by its own task.
Layout (0x30) + HashMapEntry (0x18) + SENTINEL marker (0xDEADBEEF). Lookup hashes the formID via CRC-32 (BSCRC32<u32>), masks to the power-of-2 capacity, walks the chain. crc32fast dep added to skse-rs. 4 unit tests including a synthetic-map happy path. game/form.rs also gets an opaque TESForm forward-declaration so hash_map can reference it; Task 9 replaces with the full layout.
TESForm layout (0x20) with form_id at 0x14 asserted. lookup_by_id resolves the allForms double-pointer (AE 400507), acquires the read lock (AE 400517 via ReadGuard), deref once to get the FormHashMap, call its lookup. Returns Option<*mut TESForm>; None for absent, RelocationError for infrastructure faults.
MemoryManager bindings (GetSingleton AE 11141, Allocate AE 68115, Deallocate AE 68117). BGSKeyword partial layout (0x28). BGSKeywordForm partial layout (0x18) + add_keyword function re-implementing CommonLibSSE-NG's inline AddKeyword: linear dedupe scan, allocate via MemoryManager (matches the game's own allocator), copy + append, swap with ordered writes, free old array.
AE Address Library ID 400269 — singleton slot (pointer-to-pointer). get_singleton() deref's once to return a *mut TESDataHandler. Inner layout opaque at M1 (smoke test uses TESForm::lookup_by_id, which goes through the global allForms map directly).
Wraps SKSE's MessagingInterface (kMessaging = 5): get_messaging queries the interface from SKSEInterface, register_listener plumbs a Rust callback through. is_data_loaded filter helper for callbacks that only care about one event type.
Default no-op; plugins override to interact with game state after all forms have loaded. The declare_plugin! macro now generates a messaging callback that dispatches kDataLoaded and wires up the listener inside SKSEPlugin_Load after on_load succeeds.
Replaces the stub "Hello from skse-rs" plugin with a complete smoke
test that exercises every skse-rs M1 API:
- on_load: opens the log, logs the SKSE runtime version, loads the
Address Library from the default SKSE/Plugins path.
- on_kDataLoaded: looks up Iron Sword (0x00012EB7) and the
WeapMaterialIron keyword (0x0001E718) via the global form map,
casts the weapon pointer + 0x140 to its BGSKeywordForm sub-object,
calls add_keyword, and logs the result + num_keywords readback.
BGSKeywordForm offset within TESObjectWEAP: 0x140
Verified by summing CommonLibSSE-NG static_assert sizes for each base
class in TESObjectWEAP's inheritance list (fetched from
CharmedBaryon/CommonLibSSE-NG master TESObjectWEAP.h and each base
class header):
TESBoundObject(0x30) + TESFullName(0x10) + TESModelTextureSwap(0x38)
+ TESIcon(0x10) + TESEnchantableForm(0x18) + TESValueForm(0x10)
+ TESWeightForm(0x10) + TESAttackDamageForm(0x10)
+ BGSDestructibleObjectForm(0x10) + BGSEquipType(0x10)
+ BGSPreloadable(0x08) + BGSMessageIcon(0x18)
+ BGSPickupPutdownSounds(0x18) + BGSBlockBashData(0x18) = 0x140
Cross-checked against sizeof(TESObjectWEAP) == 0x220.
Check.sh now asserts 9 required lines in log order, matching the upgraded plugin's output. README documents the full invariant.
Drops rust-ready.marker in skse-rs-smoke (first Rust-ready case). New integration-gate job (ubuntu-latest) checks for any tests/integration/**/rust-ready.marker and exports has_rust_ready. skyrim-integration's if: consumes that output — the self-hosted job now runs on PRs that touch a Rust-ready case.
Merged the nested `if let Some(stripped) = ... { if let Some(home) = ... }`
into a single let-chain (edition 2024), and fmt-canonicalized smoke
plugin whitespace. No functional changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Completes spec milestone M1 per
docs/superpowers/plans/2026-04-21-rust-kid-pivot-plan-3-skse-rs-game-interop.md:skse_rs::address_library) — pure Rust, binary-searchable, 5 fixtured unit tests + optional real-bin smoke test.skse_rs::relocation) —GetModuleHandleW(null)image base + Address Library offset.skse_rs::game::*) — TESForm (form_id only), BSReadWriteLock (read path), BSTHashMap<FormID, *mut TESForm> read-only lookup, TESDataHandler singleton, BGSKeyword, BGSKeywordForm, MemoryManager. Every struct is// M1-minimal— full surface grows as real consumers need it.SksePlugin::on_data_loaded+ declare_plugin!-generated messaging callback.0x140(sum of 14 base-class sizes from TESObjectWEAP.h), calls add_keyword, verifies readback, writes "smoke OK" to log.integration-gatejob checks forrust-ready.markerfiles;skyrim-integrationnow runs on self-hosted runners when that marker is present.Test plan
cargo test --workspace— 27 tests, 0 failures (Plan 2's 15 + Plan 3's 11 new + Cargo.lock didn't change test count).cargo clippy --all-targets -- -D warningsclean.cargo fmt --checkclean.cargo xwin check --target x86_64-pc-windows-msvc --workspaceclean.cargo xwin build -p skse-rs-smoke --releaseproducesSkseRsSmoke.dll(178KB) with all three SKSE exports.skyrim-integrationrunstests/integration/skse-rs-smoke/check.shagainst a live Skyrim and passes. Requires the Unraid runner image refreshed perdocs/src/runner-image-refresh.md. If the image refresh hasn't been done yet, the self-hosted job will fail — that's scope for you to evaluate.Notable implementation details
lookup. Insert/delete paths aren't needed at M1.add_keyworduses Skyrim's MemoryManager, notHeapAlloc— required because the gamefrees the existing array via its own allocator when the form is destroyed. Mixing allocators would crash.Next up
M1 is complete (Plan 2 + Plan 3 merged). Plan 4 begins M2 —
mora-core+mora-esp(patch format, chance RNG, mmap ESP parser, plugins.txt). Written when this lands.