Wraith Engine - DB2Gen

The data-side tool. It builds the FileDataID → path lookup tables the engine resolves modern assets through - turning the community listfile into the *FilePath.db2 tables that Engine/src/Features/DB2/DB2.cpp reads at runtime.

Part of the Wraith Engine project - bringing Legion 7.3.5 rendering to the WotLK 3.3.5a (build 12340) client without replacing the client, the data, or the protocol. See the organization page for the vision and the two-phase roadmap.

Why this exists. Legion references assets by numeric FileDataID; the 3.3.5a client addresses everything by string path. The modern-model backport has to cross that gap at load time. DB2Gen precomputes the bridge - one compact WDC1 table per asset family, keyed by FileDataID, with the path stored inline. No patched data files: the engine just reads these tables alongside the client's own.

What it does

Reads the listfile once, sorts everything into per-theme tables, writes them, and reads the first records straight back to prove the round-trip.

• Theme bucketing. Each line's extension decides its table - blp lands in textures, the model family (m2 wmo skin anim phys bone) lands in models. Extensions nobody claims are skipped.
• One WDC1 table per theme. Records are sorted by FileDataID, identical paths are de-duplicated into a shared string block, and each record is the minimal {uint32 id; uint32 pathOffset} the engine decodes - nothing it can't read.
• Native path storage. Paths are stored WoW-style with \ separators by default; --sep slash keeps the listfile's forward slashes instead.
• Built-in verification. After each table is written, its first records are decoded back through the engine's read path and printed, so a malformed write fails loudly on the spot.

The tables

Table	Extensions	Holds
TextureFilePath.db2	blp	every texture path, keyed by FileDataID
ModelFilePath.db2	m2 wmo skin anim phys bone	every model-family path, keyed by FileDataID

Edit the THEMES table at the top of db2gen.lua to add a family, split one out, or restrict a set. Each theme is independent - one table in, one table out.

Architecture

Two modules, one job each - kept deliberately small (plain loops, the bit library, dicts).

File	Role
db2gen.lua	Entry point. Reads the listfile, buckets {fdid, path} pairs by theme, drives the writer, runs the verify pass.
wdc1.lua	The format. A minimal WDC1 writer/reader for {id, path} tables - the only code that touches bytes.

db2gen.lua            entry point  -  listfile → per-theme .db2
wdc1.lua              WDC1 writer/reader (the on-disk format)
out/                  generated tables (created on first run)

WDC1 layout

A single uncompressed section, inline id at field 0, one string field. Written in order:

WDC1 header             84 bytes
field_structure[2]      8  bytes   (4 each)
records[N]              8  bytes each   →   { uint32 id; uint32 pathOffset }
string block               offset 0 = "" , then deduped NUL-terminated paths
field_storage_info[2]   48 bytes   (24 each)

A record's path is resolved by reading the NUL-terminated string at string_block_start + pathOffset. That is exactly what the engine's DB2 reader does - the writer exists to feed that one decode path and nothing more.

Running

The target is a script, not a build - no compile step.

Requirements

• LuaJIT 2.1 (the bit library is used for the little-endian packing)
• A community listfile - FileDataID;path per line, ;-delimited, lowercase forward-slash paths

luajit db2gen.lua --listfile "D:/path/to/listfile.csv" --out out

Flag	Default	Meaning
--listfile PATH	the DEFAULT_LISTFILE baked into db2gen.lua	the FileDataID;path listfile to read
--out DIR	./out, next to the script	where the .db2 tables are written
--sep backslash|slash	backslash	path separator stored in the strings - backslash is native WoW

Example run:

read 200000 lines, kept 160284 in 0.6s
  TextureFilePath.db2: 89997 rows, string table 692636 bytes, file 1412752 bytes
      verify id=1 -> world\maps\file0001.blp
      verify id=2 -> creature\file0002.blp
  ModelFilePath.db2: 70287 rows, string table 1440650 bytes, file 2003086 bytes
      verify id=4 -> item\weapon\file0004.m2
      verify id=5 -> tileset\file0005.wmo

Drop the resulting out/*.db2 where the engine reads the client's data, and its DB2 reader resolves FileDataIDs against them at runtime.

The listfile is community-maintained metadata, not game data. Point DB2Gen only at a listfile and a client you are permitted to use, and keep an untouched backup of anything you generate against.

Status

• ✅ WDC1 writer - header, records, deduped string block, field storage info
• ✅ Theme bucketing from the listfile (texture + model families)
• ✅ Stable sort by FileDataID - output is byte-identical run to run
• ✅ Round-trip verify through the engine's decode path
• 🚧 More themes (sound, lighting, client DB tables) as the backport reaches them
• 🚧 FFI record buffer for faster writes on the full multi-million-line listfile

Legal

DB2Gen is an interoperability tool. It ships no Blizzard code and no game assets - it reads a community-maintained listfile (asset metadata: numeric IDs and path strings) and emits lookup tables. It runs only against a client and data you supply and own.

World of Warcraft and Wrath of the Lich King are trademarks of Blizzard Entertainment. This project is not affiliated with or endorsed by Blizzard.

License

Released under the GNU General Public License v3.0, matching the rest of the Wraith Engine project - see LICENSE.