Skip to content

pipeline_max_coverage

Jump to bottom
Jan Boon edited this page Jul 8, 2026 · 34 revisions

title: pipeline_max coverage vs the build_gamedata Max export surface description: Gap analysis of the headless .max exporters against every Max-side export build_gamedata needs, with sized work packages for follow-up sessions published: true date: 2026-07-08T00:00:00.000Z tags: editor: markdown dateCreated: 2026-07-08T00:00:00.000Z

Written 2026-07-08 (authoring session), as the pick-up map for the next few implementation sessions. Read pipeline_max_design FIRST — especially §12 (rules for the implementing session) — and the per-format sections referenced below. The goal state: build_gamedata's 1_export.py steps can run the headless pipeline_max_* tools instead of 3ds Max, for every asset type, with corpus-proven fidelity.

How the surface was enumerated

nel/tools/build_gamedata/processes/*/1_export.py + their maxscript/*.ms scripts. Every Max-side export goes through one of these NelExport* MAXScript entry points (implemented in nel/tools/3d/plugin_max/nel_export/nel_export_script.cpp → nel_mesh_lib/): NelExportShapeEx, NelExportAnimation (used by BOTH the anim and shape processes), NelExportSkeleton, NelExportSkeletonWeight, NelExportInstanceGroup (ig + ligo), NelExportCollision (rbank + ligo), NelExportPACSPrimitives, NelExportVegetable, NelExportLodCharacter, plus ExportRykolZone (zone) and LigoExportZone (ligo). The ps process is a plain file copy (no Max involvement) despite its log line. Everything else under processes/ (rbank build, ig_light, zone_light, banks, maps, sheets, …) is post-export C++ tooling that already runs headless.

Reference outputs live in ~/pipeline_export (continents/ecosystems/common trees, mirroring the workspace config under ~/ryzomcore_leveldesign/workspace); sources in ~/ryzomcore_graphics.

Coverage table

Entry point Process(es) Output Refs in ~/pipeline_export Headless tool Status
ExportRykolZone zone .zone 6306 (incl. ligo) pipeline_max_export_zone GREEN — corpus 1068 exact + 130 x87-tier + 3 budget (§10h)
LigoExportZone + ligo igs ligo .zone/.ig bricks (in above) export_zone --ligo / export_ig GREEN — byte-identical (§10g-bis)
NelExportSkeleton skel .skel 203 pipeline_max_export_skel GREEN — legacy 99.4% drot, era classes documented (§10, §10e)
NelExportSkeletonWeight swt .swt 1 (+ corpus) pipeline_max_export_swt GREEN (§10f)
NelExportInstanceGroup ig, ligo .ig 18497 pipeline_max_export_ig GREEN — 53/54 field-exact (§10g)
NelExportAnimation anim, shape .anim 9146 pipeline_max_export_anim MOSTLY — non-biped byte-identical; biped structural-exact, in-between IK bounded (median 0.069, §10r); gaps: texture-matrix material anims (waterfalls), biped exactness (§10s)
NelExportShapeEx shape .shape 11251 pipeline_max_export_shape PARTIAL — M1 plain-CMesh path only (704 float-eq + 989 lightmap-bucketed + 383 differ, §10i); skinned/MRM, MultiLod/coarse, water NOT attempted
NelExportCollision rbank, ligo .cmb 445 (16 genuine direct; the rest are a downstream non-Max copy, out of scope) pipeline_max_export_cmb MOSTLY — direct tier 10/16 exact + 3/16 x87-tier + 2/16 Edit-Mesh-created-geometry gap + 1/16 unresolved offset (§10v); ligo tier informational, XRef nodes not resolved
NelExportPACSPrimitives pacs_prim .pacs_prim 493 pipeline_max_export_pacs_prim GREEN — 493/493 byte-identical (§10v)
NelExportVegetable veget .veget 126 — MISSING
NelExportLodCharacter clodbank .clod 105 — MISSING

Cross-cutting infrastructure that is DONE and reusable: the storage/scene layer with full T1/T2 byte-identity over every corpus (max files parse→build byte-identical), the OLE backend (§2a), the VS2008/Wine x87 reference build (§2b), Edit Mesh / Editable Poly / Edit Patch geometry decode (ig/zone/shape sessions), materials + ParamBlock2 (§10j), the biped system incl. typed keytrack editing (§10t/§10u), AppData flag reading, and the corpus-test harness patterns (*_corpus.py, fork-per-file, tolerance tiers).

The gaps, sized

A. The remaining small formats (breadth completion)

.pacs_prim and .cmb landed 2026-07-08 (§10v) — .veget and .clod close the rest of the SURFACE.

  1. .cmb — NelExportCollision DONE, mostly (§10v) — pipeline_max_export_cmb. Open follow-ups: XRef collision-node resolution (--ligo mode), Edit Mesh created-face topology decode, one unresolved direct-tier vertex-offset file, triage the 22 ligo-tier diffs.
  2. .pacs_prim — NelExportPACSPrimitives DONE (§10v) — pipeline_max_export_pacs_prim, 493/493 byte-identical.
  3. .veget — NelExportVegetable (126 refs; 817 veget-related source .max). Serializer: NL3D::CVegetableShape (nel/3d/vegetable_shape.h) — mesh + bend-weight painting (vertex colors/UV1?) + appdata params (bend mode, freq…). Reference impl: nel_mesh_lib/export_vegetable.cpp (buildVegetableShape). Plain-mesh scale; the vertex-paint channel decode is the only likely new ground.
  4. .clod — NelExportLodCharacter (105 refs; 111 *_lod.max sources under stuff/lod_actors/). Serializer: NL3D::CLodCharacterShapeBuild (nel/3d/lod_character_shape.h) — positions, tris, UVs, normals + per-vertex skin weights to a bone-name table. Reference impl: nel_mesh_lib/export_lod_character.cpp. NOTE: this needs the SKIN modifier decode (bone weights) — the same decode the shape-MRM work needs; whichever session lands first should put the skin-weight reader in pipeline_max_export_common for the other to reuse. (The .swt exporter already reads skeleton-weight data — check what it shares before writing anything new.)

B. Shape depth (the flagship remaining work — Opus-sized)

NelExportShapeEx beyond plain CMesh, in rough dependency order:

  1. Skinned + MRM meshes (CMeshMRM) — characters, fauna; the bulk of the 11251 refs. Two halves: (a) the SKIN decode (Physique/Skin modifier chunks → per-vertex bone weights; corpus era is Physique — decode lives in nel_mesh_lib/export_skinning.cpp for reference) and (b) the MRM build (NL3D::CMRMBuilder) — deterministic NeL code; link the CURRENT NeL builder like anim/shape M1 already link NeL, then measure the 2004-vs-now divergence per file and bucket tolerance tiers exactly like §10h/§10i did (expect an era tier; the VS2008 build is the instrument if it matters).
  2. CMeshMultiLod / coarse meshes — the shape_with_coarse_mesh* reference dirs exist in ~/pipeline_export/common; the coarse-mesh texture step is part of the shape process.
  3. Water shapes (CWaterShape) — few files, self-contained (see the existing nel/3d/water_* docs and the waterfall/texture-matrix notes).
  4. Shape-process animations — shape_export.ms ALSO calls NelExportAnimation for animated materials: the waterfall texture-matrix gap (see waterfall-texmat-anim finding: bExportTextureMatrix + StdUVGen U/V Offset controllers; ref anims in ~/core4_data/*_shapes/waterfall*.anim). This is a small, well-scoped export_anim extension and can ride along with any session.

C. Anim depth (parked, documented)

The biped §10s experiment list (arm-pin space decode, in-plant foot rotation probe, turn-rule probe, blend-ramp weighting, PODA leads (6)–(8) in poda_1985_notes) — pick up only when biped exactness becomes the priority; the export is already structurally exact with bounded in-between error.

Suggested session plans

Session 1 (Sonnet-scale): .pacs_prim + .cmb. DONE 2026-07-08 (§10v) — pipeline_max_export_pacs_prim (493/493 byte-identical) and pipeline_max_export_cmb (16/16 direct files T1/T2, 13/16 T3 exact-or-tolerance, 3/16 documented open gaps; ligo tier informational). pacs_prim_corpus.py/cmb_corpus.py wired into ctest. Open follow-ups listed in §10v and item A.1 above.

Session 2 (Sonnet-scale): .veget, then .clod if the skin decode goes fast. .veget first (no skinning). For .clod, put the skin-weight decode in pipeline_max_export_common with its own mini-gate so the MRM session can build on it. If the Physique decode balloons, stop, document the chunk findings in the wiki, and leave .clod for the MRM session — do not half-land a skinning decode without a gate.

Session 3 (Opus-scale): skinned/MRM shapes. The big one. Approach exactly like §10i M1: replicate CExportNel::buildShape's MRM branch against the real NeL builders, corpus-bucket the outcomes (byte-exact / float-eq / era-tier / differ), and expect several probe rounds. Reuse the skin decode from session 2 if it landed. Success metric: the character/fauna shape corpus classified into explained tiers with zero unexplained structural fails — byte-identity is the stretch goal, not the bar (see the zone precedent).

Session 4 (Sonnet-scale, anytime): waterfall texture-matrix anims + shape-process anim wiring — small, high-polish-value, mostly export_anim plumbing.

Non-negotiables for every session (from §12, restated)

  • Load the session baseline first: all pipeline_max sources + the design doc (see the pipeline-max-session-baseline memory note / §12).
  • Corpus-gate EVERYTHING: T1/T2 byte-identity must stay green over the full .max corpus for any parser change; new exporters get their own T3 vs ~/pipeline_export references with explicit tolerance tiers; per-file forked runs (parse→clean→build→disown; COFile, not CMemStream).
  • The .ms scripts + workspace config (~/ryzomcore_leveldesign/workspace) are the ground truth for WHAT gets exported from each source file and with which options — read them before the C++.
  • Emission-form preservation, typed-chunk order preservation, and the §11 hazard list apply unchanged.
  • Don't rebuild binaries mid-sweep; don't trust per-file probes over full-corpus A/B (§10r method notes).

Clone this wiki locally