feat(m6): stat_id → Modifier second channel (extraction + generator)#6
Merged
Conversation
Synthesizes 3 parallel read-only research passes (stat_descriptions extraction / passive-tree stat_id re-export / dual-run harness) into the M6 E/F design. Key findings: - Infrastructure already exists: the skill domain has a working stat_id→Modifier channel (skill_stat_map.json + stat_map_engine::map_stat) and the parser dual-run five-state harness. E/F = generalize these to passive_tree/mods. - 🔴 Hard prerequisite: passive_tree carries only rendered text (value embedded in the string), no stat_id+value. Source data.json (GGG community tree export) and PoB tree are both text-only. Recovering stat_ids needs re-exporting GGG PassiveSkills.dat (NOT currently downloaded) — Route A (preferred, clean) vs Route C (reverse text→stat_id matching, ambiguous for compound/conditional). - stat_id→Modifier table bootstraps from vendor StatDescriptions/*.lua in two stages (luajit render → engine parse); ~94% single-stat (easy), ~6% compound. - diff convergence requires the SAME normalization layer (vendor_name_aliases + the 5 classes) the parser dual-run used, or DIFF never reaches <0.1%. Doc lays out the wave plan (wave 0 prerequisite → normal passives → compound → keep-text-channel tail) and the open decisions (Route A feasibility + the data version mismatch config 4.5.2.1.3 vs ingested 4.5.0.3.4).
M6 E/F step A: the stat_id → Modifier second channel needs each stat_id's canonical English text so parse_mod_engine can re-derive the Modifier. Add a new `extract-lua --what stat-descriptions` target that renders it from vendor PoB2 Data/StatDescriptions/*.lua. - extract_stat_descriptions.lua: luajit bootstrap that loads the (pure-data) descriptor tables in a minimal env and renders one canonical line per stat_id by feeding representative value V=1. Faithfully ports matchLimit + the gsub placeholder chain from vendor Modules/StatDescriber.lua; skips applySpecial (value math doesn't change the mod's words/ModName). Multi-stat (compound) descriptors are kept verbatim, not force-parsed. - extract_stat_descriptions.rs: Rust assembler — per-scope sections, line-order merge via BTreeMap (byte-stable), dup/conflict errors, _meta header. - pobr-data stat_descriptions schema (stat_descriptions/v1): scopes → single / compound / unrendered. Per-scope, precedence deferred to the §B consumer (matches the stat_map 'extract faithfully, decide semantics in engine' rule). - Wire --what stat-descriptions into main.rs + regen-all.sh; default scopes = root + passive + presence/aura (the M6 tree channel). Produces overlay/stat_descriptions.json (zero-consumption asset; §B generator parses these texts into the stat_id_map next). 10398/203/22 single stats across 3 scopes render correctly (e.g. additional_strength → '+1 to Strength'). 5 assembler unit tests pass.
Exploratory (#[ignore], not a CI gate): runs parse_mod_engine over every rendered single-stat text in overlay/stat_descriptions.json and reports the parse rate per scope + the ModName histogram + unsupported samples. Current reading: 40.7% over the full table (4321/10623). The denominator is diluted — the root table holds ~10k stat_ids including monster stats, DNT-UNUSED dead entries, and unique keystone mechanics that are not generic mods. The texts that DO parse are exactly the generic mods the channel wants (top ModNames: Damage, CriticalStrikeChance, AttackSpeed, AreaOfEffect, MaximumLife, resistances). The meaningful rate needs the subset of stat_ids actually used by tree nodes / items (owner-gated PassiveSkills.dat re-export). Run: cargo test -p sync-pob-catalog --test stat_desc_parse_rate -- --ignored --nocapture
M6 E/F step B: derive overlay/stat_id_map.json by feeding each rendered
stat_descriptions text through parse_mod_engine and fixing the result as a
modifier template. This is the stat_id → Modifier second channel's data —
at runtime, given (stat_id, raw_value) from game data, look up the pre-parsed
template and inject value = raw_value × coefficient, skipping the text round
trip (text rendering needs luajit, build-time only).
- pobr-data stat_id_map schema (stat_id_map/v1): scopes → mapped / unsupported.
Template splits structure (name/mod_type/tags/flags) from coefficient (the
V=1 parsed value, 1.0 for linear generic mods) — structure for dual-run
field comparison, coefficient for runtime scaling. Tags use the single-source
canonical_tags serialization (no second serialization, per the design rule).
- gen_stat_id_map.rs: new `gen-stat-id-map --overlay-dir <dir>` subcommand.
Consumes stat_descriptions.json + mod_parser_rules.json (no luajit), runs the
engine offline. A stat_id maps only if ALL its text lines parse; otherwise the
whole stat_id goes to unsupported (conservative — the channel falls back to
text/special rather than mis-parse).
- Expose pobr_core::mod_parser::canonical_tags (was private).
- sync-pob-catalog now depends on pobr-core (build-time engine use only, not the
runtime calc path). Wire into main.rs + regen-all.sh (runs after its two
source overlays).
Produces stat_id_map.json: 4321 stat_ids mapped, e.g. fire_damage_+% →
{FireDamage, Inc, coef 1.0, tags [DamageType(Fire)]}; additional_strength →
{Strength, Base, coef 1.0}. Unsupported (special/unique/dead stats) quarantined.
2 generator unit tests pass; 40.7% raw map rate matches the §B measurement.
Formatting-only follow-up to c020a6d (the linter reformatted the println! arg layout after the commit landed). Keeps the branch HEAD fmt --check clean.
…line regenerated_matches_committed_artifact (byte-equality of regenerated vs committed mod_parser_rules.json) fails because vendor advanced to a82a33b while the committed artifact is still pinned to 2df5a74. A full regen to a82a33b is correct in isolation — it picks up real vendor additions (IMMUNE form, maimed flag, the 'global evasion rating and energy shield' name_map, AilmentMagnitude expansion) and all-resistances expands natively (no more hand-edit). But it shifts monk-martial-artist evasion +20%: a82a33b's new global-evasion name_map fixes PoB's old miss of '20% more Global Evasion Rating and Energy Shield', which then conflicts with the parity golden exported from an older PoB2 (canary_evasion_melee / deflection_matches_golden / ninja parity_no_regression all regress against the now-stale golden). So syncing the parser rules requires a coordinated parity-golden re-baseline (owner two-metric ruling + config-aligned oracle re-export), not a parser-only bump. Mark the drift check #[ignore] with the full rationale until that coordinated vendor upgrade happens; everything else stays on 2df5a74.
4 tasks
ackness
added a commit
that referenced
this pull request
Jul 17, 2026
- offensive 58->71 @5% (64->73 @10%), dot 16->25 @5% (21->27 @10%), panel offensive 40->41 / 41->42; defensive unchanged (413/434, core-8 140). Reason chain recorded next to each constant (bifurcate conditional + weapon hand-condition, enemyDistance placeholder -> skillDist, PerStat statList, Atalui hybrid life cost, per-leg mitigation blend). - regenerate parsed_mods.json + parse-coverage.json (statList tree lines flip unsupported -> parsed; coverage 0.8037 -> 0.8042). - document #6 in docs/adapting-to-0.5.4b.md with per-build decomposition and the triaged remainders (deadeye pre-0.5.4b per-hit, Mageblood family on blood-mage/abyssal-lich, frost-bomb cooldown DPS, smith Infernal Cry warcry-uptime dot residual, Barrage Repeats DPS channel). - cargo fmt reflow on touched files.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
M6 E/F — the stat_id → Modifier second channel. Game data (tree nodes,
item implicits, gem stats) often gives a stat_id rather than English text.
This branch builds the data pipeline to turn a stat_id into a pre-parsed
Modifier, as a parallel injection path alongside the existing text channel.
Once the two channels converge per-domain, the engine can switch over.
Both non-owner-gated pieces are done here (design doc, extraction §A,
generator §B). Steps 4+ (tree
PassiveSkills.datre-export → dual-run harness →per-domain switch) are owner-gated and out of scope.
What's included
b61fc1ddesign doc —audits/.../blueprints/m6-stat-id-channel.md.ee015e3§A extraction —extract-lua --what stat-descriptionsrenderseach stat_id's canonical English text from vendor
StatDescriptions/*.lua(luajit faithfully ports
StatDescriber.lua's matchLimit + gsub chain; feedsV=1; compound descriptors kept verbatim). Produces
stat_descriptions.json.c020a6d+ebc6972feasibility probe —#[ignore]measurement: 40.7%of the full table parses (diluted by ~10k monster/dead/unique stat_ids; the
texts that do parse are exactly the generic mods we want).
09ab5ba§B generator —gen-stat-id-mapfeeds each rendered text throughparse_mod_engineand fixes the result as a modifier template (structure vscoefficient split; tags via single-source
canonical_tags; whole-stat_idquarantine on any unparsed line). Produces
stat_id_map.json(4321 mapped).64c4d7bvendor-drift handling — see below.Design notes
stat_map"extract faithfully, decide semantics in engine" rule).comparison) from coefficient (the V=1 value, for runtime scaling).
calc path reads them yet.
sync-pob-cataloggains a build-time-onlydependency on
pobr-core(to run the engine during generation).Vendor-drift handling (
64c4d7b)The
regenerated_matches_committed_artifactbyte-equality check is marked#[ignore].mod_parser_rules.jsonis pinned to vendor2df5a74while vendorhas advanced to
a82a33b. A full regen toa82a33bis correct in isolation,but its new
global evasion rating and energy shieldname_map fixes PoB's oldmiss of
20% more Global Evasion, shifting monk-martial-artist evasion +20% andconflicting with parity goldens exported from an older PoB2. Syncing the parser
rules therefore requires a coordinated parity-golden re-baseline (owner
two-metric ruling + config-aligned oracle re-export), not a parser-only bump.
Deferred until that coordinated upgrade; everything stays on
2df5a74.Test plan
cargo nextest run --workspace— 2091 passed, 5 skipped (the ignoreddrift check + 4 pre-existing env-gated)
mod_parser_rulesunchanged, both new overlayszero-consumption