Port unique updater scripts from PoB2#9480
Port unique updater scripts from PoB2#9480EtherealCarnivore wants to merge 33 commits intoPathOfBuildingCommunity:devfrom
Conversation
Port the unique item updater scripts from PathOfBuilding-PoE2 (PRs PathOfBuildingCommunity#54, PathOfBuildingCommunity#65, PathOfBuildingCommunity#175) and fix multiple bugs for PoE1 compatibility. Scripts added: - uTextToMods.lua: converts unique item text to mod IDs (run when adding new uniques) - uModsToText.lua: converts mod IDs back to text with fresh GGPK data (run when game data updates) Changes to existing files: - mods.lua: add ModItemExclusive.lua and ModTextMap.lua generation - statdesc.lua: handle '!' negation limits and ranges starting at 0 Bug fixes over the original PoE2 scripts: - Fix uModsToText.lua not flushing mods for last item in each file - Fix mod ID regex misidentifying base type names containing hyphens (e.g. "Two-Point Arrow Quiver") as legacy mod ranges - Fix unresolved text lines losing position among ordered mods - Fix nil access on statOrder when processing legacy-only mods - Fix uTextToMods.lua greedy tag stripping pattern and mod selection - Make usedMods local and move modTextMap load outside loop - Enable all 20 PoE1 item types (original only had axe enabled) Bug fix in Item.lua: - Fix excluded/exclude variable name mismatch creating accidental global - Fix wrong Lua pattern ^%[a ]+ to correct ^[%a ]+ in jewel radius - Fix space indentation to tabs in jewel radius block
- ModItemExclusive.lua: unique/exclusive mod data generated by mods.lua - Export/Uniques/ModTextMap.lua: mod text to mod ID mapping - Export/Uniques/*.lua: intermediate mod ID format for all item types
Re-export all unique item text files using the updater scripts with current game data. Updates stat ordering, mod wording, and adds catalyst tags to amulets and rings.
|
I'm going to push some changes to this branch to start fixing a lot of these uniques:
Even with these changes, I'm just going to go through each item type one at a time so they can be inspected thoroughly without one-shotting the whole thing. There are a lot of manual fixes that are needed, which is why the uTextToMods script was left out of the PoE2 one. At the end I'll probably disable it again once we have the mod files in a good state.
Known issues:
|
This script now handles sorting mods on preference to remove as much manual mod entry as possible. There are still many caveats with using it, so item types have been commented out for now with the intent to tackle each type one at a time until we are confident in its data.
|
One thing that is near-impossible to automate is when GGG did the base item implicit upgrade. The mods are full of implicits like this: Implicits: 2
{variant:1}ElementalDamagePercentImplicitSceptreNew16[10,10]
{variant:2,3}ElementalDamagePercentImplicitSceptreNew16When you search on the trade site and sort by the implicit, you'll see that the 10% implicits are really 20%, and are likely |
|
I did some work here: https://github.com/PJacek/PathOfBuilding/tree/port-unique-updater |
|
Added jewels on top of body armour, boots, and gloves. Rest of the item types (helmets, shields, quivers, belts, rings, amulets) still to go - skipping flasks and tinctures since someone else is on those. |
|
Re-exported body, boots, gloves, and jewel through the full pipeline on Windows:
Had to use Windows for this since Test suite: 194/195 passing (1 pre-existing TradeQueryCurrency failure, unrelated). |
This reverts commit 49ac1b8.
Pattern was missing dots so mods with values like 0.2 (leech rates etc) would fail to parse and crash describeStats with nil comparison.
Wires77
left a comment
There was a problem hiding this comment.
There are a bunch of duplicated lines in these latest commits, due to the multiline bullet in my other comment. The output should look nearly identical to the existing item unless there are some mod changes we didn't catch in earlier leagues.
We'll also want to commit the Export/Uniques file for future automation
| {variant:3}20% less Effect of your Curses | ||
| Socketed Hex Curse Skills are Triggered by Doedre's Effigy when Summoned | ||
| Hexes from Socketed Skills can apply 5 additional Curses | ||
| {variant:2}20% less Effect of Curses from Socketed Hex Skills |
There was a problem hiding this comment.
Check your output for duplicated lines. These already exist above, because the one mod is three lines in one.
When a mod ID resolves to N lines of text, the export file also has N-1 plain text continuation lines after it. Without skipping those, they'd get output twice (once from the mod resolution, once as unresolved text).
…realCarnivore/PathOfBuilding into port-unique-updater-scripts
|
One last thing I've noticed on a handful of mods is that GGG has set their value to 0, like this section for Lightpoacher explains on the wiki. This affects us by making the mod not appear at all in |
Rewrote io.linesBackward to read the whole file into memory instead of the old chunk-based approach - the 4KB-chunk reader broke on Windows because text-mode \r\n conversion threw off the seek math, producing garbled lines at chunk boundaries. After resolving a multi-line mod to its ID, strip stale continuation lines from the output. Also handles sibling mods that share the same first line (Dream/Nightmare jewels have 4 mods starting with the same text but different continuations). Reverted the skipLines workaround in uModsToText - it was eating legitimate variant override lines (e.g. GrantShaperSkill_1 variants). Fixed a few ModItemExclusive wording mismatches (Violent Dead recovery rate/speed, unarmed attack/attacks pluralization, DNT tag).
|
Fix multi-line mod handling in export round-trip |
Summary
Ports the unique item updater scripts from PathOfBuilding-PoE2 (PRs #54, #65, #175), building on the work started in #8700 by @Wires77. Extends coverage to all 20 PoE1 item types and fixes multiple bugs found during testing.
New scripts
Changes to existing files
ModItemExclusive.luaandModTextMap.luageneration!negation limits and ranges starting/ending at 0Bug fixes over original PoE2 scripts
uModsToText.lua: last item in each file wasn't getting mods flushed (no separator at EOF)uModsToText.lua: mod ID regex misidentified base type names containing hyphens (e.g. "Two-Point Arrow Quiver") as legacy mod rangesuModsToText.lua: unresolved text lines lost their position among stat-ordered modsuModsToText.lua: nil access onstatOrderwhen processing legacy-only modsuTextToMods.lua: greedy{.+}tag stripping consumed multiple tags; fixed to lazy{.-}uTextToMods.lua: mod selection logic could skip better matches; separated into two-pass approachuTextToMods.lua:usedModswas global;modTextMapreloaded inside loop unnecessarilyBug fixes in Item.lua (pre-existing)
excluded/excludevariable name mismatch creating an accidental global (lines 103-143)^%[a ]+corrected to^[%a ]+for jewel radius parsingData update
Addresses review feedback from @Nightblade on #8700. Supersedes #8700.
Test plan