Fix release packaging, update LOOT API #408
Conversation
What changed was the way loot_api is initialised and the get_plugin_tags() and get_plugin_cleanliness() functions, which now need to be passed True as a second argument to retain their previous function. Also updated the install_loot_api and mktaglist scripts. They both work for me now out of the scripts folder. A bunch of file paths assumed the scripts to be in the root directory (not in scripts). No idea when they were last working. Under # 388. Catch AttributeError from older loot api: SSS Wrye Bash encountered an error. Please post the information below to the official thread at: https://afkmods.iguanadons.net/index.php?/topic/4966-wrye-bash-all-games/& or https://bethesda.net/community/topic/38798/relz-wrye-bash-oblivion-skyrim-skyrim-se-fallout-4/ Traceback (most recent call last): File "bash\bash.py", line 229, in main _main(opts) File "bash\bash.py", line 325, in _main bosh.initBosh(opts.personalPath, opts.localAppDataPath, bashIni) File "bash\bosh\__init__.py", line 3427, in initBosh initDirs(bashIni, personal, localAppData) File "bash\bosh\__init__.py", line 3239, in initDirs configHelpers = ConfigHelpers() File "bash\bosh\mods_metadata.py", line 230, in __init__ loot_api.initialise_locale('') AttributeError: 'module' object has no attribute 'initialise_locale' @@ -232,3 +232,3 @@ def __init__(self): lootDb = loot_game.get_database() - except OSError: + except (OSError, AttributeError): deprint(u'The LOOT API failed to initialize', traceback=True) Fix for refreshBashTags errors after loot api update: The error itself should be benign. The `eval_lists()` command was removed in the newer loot api versions. The loot api integration update did make use of the new way of evaluating conditions (passing `True` as a second argument to `get_plugin_tags()` and `get_plugin_cleanliness()` but failed to removed the now unnecessary and failing `eval_lists()` calls. Apparently the error would only show for Skyrim/SSE, not for FO4 or Oblivion. Under # 388.
C:\_\Python27\python.exe C:/Dropbox/eclipse_workspaces/python/wrye-bash/scripts/mktaglist.py Loaded the LOOT API v0.12.0 using wrapper version 3.0.0 Oblivion masterlist converted. Skyrim masterlist converted. Skyrim Special Edition masterlist converted. Fallout3 masterlist converted. FalloutNV masterlist converted. Fallout4 masterlist converted. Taglist generator finished. Done
+ },
+ 'bash.mods.loadOrders.colWidths': {
+ 'Index':24,
+ # 'Date':135,
+ },
#--Tes4View/Edit/Trans
Mopy/bash/basher/constants.py:
'bash.mods.loadOrders.colReverse' and 'bash.mods.loadOrders.colWidths'
are not strictly needed but due to the way settings operate better be
there from the start. The rest _are_ needed.
FFF LoBrowser
Fleshing out details - no AbstractError anymore
Masterlist!
Master List fleshing out:
I needed proper names so _get_column_name
details
Ut: we needed to import bush, kind of a smell here
NEW! mod_links (copy paste) code dropped - log_config should work
alright
EEE drop identical attributes to _AlistsMerger
@@ -589,5 +589,5 @@ def getDefaultTags():
for fileName in (u'FormID Lists.csv',u'My FormId Lists.csv'):
- textPath = dirs[u'patches'].join(fileName)
+ textPath = getPatchesPath(fileName)
if textPath.exists():
- with bolt.CsvReader(textPath) as reader:
+ with CsvReader(textPath) as reader:
for fields in reader:
getDefaultTags needs merge with base
Due to refactoring of config
Mopy/bash/patcher/patchers/special.py
@@ -523,8 +523,4 @@ class FidListsMerger(_AListsMerger,ListPatcher):
tip = _(u"Merges changes to formid lists from all active mods.")
- choiceMenu = (u'Auto',u'----',u'Deflst') #--List of possible choices for each config item. Item 0 is default.
- autoKey = (u'Deflst',)
- forceAuto = False
+ autoKey = {u'Deflst'}
iiMode = True
- selectCommands = False
- defaultConfig = {'isEnabled':False,'autoIsChecked':True,'configItems':[],'configChecks':{},'configChoices':{}}
@@ -543,23 +539,2 @@ def getDefaultTags():
- #--Config Phase -----------------------------------------------------------
- def getChoice(self,item):
- """Get default config choice."""
- choice = self.configChoices.get(item)
- if not isinstance(choice,set): choice = {u'Auto'}
- if u'Auto' in choice:
- if item in bosh.modInfos:
- bashTags = bosh.modInfos[item].getBashTags()
- choice = {u'Auto'} | ({u'Deflst',} & bashTags)
- self.configChoices[item] = choice
- return choice
-
- def getItemLabel(self,item):
- """Returns label for item to be used in list"""
- choice = map(itemgetter(0),self.configChoices.get(item,tuple()))
- if isinstance(item,Path): item = item.s
- if choice:
- return u'%s [%s]' % (item,u''.join(sorted(choice)))
- else:
- return item
-
#--Patch Phase ------------------------------------------------------------
Also
@@ -542,3 +542,3 @@ def initPatchFile(self,patchFile,loadMods):
"""Prepare to handle specified patch mod. All functions are called after this."""
- Patcher.initPatchFile(self,patchFile,loadMods)
+ super(FidListsMerger, self).initPatchFile(patchFile, loadMods)
self.srcMods = set(self.getConfigChecked()) & set(loadMods)
`
Conflict resolution in Mopy/bash/basher/gui_patchers.py
@@@ -1015,6 -752,7 +1015,13 @@@ class CBash_ListsMerger(special.CBash_L
-class FidListsMerger(special.FidListsMerger, _ListPatcherPanel):
- listLabel = _("Override Deflst Tags")
++class FidListsMerger(special.FidListsMerger, _ListsMergerPanel):
++ listLabel = _(u"Override Deflst Tags")
++ forceItemCheck = False #--Force configChecked to True for all items
++ choiceMenu = (u'Auto', u'----', u'Deflst')
++ # CONFIG DEFAULTS
++ default_isEnabled = False
+
-class ContentsChecker(special.ContentsChecker, _PatcherPanel): pass
-class CBash_ContentsChecker(special.CBash_ContentsChecker, _PatcherPanel): pass
+class ContentsChecker(special.ContentsChecker, _PatcherPanel):
+ default_isEnabled = True
+class CBash_ContentsChecker(special.CBash_ContentsChecker, _PatcherPanel):
+ default_isEnabled = True
EEE use _scanModData ?
srcMods is taken care of by super (its is self.srcs)
needs further work to use Import patchers refactored common methods
Used unicode instead of string:
@@ -457,2 +457,3 @@ class DestructiblePatcher(ImportPatcher):
- name = _("Import Destructible")
- text = _("Merges changes to destructible records.\n\nWill have to use if Destruction Environment mod is installed and active.")
+ name = _(u"Import Destructible")
+ text = _(u"Merges changes to destructible records.\n\nWill have to use if "
+ u"Destruction Environment mod is installed and active.")
@@ -460 +461 @@ class DestructiblePatcher(ImportPatcher):
- autoKey = ('Destructible',)
+ autoKey = (u'Destructible',)
Due to config refactoring:
@@ -457,3 +457,3 @@ class DestructiblePatcher(ImportPatcher):
tip = text
- autoKey = (u'Destructible',)
+ autoKey = {u'Destructible'}
Ut: same old stuff - same as before refactoring of common code is needed
@@ -3436,7 +3436,13 @@ class WeaponModsPatcher(ImportPatcher):
editOrder = 27
- name = _("Import Weapon Modifications")
- text = _("Merges changes to weapon modifications.")
+ name = _(u"Import Weapon Modifications")
+ text = _(u"Merges changes to weapon modifications.")
tip = text
autoRe = re.compile(r"^UNDEFINED$",re.I)
- autoKey = 'WeaponMods'
+ autoKey = u'WeaponMods'
+ _record_attributes = ('modelWithMods', 'firstPersonModelWithMods',
+ 'weaponMods', 'soundMod1Shoot3Ds', 'soundMod1Shoot2D', 'effectMod1',
+ 'effectMod2', 'effectMod3', 'valueAMod1', 'valueAMod2', 'valueAMod3',
+ 'valueBMod1', 'valueBMod2', 'valueBMod3', 'reloadAnimationMod',
+ 'vatsModReqiured', 'scopeModel', 'dnamFlags1.hasScope',
+ 'dnamFlags2.scopeFromMod')
@@ -3445,7 +3451,5 @@ def initPatchFile(self,patchFile,loadMods):
"""Prepare to handle specified patch mod. All functions are called after this."""
- Patcher.initPatchFile(self,patchFile,loadMods)
+ super(WeaponModsPatcher, self).initPatchFile(patchFile, loadMods)
self.id_data = {} #--Names keyed by long fid.
self.srcClasses = set() #--Record classes actually provided by src mods/files.
- self.sourceMods = self.getConfigChecked()
- self.isActive = len(self.sourceMods) != 0
self.classestemp = set()
@@ -3454,8 +3458,5 @@ def initPatchFile(self,patchFile,loadMods):
recFidAttrs_class = self.recFidAttrs_class = {}
- self.longTypes = set(('WEAP',))
+ self.longTypes = {'WEAP'}
for recClass in (MreRecord.type_class[x] for x in self.longTypes):
- recAttrs_class[recClass] = ('modelWithMods','firstPersonModelWithMods','weaponMods','soundMod1Shoot3Ds','soundMod1Shoot2D',
- 'effectMod1','effectMod2','effectMod3','valueAMod1','valueAMod2','valueAMod3',
- 'valueBMod1','valueBMod2','valueBMod3','reloadAnimationMod','vatsModReqiured','scopeModel',
- 'dnamFlags1.hasScope','dnamFlags2.scopeFromMod')
+ recAttrs_class[recClass] = WeaponModsPatcher._record_attributes
@@ -3468,5 +3469,5 @@ def initData(self,progress):
longTypes = self.longTypes & set(x.classType for x in recAttrs_class)
- progress.setFull(len(self.sourceMods))
+ progress.setFull(len(self.srcs))
cachedMasters = {}
- for index,srcMod in enumerate(self.sourceMods):
+ for index,srcMod in enumerate(self.srcs):
temp_id_data = {}
@@ -3597,8 +3598,2 @@ def buildPatch(self,log,progress):
id_data = None
- log.setHeader('= '+self.__class__.name)
- log(_("=== Source Mods"))
- for mod in self.sourceMods:
- log("* " +mod.s)
- log(_("\n=== Modified Records"))
- for type,count in sorted(type_count.iteritems()):
- if count: log("* %s: %d" % (type,count))
+ self._patchLog(log, type_count)
Also
@@ -3390,3 +3390,3 @@ class WeaponModsPatcher(ImportPatcher):
autoRe = re.compile(r"^UNDEFINED$",re.I)
- autoKey = u'WeaponMods'
+ autoKey = {u'WeaponMods'}
_record_attributes = ('modelWithMods', 'firstPersonModelWithMods',
Updated 0.8.1 taglists
Records, init files, INI Tweaks, Save Game Updates Note Mopy/bash/bosh/bsa_files.py was FO3 ready Added Fallout Images, otherwise this crashes Added cosave info SSS - see: #236 (comment) Utumno: This is a squash of various fixup commits. It has been dragging along 305/6/7 development and had to be continually patched, plus coding style, wrapping and other changes. In particular save header code was rewritten to be moved in save_files.py, chopping off dozens of lines of duplicated, nasty logic. Note that in all occurrences of: raise Exception(u'%s: Unknown byte is not 0x1B.' % path) path was undefined. Also IIUC in game.falloutnv.ess#writeMasters `out.write(ins.read(65))` should be `out.write(ins.read(70))`. Latest patches to sync with dev: - Fix unpackRef change signature - struct_unpack,pack refactoring - work by @ GandaG to add GameInfo refactoring - # 358 - Record header changes: brec.RecordHeader.plugin_form_version = 15 reported by @ zilav #342 (comment) Refactor fo3 and fnv packages into classes Move record imports and mergeclasses to init Utumno: - init(cls) - allTags is now a set - wrapping - added back some comments - dropped bashed patches from wryeBashDataFiles - I now detect them via modInfos - order main variables at the top - add back archive invalidation
FFF fallout3/nv wrapping and imports
also:
--------------------- Mopy/bash/game/fallout3/constants.py ---------------------
index cd79697..fe4172a 100644
@@ -864,8 +864,8 @@
-#-------------------------------------------------------------------------------
+#------------------------------------------------------------------------------
# CellImporter
-#-------------------------------------------------------------------------------
-cellAutoKeys = (
- u'C.Acoustic',u'C.Climate',u'C.Encounter',u'C.ImageSpace',u'C.Light',
- u'C.Music',u'C.Name',u'C.Owner',u'C.RecordFlags',u'C.Water',)#,u'C.Maps')
+#------------------------------------------------------------------------------
+cellAutoKeys = {u'C.Acoustic', u'C.Climate', u'C.Encounter', u'C.ImageSpace',
+ u'C.Light', u'C.Music', u'C.Name', u'C.Owner',
+ u'C.RecordFlags', u'C.Water'} #,u'C.Maps'}
cellRecAttrs = {
-------------------- Mopy/bash/game/falloutnv/constants.py --------------------
index e425aa5..93c68c6 100644
@@ -1105,5 +1105,5 @@
#-------------------------------------------------------------------------------
-cellAutoKeys = (
- u'C.Acoustic',u'C.Climate',u'C.Encounter',u'C.ImageSpace',u'C.Light',
- u'C.Music',u'C.Name',u'C.Owner',u'C.RecordFlags',u'C.Water',)#,u'C.Maps')
+cellAutoKeys = {u'C.Acoustic', u'C.Climate', u'C.Encounter', u'C.ImageSpace',
+ u'C.Light', u'C.Music', u'C.Name', u'C.Owner',
+ u'C.RecordFlags', u'C.Water'} #,u'C.Maps')
cellRecAttrs = {
- getDefaultTags is unused - DestructiblePatcher is-a _SimpleImporter. Made clear once I got me rid of unused self.recFidAttrs_class which simplified the guts of initData. TODO: buildPatch
Unfortunately PyCharm bugs out - so this needs careful review
|
2.2 crashes - 2.1 fails with the Both display the same mktglist error: Note the wrapper version 2.0.0 still prints as 2.0.0 The Traceback line that 2.1 (and 2.0) crashes at is: tags = lootDb.get_plugin_tags(modName.s)in |
|
OK, here's a build of the latest code but with pybind11 rolled back to the same version as in v2.0.0 of the wrapper. |
|
Success !!! I had to revert the reverting of D4id4los changes - so it was the pybind version used? Here is a link to my standalone - please make sure it works correctly for you: The whole idea is to assign tags using the taglist.yml fallback if no loot install masterlists are found - time to check it does so |
|
Well, it launches for me, but I don't see tags that are in the Skyrim SE taglist displayed in the UI. I checked with the Python version and same deal, and the last line in the BashBugDump.log is Now that we've got a build that will actually load though, I'll try to see why the integration isn't working, and go through and tidy things up for a proper release. Though I've probably only got a few hours till I'm AFK for a week. On a related note, is it possible to launch the exe in debug mode like you can the Python version? I know you can enable debug logging once launched, but that doesn't catch any init issues. |
Utumno
force-pushed
the
150-fo3-fnv-support
branch
from
ac1ab55
to
1d55502
Compare
|
It looks like something is trying to get tags before the taglist has loaded. Here's an excerpt from my log after putting in a few (The printed tag sets are |
|
Thanks @WrinklyNinja yes some cleanup is needed - loot_api_download updated also. Info on pre wx booting should be in Wrye Bash.exe.log - see: https://github.com/wrye-bash/wrye-bash/wiki/%5Bgithub%5D-Reporting-a-bug#bash-log-files This is a bit grey land those log files but no time for that now Btw the dll/api sizes were much bigger - not sure if matters |
That's due to them shipping with debug info. |
|
Ooops - will have to track Be aware that probably the tags are filtered to only include supported ones (per game - track |
I guessed so - should we link to the debug binaries in release (loot_api script for instance ?) |
They're still release binaries, just with debug info. From what I've read the size increase is the only significant impact, performance should be equivalent, and being able to use the debug symbols may come in handy again. |
|
So in they go - " use the debug symbols" is quite another story - was not able to (we need the source too and somewhere where VS can find it) nor have I a clue how to add debug info to the Wrye Bash exe. But at some point we should revisit so we can really see the line that crashes in C++ |
|
Don't forget to add a note on dll/pyd version numbers :P (say in loot_api.py ?) - maybe bump them to the same (main version) number? |
This includes a rollback of pybind11 to fix runtime crashing in some environments.
|
OK, I missed the bit in the debug log where it said it was loading an external masterlist, and it turned out that masterlist didn't have the tags, so everything was actually working. I've released v3.1.1 of the wrapper and pushed a commit for Bash to install it, this includes the debug info for the wrapper but not the for LOOT API itself, because I haven't released a version of the latter including it, and I don't want to just yet because I'm in the middle of making sorting improvements. AFAIK this should now be good to go, can you test it out? |
|
Will do ASAP and pack a standalone for people to test (along with fixes for a couple bugs reported in the forums) - have a nice AFK week! |
Utumno
force-pushed
the
150-fo3-fnv-support
branch
from
1d55502
to
299ca08
Compare
Utumno
force-pushed
the
150-fo3-fnv-support
branch
from
299ca08
to
5067fe9
Compare
|
Hi @WrinklyNinja - so I keep: or do you have a newer one I need to try? I need to release before summer comes for good... this one is working fine on my part - I build and run |
Utumno
force-pushed
the
150-fo3-fnv-support
branch
from
5067fe9
to
da043f2
Compare
|
Hmm, I should probably release an update for the API v0.13.x, I'll get it done this weekend. Would you like a new PR, or should I update this one? |
|
Well actually the beta went out with that version (I was planning to delay this but it was published by alt3rn1ty - so much the better I'd say) I would like a new pull with the new API yes (will close this one from the commit on dev when I get round to merging). This will be merged in 307 proper - I expect this to be the last beta |
Infernio
added
the
C-enhancement
Infernio
added
the
M-relnotes
The setup.py generated when packaging for release isn't necessarily recognised as executable by the OS, which was causing me problems, so run it using the Python executable.
I've also lumped in an update to the latest release of the LOOT API module, with it WB is able to start for me. Relevant to #388.