PyKotor: IsADirectoryError and KeyError

[Paisseon]

Not KOTORModSync related, but PyKotor has no issues button. Feel free to close this if you don't want to deal with issues for that!

I tried running PyKotorCLI on a heavily modified version of Snigaroo's list and while 126 mods appear to succeed (not tested in-game), 4 give errors which exit the patcher early. One mod tries to open the override directory as a file(?) and the others have three different KeyError issues.

I used /opt/homebrew/bin/python3 /Users/mobile/Downloads/PyKotor/pykotorcli.py /Users/mobile/Downloads/Payload/KOTOR.app/ /Users/mobile/Downloads/K1_Mods/$MOD_PATH to invoke pykotorcli.

The logs, as printed in my terminal, are below.

Details

[*] Installering af Escape from Taris [*] Kører PyKotorCLI... Using changes.ini path: /Users/mobile/Downloads/K1_Mods/TAR_Escape/tslpatchdata/changes.ini Parsing file list from [InstallList] Parsing stringrefs from [TLKList] Parsing 2da from [2DAList] Parsing SSF from [SSFList] Parsing GFF from [GFFList] Parsing NSS from [NSSList] Load modules... Load override... Load lips... Load textures... Load chitin... Load streammusic... Load streamsounds... Load streamvoices... Load rims... [Warning] A file named 'M12ab.mod' already exists in the 'Modules' folder. Skipping file... [Warning] A file named 'STUNT_06.mod' already exists in the 'Modules' folder. Skipping file... [Note] Copying file 'STUNT_07.mod' to the 'Modules' folder... [Note] Replacing file 'm12aa_01p.mdx' in the 'Override' folder... [Note] Replacing file 'm12aa_01p.mdl' in the 'Override' folder... [Note] Copying file 'DP_TarisDanTran.ncs' to the 'Override' folder... [Note] Copying file 'M12aa_DPMissSit.mdx' to the 'Override' folder... [Note] Copying file 'M12aa_DPMissSit.mdl' to the 'Override' folder... [Note] Copying file 'M12aa_DP_T3M4Std.mdl' to the 'Override' folder... [Note] Copying file 'M12aa_DP_T3M4Std.mdx' to the 'Override' folder... [Note] Copying file 'M12aa_DPCandSit.mdl' to the 'Override' folder... [Note] Copying file 'M12aa_DP_ZaalStd.mdx' to the 'Override' folder... [Note] Copying file 'm12aa_c07_char01.mdl' to the 'Override' folder... [Note] Copying file 'M12aa_DPCandSit.mdx' to the 'Override' folder... [Note] Copying file 'M12aa_DP_ZaalStd.mdl' to the 'Override' folder... [Note] Copying file 'm12aa_c07_char01.mdx' to the 'Override' folder... [Note] Replacing file 'k_scene_start.ncs' in the 'STUNT_07.mod' archive... [Note] Replacing file 'dp40_bdgdoor.utd' in the 'STUNT_06.mod' archive... [Note] Replacing file 'k_pebo_mgheart.ncs' in the 'M12ab.mod' archive... [Note] Replacing file 'k_pebo_sthdeath2.ncs' in the 'M12ab.mod' archive... [Note] Replacing file 'k_pebo_sthdeath3.ncs' in the 'M12ab.mod' archive... [Note] Replacing file 'k_pebo_sthdeath4.ncs' in the 'M12ab.mod' archive... [Note] Replacing file 'k_pebo_sthdeath5.ncs' in the 'M12ab.mod' archive... [Note] Replacing file 'k_pebo_sthdeath6.ncs' in the 'M12ab.mod' archive... [Note] Replacing file 'k_pebo_sthdeath7.ncs' in the 'M12ab.mod' archive... [Note] Patching 'stunt_levbridge.git' in the 'Modules/STUNT_06.mod' archive. [Note] Patching 'module.ifo' in the 'Modules/STUNT_06.mod' archive. [Note] Patching 'scene_start.dlg' in the 'Modules/STUNT_07.mod' archive. [Note] Patching 'ebo_bast_vision.dlg' in the '.' folder. Traceback (most recent call last): File "/Users/mobile/Downloads/PyKotor/pykotorcli.py", line 150, in main() File "/Users/mobile/Downloads/PyKotor/pykotorcli.py", line 95, in main installer.install() File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/config.py", line 266, in install self.write( File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/config.py", line 339, in write BinaryWriter.dump(destination, data) File "/Users/mobile/Downloads/PyKotor/pykotor/common/stream.py", line 738, in dump with path.open("wb") as file: ^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/pathlib.py", line 1044, in open return io.open(self, mode, buffering, encoding, errors, newline) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ IsADirectoryError: [Errno 21] Is a directory: '/Users/mobile/Downloads/Payload/KOTOR.app/Override'

[] Installering af Diverse Weapons
[] Kører PyKotorCLI...
Using changes.ini path: /Users/mobile/Downloads/K1_Mods/GBL_DivWeapon/tslpatchdata/changes.ini
Parsing file list from [InstallList]
Parsing stringrefs from [TLKList]
Parsing 2da from [2DAList]
Parsing SSF from [SSFList]
Parsing GFF from [GFFList]
Parsing NSS from [NSSList]
Load modules...
Load override...
Load lips...
Load textures...
Load chitin...
Load streammusic...
Load streamsounds...
Load streamvoices...
Load rims...
[Warning] A file named 'end_m01ab.mod' already exists in the 'Modules' folder. Skipping file...
[Note] Patching 'n_repsold002.utc' in the 'modules/end_m01aa.mod' archive.
[Note] Patching 'n_repsold004.utc' in the 'modules/end_m01aa.mod' archive.
[Note] Patching 'end_repsol004.utc' in the 'modules/end_m01aa.mod' archive.
[Note] Patching 'end_repsol005.utc' in the 'modules/end_m01aa.mod' archive.
[Note] Patching 'end_repsol006.utc' in the 'modules/end_m01aa.mod' archive.
[Note] Patching 'end_repsol007.utc' in the 'modules/end_m01aa.mod' archive.
[Note] Patching 'end_repsol008.utc' in the 'modules/end_m01aa.mod' archive.
[Note] Patching 'end_repsol009.utc' in the 'modules/end_m01aa.mod' archive.
[Note] Patching 'end_repsol010.utc' in the 'modules/end_m01aa.mod' archive.
[Note] Patching 'end_reinforce2.utc' in the 'modules/end_m01aa.mod' archive.
[Note] Patching 'end_reinforce3.utc' in the 'modules/end_m01aa.mod' archive.
[Note] Patching 'end_reinforce.utc' in the 'modules/end_m01aa.mod' archive.
[Note] Patching 'end_reinforce1.utc' in the 'modules/end_m01aa.mod' archive.
[Note] Patching 'n_sithsoldier004.utc' in the 'modules/end_m01aa.mod' archive.
[Note] Patching 'g_sithtroop003.utc' in the 'modules/end_m01ab.mod' archive.
[Note] Patching 'end_repsol003.utc' in the 'modules/end_m01ab.mod' archive.
[Note] Patching 'g_sithtroop002.utc' in the 'modules/end_m01ab.mod' archive.
[Note] Patching 'g_sithtroop004.utc' in the 'modules/end_m01ab.mod' archive.
[Note] Patching 'tar02_bountyh022.utc' in the 'modules/tar_m02ac.mod' archive.
[Note] Patching 'tar02_bountyh023.utc' in the 'modules/tar_m02ac.mod' archive.
[Note] Patching 'tar02_scaredm021.utc' in the 'modules/tar_m02ad.mod' archive.
[Note] Patching 'tar03_blackv039.utc' in the 'modules/tar_m03ab.mod' archive.
Traceback (most recent call last):
File "/Users/mobile/Downloads/PyKotor/pykotorcli.py", line 150, in
main()
File "/Users/mobile/Downloads/PyKotor/pykotorcli.py", line 95, in main
installer.install()
File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/config.py", line 265, in install
gff_patch.apply(template, memory, self.log)
File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/mods/gff.py", line 365, in apply
change_field.apply(gff.root, memory, logger)
File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/mods/gff.py", line 304, in apply
field_type = parent_gff_struct._fields[label].field_type()
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
KeyError: 'EquippedRes'

[] Installering af Diverse NPCs
[] Kører PyKotorCLI...
Using changes.ini path: /Users/mobile/Downloads/K1_Mods/GBL_DivNPC/tslpatchdata/changes.ini
Parsing file list from [InstallList]
Parsing stringrefs from [TLKList]
Parsing 2da from [2DAList]
Parsing SSF from [SSFList]
Parsing GFF from [GFFList]
Traceback (most recent call last):
File "/Users/mobile/Downloads/PyKotor/pykotorcli.py", line 150, in
main()
File "/Users/mobile/Downloads/PyKotor/pykotorcli.py", line 95, in main
installer.install()
File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/config.py", line 148, in install
config = self.config()
^^^^^^^^^^^^^
File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/config.py", line 142, in config
self._config.load(ini_text, self.mod_path)
File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/config.py", line 87, in load
ConfigReader(ini, CaseAwarePath(mod_path)).load(self)
File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/reader.py", line 105, in load
self.load_gff()
File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/reader.py", line 373, in load_gff
modifications_ini = dict(self.ini[file].items())
~~~~~~~~^^^^^^
File "/opt/homebrew/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/configparser.py", line 979, in getitem
raise KeyError(key)
KeyError: 'tar03_blackv037.utc'

[] Installering af Environment Maps
[] Kører PyKotorCLI...
Using changes.ini path: /Users/mobile/Downloads/K1_Mods/GBL_EnvMaps/tslpatchdata/changes.ini
Parsing file list from [InstallList]
Parsing stringrefs from [TLKList]
Parsing 2da from [2DAList]
Traceback (most recent call last):
File "/Users/mobile/Downloads/PyKotor/pykotorcli.py", line 150, in
main()
File "/Users/mobile/Downloads/PyKotor/pykotorcli.py", line 95, in main
installer.install()
File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/config.py", line 148, in install
config = self.config()
^^^^^^^^^^^^^
File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/config.py", line 142, in config
self._config.load(ini_text, self.mod_path)
File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/config.py", line 87, in load
ConfigReader(ini, CaseAwarePath(mod_path)).load(self)
File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/reader.py", line 101, in load
self.load_2da()
File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/reader.py", line 304, in load_2da
dict(self.ini[modification_id].items()),
~~~~~~~~^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/configparser.py", line 979, in getitem
raise KeyError(key)
KeyError: 'TMA27'

Thanks for the tool by the way! Much better than having to open TSLPatcher in Wine every time

[oldrepublicwizard]

Try the latest version from my master branch to see if it fixes your issue. The version you built awhile back on your last issue thread had some case-sensitive resource lookup issues.

[Paisseon]

Sorry, should have clarified-- this was using the Python script from commit 28e21b8 ("cleanup") rather than the compiled binary.

[oldrepublicwizard]

Strange - you said you had problems with 28e21b8? That should have been my most stable commit.

Looking closer at your logs...

File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/reader.py", line 101, in load
self.load_2da()
File "/Users/mobile/Downloads/PyKotor/pykotor/tslpatcher/reader.py", line 304, in load_2da
dict(self.ini[modification_id].items()),
~~~~~~~~^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/configparser.py", line 979, in getitem
raise KeyError(key)
KeyError: 'TMA27'

This implies the section [TMA27] doesn't exist in your ini, referenced by something in [2DAList]. If you can provide those sections of the INI I can give you more meaningful help on that front.

I just pushed a massive PR that completely rewrote the way PyKotor does case-insensitive pathing on unix systems. Can you try commit fix NamespaceReader and pykotorcli.py id dba61d (or later), to see if it solves your problem?

Thanks for your patience, I've been busy with school. I definitely want to stay up to date here though as I appreciate these bug reports.

[oldrepublicwizard]

"/Users/mobile/Downloads/PyKotor/pykotor/common/stream.py", line 738, in dump with path.open("wb") as file: ^^^^^^^^^^^^^^^ File "/opt/homebrew/Cellar/python@3.11/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/pathlib.py", line 1044, in open return io.open(self, mode, buffering, encoding, errors, newline) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ IsADirectoryError: [Errno 21] Is a directory: '/Users/mobile/Downloads/Payload/KOTOR.app/Override'

This implies the Override folder doesn't exist somehow. I did make the override folder optional at one point (pykotor used to hard error when it wasn't found). Whether or not I remembered to create Override everywhere that referenced it is unknown.

One mod tries to open the override directory as a file(?)

No, Python for some reason always raises a IsADirectoryError instead of FileNotFoundError on Unix systems.

Can you run the command find /path/to/kotor/directory -type d | sed -e 's;[^/]*/;|____;g;s;____|; |;g' and send the output? I'm not sure what the case-sensitive folder names are on mac.

If you can build a mac binary from either create-dirs-before-writing or dba61d, I'll add it to the project. I've released a mac binary with the project in v0.10.23

I created a new branch to fix that problem here:
https://github.com/th3w1zard1/PyKotor/tree/create-dirs-before-writing

If you find any new/different pykotor problems I've created an issues page over at https://github.com/th3w1zard1/PyKotor/issues
feel free to continue to use this thread

[oldrepublicwizard]

KOTORModSync v0.10.31 should fix this issue

EDIT: Yep I ran into the KeyError, it's due to tslpatcher using case-insensitive section names in namespaces.ini while PyKotor doesn't. This has been fixed in v0.10.31 see the changelogs

EDIT2: It seems you have another KeyError I missed on my first read-through unrelated to namespaces.ini. Please post more details

[oldrepublicwizard]

I've made all key/section lookups in INI's case-insensitive, whatever mod you were using should no longer have that KeyError.