invest-autotest fails on main

[davemfish]

The last couple of checks on main have failed at the invest-autotest step:
https://github.com/natcap/invest/actions/runs/10725072571/job/29742132887

It looks like taskgraph was somehow missing from the pyinstaller binaries?

2024-09-05T17:40:40.2746964Z Traceback (most recent call last):
2024-09-05T17:40:40.3347590Z   File "exe\hooks\rthook.py", line 6, in <module>
2024-09-05T17:40:40.4471477Z     multiprocessing.freeze_support()
2024-09-05T17:40:40.5307847Z   File "multiprocessing\context.py", line 150, in freeze_support
2024-09-05T17:40:40.5401076Z   File "multiprocessing\spawn.py", line 79, in freeze_support
2024-09-05T17:40:40.5900983Z   File "multiprocessing\spawn.py", line 122, in spawn_main
2024-09-05T17:40:40.6053253Z   File "multiprocessing\spawn.py", line 132, in _main
2024-09-05T17:40:40.6417799Z   File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
2024-09-05T17:40:40.6793002Z   File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
2024-09-05T17:40:40.6820685Z   File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
2024-09-05T17:40:40.6822927Z   File "PyInstaller\loader\pyimod02_importers.py", line 378, in exec_module
2024-09-05T17:40:40.6824143Z   File "taskgraph\__init__.py", line 3, in <module>
2024-09-05T17:40:40.6914355Z   File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
2024-09-05T17:40:40.6918706Z   File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
2024-09-05T17:40:40.6947073Z   File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
2024-09-05T17:40:40.6964247Z   File "PyInstaller\loader\pyimod02_importers.py", line 378, in exec_module
2024-09-05T17:40:40.6965395Z   File "taskgraph\Task.py", line 21, in <module>
2024-09-05T17:40:40.6966380Z   File "pkg_resources\__init__.py", line 541, in get_distribution
2024-09-05T17:40:40.6967711Z   File "pkg_resources\__init__.py", line 424, in get_provider
2024-09-05T17:40:40.6968721Z   File "pkg_resources\__init__.py", line 1063, in require
2024-09-05T17:40:40.6969701Z   File "pkg_resources\__init__.py", line 890, in resolve
2024-09-05T17:40:40.6970717Z   File "pkg_resources\__init__.py", line 931, in _resolve_dist
2024-09-05T17:40:40.6972598Z pkg_resources.DistributionNotFound: The 'taskgraph' distribution was not found and is required by the application
2024-09-05T17:40:40.6974364Z [PYI-4676:ERROR] Failed to execute script 'rthook' due to unhandled exception!

[phargogh]

Looking back through the logs, I think there's an issue with the multiprocessing setup, maybe of one of the models? There are thousands and thousands of lines of this exception repeater, and this looks like the kind of thing where we aren't protecting something with an if __name__ == '__main__' block.

I also find it strange that this is happening in the pyinstaller runtime hook ... maybe we aren't supposed to call freeze_support from within the rthook?

[phargogh]

Looks like PyInstaller 6.10 did some work in the multiprocessing rthook, so I'm guessing that this might not be an issue on our end.

Clean up the multiprocessing run-time hook. Due to changes in detection of inherited PyInstaller environments, we do not need to restore (the now renamed) _MEIPASS2 environment variable anymore, and we can remove all our custom Popen overrides. (#8634)

[phargogh]

We partially addressed this in #1624 by capping the pyinstaller version to <6.10, but haven't yet actually figured out the root cause or a true solution. At least the builds are working for now, but I'll be looking into this again soon.