Remove hard-coded dependencies for Python 3.5

[elgonzo]

While it is already possible to choose a particular embeddable/minimal Python3 version (such as the current version 3.6, for example) with the help of the --python-minimal command line parameter, the C code of the launcher unfortunately has a hard-coded dependency on python35.dll.

I am aware that i can patch the respective string resource(s) in the executable. However, i think that this should not be necessary.

In my opinion, for the 3.x launcher it would be a better approach to let it search inside the python3-minimal folder for a DLL file matching the pattern python3?.dll (using the WinAPI function FindFirstFile, for example). When such a file is found, the address pointers of the necessary exported functions should be obtained from it. If this is successful, the found DLL file is indeed the Python Core DLL.

If the Python version is further required by the launcher, it can be queried by calling Py_GetVersion() (exported by Python3?.dll). Alternatively, the version number could also be extracted from the DLL file name.

(Side note: The Python2 launcher exhibits the same issue -- it has a dependency on Python27. But this dependency is not so much of an issue there, as a "newer" Python 2.8 release will pretty much never happen.)

[zsquareplusc]

For Python 3, the best way would be to link against python3.dll which was introduced to remove the problem for extensions, that they had to link to a specific version. However, I was not able to get it working. If I remember correctly, the problem was that the functions in python3.5.dll where not found despite the "links" in python3.dll and that it was located in the same directory. But maybe that could be solved.

[elgonzo]

To resolve the problem you encountered, you can do one of the following things:

• Use LoadLibraryEx function with the LOAD_WITH_ALTERED_SEARCH_PATH flag to load python3.dll. Note that this will require specifying the full absolute path to python3.dll. This is the preferred approach, if Windows XP should still be supported.
• Call SetDllDirectory, providing the path to the python3.dll/python3?.dll before invoking LoadLibrary. This should hopefully resolve the issue. This approach is also okay, although the minimum supported OS version is Windows XP SP1.
• Call AddDllDirectory before invoking LoadLibrary. Or use LoadLibraryEx with LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR flag. Unfortunately, these two approaches are supported out-of-box only since Win8/Win2K12. WinVista/7/2K8 require update KB2533623; and WinXP does not provide support for this at all...

(Whatever approach you choose, please resist hard-coding absolute paths in the executable, as this would make it impossible to move the resulting program package to another drive or directory without patching the exe. Also, i have not confirmed either of my suggestion whether they work in a scenario with forwarder exports; but i am quite optimistic that they will work ;) )

---

The cause of the problem you encountered:

The functions exported by python3.dll are actually forwarders to functions exported by python3?.dll. For example, the forwarder for Py_Main looks like this (i am using Python 3.5 in this example):

Py_Main=python35.Py_Main

The "python35" part in the forwarder denotes the name of the DLL the function is forwarded from.

The problem manifests in how Windows tries to load the DLL when resolving such a forwarder. It will only use the file name of the referenced DLL (python35.dll) to load it. It does not use a relative path nor an absolute path, Windows attempts to load the DLL just using its file name. This means the standard DLL search paths will be used.

So, Windows will look for phyton35.dll in the directory of the launcher executable and won't find it there. Then it will look for the DLL in all the other standard DLL paths and won't find it there either. But the directory of the forwarding python3.dll is not part of the standard DLL search path, so Windows will not look there...

The solution to this problem is to alter the DLL search path rule/mode (for example by using the SetDllDirectory function) so that the python35.dll can be found by the DLL loader...

[zsquareplusc]

I did try LOAD_WITH_ALTERED_SEARCH_PATH and SetDllDirectory with no success, in both cases it can not look up Py_Main. I've also tested these things back when first wrote the launcher.exe. And for AddDllDirectory the recommended code to test and use it at run-time is almost as large as the application itself... :/

[elgonzo]

Damn, you are right. When a referenced DLL has not been loaded already when resolving export fowarders, custom DLL search paths are completely ignored by whatever DLL load routine GetProcAddress employs. I just did test it myself (with a small project using VS2015 on a Win7 x64 box), and experienced the problem as you describe. Whatever i tried, whenever invoking GetProcAddress on a forwarder, the python36.dll was only looked for in the default DLL search paths. Bummer...

Seems the only way forward is to ignore python3.dll and loading python3?.dll directly :(

(I guess i will see whether a Python bug report about this issue has already been filed; and if not i will create one, suggesting to include a dependency on python3?.dll in python3.dll to make the export forwarders work in all situations)

[elgonzo]

FYI: I filed Python issue #29399. Obviously, it won't help with dealing with the DLLs of the current 3.5/3.6 releases....

[zsquareplusc]

It does find the forwarded functions in python3?.dll if I do a SetCurrentDirectory to the Python distribution, then do the GetProcAddress calls and then restore the current directory. So this might be a workaround...

However, I also need Py_SetPath which is not exported from python3.dll :( So maybe just searching for python*.dll is better anyway. Though it feels not as secure when it loads with wildcards instead of fixed names.

[elgonzo]

Since you need to load python3?.dll anyway (strange that Py_SetPath is not exported by python3.dll, since Py_GetPath is...), no need to worry about the whole DLL loading shenanigans and SetCurrentDirectory.

You can use FindFirstFile/FindNextFile with a file name pattern like "python3?.dll". That should make it a little bit more robust, as it will only find python3.dll and any file like python35.dll. It is currently irrelevant whether you use python3*.dll or python3?.dll as file name pattern, but if the Python devs ever decide to include files like python37-some-stuff.dll, the whole thing could be a teeny weeny bit faster when locating the DLL... I mean, i just realize that i just engaged in premature micro optimization... :)

By the way, if you read the Python issue i posted above, one of the Python/Windows developers confirmed that the problem with the export forwarder resolution is not a problem anymore in Win10. Seems MS did know and fixed this issue in the new(er) Windows version(s)...

[zsquareplusc]

I've pushed an implementation that uses FindFirstFile. I'm locating the zip file as python3?.dll finds python3.dll first and python3x.dll only as second match... The zip file name is also used in the code (for Py_SetPath).

Regarding the DLL path issues, I guess all the functions only influence LoadLibrary[Ex] but not GetProcAddress when the DLL is loaded as side effect of searching a function, at least on Win 7.

[zsquareplusc]

fix released