[windows] improve python3.dll load check

[charles-zablit]

This patch improves the detection of python3x.dll.

SearchPathW does not account for SetDllDirectoryW. Therefore, when manually adding python3x.dll to the search path, SearchPathW will still return that it has not been found, resulting in a false negative.

This patch ensures that the driver uses the same mechanism as the loader to attempt to load python3x.dll using LoadLibraryW.

rdar://165123628

[llvmbot]

@llvm/pr-subscribers-platform-windows

@llvm/pr-subscribers-lldb

Author: Charles Zablit (charles-zablit)

Changes

This patch improves the detection of python3x.dll.

SearchPathW does not account for SetDllDirectoryW. Therefore, when manually adding python3x.dll to the search path, SearchPathW will still return that it has not been found, resulting in a false negative.

This patch ensures that the driver uses the same mechanism as the loader to attempt to load python3x.dll using LoadLibraryW.

---

Full diff: https://github.com/llvm/llvm-project/pull/168864.diff

1 Files Affected:

• (modified) lldb/tools/driver/Driver.cpp (+6-7)

diff --git a/lldb/tools/driver/Driver.cpp b/lldb/tools/driver/Driver.cpp
index bebf1a70d50e9..33cce42696372 100644
--- a/lldb/tools/driver/Driver.cpp
+++ b/lldb/tools/driver/Driver.cpp
@@ -477,18 +477,17 @@ bool AddPythonDLLToSearchPath() {
 #endif
 
 #ifdef LLDB_PYTHON_RUNTIME_LIBRARY_FILENAME
-/// Returns whether `python3x.dll` is in the DLL search path.
+/// Returns true if `python3x.dll` can be loaded.
 bool IsPythonDLLInPath() {
 #define WIDEN2(x) L##x
 #define WIDEN(x) WIDEN2(x)
-  WCHAR foundPath[MAX_PATH];
-  DWORD result =
-      SearchPathW(nullptr, WIDEN(LLDB_PYTHON_RUNTIME_LIBRARY_FILENAME), nullptr,
-                  MAX_PATH, foundPath, nullptr);
+  HMODULE h = LoadLibraryW(WIDEN(LLDB_PYTHON_RUNTIME_LIBRARY_FILENAME));
+  if (!h)
+    return false;
+  FreeLibrary(h);
+  return true;
 #undef WIDEN2
 #undef WIDEN
-
-  return result > 0;
 }
 #endif
 

[github-actions]

🐧 Linux x64 Test Results

• 33163 tests passed
• 494 tests skipped

[mstorsjo]

LGTM, looks reasonable.

This has a slight other effect as well; if python3.dll did exist but has missing dependencies, we'd previously mark it as found, while we now would conclude that it's not found. The error message for that would be slightly misleading though, but it's probably not an issue.

[compnerd]

This is a bit more expensive. This is going to do the search and load the module (performing any dynamic relocations required) and then follow that with an unload.

I wonder if this is appropriate given that we use this in the start up path.