Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support DLL delay-loading on Windows #13436

Merged

Conversation

HertzDevil
Copy link
Contributor

@HertzDevil HertzDevil commented May 5, 2023

This PR adds support for the /DELAYLOAD MSVC linker flag, and additionally makes almost all DLLs delay-loaded when -Dpreview_dll is used to build a load-time dynamically linked executable.

When a program is linked against a DLL import library, by default the startup code automatically calls LoadLibrary and GetProcAddress behind the scenes to perform the actual linking, and terminates with STATUS_DLL_NOT_FOUND = 0xC0000135 if a DLL cannot be found. This happens before the entry point gets called, so Crystal's runtime cannot intercept the error at all, and the parent process receives no information other than the exit code. With delayed loading, the DLLs and symbols are only loaded the first time they are used, and __delayLoadHelper2 becomes the linker function, which must be defined separately.

One way to obtain this linker function is to link against delayimp.lib from Visual C++ for a sensible default implementation with support for several user-defined hooks, and the other is to port the reference implementation to Crystal. This PR does the latter, which gives us complete control over the link process, in particular the DLL search order and error reporting. At the moment fun __delayLoadHelper2 is a verbatim port of the file Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.35.32215\include\delayhlp.cpp, except all the hooks are omitted. In the future we could enhance this function to deliver a better experience when dealing with DLLs on Windows.

When -Dpreview_dll is specified, if both the host and the target use MSVC, the compiler will now look up all the import libraries passed to the linker, and append a /DELAYLOAD linker flag for every imported DLL, reusing the logic from Crystal::Loader. No loader is actually created, and the compiler itself never loads any dynamic libraries this way. For example, running dumpbin /dependents on an empty source code compiled with -Dpreview_dll gives:

File Type: EXECUTABLE IMAGE

  Image has the following dependencies:

    KERNEL32.dll

  Image has the following delay load dependencies:

    pcre2-8.dll
    gc.dll
    libiconv.dll
    ADVAPI32.dll
    VCRUNTIME140.dll
    SHELL32.dll
    ole32.dll
    dbghelp.dll
    api-ms-win-crt-runtime-l1-1-0.dll
    api-ms-win-crt-heap-l1-1-0.dll
    api-ms-win-crt-string-l1-1-0.dll
    api-ms-win-crt-stdio-l1-1-0.dll
    api-ms-win-crt-math-l1-1-0.dll
    api-ms-win-crt-filesystem-l1-1-0.dll
    api-ms-win-crt-locale-l1-1-0.dll

The commented out FormatMessageA stub in delay_load.cr gives:

Loading `_initterm_e` from `api-ms-win-crt-runtime-l1-1-0.dll`
Loading `_set_app_type` from `api-ms-win-crt-runtime-l1-1-0.dll`
Loading `_set_fmode` from `api-ms-win-crt-stdio-l1-1-0.dll`
Loading `__p__commode` from `api-ms-win-crt-stdio-l1-1-0.dll`
Loading `_crt_atexit` from `api-ms-win-crt-runtime-l1-1-0.dll`
Loading `_configure_wide_argv` from `api-ms-win-crt-runtime-l1-1-0.dll`
Loading `_configthreadlocale` from `api-ms-win-crt-locale-l1-1-0.dll`
Loading `_initialize_wide_environment` from `api-ms-win-crt-runtime-l1-1-0.dll`
Loading `_initterm` from `api-ms-win-crt-runtime-l1-1-0.dll`
Loading `_set_new_mode` from `api-ms-win-crt-heap-l1-1-0.dll`
Loading `_get_initial_wide_environment` from `api-ms-win-crt-runtime-l1-1-0.dll`
Loading `__p___wargv` from `api-ms-win-crt-runtime-l1-1-0.dll`
Loading `__p___argc` from `api-ms-win-crt-runtime-l1-1-0.dll`
Loading `malloc` from `api-ms-win-crt-heap-l1-1-0.dll`
Loading `GC_init` from `gc.dll`
Loading `GC_set_start_callback` from `gc.dll`
Loading `GC_set_warn_proc` from `gc.dll`
Loading `GC_malloc` from `gc.dll`
Loading `memset` from `VCRUNTIME140.dll`
Loading `GC_malloc_atomic` from `gc.dll`
Loading `GC_register_finalizer_ignore_self` from `gc.dll`
Loading `GC_get_push_other_roots` from `gc.dll`
Loading `GC_set_push_other_roots` from `gc.dll`
Loading `SystemFunction036` from `ADVAPI32.dll`
Loading `_get_osfhandle` from `api-ms-win-crt-stdio-l1-1-0.dll`
Loading `_setmode` from `api-ms-win-crt-stdio-l1-1-0.dll`
Loading `GC_realloc` from `gc.dll`
Loading `GC_get_my_stackbottom` from `gc.dll`
Loading `pcre2_config_8` from `pcre2-8.dll`
Loading `memchr` from `VCRUNTIME140.dll`
Loading `memcpy` from `VCRUNTIME140.dll`
Loading `free` from `api-ms-win-crt-heap-l1-1-0.dll`
Loading `exit` from `api-ms-win-crt-runtime-l1-1-0.dll`

This behavior can be disabled if -Dno_win32_delay_load is also provided; the linker function remains available, so individual DLLs can still be delay-loaded if the respective /DELAYLOAD flag is provided explicitly.

A side effect of this change is that a missing DLL is no longer an error as long as the executable doesn't call any function from that DLL, but the most useful benefit is not having to link the entire user32.dll upfront whenever OpenSSL is used (they use MessageBoxW for certain errors).

$image_base = __ImageBase : IMAGE_DOS_HEADER
end

private macro p_from_rva(rva)
Copy link
Contributor Author

@HertzDevil HertzDevil May 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is just one caveat: for some reason LibC.image_base cannot be accessed from any LLVM module scope other than the top-level one, otherwise MSVC raises an internal linker error, and the only way out is to pass --single-module. So this cannot be a class method of Crystal::System::DelayLoad, and neither can __delayLoadHelper2 itself

src/compiler/crystal/loader/msvc.cr Outdated Show resolved Hide resolved
src/crystal/system/win32/delay_load.cr Outdated Show resolved Hide resolved
src/crystal/system/win32/delay_load.cr Outdated Show resolved Hide resolved
src/crystal/system/win32/delay_load.cr Outdated Show resolved Hide resolved
src/crystal/system/win32/delay_load.cr Outdated Show resolved Hide resolved
src/crystal/system/win32/delay_load.cr Outdated Show resolved Hide resolved
@straight-shoota straight-shoota added this to the 1.9.0 milestone May 10, 2023
@straight-shoota straight-shoota merged commit 39aae80 into crystal-lang:master May 11, 2023
40 of 45 checks passed
@HertzDevil HertzDevil deleted the feature/windows-dll-delay-load branch May 11, 2023 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants