Actually installing DLLs via @[Link]

[HertzDevil]

To facilitate load-time dynamic linking on Windows, Crystal added experimental compiler support to detect and load DLLs from more "convenient" locations via a delay-load helper, but it does not work on DLLs that import data. So let's try to actually install them instead.

To get a better idea about how other compiled languages handle this, here are some examples with SDL2:

• C/C++: The major C compilers are not responsible for installing DLLs. SDL2 has a CMakeLists.txt though, so any project that consumes SDL2 via CMake ≥3.21 should be able to use install(TARGET_RUNTIME_DLLS).
• Go: It doesn't install DLLs.
• Haskell: It doesn't install DLLs. (It uses MinGW-based toolchains, but the point is the same.)
• Rust: It doesn't install DLLs. Another example with SFML
• Zig: The link step does this via installBinFile, which can install files other than DLLs too. Presumably, this link step is automatically executed when building projects that depend on the given SDL2 bindings.

I propose adding a new dll parameter to the @[Link] compiler annotation:

@[Link(dll: "foo.dll")]
@[Link(dll: ["foo1.dll", "foo2.dll"])]
lib Foo
end

dll must be a string literal or an array literal of them. Each string on a lib indicates the name of a DLL that should be searched and installed. When building on Windows and not cross-compiling, the compiler either hardlinks or copies the DLLs to the same directory as the built executable. The search order used by the compiler, completely independent from the runtime DLL search order, would be:

• CRYSTAL_LIBRARY_PATH
• Same directory as the compiler itself
• %PATH%

There is no longer automatic DLL detection from import libraries, and all libraries must declare their DLLs manually. This is because we only want to install library DLLs, not system DLLs like kernel32.dll or vcruntime140.dll, but no detection scheme is ever going to handle that. Hence, this proposal effectively reverts all the delay-load stuffs added in recent Crystal versions.

Alternatively, if we go down the Zig route of supporting arbitrary file installs, we could spare a new annotation:

{% if flag?(:windows) %}
  {%
    search = Crystal::LIBRARY_PATH.split(";") +
      ... + # `Process.executable_path` of the compiler itself in the macro language
      env("PATH").split(";")
    dll_path = nil
    search.each do |dir|
      unless dll_path
        full = ... # `File.join(dir, "foo.dll")` in the macro language
        dll_path = full if file_exists?(full)
      end
    end
  %}
  @[Install({{ dll_path }})]
{% end %}
lib Foo
end

Of course, this further blurs the line between Crystal as a compiler and Crystal as a build system, in absence of an actual build system tailor-made for Crystal.

[beta-ziliani]

So how does your proposal work for system dlls?

I'm not sure I understand the alternative Zig-like proposal, why is the dll_path in charge of the lib user?

I'll invoke @oprypin to weigh in.

[HertzDevil]

So how does your proposal work for system dlls?

It doesn't, and not all of those DLLs are even redistributable.

why is the dll_path in charge of the lib user?

This is not 100% necessary here, @[Install] can be made as terse as @[Link(dll: ...)]; the snippet is mainly to show the minimum missing pieces needed to express the same functionality in the current macro language.

[oprypin]

I personally don't find it hard to just copy some files in my own way. Users of other programming languages also need to do it.

Automatic copying can be slightly unexpected but if it's opt-in then great! 👍
Can be a nice solution 🙂

[beta-ziliani]

It doesn't, and not all of those DLLs are even redistributable.

They will just use the same @[Link] annotation as today, right? This was my question.

If dll: would only mean "install this one", wouldn't it be better to mark it some other way? Lke @[Link("somedll", install: true).

[HertzDevil]

There is no correlation between a DLL's name and its import library's name. Using stdlib's dependencies as example:

• gc-dynamic.lib imports gc.dll
• iconv-dynamic.lib imports libiconv.dll
• ssl-dynamic.lib imports libssl-3-x64.dll
• z-dynamic.lib imports zlib1.dll

Reconciling the two sets of names means either rebuilding the DLLs, or giving the import libraries different names from non-Windows platforms. IMO neither option is desirable. And not doing this means the DLL names must be supplied somewhere, which is what dll: does.

They will just use the same @[Link] annotation as today, right? This was my question.

Yes, the meaning of @[Link]'s positional parameter remains the same

[HertzDevil]

One practical problem is @[Link] doesn't understand the dll: parameter from Crystal 1.0 to 1.10, because argument parsing is part of the compiler. So the annotations in stdlib may look like this:

@[Link("z")]
{% if flag?(:win32) && compare_versions(Crystal::VERSION, "1.11.0-dev") >= 0 %}
  @[Link(dll: "zlib1.dll")] 
{% end %}
lib LibZ
end

or like this:

@[Link("z")]
{% if flag?(:win32) && flag?(:supports_link_dll) %}
  @[Link(dll: "zlib1.dll")] 
{% end %}
lib LibZ
end

where supports_link_dll is a compile-time flag added in the same PR that implements dll:.

[straight-shoota]

I don't think this practical problem is a big issue. Stdlib and compiler version are usually expected to be equal. So there's not much to worry about for stdlib, yet it's easy to integrate safe guards.
For user code, it's probably not much of an issue either since there aren't many Windows specific library bindings anyway.

IMO compare_versions(Crystal::VERSION, "1.11.0-dev") >= 0 should be good enough and there's no need for a flag.

[HertzDevil]

There is another caveat: if the local compiler is statically built, even from a compiler with support for dll:, then no DLLs would be present under .build, which means the local compiler in turn won't find the DLLs in the original compiler unless that compiler is already in %PATH%.

I guess we could add %CRYSTAL% as a search path before %PATH%, since that is what the wrapper scripts use if a local compiler is absent?