Overriding the default dynamic library lookup path

[HertzDevil]

crystal run and the macro run create a temporary executable in Crystal's cache directory. On Windows, programs typically do not install their DLLs into a system-wide directory, instead placing them in the same directory as the executable itself. Therefore, assuming Crystal follows this convention, if Crystal is not in %PATH%, this temporary executable cannot locate the DLLs; even if Crystal is present in %PATH%, it might be preceded by other directories.

The default DLL search order does not provide an easy way to intercept the search paths. With delayed loading, however, we can call LoadLibrary multiple times until one of them returns a valid handle, for example:

# NOTE: can't use stdlib or the c runtime here
private def strlen(str)
  len = 0
  while str.value != 0
    len &+= 1
    str += 1
  end
  len
end

private def strcat(*args : *T) forall T
  buflen = 1
  {% for i in 0...T.size %}
    %len{i} = strlen(args[{{ i }}])
    buflen &+= %len{i}
  {% end %}
  buf = LibC.HeapAlloc(LibC.GetProcessHeap, LibC::HEAP_ZERO_MEMORY, buflen).as(UInt8*)

  ptr = buf
  {% for i in 0...T.size %}
    src = args[{{ i }}]
    while src.value != 0
      ptr.value = src.value
      ptr += 1
      src += 1
    end
  {% end %}

  buf
end

private def load_library(dll)
  my_path = strcat("C:\\foo\\bar".to_unsafe, "\\".to_unsafe, dll)
  begin
    LibC.LoadLibraryExA(my_path, nil, LibC::LOAD_WITH_ALTERED_SEARCH_PATH) ||
      LibC.LoadLibraryExA(dll, nil, 0)
  ensure
    LibC.HeapFree(LibC.GetProcessHeap, 0, my_path)
  end
end

fun __delayLoadHelper2(pidd : LibC::ImgDelayDescr*, ppfnIATEntry : LibC::FARPROC*) : LibC::FARPROC
  # ...
  if !hmod
    unless hmod = load_library(dli.szDll)
      # ...
    end
  end
  # ...
end

where C:\foo\bar is our Crystal installation path, or some other arbitrary directory. From here we could make it configurable:

private def load_library(dll)
  {% if (paths = env("CRYSTAL_LIBRARY_RPATH")) && !paths.empty? %}
    {% for path, i in paths.split(";") %}
      my_path = strcat({{ path + "\\" }}.to_unsafe, dll)
      hmod = LibC.LoadLibraryExA(my_path, nil, 0x8)
      LibC.HeapFree(LibC.GetProcessHeap, 0, my_path)
      return hmod if hmod
    {% end %}
  {% end %}

  LibC.LoadLibraryExA(dll, nil, 0)
end

Then set CRYSTAL_LIBRARY_RPATH=C:\foo\bar would set, at build time, the extra directory(ies) that would be searched first for dynamic libraries before the default order. If the name rings a bell, this is exactly the ELF RPATH attribute: (RPATH overrides LD_LIBRARY_PATH, RUNPATH doesn't and is less universally supported)

{% if (paths = env("CRYSTAL_LIBRARY_RPATH")) && !paths.empty? %}
  @[Link(ldflags: "-Wl,-rpath,#{paths}")]
  lib LibCrystalMain
  end
{% end %}

This in turn suggests our custom implementation of CRYSTAL_LIBRARY_RPATH on Windows should be able to expand $ORIGIN. This is usually unnecessary because the executable's directory is already the first one with the default order, but it could technically allow things like $ORIGIN\dll so that the DLLs do not pollute %PATH% even if the executable itself is added.

Naturally, a Crystal interpreter built with a custom CRYSTAL_LIBRARY_RPATH should be able to load libraries under CRYSTAL_LIBRARY_RPATH as well. This means Crystal::Loader should replicate this functionality too; on Unix, by adding those directories to .default_search_paths, and on MSVC, explicitly via #open_library similar to the delay load helper.

Finally, displaying CRYSTAL_LIBRARY_RPATH in crystal env might be a good idea.