Wishlist: Precompile-only packages

[giordano]

There are some packages which are only required, at least for some functionalities, only during precompile time, but not at runtime. Some examples:

• packages whose only API is some macros, but they don't have their own runtime, for example SnoopPrecompile.jl and MLStyle.jl
• packages used to read serialised data during precompilation, but otherwise unused during runtime. We use JSON.jl in BinaryBuilder.jl to read a JSON file to be stored in a constant, for this purpose JSON isn't needed at runtime (we happen to use JSON.jl for something else within BinaryBuilder, but this is besides the point)
• I can imagine that in some cases Clang.jl might be used to generate some wrappers during precompilation, for example if system header files are needed, although this isn't its main use case: package developers typically statically write to file the wrappers, so they don't need to be generated during precompilation, but you get the idea.

Using these packages requires always loading them at runtime, even if they aren't needed. It'd be nice not to load the precompile-only packages at all. @fredrikekre mentioned that ccall(:jl_generating_output, Cint, ()) == 1 can be used to guard precompilation code (see also #45650 for a suggestion to make this easier to use), but @KristofferC pointed out that this won't work in practice to elide the using statements. There is a recent package which tries to address this issue, DevOnly.jl, but its current implementation isn't very clean. A challenge for a built-in solution in Julia mentioned on Slack by @vchuravy is that the loaded package may have modified a method table.

There is prior-art in Lisp dialects. Emacs Lisp has an eval-when-compile macro, which executes its body only during compilation. A common use-case, mentioned also in the linked documentation, is exactly for loading some other packages which are used to generate some code with macros to be expanded during compilation (a popular package for this use-case is the cl-lib package which provides lots of Common Lisp macros):

(eval-when-compile
  (require 'my-macro-package))

Common Lisp should have something analogous like (eval-when (compile eval) ....

[vchuravy]

Yeah I don't think we can do that since the side-effects of package loading are rather broad. What I could see working is "header files", e.g. including files from a dependency without loading it.

pkgid = Base.identify_package("Random")
pkgid === nothing && error("Package not found")
pkgpath = Base.locate_package(pkgid)
pkgpath === nothing && error("Could not locate package")
include_dir = joinpath(dirname(dirname(pkgpath)), "include")
include(joinpath(include_dir, "definitions.jl"))

We could attempt to formalize this, but this would duplicate macro, function and type definitions so I am not sure it's worthwhile.