"Local artifacts" to reduce dependence on @__DIR__

[Keno]

When discussing #38682 on triage, the concern was brought up that while @__DIR__ is useful, it is quite problematic for things like PkgCompiler that may end up distributing a compiled binary without the corresponding source tree, so packages relying on @__DIR__ to load configuration or test data will be unhappy. The best alternative mechanism we currently have is artifacts, which is fully integrated with PkgCompiler. However, it is a little high friction for the use case of referring to a few small files that are version controlled with the package. Thus, a proposal was made to add a variant of artifacts that is lower friction.

In particular, we would add the ability to declare a folder in the package as a "local" artifact (by marking it appropriately in the Artifacts.toml). These local artifacts do not have a content hash, just a local relative path (and no download information, etc). They can then be referenced by name, just like any other artifact and during normal execution will just resolve the location in the local directory. However, when such a package is PkgCompile'd, PkgCompiler would go through, snapshot the referenced directory and turn these local artifacts into regular artifacts such that the rest of the PkgCompiler artifacts support just goes through.

[fredrikekre]

See JuliaLang/Pkg.jl#1727, #35207

[KristofferC]

However, when such a package is PkgCompile'd, PkgCompiler would go through, snapshot the referenced directory and turn these local artifacts into regular artifacts such that the rest of the PkgCompiler artifacts support just goes through.

An alternative is for the artifact loader (just like code loading) to understand non-content-addressed artifcacs and allow of a way to add/modify the way these are looked for.

Say we have an artifact.toml with

# Example Artifacts.toml file
[my_local_artifact]
path = "data"

The artifact string macro could expand artifact"my_local_artifact" to something like:

macro artifact_str(name)
    return quote
        if !is_local_artifact(name) 
           # do the current stuff
        end
        local_artifact_dir = get(ENV, "LOCAL_ARTIFACT_DIR", nothing)
        if local_artifact_dir === nothing
            return joinpath(artifact_path, name)
         else
            # local artifact dir is overridden, namespace via pkgslug (same as code loading)
            return joinpath(local_artifact_dir, pkgslug, name)
        end
    end
end
end

PackageCompiler would then just set LOCAL_ARTIFACT_DIR and bundle all the local artifacts in a similar way to how the content addressed artifacts are bundled. That seems slightly less awkward than having to rewrite the Artifact.toml file, make the package compile with that rewritten file etc.