[OpenSSL] Set openssldir to be a placeholder path of 255 characters

[mkitti]

OpenSSL looks for certifiates and other files in a location set at compile time by the configuration option --openssldir.

This pull request sets OPENSSLDIR to be a placeholder path that can be modified in the resulting shared library binaries if needed
similar to the configuration used by conda-forge.

Specifically it will be set to a 255 character string, "/workspace/destdir/_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place".

This changes the certificate directory and file as follows.

julia> @ccall(OpenSSL_jll.libssl.X509_get_default_cert_dir()::Cstring) |> unsafe_string
"/workspace/destdir/_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/certs"

julia> @ccall(OpenSSL_jll.libssl.X509_get_default_cert_file()::Cstring) |> unsafe_string
"/workspace/destdir/_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/cert.pem"

julia> run(`$(OpenSSL_jll.openssl()) version -d`);
OPENSSLDIR: "/workspace/destdir/_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place"

The idea is that, if needed, someone could replace this path in the binary with a null paddded string to the needed default path leading to the appropriate certificates.

xref: JuliaLang/julia#53891

[mkitti]

Example code for mangling the library.

julia> using NetworkOptions, OpenSSL_jll

julia> function mangle_openssldir()
           target_path = realpath(OpenSSL_jll.libcrypto_path) 
           org_path = target_path * ".org"
           placeholder = "/workspace/destdir/_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place"

           # Backup the file
           if !isfile(org_path)
               @info "Saving a copy of $target_path to $org_path"
               cp(target_path, org_path)
           else
               @info "Reading from $org_path"
           end
         
           s = read(org_path, String)
           r = Regex(placeholder * "([[:alpha:]\\./]*)\x00")
           # Replace matches with null terminated versions
           for m in eachmatch(r, s)
               iob = IOBuffer()
               len = write(iob, NetworkOptions.bundled_ca_roots() |> dirname)
               len += write(iob, m.captures[1])
               write(iob, zeros(UInt8, length(m.match) - len))
               replacement = String(take!(iob))
               @info "Replacing" m.match replacement
               s = replace(s, m.match => replacement)
           end
           
           # Change permissions to overwrite the file
           try
               chmod(target_path, 0o755)
               write(target_path, s)
           finally
               chmod(target_path, 0o555)
           end
       end
mangle_openssldir (generic function with 1 method)

julia> mangle_openssldir()
[ Info: Reading from /home/mkitti/.julia/artifacts/eea8b7729a71c9a71335bb0084579392e1cdd79f/lib/libcrypto.so.3.org
┌ Info: Replacing
│   m.match = "/workspace/destdir/_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place\0"
└   replacement = "/home/mkitti/.julia/juliaup/julia-1.10.4+0.x64.linux.gnu/share/julia\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
┌ Info: Replacing
│   m.match = "/workspace/destdir/_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/private\0"
└   replacement = "/home/mkitti/.julia/juliaup/julia-1.10.4+0.x64.linux.gnu/share/julia/private\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
┌ Info: Replacing
│   m.match = "/workspace/destdir/_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/certs\0"
└   replacement = "/home/mkitti/.julia/juliaup/julia-1.10.4+0.x64.linux.gnu/share/julia/certs\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
┌ Info: Replacing
│   m.match = "/workspace/destdir/_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place/cert.pem\0"
└   replacement = "/home/mkitti/.julia/juliaup/julia-1.10.4+0.x64.linux.gnu/share/julia/cert.pem\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
5263932

[giordano]

Editing an artifact in place breaks the content addressability

[mkitti]

Editing an artifact in place breaks the content addressability

Could you elaborate? I'm replacing a string with a string of the same length using null bytes for padding. How does that break content addressability?

Nonetheless, this pull request is not about editing an artifact in place, it's about setting certain default paths to be long.

[giordano]

How does that break content addressability?

You're editing the file in place. Which isn't even possible if Julia installation is read-only (as it is now in Julia master).

[mkitti]

This pull request does not implement any editing itself. It simply replaces what is currently a useless default path with a long placeholder path to enable others to use a similar solution as implemented by conda-forge. This could be useful in certain situations where Julia needs to work in the same process as Python.

Julia itself should probably use the SSL_CTX API and provide a mechanism to obtain a default Julia SSL_CTX configured to use Julia's certifiate stores.
https://docs.openssl.org/3.0/man3/SSL_CTX_load_verify_locations/

[giordano]

others to use a similar solution as implemented by conda-forge.

Sure, also spack uses this trick of automatically relocating binaries and padding paths with ridiculously long placeholders, it's a common trick, but neither of them cares about content-addressability.

[fingolfin]

I don't think this is a sustainable way forward for us.

I think we can solve this instead by setting environment variables: That is, ensure OPENSSL_CONF, OPENSSL_ENGINES, OPENSSL_MODULES are set suitably (see this issue for more info) by the JLL wrapper, as discussed here for other packages.

[mkitti]

The purpose of this is not for Julia itself, but to make it easier to integrate Julia into other environments. For example dropping stock Julia into a conda environment and then trying to use PythonCall.jl.

For Julia configuration, I recommend creating a SSL context. Environment variables can be quite fragile depending on the operating system.