[bws] add 1.0.0

[s-celles]

Closes #10609

[giordano]

Suggested change

	mkdir -p "${bindir}"
	cp ${target}/bw${exeext} ${bindir}
	chmod +x ${bindir}/*
	install -Dvm 755 "${target}/bw${exeext}" -t "${bindir}"

[giordano]

Suggested change

	build_tarballs(ARGS, name, version, sources, script, platforms, products, dependencies; julia_compat="1.6")
	build_tarballs(ARGS, name, version, sources, script, platforms, products, dependencies; julia_compat="1.6")

[giordano]

We don't support this platform at the moment

Suggested change

Platform("aarch64", "windows")

[giordano]

Suggested change

ArchiveSource("$(url_prefix)/bws-aarch64-pc-windows-msvc-$(version).zip", "457b706961d7949202e74c799d465097462161171c255646513ea76add60d169"; unpack_target = "aarch64-pc-windows-msvc"),

[giordano]

Suggested change

ArchiveSource("$(url_prefix)/bws-macos-universal-$(version).zip", "5ece899717ccd3abdb1a40fff5e34e759d8e95061845d289df31957f7bc700b0"; unpack_target = "macos-universal"),

[giordano]

I have no idea what you're doing, but this is waiting for interactive input, which is clearly not possible: https://buildkite.com/julialang/yggdrasil/builds/18090#01953777-c4d4-43c9-8db5-67604ff5bbbd/692-707

replace bws? [y]es, [n]o, [A]ll, [N]one, [r]ename:

[s-celles]

Oh! I noticed that

ArchiveSource("$(url_prefix)/bws-aarch64-unknown-linux-gnu-$(version).zip", "20a3dcb9e3ce7716a1dc3c0e1c76cea9d5e2bf75094cbb5aad54ced4304929cb"; unpack_target = "aarch64-linux-gnu"),

was duplicated. Sorry

[s-celles]

It should be ready now.

[giordano]

Uhm, I'm now having a look at the license https://github.com/bitwarden/sdk-sm/blob/4518617715c7b4b3afd3700d129d6b535b2732ff/LICENSE and I just realised it's very much not a standard open source license (not open by any means). Since I'm not a lawyer I'm particularly wary of reviewing it, since it's very much not clear to me whether we have the right to redistribute the binaries.

[s-celles]

I'm not lawyer also. I asked them some clarification. Until this I marked this PR as Draft to avoid it to be merged accidentaly

[s-celles]

To my understanding we are not really "distributing" the binary because we don't store it in repository / cloud storage.
In fact, it's final user which will download the binary (automatically) through Julia.
We are just providing an automatic "recipe" to download from the official gh repository Release of Bitwarden

[s-celles]

Here is Bitwarden support answer:

Thank you so much for getting in touch with Bitwarden's Customer Support team. 

I'd recommend reading this comment (bitwarden/clients#11611 (comment)) for more information about this topic; You can also learn more about this in this Reddit thread (https://www.reddit.com/r/Bitwarden/comments/1g7uwa2/desktop_version_2024100_is_no_longer_free/)

[s-celles]

@giordano I have an other answer from Bitwarden support

if you are not hosting the SDK binaries yourself and the final user downloads them directly from the official GitHub repository, it is generally considered acceptable usage. This is because you are not distributing the binaries yourself but rather facilitating the user's access to them. It is important to ensure that this process complies with any licensing agreements or terms of use specified by the SDK's official repository. If there are specific legal concerns, consulting with a legal professional would be advisable.

If there's anything else you need assistance with or if you have any more questions, please don't hesitate to reach out!

Any opinion? I mark this as ready for review now (removing Draft status)

[ararslan]

if you are not hosting the SDK binaries yourself and the final user downloads them directly from the official GitHub repository, it is generally considered acceptable usage. This is because you are not distributing the binaries yourself but rather facilitating the user's access to them.

The JLL package generated from this build script will host the platform-specific tarballs containing the binary artifacts as GitHub release assets; the end user would not be downloading them directly from the official Bitwarden GitHub repository.

[s-celles]

Thanks @ararslan for this clarification.
Isn't there a way to add to build script a special configuration to fit these kind of use case?

[s-celles]

I should probably add @kspearrin to this discussion to see if they can/want to do something on their side.

[ararslan]

Isn't there a way to add to build script a special configuration to fit these kind of use case?

I don't know of anything like that for Yggdrasil build recipes, but you can make your own package that isn't managed by Yggdrasil and has an Artifacts.toml file that points to the official binaries.

[s-celles]

I've never created Artifacts.toml https://pkgdocs.julialang.org/v1/artifacts/
After reading doc (quickly... and as a non native english speaker) I'm still unsure if I should

1. create Artifacts.toml like so

[bitwarden_sdk_sm]
git-tree-sha1 = "0000000000000000000000000000000000000000"

    [[bitwarden_sdk_sm.download]]
    url = "https://github.com/bitwarden/sdk-sm/releases/download/bws-v1.0.0/bws-x86_64-pc-windows-msvc-1.0.0.zip"
    sha256 = "69b8d0fb2facc8cec4dd2b8157a3496ecaaa376ee1b0fd822012192ce7437505"
    [[bitwarden_sdk_sm.download.arch]]
    arch = "x86_64"
    os = "windows"

    [[bitwarden_sdk_sm.download]]
    url = "https://github.com/bitwarden/sdk-sm/releases/download/bws-v1.0.0/bws-x86_64-unknown-linux-gnu-1.0.0.zip"
    sha256 = "9077fb7b336a62abc8194728fea8753afad8b0baa3a18723fc05fc02fdb53568"
    [[bitwarden_sdk_sm.download.arch]]
    arch = "x86_64"
    os = "linux"

    [[bitwarden_sdk_sm.download]]
    url = "https://github.com/bitwarden/sdk-sm/releases/download/bws-v1.0.0/bws-aarch64-unknown-linux-gnu-1.0.0.zip"
    sha256 = "20a3dcb9e3ce7716a1dc3c0e1c76cea9d5e2bf75094cbb5aad54ced4304929cb"
    [[bitwarden_sdk_sm.download.arch]]
    arch = "aarch64"
    os = "linux"

    [[bitwarden_sdk_sm.download]]
    url = "https://github.com/bitwarden/sdk-sm/releases/download/bws-v1.0.0/bws-x86_64-apple-darwin-1.0.0.zip"
    sha256 = "7e06cbc0f3543dd68585a22bf1ce09eca1d413322aa22554a713cf97de60495a"
    [[bitwarden_sdk_sm.download.arch]]
    arch = "x86_64"
    os = "macos"

    [[bitwarden_sdk_sm.download]]
    url = "https://github.com/bitwarden/sdk-sm/releases/download/bws-v1.0.0/bws-aarch64-apple-darwin-1.0.0.zip"
    sha256 = "5dd716878e5627220aa254cbe4e41e978f226f72d9117fc195046709db363e20"
    [[bitwarden_sdk_sm.download.arch]]
    arch = "aarch64"
    os = "macos"

or

2. a build_artifacts.jl script which should create this Artifacts.toml specifically for a given architecture.

Claude.ai code

using Pkg.Artifacts
using Downloads
using SHA
using Base.BinaryPlatforms # Import for Platform type

# This is the path to the Artifacts.toml we will manipulate
artifact_toml = joinpath(@__DIR__, "Artifacts.toml")

# Query the Artifacts.toml file for the hash bound to the name "bitwarden_sdk_sm"
# (returns `nothing` if no such binding exists)
bitwarden_hash = artifact_hash("bitwarden_sdk_sm", artifact_toml)

# If the name was not bound, or the hash it was bound to does not exist, create it!
if bitwarden_hash === nothing || !artifact_exists(bitwarden_hash)
    # Define temporary directory for downloads
    tmp_dir = mktempdir()
    
    # Download the zip for Windows
    zip_url = "https://github.com/bitwarden/sdk-sm/releases/download/bws-v1.0.0/bws-x86_64-pc-windows-msvc-1.0.0.zip"
    zip_path = joinpath(tmp_dir, "bitwarden.zip")
    println("Downloading ZIP from $zip_url")
    Downloads.download(zip_url, zip_path)
    
    # Verify the hash
    actual_hash = open(zip_path) do io
        bytes2hex(sha256(io))
    end
    expected_hash = "69b8d0fb2facc8cec4dd2b8157a3496ecaaa376ee1b0fd822012192ce7437505"
    println("Expected hash: $expected_hash")
    println("Actual hash:   $actual_hash")
    if actual_hash != expected_hash
        error("Hash mismatch for downloaded file!")
    end
    
    # Create the artifact by extracting the ZIP contents
    bitwarden_hash = create_artifact() do artifact_dir
        extract_dir = joinpath(tmp_dir, "extract")
        mkpath(extract_dir)
        
        # Extract the ZIP contents
        println("Extracting ZIP contents...")
        if Sys.iswindows()
            run(`powershell -Command "Expand-Archive -Path $zip_path -DestinationPath $extract_dir -Force"`)
        else
            run(`unzip -q $zip_path -d $extract_dir`)
        end
        
        # Create the bin directory in the artifact
        bin_dir = joinpath(artifact_dir, "bin")
        mkpath(bin_dir)
        
        # Look for the bws.exe executable in the extracted files
        found = false
        for (root, dirs, files) in walkdir(extract_dir)
            for file in files
                if file == "bws.exe"
                    src_path = joinpath(root, file)
                    dst_path = joinpath(bin_dir, file)
                    println("Found bws.exe at $src_path, copying to $dst_path")
                    cp(src_path, dst_path)
                    found = true
                    break
                end
            end
            if found
                break
            end
        end
        
        if !found
            error("Could not find bws.exe in the extracted files")
        end
        
        # On Windows, clear any read-only attributes
        if Sys.iswindows()
            exe_path = joinpath(bin_dir, "bws.exe")
            try
                run(`cmd /c attrib -R "$exe_path"`)
                println("Cleared read-only attribute from executable")
            catch e
                println("Note: Failed to modify file attributes: $e")
                println("This might not be a problem if the file is already writable")
            end
        end
    end
    
    # Now bind that hash within our Artifacts.toml for this platform.
    # We use force=true to ensure it overwrites any existing binding, if present.
    println("Binding artifact hash to 'bitwarden_sdk_sm' in Artifacts.toml...")
    bind_artifact!(artifact_toml, "bitwarden_sdk_sm", bitwarden_hash;
                   force=true,
                   download_info=[(zip_url, expected_hash)],
                   platform=Platform("x86_64", "windows"))
                   
    # If you wanted to add bindings for other platforms, you would include additional
    # calls to bind_artifact! here with different platform specifications.
    
    # Cleanup the temporary directory
    rm(tmp_dir, recursive=true)
end

# Get the path of the bitwarden artifact, either newly created or previously generated
bitwarden_path = artifact_path(bitwarden_hash)

println("\nArtifact successfully prepared!")
println("Artifact hash: $bitwarden_hash")
println("Artifact path: $bitwarden_path")

# Verify the executable exists in the expected location
exe_path = joinpath(bitwarden_path, "bin", "bws.exe")
if isfile(exe_path)
    println("Executable found at: $exe_path")
else
    println("WARNING: Executable not found at expected path!")
end

println("\nYou can now use your artifact with:")
println("julia> using Pkg.Artifacts")
println("julia> bitwarden_path = artifact\"bitwarden_sdk_sm\"")
println("julia> exe_path = joinpath(bitwarden_path, \"bin\", \"bws.exe\")")

println("\nOr in your package code:")
println("using Artifacts")
println("artifact_path = artifact\"bitwarden_sdk_sm\"")
println("bws_exe = joinpath(artifact_path, \"bin\", \"bws.exe\")")

[giordano]

https://github.com/JuliaPackaging/ArtifactUtils.jl

[s-celles]

ArtifactUtils seems to only support TAR (not ZIP)
Adding Bitwarden SDK to Artifacts.toml...

ERROR: Le canal de communication est sur le point d'être fermé.
ERROR: LoadError: This does not appear to be a TAR file/stream — malformed chksum field: "\x10y)\x8a\f\xfd(\x8b". Note: Tar.jl does not handle decompression; if the tarball is compressed you must use an external command like gzcat or package like CodecZlib.jl to decompress it. See the README file for examples.

[giordano]

Artifacts can only be tarballs anyway, that's not a limitation of ArtifactUtils specifically.

[s-celles]

Ok. So let's unzip and create a tarball.

Wondering if it's the simplest approach!

using Pkg

# First, add necessary packages if not already installed
for pkg in ["ArtifactUtils", "Downloads", "SHA", "Tar"]
    if !haskey(Pkg.project().dependencies, pkg)
        println("Installing $pkg...")
        Pkg.add(pkg)
    end
end

using ArtifactUtils, Artifacts, Downloads, SHA, Tar

println("Setting up Bitwarden SDK artifact...")

# Create temporary directories
tmp_dir = mktempdir()
extract_dir = joinpath(tmp_dir, "extract")
tar_dir = joinpath(tmp_dir, "tar")
bin_dir = joinpath(tar_dir, "bin")
mkpath(extract_dir)
mkpath(bin_dir)

# Download the ZIP file
zip_url = "https://github.com/bitwarden/sdk-sm/releases/download/bws-v1.0.0/bws-x86_64-pc-windows-msvc-1.0.0.zip"
zip_path = joinpath(tmp_dir, "bitwarden.zip")
println("Downloading ZIP from $zip_url...")
Downloads.download(zip_url, zip_path)

# Extract the ZIP file
println("Extracting ZIP file...")
if Sys.iswindows()
    run(`powershell -Command "Expand-Archive -Path $zip_path -DestinationPath $extract_dir -Force"`)
else
    run(`unzip -q $zip_path -d $extract_dir`)
end

# Find and copy the executable to the bin directory
println("Locating and copying executable...")
global found_exe = false
exe_src_path = ""

# First, look for the file directly in the extract directory
if isfile(joinpath(extract_dir, "bws.exe"))
    exe_src_path = joinpath(extract_dir, "bws.exe")
    found_exe = true
    println("Found bws.exe at $exe_src_path")
else
    # Search recursively if not found at the root
    for (root, dirs, files) in walkdir(extract_dir)
        for file in files
            if file == "bws.exe"
                exe_src_path = joinpath(root, file)
                found_exe = true
                println("Found bws.exe at $exe_src_path")
                break
            end
        end
        if found_exe
            break
        end
    end
end

# Check if we found the executable
if !found_exe
    # Try listing the directory contents to debug
    println("Could not find bws.exe. Contents of extract directory:")
    for (root, dirs, files) in walkdir(extract_dir)
        println("Directory: $root")
        for file in files
            println("  File: $file")
        end
    end
    error("Could not find bws.exe in the extracted files!")
end

# Copy the executable to the bin directory
exe_dst_path = joinpath(bin_dir, "bws.exe")
println("Copying to $exe_dst_path...")
cp(exe_src_path, exe_dst_path, force=true)

# Create a tarball from our prepared directory
println("Creating tarball...")
tar_path = joinpath(tmp_dir, "bitwarden_sdk_sm.tar")
Tar.create(tar_dir, tar_path)

# Calculate the SHA256 hash of the tarball
tar_sha256 = open(tar_path) do io
    bytes2hex(sha256(io))
end
println("Tarball SHA256: $tar_sha256")

# Now use ArtifactUtils with our prepared tarball
println("Adding artifact to Artifacts.toml...")
artifact_hash = ArtifactUtils.artifact_from_directory(tar_dir)
println("Artifact hash: $artifact_hash")

# Add the artifact to Artifacts.toml
open(joinpath(@__DIR__, "Artifacts.toml"), "w") do io
    println(io, """
    [bitwarden_sdk_sm]
    git-tree-sha1 = "$artifact_hash"
    
        [[bitwarden_sdk_sm.download]]
        sha256 = "$tar_sha256"
        url = "file://$(replace(tar_path, '\\' => '/'))"
    """)
end

# Make sure the artifact is installed
println("Ensuring artifact is installed...")
artifact_path = Pkg.Artifacts.ensure_artifact_installed("bitwarden_sdk_sm", joinpath(@__DIR__, "Artifacts.toml"))
println("Artifact installed at: $artifact_path")

# Check if the executable is in the expected location
exe_path = joinpath(artifact_path, "bin", "bws.exe")
if isfile(exe_path)
    println("Executable found at: $exe_path")
    
    # On Windows, ensure the executable is not read-only
    if Sys.iswindows()
        println("Clearing read-only attribute if present...")
        try
            run(`cmd /c attrib -R "$exe_path"`)
        catch e
            println("Note: Failed to modify file attributes: $e")
        end
    end
    
    # Test the executable
    println("Testing the executable...")
    try
        cmd = `cmd /c "$exe_path" --help`
        output = read(cmd, String)[1:min(end, 200)]
        println("Executable test output (first 200 chars):")
        println(output)
    catch e
        println("Warning: Could not execute the binary: $e")
        println("This might not be a problem if you use cmd /c in your code")
    end
else
    println("WARNING: Executable not found at expected path!")
end

println("\nArtifact setup complete!")
println("You can use it in your code with:")
println("using Artifacts")
println("bws_path = joinpath(artifact\"bitwarden_sdk_sm\", \"bin\", \"bws.exe\")")

# Cleanup
println("Cleaning up temporary files...")
rm(tmp_dir, recursive=true, force=true)