Hash Store, in-project and sketch SDKs

[jonathan-conder]

Description

We already hash SDKs published using sdkcraft try. This brings the other types of SDKs in line with that, with a view towards caching SDK snapshots across multiple workshops.

For Store SDKs, the GCS server only supports MD5 hashes and there's no easy way to download a specific SDK revision. Until we switch to the actual Store, SDKs will have either an MD5 or SHA3 hash. The hash is recomputed when downloading the SDK, since it might have changed since the initial query. Both the hash and SDK YAML are cached alongside the download, similar to try SDKs.

I added a validation step for all SDKs, which happens just before planning a launch or refresh. It checks that every SDK in the workshop has a hash and a valid sdk.yaml. Previously sdk.yaml was only parsed this early for Store SDKs. If the downloaded SDK differs from the plan, we validate the new version before installing it in the workshop.

For local SDKs, on launch/refresh the SDK files are copied to a temporary directory, to make them immutable. Instead of comparing this directory with others, we now compute a hash of the contents. The temporary directory is renamed to the hex digest (unless it already exists). If needed, a revision number is allocated, as a symlink (e.g. x42 -> abcdef1234567890). Otherwise the timestamp of the revision is bumped, using a new function Lchtimes to avoid following the link.

I fixed a minor bug in the previous local SDK logic: if a user deletes a file within an SDK just before it's copied to the temporary directory, Workshop now ignores the file instead of raising an error.

The hash of a directory is recursive. It accounts for filenames, contents, permissions and doesn't follow symlinks. The exact format is based on git:

• For regular files, the data we hash is blob <file size>\0<file contents>.
• For symlinks, the data is blob <target length in bytes>\0<link target>.
• For directories, the data is tree <entries length>\0<entries>, where entries is a sequence of items like <file mode> <file name>\0<hash>.

Self-review quick check

• Make decisions that cost a lot to reverse explicit in the PR description.
• Avoid nested conditions.
• Delete dead code and redundant comments.
• Normalise symmetries by sticking to doing identical things identically.

// one way to handle errors
if err := f(); err != nil {
   ...
}

// one way to handle multiple returns
val, err := f()
if err != nil {
   ...
}
...

• Check that coupled code elements, files, and directories are adjacent. For example, test data is stored as close as possible to a test.
• Put variable declaration and initialisation together.
• Divide large expressions into digestable and self-explanatory ones. Use multiple variables if required.
• Put a blank line between two logically different chunks of code.
• Follow the style guide for new error messages.

Docs

• I have checked and added or updated relevant documentation.
• I have checked and added or updated relevant release notes.
• I have included the technical author in the review.

Or:

• I confirm the PR has no implications for documentation.

[jonathan-conder]

Marking this as ready now, but I will try to make the local SDK changes more back-compatible

[Copilot]

Pull Request Overview

This PR introduces hashing for Store, in-project, and sketch SDKs, bringing them in line with the existing Try SDK hashing mechanism. The changes enable caching SDK snapshots across multiple workshops and add comprehensive validation of SDK definitions before installation.

Key changes:

• Store SDKs now support MD5 hashing (with SHA3 planned for future Store migration)
• Local SDKs are hashed using SHA3-384, with content-addressable storage replacing direct copies
• SDK validation is performed early in the launch/refresh process to catch configuration issues before deployment

Reviewed Changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
internal/sdk/store.go	Updates SdkResult structure to include hash fields and modifies download interface to return SDK metadata
internal/sdk/local.go	Implements content-addressable storage for local SDKs using SHA3-384 hashes with symlinked revisions
internal/fakestore/store.go	Adds MD5 hash computation and caching for Store SDK downloads
internal/workshop/workshop.go	Introduces ValidateSdkInfo function for early SDK definition validation
internal/overlord/workshopstate/request.go	Integrates SDK validation into launch and refresh workflows
internal/overlord/sdkstate/handlers.go	Updates download handlers to process and validate SDK metadata
internal/osutil/hash.go	Implements git-compatible directory hashing algorithm
internal/osutil/sys/syscall.go	Adds Lchtimes function for updating symlink timestamps without following
internal/testutil/symlinktargetchecker.go	Provides test utilities for verifying symlink targets

[github-actions]

TICS Quality Gate

✔️ Passed

workshop

All conditions passed

See the results in the TICS Viewer

The following files have been checked for this project

• internal/fakestore/store.go
• internal/osutil/cp.go
• internal/osutil/hash.go
• internal/osutil/sys/syscall.go
• internal/overlord/sdkstate/handlers.go
• internal/overlord/workshopstate/request.go
• internal/sdk/local.go
• internal/sdk/sdk.go
• internal/sdk/store.go
• internal/sdk/system/system.go
• internal/testutil/symlinktargetchecker.go
• internal/workshop/backend.go
• internal/workshop/lxd/lxd_backend_volume.go
• internal/workshop/workshop_dirs.go
• internal/workshop/workshop.go

Automatic-tests / code-coverage / TICS GitHub Action