Skip to content

sandbox: no CA bundle bound in, breaks JVM tooling (pkl loading hk packages) #346

@mattwilkinsonn

Description

@mattwilkinsonn

Symptom

Two integration tests in jj-hooks (vendored as tools/jj-hooks/ in the zireael monorepo, but the bug is in seal's sandbox, not jj-hooks) fail when run under cargo nextest inside the sandbox:

  • hk_passing_hooks_pushes
  • hk_hook_autofix_creates_fixup_ref

Both share the same fault. Stderr:

thread 'main' panicked at src/settings.rs:101:30:
Failed to load configuration: failed to analyze pkl

Caused by:
   0: failed to run pkl
   1: Failed to evaluate Pkl config at /tmp/seal/sandbox/<id>/jj-hooks-worktree-<id>/hk.pkl

      Exit code: 1

      Error output:
      –– Pkl Error ––
      Exception when making request `GET https://github.com/jdx/hk/releases/download/v1.45.0/hk@1.45.0`:
      Error during SSL handshake with host `github.com`:
      unable to find valid certification path to requested target

      1 | amends "package://github.com/jdx/hk/releases/download/v1.45.0/hk@1.45.0#/Config.pkl"
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The tests run hk run against a worktree that has hk.pkl declaring amends "package://github.com/jdx/hk/releases/download/v1.45.0/hk@1.45.0#/Config.pkl". pkl (Java) tries to download the package over HTTPS and the JVM rejects every TLS connection because it can't validate the certificate chain.

The same tests pass on plain Ubuntu CI runners and on the host machine outside the sandbox.

Root cause

bubblewrap doesn't bind-mount /etc/ssl/certs (or /etc/pki/ca-trust/, or the OS-specific path containing the CA bundle) into the sandbox by default. The JVM's SunX509 trust manager falls back to its bundled trust store, which on a fresh JDK install is typically the cacerts file from the JDK distribution — usually present in $JAVA_HOME/lib/security/cacerts.

In this sandbox setup, either:

  1. $JAVA_HOME doesn't point at a JDK with a populated cacerts, or
  2. The JDK's cacerts is present but doesn't include the root CA that GitHub's cert chain terminates at (DigiCert Global Root G2, currently).

Either way, the fix is to expose the host system's CA bundle to JVM-based tools running inside the sandbox.

Reproduction (against the standalone jj-hooks repo or zireael/tools/jj-hooks/)

cd ~/repos/jj-hooks       # or ~/repos/zireael/tools/jj-hooks
cargo nextest run --no-fail-fast -E 'test(hk_passing_hooks_pushes) | test(hk_hook_autofix_creates_fixup_ref)'

Expected: 2 passed.
Actual: 2 failed with the SSL handshake error above.

Proposed fixes (any one resolves this)

A. Bind-mount the host CA bundle into every sandbox

In the sandbox bwrap argv:

--ro-bind /etc/ssl/certs /etc/ssl/certs
--ro-bind /etc/ca-certificates /etc/ca-certificates    # if present

Then export SSL_CERT_DIR=/etc/ssl/certs so the JVM and other TLS libraries find it. Most distros put the system bundle there; update-ca-certificates populates it.

B. Set a sandbox-wide JAVA_TOOL_OPTIONS

JAVA_TOOL_OPTIONS=-Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts

This only fixes JVM tooling (pkl, lefthook isn't JVM, hk isn't JVM but its pkl config IS). Narrower fix.

C. Detect "host has a CA bundle" at sandbox spawn and emit a warning if it's missing

Less intrusive — keeps the current default-deny but tells users they need to grant the CA path explicitly. Good as a complement to A/B, not a replacement.

Workaround we're using today

In tools/jj-hooks/ we just accept the 2 failing tests as known-fails inside the sandbox. They pass on Ubuntu CI and macOS so the test signal is preserved end-to-end; only local agent runs flag them. This isn't great long-term because:

  1. The agent has to mentally subtract them every time they read nextest output.
  2. New tools added to the monorepo that use pkl (Homebrew formulae are hand-edited Ruby, but a future tool using pkl for config would hit the same wall).
  3. The TLS-handshake-failure mode is generic — any JVM tool downloading anything over HTTPS will fail, even outside test contexts (e.g. someone running pkl directly in their seal session).

Suggested priority

P2 / convenience. Doesn't break correctness — tests still pass in real CI. But it surfaces noise in every monorepo agent session that touches the jj-hooks tests, and the underlying gap (no CA bundle in sandbox) will keep biting other JVM tools.

Related

  • The monorepo migration plan at ~/notes/seal/zireael-migration-plan.md documents this as a known issue.
  • ~/repos/zireael/tools/jj-hooks/tests/push.rs has the affected tests (hk_passing_hooks_pushes, hk_hook_autofix_creates_fixup_ref).
  • Discovered during phase B of the zireael monorepo migration.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions