BE | Package Docker build context in gem (#86)

[igmarin]

Summary

• Add packaged lib/skill_bench/execution/docker/ context (minimal Dockerfile + .dockerignore)
• Expand gemspec spec.files so non-Ruby Docker assets ship in the gem
• Centralize docker context path on Constants::Sandbox.docker_context_path / Sandbox.docker_context_path
• Require Dockerfile in PackageVerifier and add packaging unit tests

Closes #86

Part of milestone container-isolation-v1

Stack

1. This PR — packaging foundation
2. BE | Define evaluator-sandbox image contract + build #87 image contract + build
3. BE | Activate container lifecycle with safe fallbacks #88 activate lifecycle / fallbacks
4. BE | Live Docker integration tests (opt-in) #89 live docker tests
5. BE | CI job for container integration #90 CI docker job
6. BE | Docs + CHANGELOG for shipped container isolation #91 docs + CHANGELOG

Test plan

• bundle exec rake test TEST=test/evaluator/sandbox_docker_packaging_test.rb (green)
• bundle exec rake test TEST=test/evaluator/sandbox_test.rb (green)
• bundle exec rake test TEST=test/evaluator/tools/run_command_test.rb (green)
• bundle exec rake package:verify includes Dockerfile
• CI green on PR

Notes

• Fail-closed host execution unchanged when Docker daemon is absent
• Security flags in start_container are not modified in this PR

Summary by CodeRabbit

• New Features

  • Added a packaged Docker sandbox based on Ruby 3.4 for isolated execution.
  • Sandbox resources are now reliably discovered from installed packages.
  • Docker sandbox execution uses a dedicated context and supports network-disabled runtime isolation.
  • Host execution remains available when Docker is unavailable.

• Bug Fixes

  • Ensured Docker sandbox files are included in releases and verified during packaging.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 5ad799ea-b3ca-485d-9575-6f7c9e272142

📥 Commits

Reviewing files that changed from the base of the PR and between 80e9605 and 15ce797.

📒 Files selected for processing (3)

• lib/skill_bench/package_verifier.rb
• ruby-skill-bench.gemspec
• test/evaluator/sandbox_docker_packaging_test.rb

🚧 Files skipped from review as they are similar to previous changes (3)

• ruby-skill-bench.gemspec
• test/evaluator/sandbox_docker_packaging_test.rb
• lib/skill_bench/package_verifier.rb

---

📝 Walkthrough

Walkthrough

The PR adds a Docker sandbox image and packages its build context in the gem. Sandbox resolves the context through a shared absolute-path helper. Tests verify the Dockerfile, gemspec packaging, and path consistency.

Changes

Docker sandbox packaging

Layer / File(s)	Summary
Docker context path integration lib/skill_bench/constants.rb, lib/skill_bench/execution/sandbox.rb	Adds a shared Docker context path constant and resolver. Sandbox uses the resolver for Docker checks and image builds.
Docker assets and packaging validation lib/skill_bench/execution/docker/*, ruby-skill-bench.gemspec, lib/skill_bench/package_verifier.rb, test/evaluator/sandbox_docker_packaging_test.rb	Adds the Ruby 3.4 Docker image, restricts the build context, packages the assets, verifies required release files, and tests path and packaging behavior.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Sandbox
  participant ConstantsSandbox
  participant DockerContext
  participant DockerDaemon
  Sandbox->>ConstantsSandbox: request docker_context_path
  ConstantsSandbox->>DockerContext: resolve packaged Docker context
  Sandbox->>DockerDaemon: build image from Docker context
  Sandbox->>DockerDaemon: check Docker availability

Loading

Possibly related issues

• Issue 83: The PR implements Docker build-context packaging and Sandbox path resolution.

Poem

A rabbit packs the Docker crate,
With paths resolved both sure and straight.
The image sleeps, the files ship too,
Tests check each path in view.
Hop, hop—sandbox is ready to go!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the primary change: packaging the Docker build context in the gem.
Linked Issues check	✅ Passed	The changes satisfy #86 by adding the Docker context, packaging assets, centralizing path resolution, verifying packaging, and preserving sandbox fallback behavior.
Out of Scope Changes check	✅ Passed	All changes support #86 and remain within Docker context packaging, path resolution, verification, and sandbox behavior.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ruby-skill-bench.gemspec`:
- Around line 20-29: Update the gem file list in the gemspec to explicitly
include lib/skill_bench/execution/docker/.dockerignore, then add the same asset
to PackageVerifier::REQUIRED_FILES and the packaging test’s Docker context asset
expectations. Keep the existing Dockerfile and other asset checks unchanged.

In `@test/evaluator/sandbox_docker_packaging_test.rb`:
- Around line 22-24: Update the Dockerfile command assertion in the sandbox
packaging test to verify the required long-running keep-alive command, rather
than accepting any CMD or ENTRYPOINT. Preserve the existing base-image assertion
and ensure the match rejects commands such as true that exit before docker exec
can run.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: ed7ef11e-e1d8-4fc5-b5a6-245ddbf4a935

📥 Commits

Reviewing files that changed from the base of the PR and between 9448ecd and 80e9605.

📒 Files selected for processing (7)

• lib/skill_bench/constants.rb
• lib/skill_bench/execution/docker/.dockerignore
• lib/skill_bench/execution/docker/Dockerfile
• lib/skill_bench/execution/sandbox.rb
• lib/skill_bench/package_verifier.rb
• ruby-skill-bench.gemspec
• test/evaluator/sandbox_docker_packaging_test.rb