Skip to content

v0.4.0 - Coding & document benchmarks, NUMA binding, cleanup

Choose a tag to compare

@JackWeiw JackWeiw released this 31 Aug 11:14
· 333 commits to main since this release

πŸš€ v0.4.0 - Coding & document benchmarks, NUMA binding, cleanup

This release adds coding (TS/Go) and document (PDF/XLSX) benchmarks, per-sandbox NUMA node binding, and arch-aware (x86_64) Dockerfiles, then removes the deprecated qemu_monitor / session-replay packages and translates remaining Chinese comments/docstrings/CLI text to English.

Features

πŸ’» Coding benchmarks (E2B)

  • TypeScript benchmark against vuejs/core β€” multi-process verify via npx tsx (default 3 concurrent processes), compiler-core baseParse trace-faithful verification
  • Go benchmark against gohugoio/hugo β€” go clean -cache before every verify for cold-compile CPU measurement
  • verify_repeat config (yaml + --coding-verify-repeat CLI) β€” repeat verify N times with independent processes
  • Per-pair edit semantics; shared DEFAULT_VERIFY_TEMPLATES pool; language-aware root discovery

πŸ“„ Document benchmarks

  • PDF and XLSX document benchmarks via document-bench-validate
  • Group document operations into phases; retry transient ready-check failures

🧬 NUMA node binding (E2B)

  • Bind each sandbox to a NUMA node via round-robin distribution across a numa_bind list
  • Normalize numa_bind to a list-of-nodes at config load; multi-node numa_bind form documented
  • numa_node_for_index contract tightened (type hints, explicit bool guard)

πŸ—οΈ Arch-aware builds

  • x86_64 Dockerfile variants for coding-ts and coding-go
  • push_to_harbor.sh made arch-aware via ARCH env var (browser / coding-ts / coding-go)

Refactor

  • Config & dockerfile reorganization: config/ and dockerfile_build/ reorganized by scenario (browser/coding/document)
  • Logging migration: replace print() with the logging module across e2b_bench and docker_bench; benchmark report emitted via logger.info, routed to stdout to preserve pre-refactor output destination
  • Round-robin cleanup guaranteed on errors

Removed

  • qemu_monitor/ package and qemu_monitor.py entry script (deprecated in v0.1.0 β†’ v0.2.0) β€” use vm_monitor
  • session-replay/ β€” use llm_replay

Notes

  • Remaining Chinese comments, docstrings, and CLI text translated to English
  • Version bumped to 0.4.0

Full changelog: v0.3.0...v0.4.0