Skip to content

1.0.0-M11

Choose a tag to compare

@github-actions github-actions released this 01 Aug 09:34
· 16 commits to master since this release
2b47094

M11 is the release where the compile server stopped being a black box. It is a long-lived JVM you never launched, shared by every workspace on your machine, and until now it had no ceiling on what it would admit, no account of where its heap went, and no vocabulary for saying why it died. All three are fixed. Along the way: Kotlin incremental compilation now actually engages, Windows works, and a family of bugs that let a broken build report success are closed.

Three months and 83 commits since M10. Most of it is hardening rather than new surface — this is the release that makes the previous one's promises hold under load.

Where these bugs came from

Almost nothing here was found by a test matrix. It was found by running bleep continuously against a codebase big enough and a workflow parallel enough to break it.

That codebase is 12.9M lines across 57,941 files in a single repository. What bleep compiles is 4.1M lines of Java and Scala, 11,896 files, 67 projects — and 3.7M of those Java lines (10,831 files, 92% of the Java) are generated and checked in, a SQL parser and AST family spanning a dozen database dialects. Codegen is not a side feature of this build; it is most of it. Testing is corpus-driven: roughly 2,300 declared test cases, several of them heavy enough to need their own heap, running against a corpus of 44,127 SQL files.

And it is never one build at a time. Development runs across a dozen or more git worktrees concurrently, each an independent client of the same shared compile server, frequently driven by several agents at once. That is the shape that produced this release's bug list — one diagnostic session caught a daemon serving 11 worktrees across 14 concurrent connections and 18 concurrent compiles, heap pegged at 12288/12288MB.

Nearly every bug fixed below is invisible to a single developer running a single build in a single checkout. One client's disconnect killing another's test forks; an IDE compile stalling a CLI compile of the same project; a heap-pressure gate that consulted a per-connection count and so concluded every one of a dozen clients was the only compile running; analyses accumulating across every workspace the daemon had ever served. You do not find these unless the daemon is genuinely shared and genuinely saturated, and you do not fix them unless it hurts every day.

That is the case for bleep on a large codebase, and it is a narrow one worth stating precisely: a build tool whose daemon is shared across every workspace on the machine can keep a dozen worktrees warm at once, which is exactly what parallel development at this scale needs — and it is also the thing that has to be correct under contention before it is worth anything. M11 is the release where that stopped being aspirational.

⚠️ Migration: everything is invalidated. Expect one full recompile per worktree.

Two on-disk changes land in this release, and between them no compile state from M10 or earlier is reusable.

Compile output moved. Per-cross-project state now lives under .bleep/projects/<cross>/, shedding the .bloop path component the old layout carried:

.bleep/builds/<variant>/.bloop/<cross>/classes     # before
.bleep/projects/<cross>/builds/<variant>/classes   # now

Source-like outputs (generated sources, generated resources) are shared across variants; build state (classes, Zinc analysis, KSP caches) is per-variant. A side benefit: generated source paths no longer collide between the Normal and BSP variants.

The Zinc analysis format changed, with zinc 2.0.4 writing consistent, reproducible analyses.

So every local analysis.zip and every remote-cache entry is invalid, and old and new clients cannot share remote-cache entries during a rollout.

Delete the whole .bleep directory in each worktree before your first M11 build:

rm -rf .bleep

Not housekeeping — leaving it in place causes build failures. The old tree holds generated sources at their v1 locations, and a project whose sources come from both layouts sees every generated file twice, so the compile stops with BleepVersion is already defined (or the equivalent for whatever your build generates). The analysis and class directories in there are already invalid for the reasons above, so there is nothing in .bleep worth keeping. It is rebuilt on the next command.

The scheduler: one loop, one budget

Compiles and forked JVMs (tests, sourcegen, KSP, platform linkers) all run on the same cores and the same RAM, but each subsystem used to gate itself independently — compiles took one semaphore, each client's test run took its own, and forked JVMs had no memory bound at all. On an 18-core / 48GB machine that arithmetic came to 216GB requested. OOM was not a scheduling failure; it was subtraction.

MachineResources is now the single authority. Every CPU- and RAM-competing operation reserves against one daemon-wide governor, and admission happens in the DAG interpreter — the one loop that already knows what should run next. Ready tasks are sorted most-unblocking-first and admitted in that order; anything that does not fit stays ready and is reconsidered the moment a task completes, which is precisely when resources come back. Previously the sort was decoration: everything past maxParallelism queued FIFO inside the governor, so the instant anything had to wait, priority stopped mattering.

Every fork now carries an -Xmx (2GB by default). It never did before — testRunnerMaxMemory defaulted to None, so HotSpot handed each fork MaxRAMPercentage=25, a quarter of the machine. A scheduler gets no say in what a process allocates, so no amount of accounting could have fixed that. On the reported 48GB machine, forks drop from 12GB to 2GB and roughly ten run concurrently where exactly one did before. CPU is very nearly the binding constraint again, which is the point.

Three deadlocks fell out of unifying this, each real and each reproduced:

  • Test forks were charged CPU twice — once by the interpreter at admission, once again by JvmPool.acquire from inside the already-admitted task. Fine until saturation, at which point every permit is held by a task queueing for a permit that cannot exist: cpu 18/18, running 18, waiting 18 with zero fork processes and an idle compute pool. Because machine CPU is daemon-wide, it starved unrelated workspaces too — compiles in another worktree sat behind parked test forks for 20+ minutes.
  • A zero-cost reservation waited on an over-commit. A compile reserves a core and zero fork-memory. When the dynamic budget retuned below current reservations (which is how we stop admitting more without evicting what runs), free memory went legitimately negative — and freeMem >= 0 is false, so every compile in the build was refused on account of a memory over-commit compiles do not contribute to. A request that consumes nothing of a dimension now always fits along it.
  • The budget was a positive feedback loop. Computed relatively, each admission inflated holding immediately while the fork's actual memory only appeared seconds later — so admitting work manufactured the evidence that there was room for more of it. The daemon log shows it climbing 19.7GB → 34GB → 48GB → 63GB of budget on a 48GB machine. Restated absolutely (physical, minus what other processes hold, minus slack), the loop cannot form.

The budget is also dynamic now: MachineMemory asks the OS what it can spare without swapping, and the governor retunes every few seconds. The question deliberately asked is not "how much is free" — on a modern OS that is near zero whether the machine is idle or desperate — but how much is held in memory that cannot simply be dropped.

One knob. The resource surface had grown to fifteen subcommands, and parallelism existed in the config model with no CLI at all while max-concurrent-compiles was CLI-only and undocumented. Two vocabularies for "how much of this machine may bleep use", and the documented one was unreachable. Now parallelism is the knob, it has a CLI, and the governor is sized from it — cores are only what it defaults to. heap-pressure-threshold and max-cached-workspaces are demoted to internal. See resource management, which gains a worked "small machines and CI" section.

Where the heap actually went

A daemon serving eleven worktrees OOM'd at -Xmx12g, and its post-GC floor climbed monotonically from 3.5GB to 8.2GB as workspaces were served. Rather than guess, the live server was measured: a class histogram showed ~4.5GB sitting in xsbti.api.* — 31.7M NameHash, 31.1M Id, 839K retained AnalyzedClass — and a forced full GC moved the live set by 68MB.

That was ZincBridge.analysisCache: unbounded, daemon-lifetime, keyed by analysis file path so it accumulated across every workspace and project. It was softly referenced on the theory that GC reclaims it under pressure. It does not — soft references are not cleared until the collector is nearly out of room, and until then they are indistinguishable from live data.

Three changes address it together:

  • Zinc 1.12.0 → 2.0.4, as zinc_3, so the for3Use213 shim goes away.
  • ConsistentFileAnalysisStore(reproducible = true) at all three sites, so identical inputs produce byte-identical analysis files — normalising the timestamps that otherwise make two worktrees' analyses differ while describing the same code.
  • AnalyzedClass is interned at deserialization, with weak values and a ReferenceQueue expunge so the interner is an index and not a retainer. Measured on real analyses before building it: 2.06x sharing within one large workspace, 2.96x within bleep's own, and — the control — a second copy of an identical workspace adds exactly zero distinct instances. Divergent branches share little; a freshly forked worktree shares everything. Against that 4.5GB, it is 2.2–3.0GB back.

Analyses also belong to their workspace now, via a shared WorkspaceKey. Evicting a build from BuildCache used to free its resolved Started — hundreds of MB — and leave that same workspace's analyses resident, which is the multi-GB part. We freed the small half and kept the large one.

Two more allocation fixes worth naming:

  • The ECJ classloader is reused across compiles. ECJ keeps the JDK module image in statics, and statics are per-classloader, so a fresh URLClassLoader per compile meant re-opening ct.sym as a zip filesystem and re-reading every JDK class it touched, every time. Isolated A/B on 800 files: 8200ms → 3300ms. End-to-end on a 2265-file Java project: -8.8%. The saving is per compile invocation, so builds with many Java projects pay it once per project.
  • macOS process footprint via proc_pid_rusage instead of spawning /usr/bin/footprint per pid every 5 seconds. Same reading, 11µs per call against 27994µs — about 2500x. (RSS was the tempting cheap fix and is wrong: on a 6GB-heap JVM phys_footprint reported 7190MB while RSS reported 2933MB, because macOS compresses pages and RSS stops counting them.)

The default heap is now min(16g, max(4g, RAM/4)) and is stated at boot, because the default surprises: it clamps to its 4g floor at or below 16GB of RAM, so a 16GB laptop and a 14GB CI runner are sized identically.

Daemon lifecycle

A healthy daemon could be declared dead, permanently. waitForServer decided readiness by grepping the shared output log for "listening" — but startServer rotates that log on every start attempt, so a second client racing to start, losing the lock, and exiting had already moved the live daemon's "listening" line into output.prev. From then on every client timed out against a daemon that was in fact answering build/initialize in ~50ms. Whole socket directories got bricked for everyone sharing them. Readiness is now a successful connect — exactly what the client ultimately needs, and immune to log rotation.

That principle generalised: the socket is the single source of truth. ensureRunning is now connect-or-spawn, and the pid-liveness/zombie/lock-file branch tree that reimplemented — and regularly disagreed with — what the socket already knows is gone.

Idle self-shutdown, default 60 minutes. Abandoned daemons otherwise linger for days across closed worktrees and version bumps; we found a jdk-25.0.0 daemon still resident beside the current jdk-25.0.1 ones.

bleep config compile-server idle-timeout 60   # minutes; 0 disables
bleep config compile-server idle-timeout-clear
bleep config compile-server read-timeout 30   # per-connection idle read

Single-spawner election ends the cold-start fork storm — a swarm of clients each forking a server JVM swamps the machine and starves the shared build server, so even unrelated compiles stall. A JVM-wide semaphore excludes concurrent fibers in one process; a FileChannel advisory lock excludes separate client processes, releasing automatically on channel close or process death. Worth noting how this landed: two earlier attempts were built, measured with the new bleep bsp-stress harness, found wanting, and reverted — before it turned out the harness itself was the problem. It simulates clients as fibers in one process, and POSIX file locks do not exclude threads within a process, so tryLock looked broken when it is in fact correct across processes. Verified directly: of 30 concurrent processes calling tryLock, exactly one acquires.

The daemon no longer inherits BLEEP_* from whichever client spawned it. ProcessBuilder.environment() starts as a copy of the spawning process, so a single BLEEP_FOO=… bleep compile became permanent state on a server that then served every other client. Demonstrated accidentally while writing the fix: one BLEEP_PARALLELISM=3 bleep compile pinned that value into the shared daemon, and unrelated test forks inherited it minutes later.

This is one half of a pair. The daemon's environment is the wrong channel for anything that belongs to one client — it is long-lived, shared, and set by whichever shell happened to cold-start it. So it stops carrying that data, and the other half — "your environment reaches your tests", under Features below — gives your environment an explicit route to the forked child instead. Non-bleep variables (PATH, HOME, proxies) are left alone — they describe the machine, which the daemon genuinely does share.

semanticdb-javac is now always on the server classpath. It is part of the JVM key naming the socket directory, and only the IDE path added it — so an IDE session and a CLI session at the same bleep version each spawned their own daemon. Two JVMs, two heaps, no shared compilation state, observed live.

One daemon, many clients

A family of bugs where state scoped to one BSP connection was used as though it were daemon-wide, or vice versa. All of them predate the multi-workspace work and are wrong under any ownership model.

  • One client's disconnect SIGKILLed every other client's forks. cancelAllActiveRequests ended with killAllChildProcesses(), walking ProcessHandle.current().children() — and current() is the daemon. A cleanup path scoped to one connection reached across all of them, destroying healthy in-flight sourcegen and test JVMs. It hid well: destroyForcibly bypassed kill-reason tracking, so the victim reported "killed by SIGKILL … not by bleep" and read exactly like an OS memory kill. It was measured and it wasn't — the killed fork sat at a flat 160MB RSS with 8GB free, and kills clustered in the 3.4–5.2s window it takes another client to disconnect.
  • ProjectLock.releaseAll() cleared process-global lock state from a per-connection finalizer, so any client disconnecting released exclusive locks still held by compiles on other connections, which then kept writing to directories they no longer owned.
  • ProjectLock keyed its bookkeeping by project name while the cross-process lock keyed on the target directory. A normal-variant and a bsp-variant compile of the same project write disjoint directories, but contended on one lock — so an IDE compile could stall a CLI compile of the same project for the full 5-minute timeout.
  • The KSP mutex map was documented as "serializing KSP runs across concurrent BSP connections" but was an instance field of a per-connection object, so it serialized a project against itself and nothing else. KSP writes to a variant-shared directory and its cancellation path deletes those outputs.
  • handleCleanCache was deleting classes/ and .zinc/ with no lock at all, concurrent with reads on other connections.

The client owns the build

The server used to load and resolve builds for itself, which meant it was guessing at what its clients wanted. That is now inverted end to end.

The client ships fully resolved projects in build/initialize; the server wraps them and uses them as given, and no longer reads bleep.yaml at all. Builds have an identity — BuildId, a SHA-256 over the payload printed with sorted keys — and are cached per daemon, keyed by (workspace, variant). Same id is a hit; a different id replaces the entry and logs both short ids, so adoption is observable rather than silent. Previously a one-shot bleep compile re-resolved the entire build on every invocation, while a client initializing with a different build silently got the old one.

bleep/buildChanged lets a live session correct its build afterwards. This made the MCP server quietly wrong: it watches bleep.yaml, reloads its own state, and holds one connection for the whole session — so after any build edit it reasoned about build N while the daemon kept compiling build N-1.

bleep bsp — the command your IDE launches — stopped being a byte pipe. It read the first message only to pick a semanticdb version and then copied bytes both ways, leaving IDEs as the last clients whose build the server had to guess at. It is now a real client: it loads and resolves the build, hands it over at initialize, watches the build files for the session's lifetime, and pushes buildChanged when they change. workspace/reload re-resolves and pushes before forwarding.

buildTarget/run is implemented rather than advertised-and-failing. The server declared runProvider and canRun and then threw MethodNotFound — no bleep client was affected, but Metals' run code lens believed the advertisement, so it was broken rather than absent. Cancellation escalates the way Bloop's forker does (destroy, 200ms, destroyForcibly) so a shutdown hook gets to run.

And diagnostics reset properly in Metals now: the memory of which files had been reported broken was scoped per-operation, but the compile that fixes an error is a different operation from the one that reported it. So the empty reset=true was never sent, and an error you had already fixed stayed on screen until something else happened to republish that file.

Silently green builds

Several independent paths let bleep report success over code that does not compile, or over tests that never ran. Each is closed, and each has a regression test.

Zinc under-invalidation — four defects in the incremental up-to-date path:

  1. A cancelled compile wrote a poisoned noop manifest. Zinc swallows CompileCancelled and returns (hasModified=false, previous), indistinguishable from up-to-date by return value, so the bridge stat'd the edited sources and recorded them as up-to-date against the old analysis. The edit stayed invisible until its content changed again.
  2. Source directories were stat'd non-recursively while output directories were walked recursively — in the same function. Adding a file in a nested package directory (exactly what code generators produce) bumped no recorded mtime.
  3. dependencyAnalyses passed only direct dependsOn edges while the compile classpath is transitive, so an API change two hops upstream was invisible — doubly so because an intermediate noop project never rewrites its analysis mtime.
  4. The cycle guard misread Zinc's detectInitialChanges probe as a re-invalidation cycle, tripping a bogus "Zinc incremental compilation bug" warning on every clean build.

Orphaned .tasty files. Zinc tracks .class files as the products of a source and deletes them when the source goes away; it does not track the sibling .tasty. Harmless under Scala 2, not under Scala 3, where .tasty is what the compiler reads to recover symbol definitions. It bites on an ordinary refactor: move a file between projects and the old project keeps serving the old API from its orphan, so dependents compile against a stale definition and the compiler reports missing members on a class whose source plainly has them. Nothing warned you and nothing but bleep clean cleared it. (This stops new orphans; a build directory that already has one needs one bleep clean of that project.)

A discovered-but-unexecuted suite reported PASSED with exit 0 — indistinguishable from success to any CI gate. Five contributing causes, closed at each layer plus a global backstop, including a JUnit 4 class routed to JUnit Platform with no vintage engine (matched no engine, executed nothing, reported green) and a cancelled suite leaving unconsumed protocol lines in a re-pooled JVM's stdout, so the next acquirer read the previous suite's terminator.

Underneath that, the (passed, failed, skipped, ignored) tuple is gone. Its all-zero value meant three unrelated things — a suite with no tests, a suite whose tests never ran, and an errored suite — which each layer then re-disambiguated with ad-hoc count arithmetic. It is now an explicit ADT:

SuiteOutcome = Executed(p, f, s, i) | Empty | NoFrameworkMatched | Errored(msg, thr)

Only the forked runner can tell these apart, so it originates there and is carried verbatim to the summary. Empty / NoFrameworkMatched / Errored each count as a failed suite with a distinct, readable reason — never a green "0 passed".

Cancellation now answers honestly. A $/cancelRequest that beat its own request was dropped on the floor, so the build ran to completion and reported Ok for something the user had cancelled. And a cancelled run answered -32603 Stream closed instead of Cancelled, because Process.destroy() closes the parent's pipe ends and the parked reader woke in getBufIfOpen and threw before the status was computed. Both orderings now converge on the right answer.

A task finishing between two state reads could be admitted twice. The scheduler snapshotted the DAG before the running set, while a completing task writes the other way around — so a task finishing in that window was visible in neither snapshot and got admitted a second time. Caught on CI by two identical link events 1ms apart; in production this is any task kind running twice.

Kotlin

Incremental compilation had never engaged in any configuration bleep shipped. Every Kotlin build was a full compile and nothing said so. Three independent bugs, each fatal alone: a runner class looked up with an arity no shipping compiler declares; ChangedFiles.Unknown, on which the compiler returns RequiresRebuild immediately; and incrementalCompilation never set on the arguments, so the trackers were dropped and the caches were never written. Plus the classpath hash living inside a directory Kotlin wipes on rebuild, so cross-language change detection erased its own state. Both fallbacks printed to a debug() gated on a flag set nowhere in the tree, which is why it was invisible.

Verified against every stable 2.x: IC engages on all ten releases from 2.2.0 up. compile gained a trailing parameter in Kotlin 2.4, so its arity is now discovered rather than pinned — pinning was the original bug, and without this bleep could not compile Kotlin 2.4 at all. Tests assert the emitted constant pool rather than merely that caches exist, because a stale class surviving an incremental round goes green while the bytecode disagrees with its source.

Separately, eight sites invented a language or platform version at compile time while two others correctly threw. All eight now throw, and BuildValidation reports every offender at build load rather than crashing mid-compile.

KSP ships, via the standalone Analysis-API runner rather than as a kotlinc plugin: kspVersion, symbolProcessors, symbolProcessorOptions, and scanForSymbolProcessors under kotlin:. Resolution and execution are a per-project DAG task with per-file change tracking. Mixed Kotlin+Java compile order flipped to kotlinc-first so the KSP→javac path round-trips.

bleep fmt gained a Kotlin branch backed by ktfmt, configured through an optional .kotlinfmt.conf mirroring .javafmt.conf. Default style is kotlinlang; google and meta are available.

Windows

M9 and M10 crashed on the first command with MissingForeignRegistrationError (#601). The native-terminal reachability metadata declared only the Unix downcalls, and none of the six kernel32 descriptors in the multi-release jar were registered. GetStdHandle is the first handle built, so the process aborted before bleep did anything. CI never caught it because the kernel32 path is guarded on being a terminal, and GitHub Actions pipes stdout — no runner is ever a console. selftest now reaches kernel32 unconditionally.

Fixing that turned the Windows job red and revealed why it had been green: the step was shell: cmd with a multi-line run:, and cmd propagates only the last command's exit code, so bleep test was discarded entirely. 42 tests had been failing on every green run. All 42 are fixed; two were genuine production bugs (a ProjectDigest that hashed differently across OSes, and an sbt path spelling that Windows read as a UNC path).

Windows now runs the same jvm3 suite as every other arch — the compiler, BSP, process-spawning and linker machinery, which had never executed there once. 177 tests → 760.

Every bleep invocation used to shell out to PowerShell and compile C# with Add-Type at runtime, just to learn where its own cache and config directories live — once at startup and from five more places in the BSP server, none memoized. That is startup cost and six hard failure points (Constrained Language Mode, AppLocker/WDAC, a non-writable TEMP, an inherited PSModulePath). The fix already existed in coursier but was unreachable: the jar carrying the FFM implementation had no MANIFEST.MF, so no Multi-Release: true, so the JDK never loaded it. coursier 2.1.25 splits it out correctly, and bleep now resolves directories through SHGetKnownFolderPath on JDK 22+. Verified per-JDK rather than assumed.

Also: CI unified on one shell for every OS; directory handles leaked in test teardown across 29 files (Files.list materialised without closing — invisible on POSIX, "delete pending" on Windows); server logs decode leniently rather than throwing on the first byte that is not UTF-8; and the Kotlin/Native prebuilt download gained a Windows entry at all (it had been silently downloading the linux-x86_64 distribution), plus timeouts, atomic writes, and self-healing on a truncated archive.

Telling you what happened

GitHub Actions annotations. Compile errors and test failures now land on the offending line in the PR diff, with a markdown summary on the job page. Getting there required making diagnostic positions structural: Diagnostic flattened position into a single file:line:col string that every consumer took apart again, and BuildDiff.diagKey recovered the file with indexOf(':') — which on C:\proj\src\Foo.scala:12:5 returns "C". Every diagnostic on Windows shared one dedup key, so a fixed error in one file cancelled out a new error in another, on every Windows build, silently. Test failures carry a source location recovered from the first stack frame belonging to the suite class, which is the failing assertion for ScalaTest, munit and utest alike with no framework-specific code.

The metrics dashboard was throwing away the answers. server-metrics and the CI summariser read the same file and had drifted: of twenty event types the server writes, the dashboard handled fourteen and dropped six via case _ => () — and the six were the ones people arrive with questions about. Five charts and six cards added. Scheduling now draws CPU-in-use against cores with running and queued on top, because saturated and starved look identical from outside and have opposite fixes. server-metrics --file opens any metrics.jsonl, so one downloaded from CI uses the same dashboard as a local one.

heap_pressure_stall was almost never heap pressure. Across two full CI runs, 74 of 74 events were recorded below the pressure threshold; the server never crossed it, peaking at 65–69% of a 4GB heap against a 0.80 gate. Every one was the admission gate's deliberate stagger — reasonable behaviour wearing a name that says the build was starved of memory, in an artifact whose whole purpose is explaining a slow build. Renamed admission_defer, carrying reason, plus delay_ms (which the gate computed and then dropped, so the event could not say whether 36 defers cost 7 seconds or 70) and others_compiling (the value the gate actually consulted, where it previously recorded a different counter that disagreed).

Forked test JVMs were recorded nowhere. The server logged what it compiled; what it ran was invisible. Five events joined on pid, charted as one lane per JVM with its suites on it — which answers "which suite was on the JVM that got killed" without a hand-join across three event types. fork_end carries killed_by separately from the exit summary, because destroyForcibly sends SIGKILL and a fork bleep killed is otherwise indistinguishable from one the OS killed.

That last point deserves its own paragraph, because it cost real time. The OS log recorded no memory kills during a run where 35 forks reportedly died of memory pressure — bleep was killing them, from six different places, and describeExit read exit 137 and asserted "sent by the OS … almost always the kernel reclaiming memory under pressure". Being confidently wrong is what made it expensive: a day of work on fork footprint multipliers, OS-reserve tuning and shared-page analysis was aimed at a failure that was never memory. Every kill site now passes a reason, and only a death bleep did not cause may be attributed elsewhere — offered as a hypothesis to check against the system log, not asserted.

Also: OOM deaths no longer freeze the JVM for minutes writing a ~5.7GB heap dump (22 of them, 116GB, nothing ever deleting them); a crashed run says FAILED instead of reporting "52 compiled, 0 failed" with Build failed: underneath it; the crash log survives the restart that was supposed to diagnose it (cleanup used to delete its own evidence); and stop-all now names each directory it deletes with its size, at warn level past 1GB, after it silently removed one holding 10GB of heap dumps.

Features

Test tags — framework-independent, because filtering happens at bleep's own suite-dispatch boundary rather than through framework-native tags. Works for ScalaTest, JUnit, munit, utest, anything the runner discovers.

projects:
  bleep-tests:
    testTags:
      slow: ["**IT"]
bleep test --only-tag slow --exclude-tag flaky

A single * stays within an FQDN segment; ** spans dots. Includes union, excludes subtract, and --only composes with --only-tag as AND. bleep list-tests annotates each suite with its tags and warns about manifest patterns matching nothing; the summary reports the projects-selected ratio and which filters were active. When a filter empties the set, the error walks the pipeline so you can tell which stage did it. Documentation.

A file:// remote-cache backend, for sharing compiled state between git worktrees on one machine. The cache only spoke S3 before, with credentials required unconditionally. Writes are atomic, keys are checked against escaping the root, and file:// skips credential resolution entirely.

Your environment reaches your tests. FOO=bar bleep test now works. It could not before: tests are forked by a long-lived daemon whose own environment belongs to whichever shell cold-started it, possibly days ago in another directory, so whatever you set in front of bleep test simply never arrived.

The client now sends its environment on the BSP request, and the daemon applies it as an overlay on the forked child only. Deliberately not by mutating its own environment — that would race between concurrent requests and leak one workspace's values into another's test run, which is the same reasoning behind the daemon dropping inherited BLEEP_* above. The hole was already there: JvmPool applies an env map to the child and already hashes it into the fork's identity, so a differing environment forces a distinct fork rather than reusing a poisoned one. Every call site simply passed an empty map.

Applied on every platform — JVM, Scala.js, Scala Native, Kotlin/JS, Kotlin/Native. The four non-JVM runners had accepted an env map all along, and because nothing ever passed one they were not honouring platform.jvmEnvironment either. Same omission in all three bleep run paths, which additionally ignored BSP's standard RunParams.environmentVariables outright.

Precedence, weakest first: NO_COLOR=1, your shell, then the build. The build deliberately outranks the ambient shell, because the environment is forwarded wholesale and letting it win would let a stray AWS_REGION in someone's profile silently override a region the build states on purpose — failing on exactly one machine. CLASSPATH is never forwarded (it is the long-classpath channel on Windows, and a forwarded copy would replace the test classpath); PWD/OLDPWD/_ are dropped as stale shell bookkeeping.

Scripts accept --flags. Trailing arguments were parsed by something that rejects any token starting with --, so bleep myscript --foo bar failed with "Unexpected option" and scripts had to resort to environment variables.

bleep-plugin-spring-boot — run, repackage (fat JAR with layered indexes, launcher class, loader manifest), and build-info.properties sourcegen for Actuator, as three plain Java classes wrapping spring-boot-loader-tools. Shipped alongside a realistic worked example and an integration test. The point is the existence proof: Spring Boot's Maven plugin is the canonical "one declaration activates a graph of bespoke build behavior", and here it is as regular classes with regular methods. See Spring Boot proves the model and the maven plugin coverage survey, which walks the top 50 Maven plugins and finds 48 covered.

A Java publishing API for bleepscript — packaging, publish targets (local ivy, Maven folder, named resolver, Sonatype Central), and wrappers over bleep's process runner and Coursier resolver, so a script in any of the three languages can fetch classpaths and fork JVMs through the same cache and logger bleep itself uses.

MCP reconnects when the daemon goes away underneath it. It used to hold one connection for the whole session, and lsp4j never recovers — observed in the wild failing every compile for three days while a healthy daemon served other workspaces. Worse, those failures surfaced as a bare -32603 Internal error, because the transport maps any unrecognised throwable to a fixed string, so the actual message never reached the agent. All tools now return proper MCP errors carrying the message and cause chain.

Under the hood

  • .bleep/ layout v2 (see the migration note above). State is keyed project-first — .bleep/projects/<cross>/ — instead of variant-first with a .bloop directory inside it, a holdover from when bleep drove Bloop rather than compiling for itself. The split that matters is source-like outputs shared across variants against build state held per variant, which is what stops generated source paths colliding between the Normal and BSP variants. BuildPaths is now the only authority for any of these locations; the server used to derive compile output from one source and the Zinc analysis directory, compile lock and test classpath from another, which agreed only by convention.
  • Every hardcoded version lives in one file (model.Versions). They had been scattered across four modules and had already drifted — the BSP server's java-semanticdb fallback disagreed with the client default, and the compiler-bridge lagged the zinc in bleep.yaml. Everything then bumped to current: Scala 2.13.18 / 3.8.4, Scala.js 1.22.0, Scala Native 0.5.12, Kotlin through 2.4.10, ECJ 3.46.0, ktfmt 0.64, scalafmt 3.11.4, Node 24.18.0, JUnit 5.14.4.
  • Version schemes travel with the libraries they are claims about. A scheme set on a project did not reach its consumers, even though dependsOn pulls that project's libraries into their resolution — so a project could resolve cleanly while every consumer failed on a conflict it neither introduced nor could see.
  • -Werror and tpolecat strict mode for every Scala project, with the last three opt-outs dropped; -source bumped 3.4 → 3.8 with migration rewrites applied across the tree.
  • bleep publish derives snapshot versions the way everything else does. It was constructing its version with dynverSonatypeSnapshots = false while the two other sites that derive from git both pass true — so publish local-ivy wrote jars at one coordinate while the client built from that same tree asked coursier for another. Nothing failed loudly; the client simply never resolved the jars it had just published and carried on against the last released server.
  • CI publishes usable dev builds. The native images CI produced were downloadable but not usable, because a bleep binary asks coursier for bleep-bsp at the version baked into it and that version was never published anywhere. CI now keeps the jars and a snapshot version resolves the whole set — binary and jars — from the CI run of its commit.
  • bleep bsp-stress, an adversarial harness that fans out N concurrent clients against a throwaway socket directory while a chaos fiber kill -9s the daemon on an interval, reporting success rate, latency, respawn coverage and peak concurrent daemons.
  • The dead single-workspace BspServer is deleted and the BSP integration suite now drives the production server through the same payload every real client sends, instead of installing build state through a back channel that existed only for tests. ReactiveTestRunner (547 lines nothing called) is gone too.

Full Changelog: v1.0.0-M10...v1.0.0-M11