docs: name the endpoint and streaming behavior agent.md left out - #79
Merged
Conversation
The reconciliation list's fetch step just said "fetch the artifact,"
with no endpoint, no mention that it rides the same mutual TLS as the
poll, and nothing about how the download is handled once it arrives.
Someone reading agent.md specifically to answer "how does the agent get
its artifact" had to already know to cross-reference publishing.md for
the URL, and the streaming behavior was not written down anywhere at
all, only visible in ExtractArchive taking an io.Reader.
Step 3 now names GET /v1/artifact/{environment}/{code_id} and the shared
mutual TLS. Added "The fetch is streamed, not staged": the response body
is piped straight into the gzip/tar extractor with no intermediate file
on disk, and the gzip is the archive format itself, not an HTTP
Content-Encoding layered on top. Verified against source before writing
it: gzip.NewReader wraps resp.Body directly, tar.NewReader reads it
header by header, no buffering step anywhere in between.
Signed-off-by: Michael Harp <mike@mikeharp.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
miharp
added a commit
that referenced
this pull request
Aug 2, 2026
#79 (streaming) and #80 (crash cleanup) were built in parallel, so the "fetch is streamed" section couldn't yet point at the crash-handling section it's really asking about. Now that both are on main, restore the cross-reference and drop the "or left behind by a crash mid-download" claim from the streaming section, since it was really about the artifact never being staged as a plain file, not a claim about what happens to the temp directory on a crash -- which is a different, now-documented, question. Signed-off-by: Michael Harp <mike@mikeharp.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
miharp
added a commit
that referenced
this pull request
Aug 2, 2026
download's temp extraction directory is cleaned up by a defer, which never runs on a hard kill: SIGKILL, an OOM kill, a power loss. Whatever partial extraction existed at that instant was left on disk, and nothing in the codebase ever noticed. reap explicitly skips anything dot-prefixed, by design, since it cannot tell an abandoned extraction apart from one a concurrently running agent is still writing into, and deleting a live one out from under it would be worse than leaking one. The next reconciliation does not help either: sync only checks whether the final directory exists, and calls download again on a fresh, differently-named temp directory each time, with no awareness the old one is still there. So a crash mid-download left a permanent leak, growing by one directory per crash, forever. Age is what actually distinguishes the two cases. A live extraction keeps creating entries under its directory, which keeps bumping that directory's own mtime; an abandoned one stops the instant the process dies and never moves again. reapStaleExtractions sweeps a dot-prefixed directory once it has sat untouched past MinAge, the bound already used elsewhere in reap for "how long before we are sure nothing still needs this", rather than introducing a second, unconfigurable threshold. performance.md's fixture unpacks in well under a second, so even the 2-hour default leaves an enormous margin before this could ever mistake a live extraction for a dead one. Called once per Once() cycle, fleet-wide rather than per-environment, since an abandoned extraction is not tied to converging any particular environment. Documents the fix in agent.md's Reaping section and the disk-sizing math in production.md, which previously had no way to account for this, and connects it back to the streaming section added in #79 now that both are on main. Signed-off-by: Michael Harp <mike@mikeharp.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Came up answering "what's the transport for getting the sealed artifact onto the agent, and is that well documented?" The design rationale (pull not push, resealing not checksum, mutual TLS as the model) is well covered — but the actual mechanics of the fetch step were thin, and split across files in a way that made them hard to find.
agent.md's reconciliation list — the doc specifically about the compiler side — just said "Fetch the artifact, extract it to a temporary directory." No endpoint, no mention it rides the same mutual TLS as the poll one line above it. The endpoint (GET /v1/artifact/{environment}/{code_id}) is documented, but only inpublishing.md, written from the publisher's side. And two mechanical details weren't written down anywhere at all:Content-Type: application/gzipis the archive format, not an HTTPContent-Encodingthe transport adds.What changed
Verified against source before writing it
ExtractArchive(internal/seal/archive.go):gzip.NewReader(r)wraps theio.Readerdirectly,tar.NewReaderreads it header-by-header viatr.Next(). No buffering step anywhere between the socket and the extracted files — confirmed this is genuinely a streaming pipeline, not just calling it one.Checks
markdownlint-cli2andcspellclean.🤖 Generated with Claude Code