fix(orphans): stop false-positives + surface EIP leaks; doc idle-stop cost (#262) - #273
Merged
Merged
Conversation
… cost (#262) Stopped instances silently keep billing (EBS + any attached Elastic IP), and 'spawn orphans' — the safety net for that — both under- and over-reported. Part 1 — orphans false-positives: state enrichment issued one batched DescribeVolumes/DescribeInstances; EC2 fails the whole call if any id is already gone, blanking every state, and a blank volume state was treated as an orphan (so one deleted volume made orphans list them all). Add per-id NotFound fallback (survivors keep real state; gone ones marked 'deleted') via new isVolumeNotFound, and only 'available' volumes are orphans now. Part 2 — EIP visibility (report-only): scanAddresses surfaces unassociated EIPs and EIPs on stopped spawn instances as 'address' orphans; 'spawn status' reports an instance's attached EIP (info while running, billing warning while stopped). Core invariant: spawn allocates no EIP so it NEVER releases one — RemoveResource refuses 'address', cleanup filters them out with a release-address hint. Part 3 — docs: --on-complete/--hibernate-on-idle help note stop's ongoing cost and recommend terminate for batch/headless; README 'Bounding cost' section + resources/orphans/cleanup commands. Tests: per-id fallback, classifier, RemoveResource-refuses-address, and a substrate-backed GetInstanceElasticIP. MINOR release. Fixes #262
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…6-5764) Newly-published advisory GO-2026-5764 in aws/protocol/eventstream (transitive via service/s3 and service/lambda) failed govulncheck on these two Lambda submodules. Bump eventstream->v1.7.8, s3->v1.97.3, lambda->v1.88.5; go mod tidy. No code change. Unrelated to the #262 fix but required to green CI.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #262. A stopped instance keeps billing for its EBS volumes and any attached Elastic IP, and
spawn orphans— the "at minimum, warn" safety net — both over-reported (listed already-deleted volumes) and under-reported (never surfaced EIPs). Confirmed live on v0.68.1.Core invariant
spawn releases only what it alone creates. An exhaustive repo sweep confirmed spawn never allocates an Elastic IP (it uses auto-assigned ephemeral public IPs, freed by EC2 on stop/terminate). So every EIP attached to a spawn instance is a user-allocated static address, and spawn must never release it (release is irreversible). EIP handling here is therefore reporting-only.
Part 1 — orphans false-positives (bug)
enrichVolumeState/enrichInstanceStateissued one batched Describe call; EC2 fails the whole call withInvalidVolume.NotFound/InvalidInstanceID.NotFoundif any single id is already gone, which blanked every resource's state — and a blank volume state was classed as an orphan. So one deleted volume madeorphansreport every volume (incl. deleted ones).isVolumeNotFound(mirrorsisInstanceNotFound); on aNotFoundbatch, fall back to a per-id sweep so survivors keep real state and gone ones are markeddeleted.IsLikelyOrphan: only a genuinelyavailablevolume is an orphan (blank/deleted is not).Part 2 — EIP visibility (report-only)
scanAddresses(separateDescribeAddressespass) surfaces unassociated EIPs and EIPs attached to a stopped spawn instance asaddressrows inorphans/resources. An EIP on a running spawn instance, or on a non-spawn instance, is not flagged.spawn status <instance>reports any attached EIP (per user request): informational while running, a billing warning while stopped.RemoveResourcerefuses to release anaddress(defense in depth);cleanupfilters addresses out and prints anaws ec2 release-addresshint instead. spawn never destroys a user EIP.Part 3 — docs (no runtime default change)
--on-complete/--hibernate-on-idlehelp note stop's ongoing EBS/EIP cost and recommendterminatefor batch/headless; README gains a "Bounding cost" section and listsresources/orphans/cleanup.Tests
Per-id fallback (deleted volume doesn't blank survivors), address classifier,
RemoveResource-refuses-address, and a substrate-backedGetInstanceElasticIPend-to-end.go build/vet/gofmtclean;pkg/awsgreen.Note: the pre-existing
cmdTestCatalogValidfailure only reproduces locally (maintainer~/.spawn/catalog.yamloverlay binds recipe apps to private ECR images) — CI has no overlay and is unaffected by this PR.MINOR release.