Skip to content

fix(aws): a subnet's occupants are traversable - #1437

Closed
lex00 wants to merge 1 commit into
mainfrom
fix/1432-containment-traversable
Closed

fix(aws): a subnet's occupants are traversable#1437
lex00 wants to merge 1 commit into
mainfrom
fix/1432-containment-traversable

Conversation

@lex00

@lex00 lex00 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

reconstructEdges treats a containment rule as a boundary hint and emits no edge — unless the rule declares a traversal name. That carve-out is #1275's, added because "a fold's first hop is sometimes exactly a containment".

AWS::EC2::Instance -> Subnet was given a name. The network interface, which is the same relationship for the same reason, was not:

{ from: "AWS::EC2::Instance",         path: "SubnetId",, viaAttr: "SubnetId" }   // traversable
{ from: "AWS::EC2::NetworkInterface", path: "SubnetId",}                        // not

So kind:EC2::Subnet !<-kind:EC2::NetworkInterface — the query the grammar exists to express, and the first thing an agent reaches for — matched 23 of 23 subnets on an estate where 8 of 13 are empty. The negation was inert, and an inert negation is indistinguishable from one that searched and found nothing.

How it presented

--explain printed the tell and nothing read it:

— 23 of 23 AWS::EC2::Subnet/AWS::EC2::SubnetRouteTableAssociation matched
  (query: kind:EC2::Subnet !<-kind:EC2::NetworkInterface)

A negation term that excludes zero of everything is worth suspecting. (That line also shows kind:EC2::Subnet substring-matching SubnetRouteTableAssociation, which is documented behaviour and a separate conversation.)

On the aws-bench negative question set, two of three trials asked this query, believed it, and answered with every subnet including the occupied ones. The trial that passed did not believe it — it cross-checked with kind:EC2::NetworkInterface --show SubnetId, dumped graph --format ir to a file, and did the join in Python. A query language that is only right when the caller distrusts it is worse than one that cannot answer at all.

Scope, and what I deliberately did not do

Scoped to "occupies a subnet" — network interface and NAT gateway — matching the exception that already existed for the instance.

Containment into a VPC stays a boundary hint. My first patch made those traversable too and reference-catalog.test.ts caught it:

// containment is never an edge
expect(edges.some((e) => e.to === "vpc")).toBe(false);

That is an explicit, deliberate invariant. Flipping it to improve a benchmark number is not a change I should make as a side effect, so I reverted it. The cost is that vpcs-with-no-running-instances stays unanswerable by the same route — worth settling on its own terms, and I have left it in the issue rather than quietly deciding it here.

The same omission exists on the ELB, ECS and RDS containments. No estate here exercises them, so I left them rather than change what I cannot test.

Verification

Four tests: the ENI reaches its subnet, the NAT gateway does too, an empty subnet is still reached by nothing, and the containment boundary is still recorded — the traversal is added, not traded for.

Full suite: 10467 passed, 5 skipped, 0 failed. Lint clean.

Closes #1432

🤖 Generated with Claude Code

https://claude.ai/code/session_014KoduiMJRyLqjFYBUbMskE

`reconstructEdges` treats a containment rule as a boundary hint and emits no
edge, unless the rule declares a traversal name — the carve-out #1275 added
because "a fold's first hop is sometimes exactly a containment".

`AWS::EC2::Instance -> Subnet` was given a name. The network interface, which is
the same relationship for the same reason, was not. Neither was the NAT gateway.

So `kind:EC2::Subnet !<-kind:EC2::NetworkInterface` — the query the grammar
exists to express, and the one an agent reaches for first — matched 23 of 23
subnets on an estate where 8 of 13 are empty. The negation was inert, and an
inert negation is indistinguishable from one that searched and found nothing.
`--explain` reported "23 of 23 matched", which is the tell, and nothing else
said anything was wrong.

Measured on the aws-bench negative question set: two of three trials asked this
query, believed it, and answered with every subnet including the occupied ones.
The one that passed did not believe it — it cross-checked with
`kind:EC2::NetworkInterface --show SubnetId`, dumped `graph --format ir` and did
the join in Python. A tool whose query language is only right when the caller
does not trust it is worse than one that cannot answer at all.

Scoped to "occupies a subnet", matching the exception that already existed.
Containment INTO a VPC stays a boundary hint: `reference-catalog.test.ts`
asserts that explicitly ("containment is never an edge"), and widening it is a
separate decision from fixing an omission. That does leave
`vpcs-with-no-running-instances` unanswerable by the same route, which is worth
settling deliberately rather than as a side effect of this.

The same omission exists on the ELB, ECS and RDS containments. No estate here
exercises them, so they are left rather than changed untested.

Closes #1432

Claude-Session: https://claude.ai/code/session_014KoduiMJRyLqjFYBUbMskE

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
lex00 added a commit to INTENTIUS/chant-bench that referenced this pull request Aug 3, 2026
5/6, up from 3/6. The subnet question went 1/3 to 3/3; the VPC question is
unchanged at 2/3, which was predicted rather than hoped for.

Two more chant defects, both pre-existing and both found by measuring rather
than reading:

- **INTENTIUS/chant#1437** — `AWS::EC2::NetworkInterface -> Subnet` was declared
  as containment with no traversal name, so `reconstructEdges` recorded a
  boundary hint and emitted no edge. `kind:EC2::Subnet
  !<-kind:EC2::NetworkInterface` — the query the grammar exists to express —
  matched 23 of 23 subnets. An inert negation is indistinguishable from one that
  searched and found nothing.
- **INTENTIUS/chant#1440** — a subnet its own stack declares is also recorded as
  a dependency, so 13 subnets replayed as 16. A duplicate always reads as empty,
  because the edges resolve to the other copy: the three false positives were
  the three OCCUPIED subnets.

Each of the four fixes was necessary and none was sufficient. Measured against
the estate at each stage, the query returned 5 (one region visible), 7 (regions
merged), 11 (negation inert), 11 (duplicates read as empty), and finally 8 —
the correct 8, keeping us-east-1's occupied default-c out.

The VPC question stays at 2/3 because containment INTO a VPC is deliberately not
traversable — `reference-catalog.test.ts` asserts it — and flipping that to
improve this number is a decision for its own change, not a side effect of this
one. Left on #1437.

## The page now shows every run

A second run of an arm is usually a different build, and which build produced a
number is the whole question this repo exists to keep answerable. Showing only
the newest quietly dropped the evidence that the number moved, so both runs are
listed with their workspace fingerprints — which is what says they are not the
same experiment.

Refs #27

Claude-Session: https://claude.ai/code/session_014KoduiMJRyLqjFYBUbMskE

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
@lex00

lex00 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by the branch above — containment is now traversable by construction rather than by a per-rule viaAttr someone has to remember, which is what made NetworkInterface -> Subnet differ from Instance -> Subnet in the first place. Closing in favour of that.

@lex00

lex00 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #1452, now merged — containment is traversable by construction rather than by a per-rule viaAttr, which is what made NetworkInterface -> Subnet differ from Instance -> Subnet in the first place.

@lex00 lex00 closed this Aug 3, 2026
@lex00
lex00 deleted the fix/1432-containment-traversable branch August 3, 2026 21:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ambient resources are recorded with no edges, so the graph over-reports every "nothing points at this" question

1 participant