Phase 3 backlog: twenty user stories, five each (proposal)

[ARSH871-bot]

Twenty user stories, five each, written so we can argue about them before any
of them becomes work. Nothing here is assigned yet — this is a proposal.

US-2 through US-19 are already in use, so these run US-20–US-39.

How I picked them

I did not invent a wish list. Each story is one of three things:

1. a gap we have measured and written down (the policy gap, the client's export)
2. something the product needs before a stranger could run it in anger
3. one of the four algorithmic problems from Phase 2 (direction only): four algorithmic problems, one per person #240, finally written as a story
   with acceptance criteria instead of a paragraph of direction

They are split along the vertical slices we already own, so nobody is working
in somebody else's file.

A note on timing, which matters more than the stories

PDR submission is Thursday 10 September. None of this should start before
then. I am posting it now because the direction is easier to argue about when
it is written down, not because there is capacity for it this week.

---

Arsh — pipeline, access control, platform

US-20 — Check my configs in CI, before the change merges

As a network engineer, I want Netwise to run in my pipeline and fail the build
when a change introduces a high-severity finding, so a bad rule never reaches a
device.

• python -m netwise check <dir> exits non-zero when a found finding is at
  or above a configurable severity, zero otherwise
• --format json writes the F-1 findings to stdout unchanged
• A status="error" finding fails the build by default — we could not check
  must never be a silent pass in automation
• Documented as a GitHub Actions snippet in README.md

US-21 — Analyse my whole network at once, not device by device

As an engineer with forty switches, I want one scan over all of them, so I see
problems that only exist between devices.

• A snapshot of N devices produces findings scoped per device with no cross-talk
• Wall-clock time and memory recorded in docs/scale.md at 10, 50 and 100 devices
• The existing multi-device-10 fixture is extended rather than replaced

US-22 — Tell me only what changed since the last scan

As someone who scans weekly, I want the diff, so I am not re-reading forty
findings I already triaged.

• Findings persisted per snapshot with a content fingerprint, not an id
  (ids are not stable identity — this is Phase 2 (direction only): four algorithmic problems, one per person #240's problem C and it blocks this)
• Output partitions into new, resolved, unchanged
• Depends on US-33; ship after it

US-23 — Make access_control read my policy

As a user with my own network, I want the access-control check to use my rules,
so it stops reporting "could not check" on every device I own.

• access_control reads rules_in_use() exactly as policy_compliance does
• With no policy supplied, behaviour is byte-identical to today
• tools/stranger_config.py re-run and the FOUND column recorded here
• Half of The policy is ours, not the user's — there is no way to state your own #87. The other half is US-26

US-24 — Show me what Netwise could not read

As a user, I want to know which parts of my config were not understood, so I
know how much of the file the result actually covers.

• Parse coverage reported as a first-class number: lines recognised / total
• Unrecognised constructs listed by kind, never silently dropped
• Feeds the coverage section already in the exported report

---

Ankeet — routing, the AI layer, vendor support

US-25 — Let me ask a follow-up

As a user, I want to ask "and what would fix that?" after an answer, so I am
not re-typing the whole question.

• A question may reference the previous answer's resolved entities only
• The grounding guarantee is unchanged: every sub-query still runs for real
• question_understood shows the fully resolved question, not my shorthand
• This is Phase 2 (direction only): four algorithmic problems, one per person #240's problem B, scoped

US-26 — Make routing read my policy

As a user, I want routing checks against my devices, not rtr-hq and
rtr-branch.

• Same shape as US-23, same no-policy guarantee
• The two together close The policy is ours, not the user's — there is no way to state your own #87; either alone only halves it

US-27 — Analyse a vendor that is not Cisco, end to end

As an engineer with a mixed estate, I want Arista, NX-OS and Juniper to work
as well as IOS does.

• The three existing vendor fixtures run through the full pipeline, not just parsing
• Any check that is IOS-specific says so as status="error", per F-4
• A per-vendor support table in README.md, generated rather than typed

US-28 — Convert my whole pfSense export, including NAT

As the client, I want my real firewall analysed, not most of it.

• NAT rules modelled or refused individually, never file-wide
• The four open questions in Discussion Open questions for Senaka (his firewall, four of them) #294 answered first — this story is
  blocked on the client, and saying so is part of the story
• Closes the remainder of The client's real export: zero quick rules, multi-interface, and 1998 elements #78

US-29 — Explain related findings as one story

As a user looking at five findings caused by one bad rule, I want one
explanation, not five.

• Findings grouped by shared evidence source before explanation
• The group explanation cites every finding in it; no finding loses its own card
• Model still never chooses what to group — grouping is deterministic

---

Shubham — policy compliance, change impact

US-30 — Tell me if my own policy contradicts itself

As someone writing a policy, I want to know that rule 3 requires what rule 7
forbids, before I run anything against a network.

• Pure logic over the policy file; no Batfish, no config
• Reports the contradicting pair with both rule numbers and the overlap
• This is Phase 2 (direction only): four algorithmic problems, one per person #240's problem D, and it is the one that needs no network at all

US-31 — Check a proposed change against my whole policy

As an engineer, I want to know whether an edit breaks any rule I have, not
just the one flow I thought about.

• analyse_change runs every policy rule across before and after
• Direction reported per rule: opening, tightening, unchanged
• Extends the existing compareFilters + differentialReachability pairing

US-32 — Tell me which of my policy rules never fire

As a policy author, I want dead rules found in my policy the way dead ACL
lines are found in my config.

• A rule that no flow in the snapshot can match is reported as a finding
• Reported as hygiene, never as a violation
• Symmetric with filterLineReachability, and states that symmetry

US-33 — Recognise the same problem across two scans

As someone tracking progress, I want to know whether this is last week's
problem or a new one.

• A content fingerprint over structured fields plus a similarity measure on the
  free-text evidence
• Measured against the known collision: rtr-us5-insecure and rtr-us5-messy
  both emit AC-001 for different problems
• US-22 depends on this. This is Phase 2 (direction only): four algorithmic problems, one per person #240's problem C

US-34 — Draft a policy from a config I already have

As a new user with no written policy, I want a starting point derived from
what my firewall already does.

• Produces a policy file, clearly marked as a draft, never applied automatically
• Every generated rule cites the config line it came from
• Refuses where the config is ambiguous instead of guessing

---

Samika — risk, interface, security

US-35 — Sign in, so my scans are mine

As one of several users on a shared install, I want my configs and results
private to me.

• Real identity on top of the session isolation from Multi-user support, half A: session isolation for uploads and the analysis cache #242
• A user can only read their own snapshots; enforced server-side and tested by
  trying to read someone else's
• No credential is ever logged, including on failure

US-36 — Rank by what it would cost me, not just what kind of bug it is

As an owner, I want the finding on my payroll server above the same finding on
a lab switch.

• Extends business_context.py beyond three tiers to a stated impact model
• The ruleset is written down before it is coded, the way severity-rules.md was
• risk still may not downgrade or drop a status="error" finding

US-37 — Give me something I can hand to an auditor

As someone facing an audit, I want a dated evidence pack, not a screenshot.

• Export includes: what was checked, what could not be checked, the policy used,
  the tool version, and the snapshot fingerprint
• The "could not check" section appears first, as it already does in the report
• Deterministic: the same snapshot and policy produce a byte-identical pack

US-38 — Let me use it with a keyboard and a screen reader

As a user who does not use a mouse, I want the whole dashboard operable.

• Every control reachable and operable by keyboard, with a visible focus state
• Landmarks and headings correct — Reorganise the dashboard into distinct areas (closes #279) #284 did this for the four areas; this
  finishes it for the cards, filters and dialogs
• An automated check in CI so it cannot regress silently

US-39 — Show me progress during a long scan

As a user scanning 50 devices, I want to know it is working.

• Per-stage progress streamed from the server, not a spinner on a guess
• A slow response can never render over a newer one — closes A late /api/findings response can render over the 'previous findings cleared' notice #250
• Cancelling actually cancels, and says so

---

What I have deliberately left out

• Anything that touches a live device. Out of scope by the client's own
  constraint, permanently.
• Application-level blocking ("block YouTube"). The client agreed this is a
  proposal-and-simulation feature at best. Writing it as a story would imply we
  can enforce it.
• A cloud or hosted mode. NFR-1 removes it, and it is the one constraint
  that shaped the whole stack.
• Anything already open. Explanations are regenerated on every /api/findings call #92, A rule with no <protocol> silently widens to 'ip'; a rule with no <type> refuses #80, Widen the lint rules and run ruff format — between sprints, not during one #93, P2: One real compliance benchmark pack -- CIS Cisco IOS #220, P2: Persistence and history -- what changed since last month? #223 and Multi-user support, half B: session isolation for the active policy (blocked on #181) #243 are existing
  issues with owners or waiting to be picked up — they are not restated here.

What I want from each of you

Reply with the one story in your five you would cut, and the one thing
you would add that I have missed. I would rather ship fifteen we all believe
in than twenty nobody has argued about.

Then, after the 10th, whichever survive get real issues with acceptance
criteria and go on a milestone.

[patelankeet2]

Replying against my own five (US-25 to US-29).

Cut: US-27 (full support for Arista, NX-OS, Juniper)

Weighed it against the same test you set for the other nineteen -- a measured gap, a real client need, or one of #240's four problems -- and it's the one of my five that fails all three. The client's actual network is Cisco plus pfSense; nothing about a third vendor serves him, and "end to end, not just parsing" across three separate vendors is the single largest unscoped chunk of work in my list. US-26 and US-28 both close gaps we have already measured and named (#87, #78). US-27 doesn't close anything that is currently broken -- it's breadth for its own sake, and breadth is the wrong thing to be buying two weeks from submission.

Add: prove the local-model boundary holds, don't just fix it once

This one is grounded in something that already happened, not invented for the exercise. 0e0f4d6 (1 September) added a real runtime guard so OLLAMA_HOST can't silently point the AI layer at a remote host -- before that fix, an operator setting that variable would have sent configuration-derived evidence off this machine without anything telling them, which is a direct breach of the project's first non-negotiable constraint. I checked ai/query.py and ai/propose.py before proposing this: neither calls a model at all, so the guard's one real call site is covered today. But the guarantee is only tested at the unit level, against _model_use_is_permitted() in isolation -- nothing proves, end to end, that zero non-loopback network calls are ever actually attempted anywhere under ai/ during a real run.

That is exactly this project's own recurring failure shape, stated in our own words already: "a guarantee that holds against an unstated condition rather than one the repo enforces." tests/test_no_configs_in_git.py's docstring says almost that sentence about constraint 1 (no config data ever committed), and I re-read it today reviewing #290. Constraint 2 (everything local) deserves the same treatment: a session-wide network-call assertion across the whole ai/ test surface, so a future refactor that adds a call site can't silently regress this the way OLLAMA_HOST almost did.

Why this pairing, for the product overall

Both halves point the same direction: US-26/US-28 spend the freed-up time on gaps we can prove are real, and the network-boundary story spends a little more of it making sure the one promise this whole product is built on -- config data never leaves the machine -- is something the test suite defends rather than something a fix happened to catch in time. That is a better trade for a final product than three more vendors nobody asked for.

[ARSH871-bot]

Submission is in, so this is no longer a proposal. All twenty are now issues, in milestone Phase 3 - market readiness, alongside sixteen new ones — 38 stories total.

US-20 #313	US-21 #314	US-22 #315	US-23 #316
US-24 #317	US-25 #318	US-26 #319	US-27 #320
US-29 #321	US-30 #322	US-31 #323	US-32 #324
US-33 #325	US-34 #326	US-35 #327	US-36 #328
US-37 #329	US-38 #330	US-39 #331

US-28 did not become a new issue. It is #78, which already exists, is already Ankeet's and already carries the client's real export in its history. Creating it again would have been the duplicate-fact defect committed by the backlog written to avoid it. It is folded into the milestone instead.

What changed from what is written above, and what did not

The stories and acceptance criteria are unchanged. They are copied out of this discussion rather than rewritten, so reviewing this thread already reviewed those issues. Two rationales were expanded during review because a test rejected them as too thin (US-26 and US-36 — the criteria themselves are untouched).

The assignments changed, and that is the part to argue with. This thread says five each. The issues say Arsh 28, Ankeet 4, Shubham 3, Samika 3.

That is deliberate. Most of this milestone is sequential — US-22 waits on US-33, US-45 on US-35, US-46 on US-22 and US-44 — and a dependency chain moves at the speed of whoever is slowest to pick up their link. Concentrating the chain removes the hand-off latency. Everyone else gets a short, self-contained slice inside a vertical they already own, with nothing in it that blocks anybody else.

It does not reduce anyone's total load, and I would rather say that here than let it be discovered. #249 measured that review, not authorship, is where this project's defects actually get caught. This split moves work from writing to reading. Every one of these still needs a current approving review from someone who did not write it. Reasoning in full in docs/design/phase-3-backlog.md on #348.

Two things in this thread I got wrong

"None of this should start before Thursday 10 September." Right at the time and now spent — that date has passed.

US-24 as written here is parse coverage (lines recognised / total). #308 has since raised a different coverage problem: the statement rendering on one surface and not another. They are not the same story, so #317 keeps this thread's scope and #308 stays its own item as US-57.

Sixteen new stories were added on top: installation, egress proof, scale ceilings, background jobs, retention, SBOM, air-gapped install, and pointing Netwise at its own deployment. None of them are in this thread because this thread was written to a different question — what would make this a better capstone rather than what would stop a stranger who paid for it asking for a refund.

Objections to any assignment are cheap to act on: it is one gh issue edit per story.