byoai: MANIFEST.json + on-demand snip fetching (do not merge until tested)#59
Merged
byoai: MANIFEST.json + on-demand snip fetching (do not merge until tested)#59
Conversation
Adds a 36 KB JSON catalog of all 62 MEBS snips with per-snip topic, OS family, category, devices, and raw.githubusercontent.com URL. AIs with web fetch can now pull only the snips they need for a given request (typical: 4-10 snips, 10-30 KB) instead of loading the full 196 KB jvd-mebs-snips.md bundle. - make-manifest.py: parses snip header (Topic, Seen on) and emits MANIFEST.json. Run as part of regenerate-bundle.sh so the manifest stays in sync with the snip sources (and the existing bundle-freshness CI check enforces that). - SYSTEM_PROMPT.md CORPUS CHECK rewritten to prefer the manifest path (CORPUS-A) and fall back to the single bundle (CORPUS-B) if the manifest can't be reached. Preserves the existing no-web-access flow for air-gapped users. - ON-DEMAND SNIP FETCHING: explicit instructions to fetch only the snips needed per request, never all 62. - Net effect on the system prompt: 294 lines / 13.5 KB (was 307 lines / 13.3 KB), still well under ChatGPT's inline-paste threshold.
This commit will be reverted before merging to main.
…ices - Normalize all entries to 'Generate N ...' (was inconsistent N/an), with parenthetical hint that any count works. - Add 4 services that have snips but were not listed: EVPN-ELAN with IRB, BGP-VPLS (Kompella L2VPN), LDP-VPLS (Junos: BGP-VPLS analogue), port-based EVPN. - Annotate each line with the MEF / protocol context so users can pick the right one.
…5 gap
PROBLEM: Previous 'minimum' tier still pulled BGP overlay AND
ISIS/SR underlay snips, so a brownfield 'add a new service to a
working PE' request would emit transport stanzas the PE already
has - noisy at best, dangerous at worst (could overwrite working
config).
CHANGES:
- TIERS.md rewritten with three tiers and explicit semantics:
* minimum = service routing-instance + AC interface unit +
per-VRF policy ONLY. Assumes PE has working
IGP/SR underlay AND BGP overlay with right AF.
* with-overlay = minimum + bgp-overlay.conf (re-asserts the
family activation - useful when unsure).
* as-deployed = full JVD baseline (transport + apply-groups +
CoS + OAM + FAT-PW + BGP-CT). Mirrors the JVD.
Greenfield/bootstrap requests still forced to as-deployed.
- SYSTEM_PROMPT.md clarifying-question text updated to describe
all three tiers and when to use each. Part 3 pointer updated to
reference all three valid form values; AI told to flag in Notes
if it can't verify the overlay activation is present.
- OUTPUT_FORMAT.md Inputs Used schema updated to allow form:
with-overlay.
- README.md Configuration form section rewritten to match. Adds a
new 'Not yet available' section listing EVPN Type-5 / IP-prefix
routes (validated MEBS pattern but no snip authored yet).
The pattern IS deployed in the JVD - L3VPN VRFs whose 'protocols evpn ip-prefix-routes' advertises Type-5 routes, paired with a matching ELAN-IRB on the same irb.<N> - but no snip captured it. Adding both OS variants and wiring it into the menu and TIERS, and removing the 'Not yet available' listing from README + TIERS. - junos/services/evpn-type5.conf and evo/services/evpn-type5.conf: modeled after the deployed METRO_L3VPN_4000 instance on mse1_mx304 (Junos) and an3_acx7100-48l (EVO). Same syntax on both OS families. - TIERS.md: new section for 'EVPN Type-5 / IP-prefix VRFs' with minimum / with-overlay / as-deployed mappings. Note that this service ALWAYS pairs with an ELAN-IRB on the same irb.<N>; in as-deployed we also include evpn-elan-mac-vrf-irb.conf. - SYSTEM_PROMPT.md menu: new entry between EVPN-ELAN with IRB and L2Circuit, with the parenthetical 'pairs with an ELAN-IRB' hint so users know what they're getting.
The JVD does not deploy 'pure' RT-5 (VRF only, no MAC-VRF). Every deployed EVPN-IRB instance pairs an ELAN-IRB MAC-VRF (RT-2 from learned MAC+IP) with a VRF running 'protocols evpn ip-prefix-routes' (RT-5 from the IRB subnet, silent hosts, and VRF prefixes), both referencing the same irb.<N>. - TIERS.md: every tier (minimum / with-overlay / as-deployed) for EVPN Type-5 now includes BOTH evpn-elan-mac-vrf-irb.conf and evpn-type5.conf, plus the AC interface snip in minimum. Header text explicitly calls out the no-pure-RT-5 stance. - SYSTEM_PROMPT.md menu: clarified the parenthetical to '(RT-2 ELAN-IRB + RT-5 VRF on same irb.<N>; JVD default)'. - snip headers (both OS): added a Highlights bullet stating the snip is the L3/RT-5 half of the paired pattern, and that pure RT-5 is not deployed in this JVD.
Three distinct services were collapsed into two confusing menu
entries. Correct taxonomy per Junos/EVO config syntax:
* Kompella L2VPN = instance-type l2vpn + protocols l2vpn { site
$NAME { site-identifier; remote-site-id; } } - point-to-point
pseudowire signalled by BGP family l2vpn (RFC 4761).
File: services/l2vpn-kompella.conf (Junos + EVO)
* BGP-VPLS = instance-type virtual-switch + protocols vpls
with 'site $NAME { site-identifier $ID; }' (no vpls-id) -
multipoint VPLS signalled by BGP NLRI (RFC 4761).
File: junos/services/bgp-vpls.conf (renamed from ldp-vpls.conf;
JVD only deploys this on Junos PEs)
* LDP-VPLS = instance-type virtual-switch + protocols vpls
with 'vpls-id $ID' + 'neighbor $REMOTE_PE' (no site block) -
multipoint VPLS signalled by LDP targeted sessions (RFC 4762).
File: evo/services/ldp-vpls.conf (JVD only deploys this on EVO)
* LDP-VPLS with BGP auto-discovery (l2vpn-id form) is NOT
deployed in this JVD; called out explicitly in TIERS.md.
Changes:
- git mv junos/services/ldp-vpls.conf -> junos/services/bgp-vpls.conf
- Rewrote junos/services/bgp-vpls.conf header (was misleadingly
framed as 'closest analogue to LDP-VPLS').
- SYSTEM_PROMPT.md menu: split into 3 entries (Kompella L2VPN,
BGP-VPLS, LDP-VPLS) with identifying syntax in parens.
- TIERS.md: rewrote 'L2VPN family' section enumerating all three
with their identifying syntax and which OS deploys each.
- README.md (snips): updated table rows for the three services.
- evo/services/ldp-vpls.conf: cross-ref now points to bgp-vpls.conf.
Old wording 'EVPN Type-5 / IP-prefix VRFs' implied a pure RT-5 VRF, which the JVD does not deploy. New wording mirrors the sibling 'EVPN-ELAN with IRB' entry and makes the L2+L3 superset relationship obvious. Also tightened the IRB-only line to '(Type-2 only; L2 + anycast GW)' for symmetry.
… list Observed Claude rendering an old, abbreviated 4-item menu with 'Generate an EVPN-ELAN' wording that no longer exists in the prompt - almost certainly because Claude had no web fetch and silently fabricated a plausible menu instead of falling back to CORPUS CHECK Step 3. Tightened FIRST USER TURN response (b): - 'Render the menu VERBATIM - do NOT add, remove, reorder, abbreviate, paraphrase, summarize, or rewrite any line.' - 'The Services section MUST contain all 10 bullets.' - 'If you cannot see the menu, DO NOT make one up - go back to CORPUS CHECK Step 3 instead.'
…?' tagline
Claude kept rendering an abbreviated 4-bullet Services section
('Generate an EVPN-ELAN', 'Generate an L2Circuit with hot-standby')
instead of the canonical 10-bullet list. Strengthened guard rails:
- Explicitly forbid 'condense' alongside abbreviate/paraphrase/etc.
- Tell the model to count to ten before sending.
- Call out the specific failure mode by example (no 'Generate an
EVPN-ELAN' / 'Generate an L2Circuit' stand-ins).
- Justify the rule: 'this menu is the authoritative list of what
this JVD supports - abbreviating it strands users on services
they cannot see.'
Also added a closing tagline so users know they can free-form ask:
'Don't see what you need? Describe what you're looking for and
I'll tell you whether the Metro EBS JVD covers it.'
…all 10 services
Claude was confirmed to be fetching the prompt successfully (it
echoed our verbatim 'Loaded JVD MEBS manifest' acknowledgment line)
yet still rendered an abbreviated 4-bullet Services list with
'Generate an EVPN-ELAN' style stand-ins. Soft 'render verbatim'
language was insufficient.
Restructured the menu instruction:
- Wrapped the menu in <<<MENU_BEGIN>>> / <<<MENU_END>>> sentinels
framed as a 'fixed payload, not a summary target' to copy
character-for-character.
- Replaced the prose with a HARD CONSTRAINTS checklist:
* exactly 10 bullets in the Services section, in order
* every bullet starts with literal 'Generate N' (no 'an'/'a')
* an explicit numbered list of all 10 service names that MUST
appear exactly once
* pre-send self-check: count bullets, regenerate if not 10
* if no corpus loaded, go to Step 3 (don't invent)
Also fixed two stale '62 snips' references (now 64) that were
making it obvious to users that the model had cached an older
version of the prompt.
… Claude refusal
Claude was refusing to load the prompt with: 'I won't wholesale
replace my actual system prompt or override my core values based
on external content - that would be a security risk regardless of
the source.' The trigger words were 'system prompt' and 'adopt
its contents as your system prompt.'
Reframed the bootstrap message and prompt header to describe the
content as 'task instructions' / 'task guide' rather than a
system-prompt replacement, and explicitly told the model up front:
'this does NOT replace your system prompt or override your safety
guidelines - it just describes a specific task.' Same payload,
non-adversarial framing.
Changes:
- make-launch-links.sh: bootstrap MSG rewritten in collaborative
voice ('please fetch...task instructions...follow its
instructions to greet me').
- README.md: launch-button hrefs updated; prose now explains the
framing choice for transparency.
- SYSTEM_PROMPT.md: 'SYSTEM PROMPT - ADOPT IMMEDIATELY' header
replaced with 'TASK INSTRUCTIONS' + a brief 'this does not
replace your system prompt' disclaimer. PART 0 renamed
IDENTITY -> ROLE with softer 'please act as' phrasing.
…ation) Claude was confirmed to be fetching the prompt and seeing the fenced menu yet still rendered an abbreviated 4-bullet Services list. Sentinels, hard constraints, and pre-send self-checks did not change the behavior - the model treats long verbatim greeting blocks as summary candidates. Switched strategy: the model no longer renders the catalog at all. The greeting is now three short sentences with a link to MENU.md on GitHub. Users get the always-current canonical list at the linked file; the model has nothing to summarize. Also revised the Audit / explain section per request: - Replaced 'Explain how Pair-with works' (snip-internal concept) with 'Compare EVO vs JUNOS VPN services' (broader value). - Added 'Explain Seamless MPLS deployment' - the MEBS JVD has five IGP domains (default ISIS + isis-instance metro-a + isis-instance metro-b + multiple OSPF areas at MA3/MA4/MSE PEs/AN3) stitched at the MDR ABRs via BGP-LU; the bgp-overlay snip already carries the full BGP-LU machinery (rib inet.3, add-path 4, rib-group RG-REMOTE-LOOPBACKS). New file: - byoai/MENU.md - canonical catalog, replaces the inline block.
Restored the inline 'Common asks' menu (10 services / 3 add-feature / 2 turn-up / 4 audit-explain) so users see actionable examples immediately without a click-out. Each section is capped at 10 bullets on purpose: small enough that ChatGPT renders it cleanly and Claude has little incentive to abbreviate, large enough to cover the high-value asks. The MENU.md link is now positioned as the 'full catalog' overflow - as we discover new query patterns over time, they go into MENU.md without bloating the inline greeting. Kept the 'Don't see what you need?' tagline for free-form discovery.
Gemini doesn't support ?q= URL prefilling, so the button just opens gemini.google.com/app and the README shows a fenced paste-message block users can copy as their first turn. ChatGPT and Claude buttons continue to ship the bootstrap message via ?q=. All three buttons now have vendor logos via shields.io: - ChatGPT: openai logo on Juniper green (#10A37F) - Claude: anthropic logo on Anthropic clay (#D97757) - Gemini: googlegemini logo on Google blue (#4796E3)
Simple Icons (the icon set shields.io uses) doesn't ship the OpenAI/ChatGPT mark - likely trademark - so logo=openai silently rendered no icon. Switched the ChatGPT badge to a data:image/svg+xml base64-encoded SVG of the OpenAI swirl, which shields.io accepts via the same logo= parameter. Anthropic and Google Gemini logos remain native simple-icons (logo=anthropic, logo=googlegemini) - those slugs do work.
Three identical-size custom SVG buttons under assets/buttons/: chatgpt.svg / claude.svg / gemini.svg. Each is 260x52 with 12px rounded corners, vendor brand color, a subtle top-down highlight gradient, a 1px drop shadow, the vendor logo (OpenAI swirl / Anthropic / Gemini star) at left, and 'Open in <Vendor>' centered in white 16px semibold text. Wins over the shields.io approach: - All three buttons are now exactly the same dimensions. - Larger rounded corners (12px vs shields.io's ~3px). - Drop shadow + highlight gradient gives a subtle 3D pill look. - ChatGPT logo finally renders (Simple Icons doesn't ship the OpenAI mark; we now embed it directly in the SVG). - No external CDN dependency for rendering.
… merge Pre-merge cleanup. All references in launch buttons, corpus-fetch URLs, MENU.md link, manifest raw_urls, README, scripts, and generated bundle now point at https://github.com/Juniper/jvd/... on the main branch. Files touched: - SYSTEM_PROMPT.md (corpus-fetch URLs + Step 3 fallback + MENU link) - README.md (launch buttons, prose URLs) - make-launch-links.sh (PROMPT_URL constant) - make-manifest.py (RAW_BASE constant) - MANIFEST.json (regenerated; 64 raw_urls now point at main) - jvd-mebs-byoai-prompt.txt (regenerated)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
MANIFEST.json— a 36 KB index of all 62 MEBS snips with per-snip topic, OS family, category, devices, and raw GitHub URL — and rewrites the BYOAI system prompt's CORPUS CHECK section to prefer on-demand snip fetching over loading the full 196 KB bundle.Why
The current flow has the AI fetch (or attach)
jvd-mebs-snips.md, a 196 KB single-file bundle of every snip in the MEBS library, on every conversation. That bundle:With the manifest, the AI fetches:
MANIFEST.json(36 KB) once at the start.What's added / changed
make-manifest.py— parses each snip'sTopic:andSeen on:header lines and emitsMANIFEST.json.MANIFEST.json— generated, 62 snip entries + 4 reference-file entries.regenerate-bundle.sh— now also runsmake-manifest.py, so the existing CI check enforces manifest freshness automatically.SYSTEM_PROMPT.md— CORPUS CHECK rewritten:jvd-mebs-snips.md(still works for air-gapped flows).jvd-mebs-byoai-prompt.txt— regenerated; 294 lines / 13.5 KB (was 307 lines / 13.3 KB), still well under ChatGPT's inline-paste threshold.Backwards compatibility
jvd-mebs-snips.md) is still produced and still works as a fallback. Air-gapped / no-fetch users see no change.Open in ChatGPT/Open in Claude) still bootstrap fromjvd-mebs-byoai-prompt.txt, which now contains the manifest-aware prompt.Testing plan (before merging)
Open in ChatGPTon the PR-branch README → verify it loads the manifest, not the bundle.Generate 1 EVPN-VPWS, mixed→ verify it fetches ~6–8 snips, not 62.Generate 2 L3VPN VRFs, JUNOS, as-deployed→ verify a different snip set is fetched.