v2.5.0 — Continuous integration: PR bot, project file, org dashboard #618
thomas-stegemann
announced in
Announcements
Replies: 2 comments
|
Bowire's latency budgets as CI gates are a concrete way to make workflow performance reviewable: a p95 threshold and error-rate threshold can fail the run before a regression reaches users. I would extend the same report with provider/model, streaming time-to-first-chunk, retry count, and cost per completed benchmark case. Parsing every threshold before load starts is especially useful because a malformed budget should fail fast rather than after a long run. I am testing an OpenAI-compatible multi-model layer around official Chinese models, where routing and usage fields need the same reproducible checks. Would a provider capability matrix or a per-workflow cost gate be more useful in Bowire's CI output? |
0 replies
|
Thx, I will consider this.
… NovaRouteAI ***@***.***> hat am 27.08.2026 20:09 CEST geschrieben:
Bowire's latency budgets as CI gates are a concrete way to make workflow performance reviewable: a p95 threshold and error-rate threshold can fail the run before a regression reaches users.
I would extend the same report with provider/model, streaming time-to-first-chunk, retry count, and cost per completed benchmark case. Parsing every threshold before load starts is especially useful because a malformed budget should fail fast rather than after a long run.
I am testing an OpenAI-compatible multi-model layer around official Chinese models, where routing and usage fields need the same reproducible checks. Would a provider capability matrix or a per-workflow cost gate be more useful in Bowire's CI output?
—
Reply to this email directly, view it on GitHub #618?email_source=notifications&email_token=B4LCHVKMM7QS7QQOBFNDNPT5MB2NBA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOBRG44TEMZZUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-18179239, or unsubscribe https://github.com/notifications/unsubscribe-auth/B4LCHVMCB764NHFTHEBADT35MB2NBAVCNFSNUABJKJSXA33TNF2G64TZHMYTEMRXGUZDMNBWHE5UI2LTMN2XG43JN5XDWMJQGY4TGMRTGWQXMAQ.
You are receiving this because you authored the thread.Message ID: ***@***.***>
|
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
v2.5 is about what happens after you leave the workbench. The things Bowire already knew — your schemas, your recordings, your benchmarks, your contracts — now show up where the work actually gets judged: in CI, on a pull request, and in an editor panel next to the code.
215 commits since v2.4.0. Every workspace, collection, recording, mock and flow loads identically. Two changes are worth reading before you upgrade — see Breaking changes.
Highlights
The workbench, in VS Code (#101)
ext install kuestenlogik.bowire-vscodeopens Bowire in an editor panel beside your code.It does not bundle a CLI — it drives one. The extension uses a
bowireyou configured, one your repository pins in a tool manifest, or one on yourPATH, and offers to fetch a verified copy when it finds none. That is the whole point: the workbench in your editor, thebowirein your terminal and the one in CI are the same binary reading the same collections.Uninstalling the extension removes a CLI it downloaded. One you installed yourself is never touched.
Design-time lint — the review comments you keep writing (#189)
Most API design problems are caught in review if the reviewer is sharp — inconsistent naming, a field that returns a password, an unbounded list with no pagination, a missing version. That scales with reviewer attention, which does not scale.
bowire lint <snapshot|url>runs a typed rule engine over the schema and reports them without a reviewer having to be sharp that day:Text for a terminal, markdown for a PR comment, JSON for whatever you build next.
--fail-ongates the build — andnoneor an unrecognised level never fails, because a lint gate is advisory and a typo should not break a build that was passing.Latency budgets as CI gates (#360, #232)
k6-style thresholds: a budget is either met or the run exits non-zero.
Budgets are plain numbers — milliseconds for the latency metrics, a
fraction for
error-rate. Metric spellings are forgiving (p(95),error_rateanderror-rateall land on the same metric), the budget isnot: a unit suffix is refused rather than guessed at, and every threshold
is parsed before the run starts so a typo in the fifth budget does not
surface after two minutes of load.
Benchmarks also gained a scheduled shape — a cron-driven run that survives a restart, with the schedule readable and pausable from the workbench. Deliberately no "create" button in the browser: a schedule carries a target URL the server will call unattended, so authoring one stays on the CLI where the operator is explicit about it.
Contract matrix in the workbench (#364)
Consumer × provider, pass/fail, in one grid. Each cell carries how many interactions backed the verdict — 11 of 12 and 0 of 12 are very different mornings — and a pair that was never run reads as a dash rather than a blank, which would look like "checked, nothing wrong".
One rollup over the artefacts you already write (#587)
Lint findings, contract results, benchmark envelopes, k6 summaries, SARIF, JUnit — Bowire already wrote all of it as structured files.
bowire report rollupreads them into one portfolio view, available identically from the CLI,GET /api/report/rollup, and thebowire.report.rollupMCP tool.Counts stay nullable on purpose: "no lint report was read" and "lint found nothing" are different statements, and flattening both to zero would let a missing report read as a clean bill of health.
A PR bot you can use (#183, #582)
The PR-report action moved to its own public repository and the Marketplace:
It posts one comment per PR — API-schema delta, test results, security findings, perf — and edits that comment on later runs instead of stacking new ones.
Secrets stay secret in CI output (#361)
Values marked secret are redacted in logs, reports, annotations and exported commands. Bowire's CI story previously printed resolved variables into all of them.
Your repo can own its Bowire configuration (#172, #616)
Collections, environments and recordings then live beside the code and commit, diff and review like any other file. Opt-in — a manifest that says nothing keeps the machine-wide store, so nothing moves under you.
Underneath, one resolver now decides where anything is stored. Fourteen files across six assemblies used to build
~/.bowire/…by hand, which meant the project opt-in reached some stores and silently missed others — the plugin directory, the proxy CA, the vuln-db cache and the MCP stores among them.Starting Bowire from another program (#615)
The OS picks a free port; Bowire writes the address it actually bound once it is listening, and deletes the file on shutdown. The file exists if and only if the workbench is bound, which makes it the address and the readiness signal.
Do not scrape the startup banner for this. It is a log line, so it disappears at a quieter log level — and it used to be printed before the bind was known to have worked, so it could announce a URL that never served. That is fixed too, but
--port-fileis the contract.Faster REST discovery (#585)
OpenAPI discovery took 5–8 s and flaked near the 8 s probe timeout. It no longer does.
Security
?workspaceId=and?storageRoot=went from the query string intoPath.Combineunchecked, across six endpoint files. A request could name any directory on the machine and have Bowire read or write there, reporting success. Both are now validated in one place: a workspace id must be a single segment from an allow-list; a storage root must be absolute, free of.., and name a directory that already exists.http/https.Uri.TryCreate(s, UriKind.Absolute)returns true for an absolute file path on Unix, so/etc/passwdparsed asfile:///etc/passwdand was accepted as an upstream — on Linux only, which is where this runs in production.Breaking changes
BrowserUiHost.HostRunnergained a parameter. Embedded hosts that substituted the runner (a test seam) now receive anonListeningcallback and must invoke it with the bound URL. Everything downstream of the address — the banner,--port-file, auto-opening a browser — hangs off that call, because with--port 0the port is not knowable before the bind.Storage paths route through
IBowirePathResolver. If you referencedDefaultBowireUserStore.UserProfileRootor built~/.bowire/…yourself, useBowirePaths.Resolve(BowireStorageScope.Data, …)or takeIBowirePathResolveras a dependency. The old property still works; it is simply no longer the thing that knows where your data is. See Storage locations.Fixes worth naming
id, so after a delete a row kept the click handlers of the workspace that had just been removed while displaying the next one. The visible symptom was a delete button that did nothing; one step further it would have renamed or deleted the wrong workspace.DELETE /api/workspace/{id}anchored its containment check on the user root, which it asked for with an empty filename — and the store rejects that, so every call ended in an unhandledArgumentExceptionbefore a single byte was deleted. The check anchors on the workspaces folder now, which also closes a gap the old prefix test left open: a sibling directory whose name merely starts with the root's.bowire contract publishmet a hand-edited recording with a stack trace. The catch around the recording loader listedIOExceptionbut notInvalidDataException, which lives inSystem.IOand derives fromSystemException— so every rejection the loader makes (unsupported format version, an ambiguous--select, a store with no recordings) escaped as an unhandled exception instead of the intended message and exit 65.Acknowledgements
The competitive-research inputs behind the threshold gates (k6) and the redaction work (Hurl) came from a mid-2026 survey pass; both landed as table stakes rather than as differentiators, which is the right way round.
The full commit list, contributors, and compare-URL diff are auto-generated below.
Added
9e8718e)5b75f0b)4423dcb)29f4d1c)536b853)95a41a0)78e4af3)8430e90)1ab05e4)ef4ac90)77714fe)0d5e8ad)062ffe4)ef9a0a8)d86f781)68884b8)7adef56)6bdc543)fbaa7bc)a71d4f1)1d62165)bowire bench runwith threshold gates and k6 export (Benchmarks: k6-style thresholds as CI gates — fail bowire test / bench runs on latency budgets #360) (29a8fb0)7065467)d86566c)047ca6b)d99496d)d431b38)348d6df)32b096f)6711abd)b0037e2)2afd448)b9b0739)75fbe88)4ba20fc)49fd7e4)javascriptflag (codecov: dedicated 'javascript' flag (lcov upload from node:test coverage) #367) (c1f91fc)61f53e2)Changed
4db3e60)2cef9e8)7e32c2c)aec1f42)65876e6)82431e0)1ca1e2d)Fixed
bowire discoverdown (397924b)7f5c26e)6e7c132)8f1e199)55b5217)46d62f0)8f0eef4)06ae24b)f98a3d1)9a6fb90)a48d3cb)c159f33)0ea922b)6365233)bb63290)fc2f36a)7989554)efee6c6)3a371bd)309f905)b6919a0)f2410f0)472dbbb)0df02d3)42bd3b0)574e6be)pluginmanagement group (2d5e74f)964b39e)8ec2884)Documentation
d342d73)46dc961)6d0177e)c33fac5)b201c76)8ae88b7)2f2471f)5b785de)8ae1547)d85ad66)c70d0f8)9c2c1d0)de30772)9704513)fd7d0b0)fe4f4f7)79d4c2d)eeb3b76)Other
6b0a528)8afda33)c5fda2d)660896d)77a032a)ed5b717)Maintenance — 153 maintenance commits · 19 dependency updates
9a5d500)74ad82e)1505486)188c77c)013411f)5d910c0)92778b5)aca6b35)966684c)f3a5144)1ed67cb)593df48)bd7c2b4)fbb69b2)84e2417)5b5c7da)876ac52)f99d3af)656d1b8)e17fd06)43fa62f)ef3147e)89a9c32)7c00f35)6e94213)89ee9b0)e5379f6)0836232)edf66df)2e51bda)060e201)3198c1e)e7d0656)314df18)50e9d70)oast serverefuses, and what its banner promises (946773d)d6a30c0)13bf75a)72bb4f9)a8b66d2)b5bfb57)14f7b09)27b1f55)fe00d19)8a7b1ae)2e001f3)9904072)e794236)97232b0)418fe8f)3f2e81a)a2171f2)c447034)0bf81e6)8365391)5873c2a)de2759f)976e51d)d2a8e3f)b582bdc)209dc3c)378cd6f)1688b12)377b16d)f137fdd)f72d88a)9211649)bd0b7b8)e55f9b0)d007551)20d878a)26f4c5c)d9e2688)5bc6674)31dc3e4)a9e25b8)1fd1903)7231099)6a6a437)3c2bf7e)a590059)4c4f4b2)48dde2a)9ce600f)2c70744)0c27fc2)71e74a3)a6319a5)e2d83a6)3fe1412)0e31b01)d96a1e7)4fe60b1)8a31395)cffecec)c63fa7f)8e38d4e)c390f4c)d808588)4fd30b6)81b4494)e2a44d3)c1274ee)b97ce6d)1d7312c)0e34883)53217c5)8a0956d)e99eec6)f3ff2f0)eabb38c)622ec9a)37ad256)c8d2cd4)c839f35)8d7d1a2)bb7e8f1)8a691ec)ab62c70)223abd0)140c0a1)b90cda6)d1a766d)30c4b9c)fa45c6b)aeae8be)10a8bf8)d1c483e)058f0d2)3d74a89)c387ed1)6bbdcbd)4ec5b2b)6965123)5f31ed8)234cb7c)420f8c1)7fa8726)a870538)6c0ca0d)eca4cb5)26f7b2a)97b6c3d)aa94031)a1bb587)6d5a71b)454f1c8)a9a2957)28d9eb4)396d80f)ff9a766)63da899)48b8529)Dependency updates
6e04f6b)517938f)277bbc0)9e50de8)145ba51)f7ec595)8371002)f56a84c)bee2811)cc0c33b)f1dd72d)d5fffb4)d86da70)6a29a49)37a230a)112acb9)4b97386)8aa2eb3)6e6f670)Contributors: Thomas Stegemann
Full diff: v2.4.0...v2.5.0
This discussion was created from the release v2.5.0 — Continuous integration: PR bot, project file, org dashboard.
All reactions