Skip to content

Releases: HarvardMadSys/hybridInference

20261001

Choose a tag to compare

@github-actions github-actions released this 01 Oct 02:42
c997711

What's Changed

  • feat(routing): keep prompts over a max input off the offload route by @1a1a11a in #1497

Full Changelog: 20260930.2...2026100

20260930.2

Choose a tag to compare

@github-actions github-actions released this 30 Sep 23:53
e1ba3f2

What's Changed

  • feat(routing): hold a local engine's queue in the gateway by @1a1a11a in #1496

Full Changelog: 20260930.1...20260930.2

20260930.1

Choose a tag to compare

@github-actions github-actions released this 30 Sep 20:44
d49bdfb

What's Changed

  • fix(routing): don't cut a request short for an offload route it cannot use by @1a1a11a in #1495

Full Changelog: 2026093...20260930.1

20260930

Choose a tag to compare

@github-actions github-actions released this 30 Sep 02:39
af442ea

What's Changed

  • fix(routing): retry a route cut short for a failed offload route by @1a1a11a in #1494
  • fix(auth): end the cloud agent session on FI sign-out and sign-in by @williamnixon20 in #1487

Full Changelog: 20260929.1...2026093

20260929.1

Choose a tag to compare

@github-actions github-actions released this 29 Sep 22:50
3899c51

What's Changed

  • fix(routing): cap an endpoint pin at one day so weight changes reach busy callers by @1a1a11a in #1486
  • feat(admin): count the past day's offloaded requests on Recent Requests by @1a1a11a in #1492
  • fix(routing): count an offload that a later route served by @1a1a11a in #1493

Full Changelog: 2026092...20260929.1

20260929

Choose a tag to compare

@github-actions github-actions released this 29 Sep 14:36
52c3ac3

What's Changed

  • fix(routing): skip fallback routes too narrow for a prompt already refused as too long by @1a1a11a in #1475
  • fix(tests): give every test an empty request context by @1a1a11a in #1476
  • fix(tests): drop the unit-tier aiohttp stub that replaced the real module by @1a1a11a in #1477
  • refactor: drop the aiohttp fallbacks that existed only for the unit-test stub by @1a1a11a in #1478
  • feat(routing): offload a queued request to an admin-chosen route by @1a1a11a in #1480
  • feat(routing): offload a stream its engine has not started, and stop feeding a stalled engine by @1a1a11a in #1481
  • fix(routing): make the engine wait per request and see held-back output by @1a1a11a in #1482
  • feat(alerts): mute one alert type at a time from the dashboard by @1a1a11a in #1483
  • fix(alerts): re-read an alert-mute lookup that raced a write by @1a1a11a in #1484
  • docs(developer): rewrite the developer guide in plain language by @realtmxi in #1485

Full Changelog: 2026092...2026092

20260927

Choose a tag to compare

@github-actions github-actions released this 27 Sep 13:45
f266412

What's Changed

  • ci: run pull_request jobs on GitHub-hosted runners by @realtmxi in #1348
  • docs(readme): link the RouteWise docs and drop the repeated lab link by @realtmxi in #1413
  • fix(serving): read the session Claude Code actually sends today by @1a1a11a in #1391
  • fix(alerts): stop naming a broken endpoint's victims as its offenders by @1a1a11a in #1415
  • fix(routing): admit one probe to a half-open circuit, not the whole burst by @1a1a11a in #1418
  • fix(auth): name the caller behind an ip_blocked row by @1a1a11a in #1420
  • fix(alerts): measure quota overrun against the cap that is actually enforced by @1a1a11a in #1422
  • fix(serving): redact internal error detail from responses, preserve every status code by @1a1a11a in #1423
  • feat(alerts): name the sources and accounts behind an auth failure spike by @1a1a11a in #1424
  • fix(ci): tolerate out-of-order Actions job timestamps in timing summary by @1a1a11a in #1425
  • fix(routewise): cap probing at one request per worker, not one per model by @1a1a11a in #1429
  • fix(adapters): repair malformed tool_call arguments before they reach upstream by @1a1a11a in #1432
  • fix(alerts): log refused Slack deliveries and bound the retries they trigger by @1a1a11a in #1435
  • fix(routing): surface a fallback route's client 400 instead of a dead primary's 500 by @1a1a11a in #1434
  • feat(routing): make silently-excluded routes and health verdicts observable by @1a1a11a in #1437
  • fix(serving): detect and log tool calls the gateway streams out with unparseable arguments by @1a1a11a in #1433
  • feat(alerts): page on client-error bursts and per-model stream failures by @1a1a11a in #1436
  • perf(routewise): stop per-request routing cost from scaling with traffic by @1a1a11a in #1426
  • fix(alerts): stop logging the Slack webhook URL on every alert post by @1a1a11a in #1438
  • fix(tokens): stop tiktoken from crashing on special-token text by @1a1a11a in #1441
  • feat(alerts): put the request id on firing pages by @1a1a11a in #1442
  • fix(alerts): honour every state_changes knob, not just one by @1a1a11a in #1439
  • docs: drop dead D1 and ollama references by @1a1a11a in #1443
  • feat(adapters): cap outbound concurrency per provider key, AIMD on 429 by @1a1a11a in #1448
  • feat(admin): show the live adaptive upstream concurrency limits by @1a1a11a in #1449
  • feat(streaming): detect a stalled upstream before the proxy's clock does by @1a1a11a in #1447
  • fix(frontend): let prettier own formatting, unblock next.config.js edits by @1a1a11a in #1446
  • refactor: retire nginx and grafana by @1a1a11a in #1444
  • fix(ci): stop the example stacks stranding a Docker volume per run by @1a1a11a in #1450
  • chore: promote dev to main for the production lock bump by @1a1a11a in #1451
  • chore: drop dead component and unused runtime deps by @1a1a11a in #1452
  • fix(alerts): stop announcing a recovery for auth_ip_blocked by @1a1a11a in #1453
  • fix(adapters): read the upstream's SSE error frame instead of blaming framing by @1a1a11a in #1445
  • docs(readme): add the Slack workspace invitation by @realtmxi in #1455
  • refactor(routing): add hybrid router abstraction over pluggable backends by @realtmxi in #1454
  • docs(routing): pin hybrid design to the #1454 merge by @realtmxi in #1458
  • docs(developer): document the endpoint execution boundary from #1454 by @realtmxi in #1459
  • docs(architecture): illustrate routing contracts and hybrid topologies by @realtmxi in #1460
  • docs(agents): add hierarchical guidance convention and routing pilot by @B-A-M-N in #1456
  • feat(routewise): add prefill-load-aware routing by @B-A-M-N in #1419
  • fix(storage): close hard-delete erasure race with advisory-lock fence (#1421) by @B-A-M-N in #1428
  • fix(routewise): validate prefix-cache locality with observed cache reuse by @B-A-M-N in #1417
  • fix(serving): resolve client IP only across trusted proxy boundaries (#1036) by @B-A-M-N in #1427
  • docs(readme): cite RouteWise with its EuroSys '27 ACM metadata by @realtmxi in #1467
  • feat(frontend): support distribution-owned public UI through the upstream build by @realtmxi in #1465
  • fix(tokens): upgrade tiktoken so unsplittable runs stop freezing the gateway by @realtmxi in #1470
  • Revise Consent Language by @williamnixon20 in #1469
  • fix(streaming): answer a pre-first-byte upstream failure with its real status by @1a1a11a in #1474

New Contributors

Full Changelog: 2026091...2026092

20260910

Choose a tag to compare

@github-actions github-actions released this 10 Sep 13:44
950f9c0

What's Changed

  • fix(admin): require authentication for routing disclosure by @realtmxi in #1380
  • fix(auth): fail fast when required secrets are missing by @realtmxi in #1381
  • fix(ci): isolate Buildx config and clear frontend dependency audit by @realtmxi in #1383
  • fix(console): restore FreeInference landing and scope developer home to example by @realtmxi in #1385
  • feat(backend): load trusted startup adapter extensions by @realtmxi in #1384
  • fix(console): compact example models and prioritize signup by @realtmxi in #1386
  • fix(routing): scope RouteWise route state to the model it routes by @realtmxi in #1388
  • fix(console): restore the Agents tile for standalone sites by @realtmxi in #1390
  • fix(embeddings): honor route-level endpoint paths by @realtmxi in #1389
  • docs: remove deployment-specific quota setup guidance by @realtmxi in #1394
  • docs(community): add contribution, issue and pull-request entry points by @realtmxi in #1392
  • fix(backend): enforce RAG policy and prevent stale allowlist cache writes by @realtmxi in #1393
  • docs(configuration): document runtime configuration from the admin console by @realtmxi in #1397
  • feat(grants): report one grant's usage over a request-start window by @realtmxi in #1396
  • docs: point at the cloud agent repository by its new name by @realtmxi in #1354
  • fix(auth): enforce distribution public signup policy by @realtmxi in #1387
  • fix(console): fail explicitly when site configuration cannot load by @realtmxi in #1382
  • docs: fold CLAUDE.md into AGENTS.md as the single agent guide by @1a1a11a in #1401
  • fix(deploy)!: default console to loopback and document upgrade paths by @realtmxi in #1400
  • feat(admin): derive route targets from the deployment, not a built-in vendor list by @realtmxi in #1398
  • feat(quotas): report provider quotas through an extension-filled fetcher registry by @realtmxi in #1399
  • test: name the shared fixtures after what they test, not after our models by @realtmxi in #1402
  • Revert "test: name the shared fixtures after what they test, not after our models" by @realtmxi in #1403
  • Remove source labels from provider registry by @realtmxi in #1404
  • perf: Clarify provider actions and open the selected provider’s keys by @realtmxi in #1405
  • fix(console): preserve mixed quota rows and source units by @realtmxi in #1406
  • docs(config): remove fixed local API key from env example by @realtmxi in #1407
  • docs: sanitize historical deployment verification details by @realtmxi in #1409
  • feat(agent): expose deployment-owned access permissions by @realtmxi in #1408
  • docs(readme): lead with deployment benefits and admin console by @realtmxi in #1410
  • feat(identity): omit account profiles from Agent tokens by @realtmxi in #1411
  • docs: fix deployment examples and publication checks by @realtmxi in #1412

Full Changelog: 2026090...2026091

20260908

Choose a tag to compare

@github-actions github-actions released this 08 Sep 02:36
2b90992

What's Changed

  • feat(observability): surface auth-failure IP blocks, and let an operator lift one by @1a1a11a in #1379
  • feat(console): make the developer home the only homepage, and hide unconfigured providers by @realtmxi in #1359

Full Changelog: 20260907.1...2026090

20260907.1

Choose a tag to compare

@github-actions github-actions released this 07 Sep 19:14
991bd10

What's Changed

  • fix(adapters): keep the system message first for OpenAI-style upstreams by @1a1a11a in #1377
  • fix(streaming): keep the SSE frame an upstream leaves unterminated at EOF by @1a1a11a in #1376
  • feat(admin): classify client disconnects apart from errors in Recent Requests by @1a1a11a in #1378

Full Changelog: 2026090...20260907.1