HybridRouter 0.4.0

HybridRouter 0.4.0 makes HybridRouterKit ready for other apps: a one-line HybridAssistant with privacy checks on by default, failures that retry and fall back instead of ending the turn, and long chats that keep working, with the model the user picked.

.package(url: "https://github.com/jaichaudhry/HybridRouter", from: "0.4.0")

let assistant = HybridAssistant(
    instructions: "You are a helpful travel assistant.",
    cloud: .claude(apiKey: key)          // or .museWonder(apiKey:), or .off for Apple's models only
)
let reply = await assistant.respond(to: "Plan three days in Kyoto")
print(reply.decision?.reason ?? "", reply.text)

HybridAssistant

• Privacy checks on by default. Every message is checked on-device first. Card numbers, IDs, contact details, health terms, and messages with an image or isPrivate: true never reach the cloud model.
• respond(to:) or streaming send(_:), which reports the route, the reply as it grows, and the finished reply with its cost.
• Cloud options: .claude(apiKey:), .claudeAuth(.appAttest(clientID:)) for shipped apps, .museWonder(apiKey:), any ModelProvider, or .off.
• Defaults built in: network tracking, the exact-answer rule and a route-mix and cost summary.
• Under the hood: RoutedSession and TurnCoordinator, still available for full control.

Install steps, a SwiftUI example and the full API are in docs/hybridrouterkit.md.

Failures retry and fall back

• Retries: a rate limit, timeout or dropped connection is retried once on the same route, after about a second.
• Fallback: after that, the turn moves up a route, or, when nothing above can help, falls back to a smaller model instead of failing. For example, Claude stays rate limited, or the phone goes offline mid-reply.
• Every attempt re-checks the network and route availability, and a turn never tries the same route twice.
• Choices are kept: a route the user forced is retried but never moved, and private turns never reach the cloud model.

Long chats keep working

The on-device model has a 4K-token window, and every request carries the whole chat.

• A model the user picks always stays. When the chat is too long for it, it answers from the most recent messages that fit, instead of failing.
• Auto still moves up to a model that sees the whole chat, and now says so. The app's new Settings › Long chats (or Router(whenTooLong: .trimHistory)) makes Auto stay instead.
• When no available model fits the whole chat, it answers from recent messages. This includes long private chats without Apple Cloud, which used to fail with "Too long to keep private".
• Every reply says when length mattered: "This chat is too long for On-device, so Claude answered with the whole conversation", or "…so it saw only the last 3 of 8 earlier exchanges".
• HistoryPolicy can also cap what a route sees, by turns or tokens (.recentTokens(2_000)). It's off by default.

Fixes

• Skips the classifier when only one route can answer, saving about 0.6 s per message for on-device-only apps.
• Reasons name the model actually serving the cloud route ("Forced: Muse Wonder"), including third-party providers.
• A provider passed for the wrong route makes that route unavailable instead of crashing.
• A message sent mid-reply stops cleanly instead of bouncing through every route.
• No false "cut off" warning after tool calls.
• The context estimate counts instructions, the route note, tool descriptions and images.
• PrivacyGuard compiles its patterns once instead of on every message.
• Cloud replies without a listed price are reported (SessionSummary.unpricedSteps) instead of counting as free.

Breaking changes

0.3	0.4
Route.claude	Route.cloud (the saved name is still "claude", so old results files and transcripts load)
RoutedSession(…, claude:)	cloud:; the old label still works, with a deprecation warning
SessionSummary(claudeModelID:), .allClaudeCost	cloudModelID:, .allCloudCost, both with deprecated aliases
EscalationOutcome.retry(decision)	.retry(decision, delay:)
Route.cloud.displayName was "Claude"	"Cloud model"; use session.labels.name(.cloud) for the serving model's name
FailureCause	adds .network

Routing quality is unchanged from 0.3.0: the router, classifier and exact-answer rule are the same, so the 95% at $0.62 per 100 prompts result still holds.

Help wanted

Apple Private Cloud Compute is still unmeasured while the entitlement is pending. If your team has it, you can add the numbers in about 15 minutes: see issue #1.

Full changelog: 0.3.0...0.4.0