Proposal: LiteRT-LM backend (Gemma 4 and other .litertlm models, on-device)

[john-rocky]

Proposal

Add a LiteRT package trait and a LiteRTLanguageModel backend that runs .litertlm models (Gemma 4 E2B/E4B, and anything else on the Hub with a .litertlm) fully on-device via Google's LiteRT-LM runtime.

Why

• It fills a runtime gap. AnyLanguageModel covers Apple FM, Core ML, MLX, and llama.cpp on-device, but not Google's on-device stack. LiteRT-LM is the runtime behind Gemma 4 on-device deployments, with Metal GPU acceleration on Apple platforms.
• Cross-platform parity. Teams shipping the same model on iOS and Android (where LiteRT-LM is the standard path) get identical weights and runtime behavior on both, behind the Foundation Models-shaped API this package already provides.
• It's device-proven. The underlying Swift package (swift-litert-lm) runs Gemma 4 E2B on an iPhone 17 Pro at ~50 tok/s decode on the Metal GPU, with text + image (+ audio/video in its own API) verified on hardware.

Shape of the change

Follows the existing backend pattern exactly:

• Package.swift: a LiteRT trait; john-rocky/swift-litert-lm dependency gated by .when(platforms: [.iOS, .macOS], traits: ["LiteRT"]). Default builds are unaffected (no trait, no dependency).
• Sources/AnyLanguageModel/Models/LiteRTLanguageModel.swift behind #if LiteRT: a LanguageModel conformance that maps the session transcript to LiteRT-LM conversations (system message + history + trigger), with:
  • respond / streamResponse for text
  • image segments for models with a vision tower
  • structured generation via schema-in-prompt + JSON extraction
  • tool calling for respond (prompt-driven, with the standard ToolExecutionDecision delegate flow)
• Env-gated tests (LITERT_TEST_MODEL), same style as the MLX suite.

Model weights are downloaded from the Hub on first use (e.g. litert-community/gemma-4-E2B-it-litert-lm, ~2.6 GB) and cached; engine bring-up is lazy.

I have this working (builds with --traits LiteRT on macOS; default-trait build unaffected) and am opening a PR alongside this issue — happy to adjust the approach based on feedback.