Skip to content

v0.1.0-alpha.21 — Structured-output reliability + OTel observability hooks (additive)

Pre-release
Pre-release

Choose a tag to compare

@baabakk baabakk released this 19 Jun 04:52

What's in this release

Two coordinated themes, all additive, no breaking changes.

Structured-output reliability (closes #46, #47, #48)

Three changes driven by ADW's 2026-06-18 LLM Provider Test Report showing that DeepInfra + Parasail were the only cheap-tier providers dropping required fields on the first structured-output attempt:

  • Per-call strict?: boolean on GenerateStructuredOptions and StreamStructuredOptions. Precedence: per-call > adapter-level > auto-detect. Lets registries with one adapter alias per provider flip strict on/off per call based on the schema shape (closed-shape → strict, z.record(...) → json_object). Plumbed through the 5 structured-output capability factories (createClassifier / createScorer / createExtractor / createAnalyzer / createPlanner).
  • Strict-mode allowlist in autoDetectStrictResponseFormat extended to api.deepinfra.com and api.parasail.io. ADW's 8-calls-per-provider sweep: deepseek-flash 2/8 → 0/8 retries, gemma-31b 8/8 → 0/8, mimo-parasail 3/8 → 0/8.
  • Bundled pricing entries for three compat models in active production use against the verified-OK matrix: deepseek-ai/DeepSeek-V4-Flash ($0.10 / $0.20 per 1M, DeepInfra), google/gemma-4-31B-it ($0.10 / $0.20 per 1M, DeepInfra), XiaomiMiMo/MiMo-V2.5 ($0.14 / $0.28 per 1M, Parasail). Consumers no longer need a parallel pricingOverrides table for these.

The isJsonModeRejection matcher fix from #46 section 1 was already in main as 9cb9a82 and is now bundled in alpha.21 as well.

OTel-aligned observability hooks

RegistryOptions.observability accepts a bundle of five fire-and-forget hooks aligned with OpenTelemetry's `gen_ai.*` semantic-conventions taxonomy so downstream pipelines (Honeycomb, Datadog, OTel Collector, custom OTLP exporters) can map them onto spans + metrics without re-deriving fields:

```ts
const registry = createRegistryFromEnv({
// ...existing options...
observability: {
onCost: (e) => { /* per-call USD breakdown / },
onTokenUsage: (e) => { /
per-call token counts / },
onFallback: (e) => { /
fired when chain advances / },
onCacheHit: (e) => { /
fired when cached_tokens > 0 / },
onValidationRetry: (e) => { /
type only in alpha.21 */ },
},
});
```

All hooks sync OR async, errors swallowed (observability can't break inference).

Emission coverage

Hook Emitted in alpha.21? Where
`onCost` Every successful `generateText` / `generateStructured` / `runAgent`
`onTokenUsage` Every successful `generateText` / `generateStructured` / `runAgent`
`onCacheHit` When the response reports `cacheReadTokens > 0`
`onFallback` When `walkChain` advances from one alias to the next
`onValidationRetry` Type only (emission deferred to alpha.22) Use adapter `onRetry` with `reason === "validation-feedback"` today

Stream methods do not emit cost yet (streamed cost surfacing is the alpha.22 follow-up).

Backwards compatibility

All four additions are additive. Calls without `strict`, registries without `observability`, adapters against allowlisted baseURLs unchanged. Bundled pricing additions are pure-additive.

Tests

  • 14 new adapter-openai tests (per-call strict precedence + DeepInfra/Parasail allowlist + bundled pricing)
  • 22 new core observability-hooks tests (4 emitted hooks + 3 emit helpers + deriveCacheHit pure helper + Registry backwards-compat regression)
  • All existing 505 unit tests across core + capabilities + 5 adapters pass

What's next (alpha.22 plan)

  • `onValidationRetry` Registry-level emission
  • Streamed cost surfacing for `onCost` / `onTokenUsage`
  • Optional Cerebras pricing entries pending rate confirmation
  • LP-REQ-01 resilient fallback preset (carried over from alpha.20.1)

Install: `pnpm add @llm-ports/core@alpha @llm-ports/adapter-openai@alpha @llm-ports/capabilities@alpha`