docs(readme): add any-llm platform links#907
Merged
Merged
Conversation
dpoulopoulos
approved these changes
Mar 10, 2026
| Switch between OpenAI, Anthropic, Mistral, Ollama, and more without changing your code. | ||
|
|
||
| [Documentation](https://mozilla-ai.github.io/any-llm/) | [Try the Demos](#-try-it) | [Contributing](#-contributing) | ||
| [Documentation](https://mozilla-ai.github.io/any-llm/) | [Platform (Beta)](https://any-llm.ai/) | [Try the Demos](#-try-it) | [Contributing](#-contributing) |
Member
There was a problem hiding this comment.
Have we merged the two docs?
Member
Author
There was a problem hiding this comment.
not yet, I'll update once it's done 👍
Merged
11 tasks
njbrake
added a commit
that referenced
this pull request
Jun 11, 2026
…2e into the matrix (#1127) ## Description Fixes two `otari` provider bugs and wires the `otari` provider into the integration matrix as an end-to-end test against the hosted gateway (`api.otari.ai`), running it like any other hosted provider via conftest rather than a separate job. ### Provider bug fixes (any-llm) - **`max_completion_tokens`**: the base OpenAI layer remaps `max_tokens` to `max_completion_tokens`, which the otari gateway 502s on for every upstream. `OtariProvider` now remaps it back to `max_tokens` (the base's documented override point). Reported upstream: mozilla-ai/otari-ai#1062. - **Pydantic `response_format`**: otari has no `parse()` helper, so the base's structured-output path passed the model class to the SDK and failed. The provider now sends a json_schema dict and reconstructs a `ParsedChatCompletion` (with `.parsed`). Both have unit tests and were verified against the live endpoint. ### e2e wiring - Reclassify otari as a non-local provider so it runs in `run-integration-tests`; gateway stays local. - otari test models point at `anthropic:claude-haiku-4-5` (otari uses `provider:model` IDs; Anthropic is what the hosted test account serves reliably). - New `tests/integration/conftest.py`: skips otari/gateway when no endpoint is configured (otari authenticates via `OTARI_AI_TOKEN` / platform mode), and skips capabilities the test account or provider can't serve yet, each pointing at a tracking issue. Mirrors the existing per-provider capability skips (LMSTUDIO, LLAMAFILE, HUGGINGFACE). - Add `OTARI_AI_TOKEN` to the job env and `otari` to `EXPECTED_PROVIDERS`; normalize `EXPECTED_PROVIDERS` parsing. ### Verified live `6 passed, 19 skipped, 0 failed` against `api.otari.ai` (completion, parallel completion, streaming, messages-streaming, structured output base-model + dataclass). ### Tracking issues filed for the rest - Account/gateway gaps: mozilla-ai/otari-ai#1036 (embeddings), #758 (`/v1/models`), #907 (no OpenAI / Responses API), plus image/pdf 502s and reasoning. - otari SDK schema bug (reasoning string): mozilla-ai/otari#143. - any-llm otari deserialization bugs (tool_calls, non-streaming messages): #1126. The skip entries point at these and should be removed as each is fixed. > **Requires** an `OTARI_AI_TOKEN` repo secret (the `tk_b_...` platform token) for the otari e2e to authenticate. Supersedes #1124 (skip-gate, folded in) and #1125 (separate-job draft). ## PR Type - 🐛 Bug Fix - 🚦 Infrastructure ## Relevant issues Fixes #1120 Related: #1126 ## Checklist - [x] I understand the code I am submitting. - [x] I have added unit tests that prove my fix/feature works - [x] I have run this code locally and verified it fixes the issue. - [x] New and existing tests pass locally - [ ] Documentation was updated where necessary - [x] I have read and followed the [contribution guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md) - [ ] **AI Usage:** - [ ] No AI was used. - [ ] AI was used for drafting/refactoring. - [x] This is fully AI-generated. ## AI Usage Information - AI Model used: Opus 4.8 - AI Developer Tool used: Claude Code - Any other info you'd like to share: The provider fixes and the e2e scoping were validated against the live otari.ai endpoint. Drafted via back-and-forth with @njbrake. - [x] I am an AI Agent filling out this form (check box if true) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Improved provider support for structured outputs, including JSON-schema-based response formatting and parsing. * **Tests** * Conditional skipping for integration tests when gateway credentials are absent. * Expanded unit tests covering completion parameter mapping and structured-output handling. * **Chores** * Tighter test environment parsing and local provider routing updates. * Updated CI integration workflow to export gateway token configuration. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This was referenced Jun 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add clear references to the hosted any-llm platform in the README so users can discover the managed option alongside the OSS library and gateway docs.
PR Type
Relevant issues
N/A
Checklist
AI Usage Information
When answering questions by the reviewer, please respond yourself, do not copy/paste the reviewer comments into an AI system and paste back its answer. We want to discuss with you, not your AI :)