[fix] AI provider compatibility: local vLLM, Gemini, Meshy CORS + settings UX - #55
Merged
Conversation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…calls) - toolCallText.js: recover tool calls emitted as plain text (Qwen XML, Hermes JSON, fenced JSON gated on real tool names); hold streamed markup back from the transcript - client.js: non-streaming request path; detect unusable streamed turns (empty or invented tool name with no args - vLLM tool-call parser mismatch) and retry unstreamed, remember per provider for the session; read reasoning deltas for a thinking indicator - tools.js: repairToolCall fixes near-miss/invented tool names by alias and argument shape; system prompt forbids inventing tool names and repeating successful calls; uuids come from the scene summary - assistant.js: undo summary counts distinct objects (not tool calls), tool failures surface as errors, aiStatus thinking line - Settings: per-provider Stream responses toggle + Temperature field - new e2e suite ai-local-model (13 checks) covering all three broken server shapes; existing ai-assistant/mesh-generation suites green Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Gemini (OpenAI-compat): echo extra_content (thought_signature) back on
replayed tool_calls - dropping it 400s the follow-up turn ("missing
thought_signature" / "function_response.name cannot be empty"); tool
result messages carry name; verified live incl. multi-turn + streaming
- providers: normalize pasted Gemini shapes (native /v1beta or
/v1beta/models base -> /v1beta/openai; strip models/ prefix and
:generateContent suffix); gemini preset default gemini-flash-latest
- Meshy: assets.meshy.ai serves the finished GLB without CORS headers so
browsers cannot fetch it - fetchResult falls back to an asset proxy
(per-provider assetProxy field, default VITE_ASSET_PROXY); real HTTP
errors are not retried through the proxy; Settings input + note
- e2e: mesh-generation gains a CORS-blocked -> proxy fallback case (9
checks); ai suites green; svelte-check 485/72 held
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- '' saved by the Settings form must fall through to the default proxy: first non-EMPTY wins (|| not ??) - the user-hit bug - builds without the gitignored .env (CI/Pages) get no VITE_ASSET_PROXY; derive the default from VITE_PEER_HOST (https://<host>/proxy) - the proxy ships on the self-hosted peer server box by design - e2e: empty-field fallback case (default proxy intercepted + served) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- assets.meshy.ai is a KNOWN no-CORS host: even a caught direct-fetch failure paints a red CORS error in the devtools console per download; skip the doomed attempt and fetch through the asset proxy directly - direct fetch remains the fallback if the proxy is down, and the first attempt for unknown hosts - e2e: proxy-first case asserts ZERO direct requests to assets.meshy.ai Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- vite devAssetProxy plugin: same allowlisted CORS relay as the peerjs box route, dev-only (configureServer); https-only, assets.meshy.ai allowlist, redirects refused, 256MB cap - meshy adapter prefers the same-origin /proxy under import.meta.env.DEV (provider field still wins), so local dev works with the EC2 box off; built deployments keep VITE_ASSET_PROXY / peer-host derivation - e2e default-proxy case intercepts the same-origin dev default too Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- assetProxyCandidates: ordered non-empty candidates tried until one succeeds - provider field -> dev /proxy -> VITE_ASSET_PROXY (the proxy.theprototype.app Worker) -> derived peerjs-box /proxy; the box route stays as automatic fallback if the Worker is removed or over the free-tier quota - .env/.env.example: VITE_ASSET_PROXY=https://proxy.theprototype.app (Worker source + wrangler config live in the infra repo worker/) - e2e: dead-primary -> next-candidate fallback case (15 checks total) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Test connection result renders under the button (green check / red
cross + description) instead of a toast; cleared on form open
- Chrome's password manager saved the provider base-url + API key as a
login pair and autofilled it into the Connect peer-id box, and its
synthetic key events (key === undefined) crashed three global
handlers with 'Cannot read toLowerCase of undefined':
- editorNavigation/voiceChat/shortcuts now guard event.key
- provider key fields get autocomplete=new-password, url/label/model
autocomplete=off, Connect peer input autocomplete=off + name
- verified: synthetic undefined-key events, red/green inline results,
clean meshy-provider save, peer-input attrs (scratch playwright)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- the deployed site is built by the cloud repo's build-site.mjs from a CLEAN core clone (no .env), so VITE_ASSET_PROXY was never baked and the chain fell through to the disabled box route - prod Meshy died - '/proxy' is now a candidate on EVERY origin, not just dev: the vite middleware serves it locally, a Worker route (theprototype.app/proxy*, infra repo wrangler.toml) serves it on the deployed hostname - zero env needed, no CORS at all - viaProxies rejects 200 text/html (a static host's SPA fallback for unknown paths is "no proxy here", not a model) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… fix - model combobox in the provider form: suggestions from the endpoint's GET /models (works for vLLM / Gemini compat / Grok alike; Gemini's models/ prefix normalized), type-to-filter, scrollable list on focus, free text still allowed (aliases / custom ids); list persisted on the provider so Edit works after a reload, silently refreshed on Edit open - Test connection verdict: green "Connected - N models" + green "model X - Configuration OK" (listed, or verified by a 1-token completion probe for unlisted aliases), red "no model selected" / "did not respond"; endpoints without /models ask for a model id first - fixLight: the import path (STL/GLB/Meshy) now honors the same classic-preset-only guard /create already had - imports under a lit environment preset no longer nag "There is no light in the scene" - verified live against Gemini (57 models, alias probe, bogus-model red) + all AI e2e suites green Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- switching the provider preset (Grok/Gemini/Custom) no longer carries the typed API key across - each preset keeps its own key entry while the form is open (session-scoped map; only Save persists a key), and endpoint state (suggestions, test verdict) resets on switch - leaving the API-key or base-URL field auto-fetches the model list, so the picker fills without a Test connection click; sequence guard keeps a slow response for a previous endpoint from leaking in; "fetching list..." placeholder while in flight - base URL normalized (Gemini paste shapes) in the test + fetch paths Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes the AI assistant work reliably across all provider types and fixes the Meshy GLB download pipeline end to end.
Local models (vLLM / Qwen)
Gemini
extra_content(thought_signature) back on replayed tool_calls — dropping it 400s every follow-up turn; tool results carryname/v1beta→/v1beta/openai, stripmodels/prefix and:generateContentsuffix)Meshy (assets.meshy.ai sends no CORS headers)
/proxy(vite dev middleware locally; Cloudflare Worker routetheprototype.app/proxy*in prod) →VITE_ASSET_PROXY(proxy.theprototype.app Worker) → derived peerjs-box route (opt-in there)Settings UX
Tests
🤖 Generated with Claude Code