Add custom provider model selection and capability options - #112
Conversation
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
Important
The override plumbing is sound on the desktop runtime path, but three gaps should be addressed before merge: the per-message image limit counts assistant-displayed images (a chat can become permanently unable to generate), the Bot catalog and Telegram picker ignore the new overrides, and the stale manual-add path drops previously configured manual models and overrides.
Reviewed changes
- Portable model intent:
customModelOptionsis split out ofmodelMetadataon save and re-merged on read, with explicit-reset semantics insaveProviderand validation inisPortableProvider. - Runtime enforcement:
resolveProviderRuntimeLimitsapplies context/output/reasoning/vision overrides (maxImages: 0forces text-only), and generation enforces the image limit and removes tools whentoolCall === false. - Display and native projection:
withProviderFallbackoverlays overrides and exposesdetectedCapabilities;AidenRemoteModelServicenow honors overrides forsupportsImages. - Settings UI: new
CustomModelOptionsEditorunder a "More options" disclosure with manual model IDs, per-model switches, numeric limits, and reset; model visibility moved into the same disclosure. - Tests: focused
custom-model-options.test.ts, config-store restart/reset test, Android/iOS decode tests, and an Electron e2e spec covering save/rediscovery/reset/manual entry.
ℹ️ Nitpicks
- The new
toolCall === falseruntime gate (llm-client.ts:2068-2071) has no test; only its persistence is covered. A focused test that tools are actually removed would pin the behavior.
DeepSeek Flash ｜ 𝕏
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
Reviewed 96958601 since the prior pullfrog review at 7351057e — the four inline concerns are addressed and pinned by new regressions.
- Role-aware image limit: The limit now counts only user-authored attachments, so assistant-displayed images can no longer block generation.
- Bot and Telegram projection: Bot inventory applies
overrides.vision ?? visionwithmaxImages !== 0, and Telegram model choices route through the new override-awaretelegramModelChoice. - Numeric override clearing: Clearing a numeric field deletes the key instead of storing explicit
undefined, restoring the detected placeholder before save. - Stale manual-add: Adding a manual ID while discovery is stale now retains prior manual IDs and their overrides.
- Tool policy extraction:
applyCustomModelToolPolicy/prepareCustomModelToolContextcentralize thetoolCall: falsegate, with a harness test showing base and extension tools are removed from a frozen snapshot without mutating it. - Regression coverage: Added focused tests for assistant images, Telegram reasoning overrides, Bot image flags, deferred browser discovery, and the Electron save/rediscovery/reset/manual flows.
Verified with npm run type-check, the focused custom-model-options, telegram-controls, and bot-capability-inventory-ports suites, and the new harness test — all pass.
DeepSeek Flash ｜ 𝕏
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
Reviewed 2c9e90b1 since the prior pullfrog review at 9695860 — a test-only follow-up that removes the CI race in the Git push cancellation fixture.
- Deterministic cancellation handshake: the wrapper now writes its completion marker as soon as
git pushexits successfully and stays alive, so the test aborts into the cancellation-reconciliation path instead of racing a two-second process-exit window. - Bounded wait and cleanup drain:
waitForFile(marker, 15_000)replaces the fixed 150-attempt poll, and afinallyblock aborts and awaits the operation so fixture cleanup never races a live push;pushTimeoutMsrises to30_000as a backstop above the marker wait. - Papercuts note: records the pre-existing fixture race for future reference.
Verified locally by running the updated test four times and the full push-related subset of git.test.ts (19/19 pass). The head-SHA E2E, Android, and verify checks were still in progress at review time.
DeepSeek Flash ｜ 𝕏
a43dd15 to
f1e4fde
Compare
There was a problem hiding this comment.
Caution
The corrected merge f1e4fde still leaves main/services/pi-agent-runtime-harness.test.ts unparseable: the conflict resolution dropped the closing }); of the custom tool disabling… test, so tsc --noEmit and test:compaction (run by npm test) fail. The package.json and .papercuts/troubleshooting.md resolutions are clean.
Reviewed changes
Reviewed f1e4fde since the prior pullfrog review at 2c9e90b1 — a re-created merge of origin/main that replaces the conflict-marker commit a43dd156.
- Resolved merge conflicts:
package.jsonand.papercuts/troubleshooting.mdno longer contain markers; thetestscript keepsnpm run test:custom-model-options &&on top of main's full test list, with no main tests dropped. - Harness test still broken: the
custom tool disabling…test lost its closing});, so main's followingprovider diagnostics…test nests inside it and the file ends with an unclosed callback.
DeepSeek Flash ｜ 𝕏
…ed in merge resolution
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes
Reviewed 79f0d54f since the prior pullfrog review at f1e4fde — a one-line fix that closes the harness test left unclosed by the merge resolution.
- Restored the harness test closing brace: the
custom tool disabling…test inmain/services/pi-agent-runtime-harness.test.tsnow ends with});, so the followingprovider diagnostics…test is parsed as a sibling and the file no longer breakstsc --noEmitortest:compaction.
Verified locally at the head: npm run type-check exits clean and npx tsx --test main/services/pi-agent-runtime-harness.test.ts passes 60/60. The remaining head-SHA checks were still running at review time.
DeepSeek Flash ｜ 𝕏
1 participant
Custom and Tailscale endpoints can now expose More options for model visibility, manual model IDs, and per-model capabilities and limits. Users can override vision, reasoning, tool calling, open weights, server video support, context length, output tokens, and images per message, or reset to detected capabilities.
User-authored model settings persist in portable configuration separately from discovery metadata. Explicit rediscovery preserves overrides and manual IDs; manual setup also works without a discovery endpoint. Runtime limits, tool availability, and native image capabilities respect the overrides. Video records server support; Aiden does not add video uploads.
The existing Git push-cancellation test now waits for an explicit completion marker and drains cancellation before fixture cleanup, fixing a timing race exposed by CI. Its upstream-safety assertions are unchanged.
Validation: