Skip to content

feat: Support inline code completion (FIM)#12262

Draft
roomote-v0[bot] wants to merge 1 commit intomainfrom
feature/fim-inline-completion
Draft

feat: Support inline code completion (FIM)#12262
roomote-v0[bot] wants to merge 1 commit intomainfrom
feature/fim-inline-completion

Conversation

@roomote-v0
Copy link
Copy Markdown
Contributor

@roomote-v0 roomote-v0 Bot commented May 3, 2026

Related GitHub Issue

Closes: #12261

Description

This PR attempts to address Issue #12261 by implementing FIM (Fill-in-the-Middle) inline code completion support for Roo Code. Feedback and guidance are welcome.

Key implementation details:

  • FimTokenFormatter: Maps model families (DeepSeek, CodeLlama, StarCoder, Mistral/Codestral, Qwen) to their respective FIM special tokens. Falls back to generic <|fim_prefix|>/<|fim_suffix|>/<|fim_middle|> tokens for unknown models.

  • FimApiClient: Lightweight API client supporting three endpoint patterns:

    • /v1/completions (OpenAI-compatible, DeepSeek)
    • /api/generate (Ollama native FIM)
    • /v1/fim/completions (Mistral/Codestral)
  • FimCompletionProvider: VS Code InlineCompletionItemProvider with debouncing (configurable, default 300ms), LRU-style caching (50 entries, 10s TTL), request cancellation via AbortController, and silent error handling.

  • FimService: Orchestrator that manages the provider lifecycle -- activates/deactivates based on settings changes, passes config updates without re-registering.

  • Settings added to GlobalSettings: fimEnabled, fimProvider, fimModelId, fimBaseUrl, fimDebounceMs, fimMaxTokens, fimApiKey (stored as secret).

Design decisions:

  • FIM model is configured independently from the chat model so users can use a cheap/fast FIM-specialized model (e.g., DeepSeek Coder) alongside a more capable chat model
  • Uses the legacy /v1/completions endpoint rather than chat completions, as FIM models work best with raw prefix/suffix tokens
  • The FIM provider is a standalone service that does not interact with the chat/task system

Test Procedure

  • 40 unit tests covering all four components (FimTokenFormatter, FimApiClient, FimCompletionProvider, FimService)
  • Run: cd src && npx vitest run services/fim/__tests__/
  • All tests pass; lint and type checks pass across the monorepo

Pre-Submission Checklist

  • Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
  • Scope: My changes are focused on the linked issue (one major feature/fix per PR).
  • Self-Review: I have performed a thorough self-review of my code.
  • Testing: New and/or updated tests have been added to cover my changes (if applicable).
  • Documentation Impact: I have considered if my changes require documentation updates (see "Documentation Updates" section below).
  • Contribution Guidelines: I have read and agree to the Contributor Guidelines.

Documentation Updates

  • Yes, documentation updates are required: Users will need guidance on configuring FIM settings (provider, model, API key, base URL). A settings UI section is planned as a follow-up.

Additional Notes

  • This is the backend/core implementation. A settings UI (webview section for toggling FIM and configuring provider/model) would be a natural follow-up PR.
  • The vscode mock was extended with InlineCompletionTriggerKind, InlineCompletionItem, and registerInlineCompletionItemProvider to support these tests.

Interactively review PR in Roo Code Cloud

Implements a lightweight FIM completion provider that registers as a
VS Code InlineCompletionItemProvider to provide ghost-text suggestions
as the user types, similar to GitHub Copilot.

Key components:
- FimTokenFormatter: Maps model families to FIM token formats (DeepSeek,
  CodeLlama, StarCoder, Mistral/Codestral, Qwen, generic)
- FimApiClient: Lightweight API client supporting /v1/completions,
  Ollama /api/generate, and Mistral /v1/fim/completions endpoints
- FimCompletionProvider: VS Code InlineCompletionItemProvider with
  debouncing, caching, and cancellation support
- FimService: Orchestrator managing provider lifecycle based on settings

Settings added to GlobalSettings:
- fimEnabled, fimProvider, fimModelId, fimBaseUrl, fimDebounceMs,
  fimMaxTokens, fimApiKey (secret)

Supported providers: openai-compatible, deepseek, mistral, ollama

Closes #12261
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ENHANCEMENT] Support inline code completion (FIM)

1 participant