[ENHANCEMENT] Support inline code completion (FIM)

[tongkangyu]

Problem (one or two sentences)

Users must install additional plugins (e.g., Continue) just for real-time inline code completions, breaking a unified AI coding experience inside Roo Code.

Context (who is affected and when)

Roo Code excels at chat-based coding assistance, but when users need fast, predictive inline completions while typing, they must switch to another tool. This affects all developers who want a single-tool workflow, especially those with limited resources or who use affordable FIM-capable models like DeepSeek.

Desired behavior (conceptual, not technical)

Roo Code should optionally offer ghost-text-style inline completions as the user types, similar to GitHub Copilot. The user simply presses Tab to accept a suggestion, without opening a chat panel or installing another plugin. The completion model can be configured separately from the chat model.

Constraints / preferences (optional)

No response

Request checklist

• I've searched existing Issues and Discussions for duplicates
• This describes a specific problem with clear context and impact

Roo Code Task Links (optional)

No response

Acceptance criteria (optional)

No response

Proposed approach (optional)

Introduce a lightweight completion provider that calls a FIM-compatible API (e.g., /v1/completions with <> / <<>> tokens). Re-use any existing editor hooks for inline suggestions, or adapt patterns from open-source plugins like Continue.

Trade-offs / risks (optional)

Need to handle cases where the FIM model returns low-quality suggestions without distracting the user.

Could overlap with the chat panel's "generate code" features, but the two serve different interaction modes (passive vs. active).

[roomote-v0]

I've analyzed this issue and explored the codebase. Here's my proposed implementation plan for inline code completion (FIM) support:

Overview

This feature would register a VS Code InlineCompletionItemProvider that calls a FIM-compatible API to provide ghost-text suggestions as the user types.

Proposed Implementation Plan

1. Add FIM Configuration Types (packages/types/)

• Add new settings to GlobalSettings for FIM configuration:
  • fimEnabled: boolean - toggle the feature on/off
  • fimProvider: ProviderName - which provider to use for FIM (separate from chat)
  • fimModelId: string - model ID for FIM completions (e.g., deepseek-coder, codestral)
  • fimDebounceMs: number - debounce delay (default ~300ms)
  • fimMaxTokens: number - max tokens for completions (default ~128)
  • fimApiKey: string (secret) - separate API key for the FIM provider
  • fimBaseUrl: string - optional custom base URL

2. Create FIM Completion Provider (src/services/fim/)

• FimCompletionProvider.ts - Implements vscode.InlineCompletionItemProvider
  • Extracts prefix (text before cursor) and suffix (text after cursor) from the active document
  • Debounces requests to avoid flooding the API
  • Calls FIM API using the /v1/completions endpoint with FIM tokens (e.g., <|fim_prefix|>, <|fim_suffix|>, <|fim_middle|> for DeepSeek/CodeLlama style, or provider-specific tokens)
  • Returns InlineCompletionItem with the generated text
  • Supports cancellation via CancellationToken
• FimApiClient.ts - Lightweight API client for FIM requests
  • Supports the standard /v1/completions endpoint (not chat completions)
  • Handles FIM token formatting per model family
  • Reuses existing provider infrastructure where possible (OpenAI SDK for compatible endpoints)
• FimTokenFormatter.ts - Maps model families to their FIM token format
  • DeepSeek/CodeLlama: <|fim_prefix|>, <|fim_suffix|>, <|fim_middle|>
  • Codestral/Mistral: native FIM endpoint
  • StarCoder: <fim_prefix>, <fim_suffix>, <fim_middle>
  • Configurable for custom models

3. Register the Provider (src/extension.ts)

• Register FimCompletionProvider as an InlineCompletionItemProvider during extension activation
• Listen for configuration changes to enable/disable the provider dynamically
• Dispose the provider when disabled or on extension deactivation

4. Add Settings UI (webview-ui/)

• Add a "Code Completion" section in settings
  • Toggle to enable/disable inline completions
  • Provider selector (reuse existing provider dropdown pattern)
  • Model ID input
  • API key input
  • Debounce delay slider
  • Max tokens input

5. Quality Controls

• Filter low-confidence completions (empty or whitespace-only responses)
• Cache recent completions to avoid redundant API calls
• Respect .rooignore and file-type exclusions
• Add telemetry for completion acceptance rate

Key Design Decisions

• Separate from chat model: The FIM model is configured independently so users can use a cheap/fast FIM-specialized model (e.g., DeepSeek Coder) while keeping a more capable chat model
• Use /v1/completions endpoint: FIM models work best with the legacy completions endpoint rather than chat completions, as it supports raw prefix/suffix tokens
• Lightweight: The FIM provider is a standalone service that doesn't interact with the chat/task system
• Progressive rollout: Start with OpenAI-compatible FIM endpoints (covers DeepSeek, Ollama, LM Studio, etc.) and add provider-specific endpoints later

Estimated Scope

• ~8-10 new files
• Modifications to packages/types/, src/extension.ts, and settings UI
• Unit tests for the FIM provider, token formatter, and API client

This approach aligns with the issue's suggestion to adapt patterns from open-source plugins like Continue, while fitting into Roo Code's existing architecture.

Please mention @roomote in a comment saying that you approve this plan, and I'll proceed with the implementation.

[tongkangyu]

I approve this plan @roomote

[roomote-v0]

Thanks for the approval @tongkangyu! I've seen your comment and will start implementing the FIM inline code completion feature based on the plan. I'll open a draft PR to share progress shortly.

[roomote-v0]

Opened a draft PR with my attempt at implementing FIM inline code completion: #12262

The PR includes the core backend implementation (token formatting, API client, completion provider, service orchestrator) with 40 unit tests. A settings UI for configuring the FIM provider/model would be a natural follow-up. Feedback welcome!