feat: Support inline code completion (FIM)#12262
Draft
roomote-v0[bot] wants to merge 1 commit intomainfrom
Draft
feat: Support inline code completion (FIM)#12262roomote-v0[bot] wants to merge 1 commit intomainfrom
roomote-v0[bot] wants to merge 1 commit intomainfrom
Conversation
Implements a lightweight FIM completion provider that registers as a VS Code InlineCompletionItemProvider to provide ghost-text suggestions as the user types, similar to GitHub Copilot. Key components: - FimTokenFormatter: Maps model families to FIM token formats (DeepSeek, CodeLlama, StarCoder, Mistral/Codestral, Qwen, generic) - FimApiClient: Lightweight API client supporting /v1/completions, Ollama /api/generate, and Mistral /v1/fim/completions endpoints - FimCompletionProvider: VS Code InlineCompletionItemProvider with debouncing, caching, and cancellation support - FimService: Orchestrator managing provider lifecycle based on settings Settings added to GlobalSettings: - fimEnabled, fimProvider, fimModelId, fimBaseUrl, fimDebounceMs, fimMaxTokens, fimApiKey (secret) Supported providers: openai-compatible, deepseek, mistral, ollama Closes #12261
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related GitHub Issue
Closes: #12261
Description
This PR attempts to address Issue #12261 by implementing FIM (Fill-in-the-Middle) inline code completion support for Roo Code. Feedback and guidance are welcome.
Key implementation details:
FimTokenFormatter: Maps model families (DeepSeek, CodeLlama, StarCoder, Mistral/Codestral, Qwen) to their respective FIM special tokens. Falls back to generic
<|fim_prefix|>/<|fim_suffix|>/<|fim_middle|>tokens for unknown models.FimApiClient: Lightweight API client supporting three endpoint patterns:
/v1/completions(OpenAI-compatible, DeepSeek)/api/generate(Ollama native FIM)/v1/fim/completions(Mistral/Codestral)FimCompletionProvider: VS Code
InlineCompletionItemProviderwith debouncing (configurable, default 300ms), LRU-style caching (50 entries, 10s TTL), request cancellation via AbortController, and silent error handling.FimService: Orchestrator that manages the provider lifecycle -- activates/deactivates based on settings changes, passes config updates without re-registering.
Settings added to
GlobalSettings:fimEnabled,fimProvider,fimModelId,fimBaseUrl,fimDebounceMs,fimMaxTokens,fimApiKey(stored as secret).Design decisions:
/v1/completionsendpoint rather than chat completions, as FIM models work best with raw prefix/suffix tokensTest Procedure
cd src && npx vitest run services/fim/__tests__/Pre-Submission Checklist
Documentation Updates
Additional Notes
InlineCompletionTriggerKind,InlineCompletionItem, andregisterInlineCompletionItemProviderto support these tests.Interactively review PR in Roo Code Cloud