LLM-LSP: Hybrid Neural-Classical Code Completion

Abstract

LLM-LSP is an innovative code completion system that combines the power of Large Language Models (LLMs) with traditional Language Server Protocol (LSP) suggestions. This hybrid approach leverages probabilistic token selection to merge context-aware suggestions from neural models with statically analyzed completions from LSP servers.

Introduction

Modern code completion systems typically fall into two categories:

Traditional static analysis-based completions (LSP)
Neural network-based suggestions (LLMs)

This project bridges these approaches by implementing a novel probability-weighted completion system that can utilize both sources simultaneously, potentially offering more accurate and contextually relevant suggestions.

Technical Architecture

Core Components

Probability Integration System
- Implements Levenshtein distance-based similarity matching
- Combines LLM token probabilities with LSP suggestion rankings
- Uses configurable weights to balance between neural and classical suggestions
LSP Integration
- Real-time LSP suggestion collection
- Conversion of LSP rankings to probability space
- Similarity threshold-based matching with LLM tokens
LLM Interface
- Currently uses OpenAI's GPT-4 API (configurable)
- Structured for easy adaptation to local models (e.g., Llama)
- Streaming response handling for real-time completions

Key Features

Weighted probability combination algorithm
Real-time streaming completions
Configurable similarity thresholds
Extensible architecture for multiple LLM backends
Detailed logging system for debugging and analysis

Implementation Details

Probability Combination Algorithm

local lsp_weight = 0.3  -- Adjustable weight for LSP vs LLM balance
local similarity_threshold = 0.7  -- Minimum similarity score for matching

The system uses a weighted combination approach:

LSP suggestions are converted to a probability space
LLM token probabilities are preserved
Similarity matching identifies related suggestions
Final probabilities are computed using configurable weights

Future Potential

Local LLM Integration
- Integration with Llama and other local LLMs
- Reduced latency and improved privacy
- Custom fine-tuning possibilities
Advanced Probability Models
- Bayesian integration of multiple suggestion sources
- Dynamic weight adjustment based on context
- Learning from user selections
Performance Optimizations
- Caching mechanisms for frequent completions
- Parallel processing of multiple suggestion sources
- Optimized similarity matching algorithms

Usage

Set up environment variables:

export OPENAI_API_KEY="your-api-key"
export OPENAI_API_URL="your-api-url"  # Optional, defaults to OpenAI endpoint

In Neovim:
```
:AIAssist
```

Research Applications

This project serves as a proof of concept for:

Hybrid neural-classical code completion systems
Probability-based integration of multiple suggestion sources
Real-time streaming completion in editor environments

Contributing

Contributions are welcome, particularly in the following areas:

Integration with additional LLM backends
Improved probability combination algorithms
Performance optimizations
Documentation and testing

License

MIT License

Citation

If you use this work in your research, please cite:

@software{llm_lsp,
  title = {LLM-LSP: Hybrid Neural-Classical Code Completion},
  author = {Amirsalar Safaei Ghaderi},
  year = {2024},
  url = {https://github.com/amirsalarsafaei/llm-lsp.nvim}
}

License

amirsalarsafaei/llm-lsp.nvim

ErrorLooks like something went wrong!

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Error
Looks like something went wrong!

Packages