Repository navigation
mcp-slim & loraspectra: Open-source MCP stdio token optimizer and .safetensors LoRA spectral profiler for LocalAI #13200
SalyyS1
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hi LocalAI community,
When running LocalAI with Model Context Protocol (MCP) servers and custom
.safetensorsLoRA adapters on local hardware, we kept running into two bottlenecks:github,playwright,filesystem) inject 15,000 to 45,000+ tokens oftools/listJSON schemas and nested$refpointers that overflow local context windows and cause smaller quantized models to fail tool validation.r=32orr=64) waste VRAM and inference bandwidth when>99%of their spectral energy sits in the top 4 to 8 singular values.We released two lightweight open-source tools to solve both:
1. mcp-slim: Zero-dependency MCP stdio proxy & SecretGuard
tools/listschemas andtools/calloutputs by 50% to 98.7% (--toon,--dynamic-toolsetsBM25 search).$ref/$defspointers (--flatten-schemas) and coerces stringified tool arguments (--coerce-args) for local LLMs.--redact-secrets).2. loraspectra: Zero-dependency
.safetensorsLoRA spectral auditor & SVD rank prunererank), and dead/saturated ranks on CPU in milliseconds without PyTorch or GPU.--energy 0.99).Links:
All reactions