Repository navigation
Releases
IcyCache v0.4.0 - multilingual semantic anchors + a test suite that runs
The test suite runs again, and the semantic cache finally works in every major language.
Added
-
A multilingual anchor engine (
src/icy_lang.py) driven by 24 language profiles
(config/lang_profiles.json). The semantic cache needs "soft anchors" (named entities) to tell
"a DNS cache" from "a web cache". Casing only marks an entity in some languages, so the
engine now:- tokenises per script (Latin / CJK / Cyrillic / Arabic / Devanagari / Bengali) — without
this,DNSキャッシュwas a single token and the acronym was invisible; - detects the language (script first, then distinct stop-word hits — at least two hits and a
lead over the second candidate, otherwise "unknown"); - applies three casing models:
sentence(en, fr, es, pt, it, vi, tr — a capitalised word that
is not sentence-initial is an entity),noun(de, nl, pl, sv, da, cs, hu, fi, no, ru, uk —
all nouns are capitalised, so they are filtered against a common-word list),none
(zh, ja, ko, ar, hi, bn — no casing, only Latin acronyms and CamelCase count).
No language is hardcoded in the source any more — adding one is a single JSON key.
- tokenises per script (Latin / CJK / Cyrillic / Arabic / Devanagari / Bengali) — without
-
Per-language soft anchors on the MCP path.
mcp_server.pycompared the normalised prompt,
which is lower-case and therefore carried zero soft anchors: the "Montreal ≠ Quebec City"
guard was completely inert there. The raw text is now used on both the lookup and the indexing
side. -
tests/conftest.pyisolates the four test scripts from pytest collection, and
pyproject.tomlgains a[tool.pytest.ini_options]section (testpaths,python_files,
asyncio_mode).
Fixed
pytest tests/no longer dies silently. It exited with code 1 and zero output: three test
files are scripts, not pytest modules —test_cacheforge.pyandtest_cacheforge_ext.pycall
sys.exit()at import whenCF_BASEis missing, andtest_prefix_proxy.pyends with
os._exit(), which killed the whole pytest process.granite_live_test.pymatched*_test.py
and was collected, starting a live LM Studio benchmark during collection.- The two async tests could never pass.
pytest-asynciowas missing from thedevextra and
from the virtualenv, sotests/test_upstream_recovery.pyfailed with "async def functions are
not natively supported". - The French word "plus" was corrupting every prompt.
normalize_numbers()replaced
" plus "with"+"everywhere: "Donne-moi plus de détails" became "Donne-moi+de détails".
It changed the exact-cache key and the embedding input. Operators are now converted only
between two numeric operands, in a single shared implementation used byicy_normalize.pyand
icy_semantic.py(the two copies had drifted apart). - The path anchor swallowed every slash.
(?:\./|/)[^\s]+matched"A/B"→"/b"and
"12/24"→"/24", so two paraphrases could never share their hard anchors and the semantic
cache could never answer. A path now requires a drive root, a relative path, or at least two
POSIX segments. - Named entities were detected by capitalisation alone, which in French marks a sentence
start, not an entity.extract_soft()now uses acronyms, internal CamelCase and
capitalisation-outside-sentence-start, filtered per language.
Changed
- The
RELAXacceptance path now requires shared grounding (identical hard anchors, or at
least one common strong entity). Without it, "Montreal" and "Quebec City" were accepted at
0.99 similarity. - Version bumped to 0.4.0 —
pyproject.toml,package.json, the README badge and
docs/INSTALL.mdnow match this release.
⚠ Breaking
- The cache key format changed. The entries already in
db/cache.dbwere written with the
old normalisation and will never be read again. They expire by TTL (30 days) or can be dropped
withDELETE FROM cache; VACUUM;.
icycache 0.3.2 - prepared for publication (registry publish pending)
Install (once the registry publish lands)
# PyPI
uv tool install icycache # or: pip install icycache
# npm
npx -y icycacheUntil the first publish, those commands return a 404 — install from source with
pip install "icycache @ git+https://github.com/MlkBlk-IT/icycache.git".
What changed
Prepared for publication. The repository, the PyPI distribution and the npm package all carry
the same name — icycache. The wheel and the sdist are built and validated (twine check), the
v0.3.2 tag points at this release, and the first PyPI/npm publish is still pending. PyPI
publication uses trusted publishing (OIDC): the workflow mints a short-lived token from the GitHub
Actions identity, so no API token is stored anywhere.
Fixed.
- The README no longer claims the package is unpublished — v0.3.1 documented the commands as
"not published yet"; they now resolve to the real registries once the publish lands. docs/INSTALL.mdpointed atdist/icycache-0.2.0-py3-none-any.whl— the current version is
0.3.2.
Not changed. No code change: the gateway, the MCP layer, the semantic cache and the
normalization engine are exactly as shipped in v0.3.0.
IcyCache v0.3.1 — install instructions corrected
The install instructions were not true yet
Fixed
- Nothing was published on PyPI or npm.
pip install icycacheandnpx -y icycacheboth returned a 404 while the README presented them as working commands. The README now says so explicitly and gives the install-from-source command that works today:pip install "icycache @ git+https://github.com/MlkBlk-IT/icycache.git" - Unlike
loom-mcp(which belongs to other projects on both registries), the namesicycacheandicycache-mcpare free on PyPI and npm — so these commands are the final ones and will work as written once the first publish happens.
Changed
- Version bumped to 0.3.1.
Not changed
No code change: the gateway, the MCP layer, the semantic cache and the normalization engine are exactly as shipped in v0.3.0.
Full details: UPDATE-NOTES.md
IcyCache v0.3.0 — DELTA-002
DELTA-002 — upstream resilience, opt-in semantic cache, tools registry
One correction pass over the request path, covering three work items that all touch the same handler. Doing it as a single delta avoids merge conflicts and half-applied intermediate states.
Fixed
- BUG-001 (High) — the gateway stayed stuck on
502after a single upstream400. The pooled httpx client reused a socket the upstream had already closed, and httpx does not retry a mid-requestRemoteProtocolError, so every later call hung until the 120 s timeout.
New modulesrc/upstream_client.py: always drain the response, recycle the client on transport errors, retry once, and probe/v1/modelsbefore reusing a suspect pool. Error messages now carry the status code and the body instead of an empty string. - Number normalization —
"two plus two"→2+2, applied before hashing and before embedding. 11 new rules: 1111 → 1122.
Added
- Opt-in semantic cache (
src/icy_semantic.py,config/semantic_cache.json) — embeddings + Qdrant store. Only the last user message is compared semantically; model, temperature, system prompt and tools stay exactly hashed and act as a hard filter. Anchor grounding is what prevents "Montreal" from matching "Quebec City" at a ~0.97 cosine score. Indexing runs in a background thread, and every semantic-layer error degrades to a silent miss, never a 500. - Tools registry (
config/tools_registry.json) — the 100 verified MCP tool anchors from DELTA-001, asserted at startup (total == 100). Thetoolsarray is part of the exact cache key, so a tool response is never served for another tool set. - Tests —
test_semantic_cache.py,test_upstream_recovery.py,test_anchors.py. - Docs — BUG-001, BUG-002, DELTA-001 and DELTA-002 (with an English summary).
Changed
README.md— English, every count aligned with the code (1122 rules, 18 upstreams, 11 MCP tools), semantic cache documented as opt-in, DELTA-002 notice at the top.UPDATE-NOTES.md— rewritten in English as the release annotation.config/normalization_rules.json—total_rulesmetadata corrected to 1122.- Version bumped to 0.3.0 (
pyproject.toml,package.json).
Removed
Pre-rename artefacts (CacheForge_Project_Documentation.docx, start_cacheforge.ps1, _loop3d_new.js), the one-shot patch scripts under tools/, and the unused dashboard capture assets under static/.
Install
pip install "icycache @ git+https://github.com/MlkBlk-IT/icycache.git"The package was not on PyPI/npm when this release was published — the names
icycacheand
icycache-mcpare free on both registries, so those commands are the final ones. See v0.3.1.
Honest status
- The semantic cache ships DISABLED (
semantic_enabled: false). Turning it on requires an embedding endpoint (LM Studio on:1234) and a running Qdrant, plus a hit-ratio and false-positive campaign over 200+ real requests before the threshold drops below 0.95. - The exact-hash fast path is untouched and stays the first branch — no regression is possible from this release.
Full details: UPDATE-NOTES.md · docs/DELTA-002-unified-fix.md
IcyCache v0.2.0 — installable universal LLM gateway
IcyCache v0.2.0 — installable universal LLM gateway
An HTTP gateway that sits between any client and any upstream — local or cloud — caches the responses and serves them back instantly. No CPU regeneration, no paid API required, no GPU.
Install
pip install icycache # or: uv tool install icycache
npx -y icycache-mcp # MCP server on stdio, for Node-only clientsRun it:
icycache serve # the gateway, http://127.0.0.1:8080
icycache mcp # the MCP layer (SSE :8081 / HTTP :8084 / stdio)
icycache doctor # diagnostic: paths, config, upstreams, databaseFull guide: docs/INSTALL.md
What's in the box
- Universal multi-upstream routing — 18 upstreams (OmniRoute, LM Studio, Ollama, OpenAI, Anthropic, NVIDIA, Groq, Cerebras, OpenRouter, Kilo, Nous, Copilot, AI Gateway, HuggingFace, Google AI Studio, llm7.io, Mistral, Cloudflare). Pick one by URL prefix,
X-IcyCache-Upstreamheader, body field, or the configured default — never an implicit fallback - 11 MCP tools —
get_cache_stats,search_cached_prompts,normalize_preview,proxy_tool,proxy_tools,list_proxy_servers,list_proxy_tools,clear_cache,clear_tool_cache,get_tool_cache_stats,check_tool_loop - MCP tool cache with per-server TTL policy and a loop guard
- Normalization — NFKD → lowercase → Aho-Corasick → accents → punctuation, 1111 rules
- Live observability — WebSocket
/ws/eventsfor token deltas,/api/metrics,/api/metrics/export, and a 3D metrics dashboard - Real token accounting — usage comes from the upstream, never estimated
- Three transports at once — stdio, SSE (
:8081), Streamable HTTP (:8084) - Portable paths — the installed package never writes into
site-packages:db/andlogs/go to%LOCALAPPDATA%\IcyCache(Windows) or~/.local/share/IcyCache
Assets
| File | What it is |
|---|---|
icycache-0.2.0-py3-none-any.whl |
Wheel — pure Python, ~296 KB |
icycache-0.2.0.tar.gz |
Source distribution — code + configs + dashboard + tests, ~319 KB |
Both were verified by installing them into a clean virtualenv: /health, /api/status, /dashboard and the /ws/events WebSocket all answered, and the MCP stdio handshake returned the 11 tools through the npm wrapper.
Honest status
- The semantic cache is off at runtime.
config/icycache.gateway.jsoncarries twocacheblocks; the code reads the root one (enable_semantic: false, 7-day TTL) and the nestedgateway.cacheblock (enable_semantic: true, 0.80 threshold, 30-day TTL) is never read. Fixing this is the next task. - The gateway itself has no green suite.
tests/run_isolated.pyrunstest_cacheforge.pyandtest_cacheforge_ext.py(both ALL PASS), buttest_gateway.pyandtest_prefix_proxy.pyare not in the runner. pytestalone does not work —test_cacheforge.pyexits ifCF_BASEis missing. Usetests/run_isolated.py.
See PROGRESS.md for the full breakdown.
License
MIT