Skip to content

Releases

IcyCache v0.4.0 - multilingual semantic anchors + a test suite that runs

Choose a tag to compare

@MlkBlk-IT MlkBlk-IT released this 03 Nov 18:46

The test suite runs again, and the semantic cache finally works in every major language.

Added

  • A multilingual anchor engine (src/icy_lang.py) driven by 24 language profiles
    (config/lang_profiles.json).
    The semantic cache needs "soft anchors" (named entities) to tell
    "a DNS cache" from "a web cache". Casing only marks an entity in some languages, so the
    engine now:

    • tokenises per script (Latin / CJK / Cyrillic / Arabic / Devanagari / Bengali) — without
      this, DNSキャッシュ was a single token and the acronym was invisible;
    • detects the language (script first, then distinct stop-word hits — at least two hits and a
      lead over the second candidate, otherwise "unknown");
    • applies three casing models: sentence (en, fr, es, pt, it, vi, tr — a capitalised word that
      is not sentence-initial is an entity), noun (de, nl, pl, sv, da, cs, hu, fi, no, ru, uk —
      all nouns are capitalised, so they are filtered against a common-word list), none
      (zh, ja, ko, ar, hi, bn — no casing, only Latin acronyms and CamelCase count).

    No language is hardcoded in the source any more — adding one is a single JSON key.

  • Per-language soft anchors on the MCP path. mcp_server.py compared the normalised prompt,
    which is lower-case and therefore carried zero soft anchors: the "Montreal ≠ Quebec City"
    guard was completely inert there. The raw text is now used on both the lookup and the indexing
    side.

  • tests/conftest.py isolates the four test scripts from pytest collection, and
    pyproject.toml gains a [tool.pytest.ini_options] section (testpaths, python_files,
    asyncio_mode).

Fixed

  • pytest tests/ no longer dies silently. It exited with code 1 and zero output: three test
    files are scripts, not pytest modules — test_cacheforge.py and test_cacheforge_ext.py call
    sys.exit() at import when CF_BASE is missing, and test_prefix_proxy.py ends with
    os._exit(), which killed the whole pytest process. granite_live_test.py matched *_test.py
    and was collected, starting a live LM Studio benchmark during collection.
  • The two async tests could never pass. pytest-asyncio was missing from the dev extra and
    from the virtualenv, so tests/test_upstream_recovery.py failed with "async def functions are
    not natively supported"
    .
  • The French word "plus" was corrupting every prompt. normalize_numbers() replaced
    " plus " with "+" everywhere: "Donne-moi plus de détails" became "Donne-moi+de détails".
    It changed the exact-cache key and the embedding input. Operators are now converted only
    between two numeric operands, in a single shared implementation used by icy_normalize.py and
    icy_semantic.py (the two copies had drifted apart).
  • The path anchor swallowed every slash. (?:\./|/)[^\s]+ matched "A/B" → "/b" and
    "12/24" → "/24", so two paraphrases could never share their hard anchors and the semantic
    cache could never answer. A path now requires a drive root, a relative path, or at least two
    POSIX segments.
  • Named entities were detected by capitalisation alone, which in French marks a sentence
    start, not an entity. extract_soft() now uses acronyms, internal CamelCase and
    capitalisation-outside-sentence-start, filtered per language.

Changed

  • The RELAX acceptance path now requires shared grounding (identical hard anchors, or at
    least one common strong entity). Without it, "Montreal" and "Quebec City" were accepted at
    0.99 similarity.
  • Version bumped to 0.4.0 — pyproject.toml, package.json, the README badge and
    docs/INSTALL.md now match this release.

⚠ Breaking

  • The cache key format changed. The entries already in db/cache.db were written with the
    old normalisation and will never be read again. They expire by TTL (30 days) or can be dropped
    with DELETE FROM cache; VACUUM;.

icycache 0.3.2 - prepared for publication (registry publish pending)

Choose a tag to compare

@MlkBlk-IT MlkBlk-IT released this 26 Oct 05:18

Install (once the registry publish lands)

# PyPI
uv tool install icycache        # or: pip install icycache

# npm
npx -y icycache

Until the first publish, those commands return a 404 — install from source with
pip install "icycache @ git+https://github.com/MlkBlk-IT/icycache.git".

What changed

Prepared for publication. The repository, the PyPI distribution and the npm package all carry
the same name — icycache. The wheel and the sdist are built and validated (twine check), the
v0.3.2 tag points at this release, and the first PyPI/npm publish is still pending. PyPI
publication uses trusted publishing (OIDC): the workflow mints a short-lived token from the GitHub
Actions identity, so no API token is stored anywhere.

Fixed.

  • The README no longer claims the package is unpublished — v0.3.1 documented the commands as
    "not published yet"; they now resolve to the real registries once the publish lands.
  • docs/INSTALL.md pointed at dist/icycache-0.2.0-py3-none-any.whl — the current version is
    0.3.2.

Not changed. No code change: the gateway, the MCP layer, the semantic cache and the
normalization engine are exactly as shipped in v0.3.0.

IcyCache v0.3.1 — install instructions corrected

Choose a tag to compare

@MlkBlk-IT MlkBlk-IT released this 26 Oct 04:23

The install instructions were not true yet

Fixed

  • Nothing was published on PyPI or npm. pip install icycache and npx -y icycache both returned a 404 while the README presented them as working commands. The README now says so explicitly and gives the install-from-source command that works today:
    pip install "icycache @ git+https://github.com/MlkBlk-IT/icycache.git"
  • Unlike loom-mcp (which belongs to other projects on both registries), the names icycache and icycache-mcp are free on PyPI and npm — so these commands are the final ones and will work as written once the first publish happens.

Changed

  • Version bumped to 0.3.1.

Not changed

No code change: the gateway, the MCP layer, the semantic cache and the normalization engine are exactly as shipped in v0.3.0.

Full details: UPDATE-NOTES.md

IcyCache v0.3.0 — DELTA-002

Choose a tag to compare

@MlkBlk-IT MlkBlk-IT released this 26 Oct 04:18

DELTA-002 — upstream resilience, opt-in semantic cache, tools registry

One correction pass over the request path, covering three work items that all touch the same handler. Doing it as a single delta avoids merge conflicts and half-applied intermediate states.

Fixed

  • BUG-001 (High) — the gateway stayed stuck on 502 after a single upstream 400. The pooled httpx client reused a socket the upstream had already closed, and httpx does not retry a mid-request RemoteProtocolError, so every later call hung until the 120 s timeout.
    New module src/upstream_client.py: always drain the response, recycle the client on transport errors, retry once, and probe /v1/models before reusing a suspect pool. Error messages now carry the status code and the body instead of an empty string.
  • Number normalization — "two plus two" → 2+2, applied before hashing and before embedding. 11 new rules: 1111 → 1122.

Added

  • Opt-in semantic cache (src/icy_semantic.py, config/semantic_cache.json) — embeddings + Qdrant store. Only the last user message is compared semantically; model, temperature, system prompt and tools stay exactly hashed and act as a hard filter. Anchor grounding is what prevents "Montreal" from matching "Quebec City" at a ~0.97 cosine score. Indexing runs in a background thread, and every semantic-layer error degrades to a silent miss, never a 500.
  • Tools registry (config/tools_registry.json) — the 100 verified MCP tool anchors from DELTA-001, asserted at startup (total == 100). The tools array is part of the exact cache key, so a tool response is never served for another tool set.
  • Tests — test_semantic_cache.py, test_upstream_recovery.py, test_anchors.py.
  • Docs — BUG-001, BUG-002, DELTA-001 and DELTA-002 (with an English summary).

Changed

  • README.md — English, every count aligned with the code (1122 rules, 18 upstreams, 11 MCP tools), semantic cache documented as opt-in, DELTA-002 notice at the top.
  • UPDATE-NOTES.md — rewritten in English as the release annotation.
  • config/normalization_rules.json — total_rules metadata corrected to 1122.
  • Version bumped to 0.3.0 (pyproject.toml, package.json).

Removed

Pre-rename artefacts (CacheForge_Project_Documentation.docx, start_cacheforge.ps1, _loop3d_new.js), the one-shot patch scripts under tools/, and the unused dashboard capture assets under static/.

Install

pip install "icycache @ git+https://github.com/MlkBlk-IT/icycache.git"

The package was not on PyPI/npm when this release was published — the names icycache and
icycache-mcp are free on both registries, so those commands are the final ones. See v0.3.1.

Honest status

  • The semantic cache ships DISABLED (semantic_enabled: false). Turning it on requires an embedding endpoint (LM Studio on :1234) and a running Qdrant, plus a hit-ratio and false-positive campaign over 200+ real requests before the threshold drops below 0.95.
  • The exact-hash fast path is untouched and stays the first branch — no regression is possible from this release.

Full details: UPDATE-NOTES.md · docs/DELTA-002-unified-fix.md

IcyCache v0.2.0 — installable universal LLM gateway

Choose a tag to compare

@MlkBlk-IT MlkBlk-IT released this 25 Oct 16:43

IcyCache v0.2.0 — installable universal LLM gateway

An HTTP gateway that sits between any client and any upstream — local or cloud — caches the responses and serves them back instantly. No CPU regeneration, no paid API required, no GPU.

Install

pip install icycache          # or: uv tool install icycache
npx -y icycache-mcp           # MCP server on stdio, for Node-only clients

Run it:

icycache serve                # the gateway, http://127.0.0.1:8080
icycache mcp                  # the MCP layer (SSE :8081 / HTTP :8084 / stdio)
icycache doctor               # diagnostic: paths, config, upstreams, database

Full guide: docs/INSTALL.md

What's in the box

  • Universal multi-upstream routing — 18 upstreams (OmniRoute, LM Studio, Ollama, OpenAI, Anthropic, NVIDIA, Groq, Cerebras, OpenRouter, Kilo, Nous, Copilot, AI Gateway, HuggingFace, Google AI Studio, llm7.io, Mistral, Cloudflare). Pick one by URL prefix, X-IcyCache-Upstream header, body field, or the configured default — never an implicit fallback
  • 11 MCP tools — get_cache_stats, search_cached_prompts, normalize_preview, proxy_tool, proxy_tools, list_proxy_servers, list_proxy_tools, clear_cache, clear_tool_cache, get_tool_cache_stats, check_tool_loop
  • MCP tool cache with per-server TTL policy and a loop guard
  • Normalization — NFKD → lowercase → Aho-Corasick → accents → punctuation, 1111 rules
  • Live observability — WebSocket /ws/events for token deltas, /api/metrics, /api/metrics/export, and a 3D metrics dashboard
  • Real token accounting — usage comes from the upstream, never estimated
  • Three transports at once — stdio, SSE (:8081), Streamable HTTP (:8084)
  • Portable paths — the installed package never writes into site-packages: db/ and logs/ go to %LOCALAPPDATA%\IcyCache (Windows) or ~/.local/share/IcyCache

Assets

File What it is
icycache-0.2.0-py3-none-any.whl Wheel — pure Python, ~296 KB
icycache-0.2.0.tar.gz Source distribution — code + configs + dashboard + tests, ~319 KB

Both were verified by installing them into a clean virtualenv: /health, /api/status, /dashboard and the /ws/events WebSocket all answered, and the MCP stdio handshake returned the 11 tools through the npm wrapper.

Honest status

  • The semantic cache is off at runtime. config/icycache.gateway.json carries two cache blocks; the code reads the root one (enable_semantic: false, 7-day TTL) and the nested gateway.cache block (enable_semantic: true, 0.80 threshold, 30-day TTL) is never read. Fixing this is the next task.
  • The gateway itself has no green suite. tests/run_isolated.py runs test_cacheforge.py and test_cacheforge_ext.py (both ALL PASS), but test_gateway.py and test_prefix_proxy.py are not in the runner.
  • pytest alone does not work — test_cacheforge.py exits if CF_BASE is missing. Use tests/run_isolated.py.

See PROGRESS.md for the full breakdown.

License

MIT