Skip to content

v5.0.0 — Msty 2.4.0+ Architecture, Bloom, Streamable HTTP

Choose a tag to compare

View all tags
@M-Pineapple M-Pineapple released this 18 Feb 10:58
· 13 commits to main since this release
v5.0.0
a47877b

Msty Admin MCP v5.0.0 — Release Notes

Release date: 18 February 2026
Minimum requirements: Python 3.10+, MCP SDK v1.0.0+, Msty Studio Desktop 2.4.0+

What's new

Bloom Behavioral Evaluation (Phase 6)

v5.0.0 introduces Bloom, a behavioral evaluation framework for local LLMs. Bloom tests models for problematic patterns like sycophancy, hallucination, and overconfidence, then scores them on a 0.0–1.0 scale. When a local model falls below quality thresholds, Bloom recommends handing the task off to Claude.

Six new tools power Bloom:

  • bloom_evaluate_model — Run multi-turn evaluations against specific behaviors
  • bloom_check_handoff — Get a confidence-scored recommendation on whether to escalate to Claude
  • bloom_get_history — Review past evaluation results
  • bloom_list_behaviors — See all evaluable behaviors (sycophancy, hallucination, overconfidence, and more)
  • bloom_get_thresholds — View quality thresholds per task category
  • bloom_validate_model — Check whether a model is suitable for Bloom evaluation

Port-based service discovery

The entire service layer has been rewritten for Msty 2.4.0+. Services are now discovered automatically via well-known ports rather than filesystem scanning:

Service Port
Local AI (Ollama) 11964
MLX 11973
LLaMA.cpp 11454
Vibe CLI Proxy 8317

No manual configuration required. The server detects what's running and adapts.

Streamable HTTP transport

In addition to the default stdio transport, v5.0.0 supports Streamable HTTP:

msty-admin-mcp --transport streamable-http  # Runs on http://localhost:8000

This enables remote access and integration with HTTP-based MCP clients.

Intelligence and calibration tools

Phase 4 (Intelligence) and Phase 5 (Calibration) bring structured model analytics:

Intelligence (5 tools):

  • Model performance metrics with time-series tracking
  • Conversation pattern analysis
  • Side-by-side model response comparison
  • Knowledge stack optimisation suggestions
  • Persona improvement recommendations

Calibration (4 tools):

  • Category-specific model quality tests
  • Response quality scoring (0.0–1.0)
  • Handoff trigger pattern detection
  • Historical calibration results

SQLite metrics storage

All performance data, calibration results, and conversation analytics are persisted to a local SQLite database at ~/.msty-admin/msty_admin_metrics.db. Created automatically on first run.

Complete tool inventory (36 tools)

Phase Tools Count
1 — Foundational Installation detection, database queries, health monitoring, tool listing, provider discovery, server status 6
2 — Configuration Config export/import, Claude sync, persona generation 4
3 — Service Integration Multi-backend orchestration across Ollama, MLX, LLaMA.cpp, and Vibe CLI Proxy 11
4 — Intelligence Performance metrics, conversation analysis, model comparison, optimisation 5
5 — Calibration Quality testing, response scoring, handoff triggers, history 4
6 — Bloom Evaluation Behavioral evaluation, handoff checks, threshold management 6

Breaking changes

  • Msty 2.4.0+ required. Port-based discovery replaces filesystem scanning. For older Msty versions, use v4.x.
  • Module restructure. Internal package layout changed — src/bloom/, src/calibration/, src/intelligence/ are new subpackages.
  • analyse_msty_health now returns structured JSON with service connectivity status per backend.

Migration from v4.x

  1. Update Msty Studio Desktop to 2.4.0+
  2. Pull the latest code or pip install --upgrade msty-admin-mcp
  3. No environment variable changes needed — defaults match Msty 2.4.0 ports
  4. Bloom tools require ANTHROPIC_API_KEY if you want Phase 6 capabilities

Performance expectations (Apple Silicon)

Task Model Latency Throughput
Simple chat llama3.2:3b 200–300ms 15–20 tok/s
Complex reasoning llama3.2:7b 500–800ms 8–12 tok/s
Calibration test llama3.2:7b 5–10s —
Bloom evaluation llama3.2:7b 30–60s —

Links

Assets 2
Loading