Skip to content

DataMind v1.0.0

Latest
Latest

Choose a tag to compare

View all tags
@haolpku haolpku released this 04 Sep 13:08
v1.0.0
01a481f

DataMind v1.0.0

DataMind v1.0.0 is the first stable release of the inference-time data plane:
two cooperating agents share five typed data surfaces without collapsing writes
and reads into one unrestricted tool set.

Stable core: the native backend, local profile workflow, safety hooks, and
PyPI package are the supported v1.0.0 baseline. The optional SDK/CCR backend
and remote database dialects remain integration points that should be validated
in the target environment.

Installation

pip install datamind

Optional extras:

pip install 'datamind[mysql]'         # MySQL dialect support
pip install 'datamind[postgres]'      # PostgreSQL dialect support
pip install 'datamind[voyage]'        # Voyage embeddings
pip install 'datamind[huggingface]'   # Local BGE / e5 embeddings

What is new in v1.0.0

Inference-time data plane

The runtime presents five complementary surfaces to the agents:

  • Knowledge base (RAG): Chroma + BM25 retrieval with reciprocal-rank fusion.
  • Database: SQLAlchemy-backed SQLite, MySQL, and PostgreSQL dialects.
  • Graph: NetworkX graph storage with JSON persistence.
  • Skills: profile-scoped SKILL.md procedures plus safe utility tools.
  • Memory: scope-typed SQLite memory for global, profile, and session data.

StoreAgent receives 11 write tools and routes incoming data to the right
surface. RetrieveAgent receives 19 read/utility tools and remains strictly
read-only. The separation is enforced in code by two physical registries before
the model can see a tool.

Scope-typed Memory

Memory entries carry explicit scope, kind, and status fields:

  • scope: global, profile, or session
  • kind: preference, decision, workflow, summary, skill, or fact
  • status: active or archived

Recall combines scope-conditioned retrieval results, which helps keep
profile-specific preferences from leaking across tenants or sessions. Existing
v0.2 memory stores are migrated automatically on first open.

Hook-based safety layer

Every tool call passes through a shared HookChain that can return Allow,
Deny, AskUser, or Rewrite:

  • PathAllowlistHook resolves symlinks and blocks file access outside the
    active working directory or profile data directory.
  • DestructiveSqlHook uses sqlglot to detect destructive SQL operations.
    DELETE, UPDATE, DROP TABLE, TRUNCATE, and ALTER require confirmation;
    DROP DATABASE and DROP SCHEMA are denied.
  • AuditLogHook records tool calls in storage/<profile>/audit.jsonl with a
    hash chain for tamper detection. Secret-shaped fields such as api_key,
    password, and token are redacted automatically.

The browser UI displays a confirmation modal when user approval is needed.
Streaming responses include hook_asks_user and hook_denied events.

Protocol-neutral agent loops

The native loop supports Anthropic /v1/messages and OpenAI
/v1/chat/completions, including tool calls and streaming. The optional SDK
backend remains available for Claude Agent SDK + claude-code-router (CCR)
deployments; the backend is selected with DATAMIND__AGENT__BACKEND.

PyPI packaging

The package includes the runtime code, browser UI, and default skill catalog
needed for a standard pip install datamind workflow:

  • datamind/ Python package
  • datamind/static/app.html
  • datamind/skills/{code-review,db-ops-sop}/SKILL.md

Release publishing is configured through GitHub Actions with PyPI Trusted
Publishing, so package uploads do not require long-lived PyPI tokens.

Fixes and release hygiene

  • Added the repository-level Apache License 2.0 text and kept package metadata
    aligned with it.
  • Unified the package, documentation, and release version at 1.0.0.
  • Added a README hero diagram for the StoreAgent → data plane → RetrieveAgent
    concept.
  • Added a Python 3.11/3.12 CI workflow with pytest, HTTP API end-to-end
    coverage, and a deterministic real SQLite verification command.
  • Documented the native/SDK support matrix, stable API, public deployment
    security boundary, and terminology for inference-time data.
  • Switched packaging metadata to source the version from
    datamind.__version__.
  • Fixed an empty-wheel packaging issue caused by an overly broad package
    exclusion rule.

Quick start

pip install datamind

export DATAMIND__LLM__API_BASE=https://your-gateway.example.com
export DATAMIND__LLM__API_KEY=sk-...
export DATAMIND__LLM__MODEL=claude-sonnet-4-6

datamind chat

To run the browser UI locally:

python -m uvicorn datamind.server:app --port 8000
# Open http://127.0.0.1:8000

Verification

The v1.0.0 release was checked with:

161 passed, 5 skipped

The five skipped tests cover the optional claude_agent_sdk integration when
that dependency is not installed. Additional smoke checks cover the CLI,
package build, PyPI metadata, server route registration, memory, and safety
hooks.

Known limitations

  • BYOP HTTP sinks: chunk, vector, and graph payload schemas are defined, but
    HTTP sink endpoints and adapter examples are still in progress. Current ingest
    flows accept local file paths.
  • MySQL / PostgreSQL: dialect support is available through optional extras;
    run environment-specific validation before production use.
  • SDK backend: DATAMIND__AGENT__BACKEND=sdk requires the optional
    claude-agent-sdk package and (for OpenAI-format upstreams) CCR.
  • Large deployments: this release has not yet been validated against
    large-scale production workloads.

Roadmap

  • HTTP sink endpoints for chunk, vector, and graph ingestion.
  • Additional ETL adapter examples for common orchestration tools.
  • Broader database validation for MySQL and PostgreSQL.
  • Expanded safety and recovery evaluations for hook-governed execution.

Links

Please include the relevant smoke-check output and a redacted Settings()
snapshot when reporting a bug.

Assets 2
Loading