Persistent long-term memory via self-hosted Dakera — following Engram PR pattern

[ferhimedamine]

Overview

This proposes adding persistent long-term memory to Dify via Dakera — a self-hosted, decay-weighted vector memory server. The implementation follows the pattern established by the Engram draft PR (weaviate integration) and is fully opt-in behind environment variables.

The problem

Dify's TokenBufferMemory (api/core/memory/token_buffer_memory.py) is session-scoped and stored in the Dify database — it does not survive beyond a conversation window. Users building production AI applications need memory that:

• Persists across weeks and months of usage
• Works across multiple Dify conversation sessions and app instances
• Is semantically searchable (vector similarity, not just keyword match)
• Stays fully on-premise (EU GDPR compliance, enterprise air-gap requirements)

Proposed implementation

Following the Engram PR pattern exactly — new files, no breaking changes, fully opt-in:

New files:

api/core/memory/dakera/__init__.py
api/core/memory/dakera/dakera_memory.py
api/events/event_handlers/store_dakera_memory_when_message_created.py

Modified files:

• api/core/app/apps/chat/app_runner.py — inject recalled memories before organize_prompt_messages()
• api/configs/feature/__init__.py — add Pydantic Field declarations for new env vars

Environment variables (all off by default):

DAKERA_ENABLED=false
DAKERA_ENDPOINT=http://your-dakera-host:3000
DAKERA_API_KEY=dk_your_key
DAKERA_RECALL_TOP_K=5

Core class:

# api/core/memory/dakera/dakera_memory.py
class DakeraMemory:
    def __init__(self, endpoint: str, api_key: str, recall_top_k: int = 5):
        self._endpoint = endpoint.rstrip("/")
        self._headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
        self._top_k = recall_top_k

    def store(self, content: str, user_id: str, app_id: str) -> None:
        """Persist a message to long-term memory. Never raises — logs on failure."""
        try:
            requests.post(
                f"{self._endpoint}/v1/memories",
                headers=self._headers,
                json={"content": content, "agent_id": app_id, "session_id": user_id},
                timeout=5,
            )
        except Exception:
            logger.warning("Dakera store failed — continuing without persistence")

    def recall(self, query: str, user_id: str, app_id: str) -> list[str]:
        """Return relevant memories for query. Returns [] on any error."""
        try:
            resp = requests.post(
                f"{self._endpoint}/v1/memories/search",
                headers=self._headers,
                json={"query": query, "agent_id": app_id, "top_k": self._top_k},
                timeout=5,
            )
            return [r["content"] for r in resp.json().get("results", [])]
        except Exception:
            return []

Signal handler:

# api/events/event_handlers/store_dakera_memory_when_message_created.py
from events.message_event import message_was_created

@message_was_created.connect
def handle(sender, **kwargs):
    from configs import dify_config
    if not dify_config.DAKERA_ENABLED:
        return
    message = kwargs.get("message")
    application_generate_entity = kwargs.get("application_generate_entity")
    # extract text content and persist via DakeraMemory.store()

Differentiation vs Weaviate Engram approach

	Weaviate Engram	Dakera
Hosting	Cloud SaaS	Fully self-hosted
Data residency	External	Your infrastructure
Pricing	Subscription	Open source / free
Memory decay	Time-based	Decay-weighted (recency + access frequency)
Extra	—	Knowledge graph, entity extraction, MCP server (14 tools)

Next step

Happy to submit a PR implementing the above once maintainers confirm this direction is welcome. The implementation is complete on our side — we are waiting for guidance on the preferred integration pattern before opening a formal PR.