Skip to content

lighthouse-web3/memory

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Lighthouse Memory

Durable long-term memory for AI agents, stored on Walrus via the Lighthouse x Walrus integration — or on Lighthouse's standard IPFS + Filecoin network, selectable per store with the network parameter (ipfs-walrus | ipfs-filecoin).

Agents get remember / recall / forget primitives whose contents live as verifiable blobs on the Walrus network (on Sui), addressed by IPFS-compatible CIDs, and retrievable through Lighthouse gateways. Memory survives sessions, machines, and runtimes.

How it works

agent ──(MCP tools / TS API)──► Memory
                                   │
                     ┌─────────────┼──────────────────┐
                     ▼             ▼                  ▼
              memory blob    local search index   index snapshot
              (JSON on       (~/.walrus-agent-    (also on Walrus,
               Walrus, CID)    memory/*.json)       for recovery)
  • Memories are batched into immutable JSON blobs on Walrus, uploaded through upload-walrus.lighthouse.storage and readable by anyone with the CID at https://gateway-walrus.lighthouse.storage/ipfs/<CID>. Writes buffer locally as pending and flush as one blob per flushEvery memories (default 10) — each blob also carries a full index snapshot, so a flush is simultaneously a backup. Pending memories are searchable immediately but live only on the local machine until flushed (flush() / memory_flush).
  • A local index (content, tags, CIDs, embedding vectors) makes recall fast without network round trips.
  • Semantic recall by default: memories are embedded locally with all-MiniLM-L6-v2 (via @huggingface/transformers, ~25 MB model downloaded once, no API key, nothing leaves the machine). Recall ranks by 0.7 × cosine similarity + 0.3 × keyword/tag overlap; memories stored before embeddings existed are embedded lazily on first recall. If the model can't load, recall degrades to keyword scoring.
  • Every batch blob embeds the full index, so the index can be rebuilt from the network (rebuild) — restore full memory on a brand-new machine from just your API key. Legacy one-blob-per-memory files and standalone snapshots from older versions are still read.
  • Walrus-native blob IDs for any memory are available via blobIds() for Sui ecosystem tooling.

Note: memories are stored unencrypted in this version — anyone with a CID can read it. Lighthouse's encrypted-upload support for Walrus is "coming soon" in their docs; don't store secrets in the meantime.

Setup

1. Get a Lighthouse Walrus API key

  1. Open the Lighthouse Walrus Dashboard and sign in with a Sui wallet (Slush).
  2. Go to API Keys, create a key, and copy it.

(Programmatic alternative: sign an auth message with a Sui keypair — see Create an API Key.)

2. Install

npm install
npm test        # offline smoke test, no API key needed

3. Configure

Env var Required Default Purpose
LIGHTHOUSE_API_KEY yes Lighthouse API key
MEMORY_NETWORK no ipfs-walrus Storage backend: ipfs-walrus or ipfs-filecoin (aliases walrus/filecoin accepted)
MEMORY_NAMESPACE no default Isolates memories per agent/project
MEMORY_AGENT no agent Agent id recorded on each memory
MEMORY_DIR no ~/.lighthouse-memory Local index location
MEMORY_FLUSH_EVERY no 10 Upload a batch blob once this many memories are pending (1 = upload on every write)
MEMORY_EMBEDDINGS no local Set off to disable semantic search (keyword-only)

Use as MCP server (any agent, automatically)

Run the server over stdio:

LIGHTHOUSE_API_KEY=lh_... npm run mcp

Add to Claude Code:

claude mcp add walrus-memory \
  --env LIGHTHOUSE_API_KEY=lh_... \
  -- npx tsx /path/to/Memory-1/src/mcp-server.ts

The agent then has these tools and will store/recall memory on Walrus on its own:

Tool What it does
memory_remember Store content (+tags, metadata); buffers locally, flushes to Walrus in batches
memory_flush Force all pending memories onto Walrus as one batch blob now
memory_recall Semantic + keyword search over memories (flushed and pending), optional tag filter
memory_list Newest-first listing
memory_get Fetch the full record from Walrus by id or CID
memory_forget Remove from index; deletes the blob once no other memory shares it
memory_rebuild_index Restore the index from Walrus on a new machine
memory_status Namespace, count, index path, last snapshot CID

Use as a library

import { Memory, LighthouseClient } from '@lighthouse-web3/memory'

const memory = new Memory(
  new LighthouseClient(process.env.LIGHTHOUSE_API_KEY!, {
    network: 'ipfs-walrus', // or 'ipfs-filecoin'
  }),
  { namespace: 'support-bot', agent: 'triage-agent' }
)

const stored = await memory.remember(
  'Customer ACME is on the enterprise plan; escalations go to the sev-1 channel.',
  { tags: ['customer', 'escalation'] }
)
// stored.cid → IPFS-compatible CID, backed by a Walrus blob on Sui
// stored.gatewayUrl → https://gateway-walrus.lighthouse.storage/ipfs/<CID>

const matches = await memory.recall('how do I escalate for ACME?')
const record = await memory.get(matches[0].id)     // full record from Walrus
const blobIds = await memory.blobIds(matches[0].id) // Walrus-native blob IDs

Live demo (uploads real blobs):

LIGHTHOUSE_API_KEY=lh_... npm run demo

Durability model

Failure Recovery
Session ends Flushed memories are on Walrus; pending memories are in the local index — nothing lost
Local machine lost before flush Pending (unflushed) memories are gone — call memory_flush before ending important sessions, or set MEMORY_FLUSH_EVERY=1
Local index deleted rebuild() restores from the newest batch blob's embedded index snapshot
Snapshots unreadable rebuild() falls back to re-reading every memory blob via the Lighthouse file list
New machine Set LIGHTHOUSE_API_KEY, run memory_rebuild_index

Deletion (forget) stops Lighthouse from renewing Walrus storage for that blob; the content remains readable until the current storage period expires, then is reclaimed.

Choosing a network

ipfs-walrus (default) ipfs-filecoin
Backing Walrus blobs on Sui IPFS + Filecoin deals
Gateway gateway-walrus.lighthouse.storage gateway.lighthouse.storage
Quota accounting ~63 MB per blob (erasure coding) Actual file size
Walrus blob IDs Yes (blobIds()) No (returns [])

Each network keeps its own local index per namespace, so the same namespace on both networks never mixes. Memories written to one network are recalled/rebuilt from that network only.

Storage quota caveat (free tier)

Walrus erasure-codes every blob, so each upload counts ~63 MB against your Lighthouse Walrus quota regardless of its actual size (walrusFileSizeInBytes in the file list). The free tier is ~100 MB — roughly one blob. Batching is how this stays workable: one blob carries flushEvery memories and the index snapshot. Raise MEMORY_FLUSH_EVERY to pack more memories per blob (a blob holds tens of thousands of memories before its real size matters), or buy storage in the Lighthouse dashboard (DataCap / x402 stablecoin payments).

References

About

Durable long-term memory for AI agents via Lighthouse — TS library + MCP server with local semantic search

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors