Skip to content

AI integration

Jump to bottom
aichannode edited this page Jun 1, 2026 · 3 revisions

AI Integration — Design Decision: "Living Pets"

Decision doc for adding AI to CryptoPets. Status: proposed. Author prompt: "AI Pets with Memory — chat with your pet, it remembers, its personality evolves, on-chain traits affect behavior."

TL;DR — my verdict

Build it. One feature, done deeply. But reframe it.

The idea as written ("chat + memory + evolving personality") is good, but if you build it literally you'll end up with a generic LLM chatbot wearing an NFT skin — nothing about it is Web3, and it could be cloned in a weekend by anyone with an OpenAI key. The part that is genuinely yours, that cannot be copied without your contracts, is this:

A pet's personality is deterministically seeded by its on-chain DNA and shaped by its real combat/breeding history. Two pets with the same DNA behave the same way; a pet that has lost 40 battles is bitter; a freshly bred legendary is arrogant. The chain is the source of the soul.

So the headline isn't "chat with your pet." It's "your pet is alive, and its life is on-chain." Chat and memory are how you experience that — not the product itself.

I'd name it Living Pets (or "Soulbound Personalities") rather than "AI chat."

On-chain vs off-chain — the definitive answer

This is the question that matters most, so here it is with no hand-waving.

The one rule

For any piece of data, ask: "Does someone need to verify this without trusting my server, AND is it small?"

  • Yes → on-chain. (identity, ownership, achievements)
  • No → off-chain. (conversations, summaries, anything big/private/chatty)

The headline you need to internalize

The MVP adds ZERO new on-chain data. Everything the AI reads from the chain already exists in your contracts today. The only new storage you build is the off-chain database. On-chain writes only appear in the optional Phase 3 (anchoring a memory hash — never the content).

So "on-chain traits affect behavior" does not mean "write personality to the chain." It means read the traits that are already on-chain and feed them to the AI. The chain is the input, not a new place to store AI stuff.

Per-field classification (every piece of data in this feature)

Data Where it lives New? Notes
dna On-chain already exists Seeds innate personality. Backend reads it.
rarity On-chain already exists Archetype/tone.
level On-chain already exists Maturity/vocabulary.
winCount / lossCount On-chain already exists Confidence vs bitterness.
name On-chain already exists Used in the persona prompt.
owner On-chain already exists The auth check: only owner may chat.
Personality trait vector (curiosity, aggression…) Nowhere — computed n/a Pure function of dna+rarity. Deterministic, so it is recomputed on demand, never stored. This is the trick: it's "on-chain-derived" without being on-chain-stored.
Lived modifiers (confidence from win rate…) Nowhere — computed n/a Pure function of level/winCount/lossCount. Recomputed each request.
Conversation messages (full transcript) Off-chain DB NEW Large, private, written every turn. Never on-chain.
Episodic memory summary Off-chain DB NEW The "it remembers" text. Regenerated periodically.
Trait snapshots over time (for "evolution" history) Off-chain DB NEW Optional; lets you show "how the pet changed."
Rate-limit counters (msgs/day per wallet) Off-chain DB NEW Operational data, never on-chain.
LLM system prompt Nowhere — assembled per request n/a Built from on-chain reads + DB summary at call time.
Memory hash (tamper-evidence) On-chain (Phase 3 only) NEW, optional A single 32-byte hash + tx, NOT the content. The only new on-chain write in the whole design, and only if you want the "provably remembers" badge.

Why each category is where it is

  • On-chain (the existing pet fields): these define identity and achievement. They must be trustless (you can't fake your win count), portable (the pet's nature travels with the NFT to any wallet/marketplace), and they're tiny. That's textbook on-chain data — and you already built it.
  • Computed (personality): because personality is a pure, deterministic function of on-chain data, storing it would be redundant and would risk drift. Anyone can recompute it and get the identical result — that is the proof. Recompute, don't store.
  • Off-chain DB (chat + memory): transcripts are large, grow every message, are private to the owner, and nobody needs to cryptographically verify them. Putting them on-chain would be expensive, slow, and a privacy disaster. A normal database is the correct tool.
  • On-chain hash (Phase 3, optional): the one place the two worlds meet. You hash the off-chain memory and store only the 32-byte fingerprint on-chain. Now you can prove memory wasn't tampered with, without putting private chat on a public ledger.

Picture it

        ON-CHAIN (already exists, READ-ONLY for this feature)
        ┌───────────────────────────────────────────────┐
        │  dna · rarity · level · winCount · lossCount   │
        │  name · owner                                  │
        └───────────────┬───────────────────────────────┘
                        │ backend reads (ethers / anchor)
                        ▼
        COMPUTED IN MEMORY (stored nowhere, deterministic)
        ┌───────────────────────────────────────────────┐
        │  innatePersonality(dna, rarity)                │
        │  livedModifiers(level, winCount, lossCount)    │
        │  → assembled into the LLM system prompt        │
        └───────────────┬───────────────────────────────┘
                        │ + recent turns & summary from DB
                        ▼
        OFF-CHAIN DB (the ONLY new storage in the MVP)
        ┌───────────────────────────────────────────────┐
        │  conversations · episodic summary              │
        │  trait snapshots · rate-limit counters         │
        └───────────────┬───────────────────────────────┘
                        │ (Phase 3, optional)
                        ▼
        ON-CHAIN (the ONLY new write — a hash, never content)
        ┌───────────────────────────────────────────────┐
        │  memory_hash (32 bytes) + anchoring tx         │
        └───────────────────────────────────────────────┘

Why this is the right single bet

Criterion Verdict
Differentiated ✅ Tied to YOUR contracts (DNA/stats), not a generic LLM wrapper
Uses what you already have ✅ dna, rarity, level, winCount, lossCount already exist in shared/src/types/pet.ts
Demoable in 30 seconds ✅ "Watch the same prompt produce different answers from two different pets"
Reuses your auth ✅ Wallet → JWT already done in backend/ + @shared/core
Reasonable scope ⚠️ Needs a DB and an LLM proxy — you have neither yet (backend is in-memory)
On-chain integrity ⚠️ The hard part. See the "honesty problem" below.

Compared to scattering 5 features (AI-generated art, price prediction, auto-battle bots, NFT recommendations, etc.), this is the only one that makes a reviewer go "wait, how does it know that?" — because the answer is your chain.

The honesty problem (and how we solve it)

The pitch says the pet "remembers" and its "personality evolves." If that memory lives only in Postgres, the claim is marketing, not engineering. A skeptical reviewer (the kind you want to impress) will ask: "so it's just a chatbot with a database?" You need a real answer.

Solution: a three-layer soul, each layer honest about where it lives.

┌───────────────────────────────────────────────────────────────┐
│ Layer 1 — INNATE (on-chain, immutable)                          │
│   Derived purely from dna + rarity. Deterministic.              │
│   Same DNA → same base personality, forever, for anyone.        │
│   "This is provably the pet's nature. I didn't make it up."     │
├───────────────────────────────────────────────────────────────┤
│ Layer 2 — LIVED (on-chain facts, off-chain interpretation)      │
│   level, winCount, lossCount, breeding lineage, age.            │
│   These ARE on-chain. The AI just narrates them.                │
│   "It's confident because it has 30 wins — verify on-chain."    │
├───────────────────────────────────────────────────────────────┤
│ Layer 3 — MEMORY (off-chain, optionally anchored)               │
│   Conversation history + episodic summaries in a DB.            │
│   To make "it remembers" provable: periodically hash the        │
│   memory state and anchor the hash on-chain (cheap) or sign it  │
│   with the backend key. "Tamper-evident memory."                │
└───────────────────────────────────────────────────────────────┘

Layers 1 and 2 give you the unfakeable Web3 story. Layer 3 is where the LLM and DB do their work, and the anchoring keeps it from being "just a database."

For an MVP, Layers 1–2 plus a plain DB (Layer 3 without anchoring) is enough to ship and demo. Anchoring is the v2 flourish that wins the "is this really Web3?" argument.

How personality is derived (the core algorithm)

This is the part to get right, because it's the whole differentiator. It must be deterministic (same inputs → same personality) and explainable (you can point at the trait and say where it came from).

Step 1 — DNA → trait vector (deterministic, on-chain-grounded)

Your dna is a bigint (EVM uint256 / Solana u64). Slice it into fields, the same way your existing utils/pets/cosmetics.ts already turns DNA into appearance. Reuse that pattern:

// pseudo — lives in shared/src/utils/pets/personality.ts
interface Personality {
  curiosity: number;    // 0–100, from dna bits [0..7]
  aggression: number;   // from dna bits [8..15]
  loyalty: number;      // from dna bits [16..23]
  humor: number;        // from dna bits [24..31]
  archetype: string;    // bucketed from rarity ("trickster", "stoic", "regal"...)
}

function innatePersonality(pet: Pet): Personality { /* pure function of dna+rarity */ }

Because it's a pure function of on-chain data, anyone can recompute it and get the same answer. That's the "provable" part.

Step 2 — lived history → modifiers (on-chain facts)

winRate = winCount / (winCount + lossCount)
  high winRate  → +confidence, +arrogance
  high lossCount → +caution or +bitterness
level           → +maturity / vocabulary
rarity          → tone (legendary = regal, common = scrappy)
freshly bred    → naive / energetic

Step 3 — compose the system prompt

The backend assembles a system prompt from Layers 1–2 + a running summary of Layer 3, then calls the LLM. The pet "is" its prompt. Example skeleton:

You are {name}, a {archetype} CryptoPet (rarity {rarity}, level {level}).
Core nature: curiosity {curiosity}, aggression {aggression}, loyalty {loyalty}.
You have won {winCount} battles and lost {lossCount}; you are {confident|wary}.
You remember: {episodic_summary}.
Stay in character. Never break the fourth wall about being an AI.

Architecture (mapped onto what you already have)

You already have wallet→JWT auth and a chain-agnostic Pet model. The new pieces are a persistence layer, an LLM proxy, and one shared hook.

Frontend / Mobile
  └─ useChatWithPet()  ← NEW shared hook in @shared/core (chain-agnostic)
        │  POST /api/pets/:petId/chat   (Bearer JWT, the auth you already have)
        ▼
Backend (Express)  ← extend the EXISTING service
  ├─ verify JWT + assert caller owns petId (read owner from chain via ethers/anchor)
  ├─ load Pet from chain  → innatePersonality() + livedModifiers()
  ├─ load memory from DB  → episodic summary
  ├─ build system prompt  → call LLM (Anthropic Claude — you're already in that ecosystem)
  ├─ stream answer back
  └─ write turn to DB; periodically summarize + (v2) anchor hash on-chain
        │
        ▼
  Persistence (NEW): Postgres / SQLite / Supabase
    - conversations(pet_id, owner, role, content, ts)
    - pet_memory(pet_id, summary, traits_snapshot, memory_hash, anchored_tx?)
        │
        ▼
  LLM provider: Anthropic Claude (recommend claude-haiku-4-5 for cost, opus for "wow")

Why these choices

  • LLM = Claude. Cheapest credible quality at claude-haiku-4-5; use a bigger model only for the demo. Prompt-cache the static persona block (it's identical across a conversation) to cut cost dramatically.
  • Backend, not frontend, calls the LLM. Never ship an API key to the client, and you need server-side ownership checks + rate limiting anyway.
  • Ownership check is the security boundary. The pet's owner is on-chain (ownerOf / PetAccount.owner). Only the owner can chat as / persist memory for a pet. This reuses the address inside your JWT.
  • Persistence: your backend is currently in-memory. This feature forces a real DB. Supabase or a single Postgres on Render (where the backend already deploys) is the path of least resistance.

Memory design (so "it remembers" is real, not hand-wavy)

Don't dump full transcript into the prompt — it gets expensive and dumb fast. Use two-tier memory:

  1. Working memory: last N turns, verbatim.
  2. Episodic memory: a rolling LLM-written summary ("Owner named me after their dog. I lost to pet #412 and hold a grudge. I like jokes about fish."). Regenerated every K turns and stored in pet_memory.summary.

Optional v2: vector recall — embed each turn, retrieve the top-k relevant past memories per message. Only add this if simple summaries feel thin; it's a real cost/complexity jump.

Personality "evolves" = the trait snapshot is recomputed when on-chain stats change. After a battle (you already emit events / have watchers like vrf-fulfill-watcher.ts), recompute lived-modifiers and let the next summary reflect it: "After my 10th win I've grown cocky." The evolution is anchored in real chain events, not random drift — which is exactly what makes it defensible.

Phased plan

Phase 0 — spike (½ day)

Pure function innatePersonality(pet) in @shared/core + a script that prints personalities for 5 random DNAs. Prove the determinism + that different DNA feels different. No UI, no LLM yet. This de-risks the whole idea.

Phase 1 — MVP chat (the demoable thing)

  • Add Postgres to backend; conversations + pet_memory tables.
  • POST /api/pets/:petId/chat with JWT + ownership check + Claude call.
  • useChatWithPet() in @shared/core; a chat panel in frontend/src/components/pet/ (you already have pet-interactions/ — add a chat-panel sibling to battle/breed/level-up).
  • System prompt from Layers 1–2. Working memory only.
  • Ship this. It already demos "same prompt, two pets, different souls."

Phase 2 — real memory

  • Episodic summarization every K turns.
  • Recompute lived-modifiers on battle/breed/level-up so personality shifts.
  • Prompt caching for cost.

Phase 3 — provable memory (the Web3 flex)

  • Hash pet_memory state; anchor the hash on-chain (a cheap event/PDA write) or sign it with the backend key. Show a "memory verified ✓ block #…" badge in the UI. This is the answer to "isn't this just a chatbot?"

Phase 4 (optional, high-wow) — pet-to-pet

Let two owned pets "talk" before a battle, or generate trash-talk based on both personalities. Cheap to add once Phase 1 exists, great for screenshots.

Cost & safety (don't skip — reviewers will ask)

  • Rate limit per wallet (e.g. N messages/day) — tied to the JWT address. Prevents your LLM bill from being a DoS vector.
  • Prompt-cache the persona block (static per conversation) → big savings.
  • Content safety: keep pets in-character; add a light system-prompt guard and the provider's moderation. It's a kids-friendly pet game in tone.
  • Cost ceiling: Haiku + caching + short context ≈ fractions of a cent per message. Budget is a non-issue at demo scale; the rate limit protects you at real scale.
  • Graceful degradation: if the LLM is down, fall back to a templated personality blurb derived from Layers 1–2 (which need no LLM at all).

What I would NOT do

  • ❌ Don't store personality/memory only in a DB and call it Web3.
  • ❌ Don't put chat memory on-chain directly (gas/rent insane, privacy bad) — anchor hashes, not content.
  • ❌ Don't let the client call the LLM (key leakage, no ownership check).
  • ❌ Don't build 5 AI features. Build this one until it's polished.
  • ❌ Don't make personality random — it must be deterministic from DNA, or the whole "provable soul" story collapses.

Open decisions for you

  1. DB choice: Supabase (fastest), Render Postgres (same host as backend), or SQLite (simplest, but weak for prod)? — I'd start Supabase or Render PG.
  2. LLM tier for the demo: Haiku everywhere, or Opus for the "wow" recording? — Haiku in prod, record the demo with Opus.
  3. Anchoring in scope for v1, or v2? — v2; ship chat first.
  4. Both chains day one, or EVM-first? The personality math is chain-agnostic; only the ownership read differs. — EVM-first, Solana right after, since the shared hook makes it cheap.
  5. Voice/TTS? Big "wow," but scope creep. — Skip for now.

Bottom line

Yes — make CryptoPets your flagship and make Living Pets its one strong feature. But win the argument that separates you from "ChatGPT with a JPEG": the pet's nature is computed from on-chain DNA, its growth is driven by on-chain battles, and its memory is tamper-evident. Lead with that. The chat is just the window you look through.

Start with Phase 0 (the deterministic personality function) — it's half a day and it tells you immediately whether the magic is there.

If you want, I can implement Phase 0 next: innatePersonality() in @shared/core plus a small script that prints souls for sample DNAs, so you can feel it before committing to the backend work.

Clone this wiki locally