AI integration

AI Integration — Design Decision: "Living Pets"

Decision doc for adding AI to CryptoPets. Status: proposed. Author prompt: "AI Pets with Memory — chat with your pet, it remembers, its personality evolves, on-chain traits affect behavior."

TL;DR — my verdict

Build it. One feature, done deeply. But reframe it.

The idea as written ("chat + memory + evolving personality") is good, but if you build it literally you'll end up with a generic LLM chatbot wearing an NFT skin — nothing about it is Web3, and it could be cloned in a weekend by anyone with an OpenAI key. The part that is genuinely yours, that cannot be copied without your contracts, is this:

A pet's personality is deterministically seeded by its on-chain DNA and shaped by its real combat/breeding history. Two pets with the same DNA behave the same way; a pet that has lost 40 battles is bitter; a freshly bred legendary is arrogant. The chain is the source of the soul.

So the headline isn't "chat with your pet." It's "your pet is alive, and its life is on-chain." Chat and memory are how you experience that — not the product itself.

I'd name it Living Pets (or "Soulbound Personalities") rather than "AI chat."

On-chain vs off-chain — the definitive answer

This is the question that matters most, so here it is with no hand-waving.

The one rule

For any piece of data, ask: "Does someone need to verify this without trusting my server, AND is it small?"

Yes → on-chain. (identity, ownership, achievements)

No → off-chain. (conversations, summaries, anything big/private/chatty)

The headline you need to internalize

The MVP adds ZERO new on-chain data. Everything the AI reads from the chain already exists in your contracts today. The only new storage you build is the off-chain database. On-chain writes only appear in the optional Phase 3 (anchoring a memory hash — never the content).

So "on-chain traits affect behavior" does not mean "write personality to the chain." It means read the traits that are already on-chain and feed them to the AI. The chain is the input, not a new place to store AI stuff.

Per-field classification (every piece of data in this feature)

Data	Where it lives	New?	Notes
`dna`	On-chain	already exists	Seeds innate personality. Backend reads it.
`rarity`	On-chain	already exists	Archetype/tone.
`level`	On-chain	already exists	Maturity/vocabulary.
`winCount` / `lossCount`	On-chain	already exists	Confidence vs bitterness.
`name`	On-chain	already exists	Used in the persona prompt.
`owner`	On-chain	already exists	The auth check: only owner may chat.
Personality trait vector (curiosity, aggression…)	Nowhere — computed	n/a	Pure function of `dna`+`rarity`. Deterministic, so it is recomputed on demand, never stored. This is the trick: it's "on-chain-derived" without being on-chain-stored.
Lived modifiers (confidence from win rate…)	Nowhere — computed	n/a	Pure function of `level`/`winCount`/`lossCount`. Recomputed each request.
Conversation messages (full transcript)	Off-chain DB	NEW	Large, private, written every turn. Never on-chain.
Episodic memory summary	Off-chain DB	NEW	The "it remembers" text. Regenerated periodically.
Trait snapshots over time (for "evolution" history)	Off-chain DB	NEW	Optional; lets you show "how the pet changed."
Rate-limit counters (msgs/day per wallet)	Off-chain DB	NEW	Operational data, never on-chain.
LLM system prompt	Nowhere — assembled per request	n/a	Built from on-chain reads + DB summary at call time.
Memory hash (tamper-evidence)	On-chain (Phase 3 only)	NEW, optional	A single 32-byte hash + tx, NOT the content. The only new on-chain write in the whole design, and only if you want the "provably remembers" badge.

Why each category is where it is

On-chain (the existing pet fields): these define identity and achievement. They must be trustless (you can't fake your win count), portable (the pet's nature travels with the NFT to any wallet/marketplace), and they're tiny. That's textbook on-chain data — and you already built it.
Computed (personality): because personality is a pure, deterministic function of on-chain data, storing it would be redundant and would risk drift. Anyone can recompute it and get the identical result — that is the proof. Recompute, don't store.
Off-chain DB (chat + memory): transcripts are large, grow every message, are private to the owner, and nobody needs to cryptographically verify them. Putting them on-chain would be expensive, slow, and a privacy disaster. A normal database is the correct tool.
On-chain hash (Phase 3, optional): the one place the two worlds meet. You hash the off-chain memory and store only the 32-byte fingerprint on-chain. Now you can prove memory wasn't tampered with, without putting private chat on a public ledger.

Picture it

        ON-CHAIN (already exists, READ-ONLY for this feature)
        ┌───────────────────────────────────────────────┐
        │  dna · rarity · level · winCount · lossCount   │
        │  name · owner                                  │
        └───────────────┬───────────────────────────────┘
                        │ backend reads (ethers / anchor)
                        ▼
        COMPUTED IN MEMORY (stored nowhere, deterministic)
        ┌───────────────────────────────────────────────┐
        │  innatePersonality(dna, rarity)                │
        │  livedModifiers(level, winCount, lossCount)    │
        │  → assembled into the LLM system prompt        │
        └───────────────┬───────────────────────────────┘
                        │ + recent turns & summary from DB
                        ▼
        OFF-CHAIN DB (the ONLY new storage in the MVP)
        ┌───────────────────────────────────────────────┐
        │  conversations · episodic summary              │
        │  trait snapshots · rate-limit counters         │
        └───────────────┬───────────────────────────────┘
                        │ (Phase 3, optional)
                        ▼
        ON-CHAIN (the ONLY new write — a hash, never content)
        ┌───────────────────────────────────────────────┐
        │  memory_hash (32 bytes) + anchoring tx         │
        └───────────────────────────────────────────────┘

Why this is the right single bet

Criterion	Verdict
Differentiated	✅ Tied to YOUR contracts (DNA/stats), not a generic LLM wrapper
Uses what you already have	✅ `dna`, `rarity`, `level`, `winCount`, `lossCount` already exist in `shared/src/types/pet.ts`
Demoable in 30 seconds	✅ "Watch the same prompt produce different answers from two different pets"
Reuses your auth	✅ Wallet → JWT already done in `backend/` + `@shared/core`
Reasonable scope	⚠️ Needs a DB and an LLM proxy — you have neither yet (backend is in-memory)
On-chain integrity	⚠️ The hard part. See the "honesty problem" below.

Compared to scattering 5 features (AI-generated art, price prediction, auto-battle bots, NFT recommendations, etc.), this is the only one that makes a reviewer go "wait, how does it know that?" — because the answer is your chain.

The honesty problem (and how we solve it)

The pitch says the pet "remembers" and its "personality evolves." If that memory lives only in Postgres, the claim is marketing, not engineering. A skeptical reviewer (the kind you want to impress) will ask: "so it's just a chatbot with a database?" You need a real answer.

Solution: a three-layer soul, each layer honest about where it lives.

┌───────────────────────────────────────────────────────────────┐
│ Layer 1 — INNATE (on-chain, immutable)                          │
│   Derived purely from dna + rarity. Deterministic.              │
│   Same DNA → same base personality, forever, for anyone.        │
│   "This is provably the pet's nature. I didn't make it up."     │
├───────────────────────────────────────────────────────────────┤
│ Layer 2 — LIVED (on-chain facts, off-chain interpretation)      │
│   level, winCount, lossCount, breeding lineage, age.            │
│   These ARE on-chain. The AI just narrates them.                │
│   "It's confident because it has 30 wins — verify on-chain."    │
├───────────────────────────────────────────────────────────────┤
│ Layer 3 — MEMORY (off-chain, optionally anchored)               │
│   Conversation history + episodic summaries in a DB.            │
│   To make "it remembers" provable: periodically hash the        │
│   memory state and anchor the hash on-chain (cheap) or sign it  │
│   with the backend key. "Tamper-evident memory."                │
└───────────────────────────────────────────────────────────────┘

Layers 1 and 2 give you the unfakeable Web3 story. Layer 3 is where the LLM and DB do their work, and the anchoring keeps it from being "just a database."

For an MVP, Layers 1–2 plus a plain DB (Layer 3 without anchoring) is enough to ship and demo. Anchoring is the v2 flourish that wins the "is this really Web3?" argument.

How personality is derived (the core algorithm)

This is the part to get right, because it's the whole differentiator. It must be deterministic (same inputs → same personality) and explainable (you can point at the trait and say where it came from).

Step 1 — DNA → trait vector (deterministic, on-chain-grounded)

Your dna is a bigint (EVM uint256 / Solana u64). Slice it into fields, the same way your existing utils/pets/cosmetics.ts already turns DNA into appearance. Reuse that pattern:

// pseudo — lives in shared/src/utils/pets/personality.ts
interface Personality {
  curiosity: number;    // 0–100, from dna bits [0..7]
  aggression: number;   // from dna bits [8..15]
  loyalty: number;      // from dna bits [16..23]
  humor: number;        // from dna bits [24..31]
  archetype: string;    // bucketed from rarity ("trickster", "stoic", "regal"...)
}

function innatePersonality(pet: Pet): Personality { /* pure function of dna+rarity */ }

Because it's a pure function of on-chain data, anyone can recompute it and get the same answer. That's the "provable" part.

Step 2 — lived history → modifiers (on-chain facts)

winRate = winCount / (winCount + lossCount)
  high winRate  → +confidence, +arrogance
  high lossCount → +caution or +bitterness
level           → +maturity / vocabulary
rarity          → tone (legendary = regal, common = scrappy)
freshly bred    → naive / energetic

Step 3 — compose the system prompt

The backend assembles a system prompt from Layers 1–2 + a running summary of Layer 3, then calls the LLM. The pet "is" its prompt. Example skeleton:

You are {name}, a {archetype} CryptoPet (rarity {rarity}, level {level}).
Core nature: curiosity {curiosity}, aggression {aggression}, loyalty {loyalty}.
You have won {winCount} battles and lost {lossCount}; you are {confident|wary}.
You remember: {episodic_summary}.
Stay in character. Never break the fourth wall about being an AI.

Architecture (mapped onto what you already have)

You already have wallet→JWT auth and a chain-agnostic Pet model. The new pieces are a persistence layer, an LLM proxy, and one shared hook.

Frontend / Mobile
  └─ useChatWithPet()  ← NEW shared hook in @shared/core (chain-agnostic)
        │  POST /api/pets/:petId/chat   (Bearer JWT, the auth you already have)
        ▼
Backend (Express)  ← extend the EXISTING service
  ├─ verify JWT + assert caller owns petId (read owner from chain via ethers/anchor)
  ├─ load Pet from chain  → innatePersonality() + livedModifiers()
  ├─ load memory from DB  → episodic summary
  ├─ build system prompt  → call LLM (Anthropic Claude — you're already in that ecosystem)
  ├─ stream answer back
  └─ write turn to DB; periodically summarize + (v2) anchor hash on-chain
        │
        ▼
  Persistence (NEW): Postgres / SQLite / Supabase
    - conversations(pet_id, owner, role, content, ts)
    - pet_memory(pet_id, summary, traits_snapshot, memory_hash, anchored_tx?)
        │
        ▼
  LLM provider: Anthropic Claude (recommend claude-haiku-4-5 for cost, opus for "wow")

Why these choices

LLM = Claude. Cheapest credible quality at claude-haiku-4-5; use a bigger model only for the demo. Prompt-cache the static persona block (it's identical across a conversation) to cut cost dramatically.
Backend, not frontend, calls the LLM. Never ship an API key to the client, and you need server-side ownership checks + rate limiting anyway.
Ownership check is the security boundary. The pet's owner is on-chain (ownerOf / PetAccount.owner). Only the owner can chat as / persist memory for a pet. This reuses the address inside your JWT.
Persistence: your backend is currently in-memory. This feature forces a real DB. Supabase or a single Postgres on Render (where the backend already deploys) is the path of least resistance.

Memory design (so "it remembers" is real, not hand-wavy)

Don't dump full transcript into the prompt — it gets expensive and dumb fast. Use two-tier memory:

Working memory: last N turns, verbatim.
Episodic memory: a rolling LLM-written summary ("Owner named me after their dog. I lost to pet #412 and hold a grudge. I like jokes about fish."). Regenerated every K turns and stored in pet_memory.summary.

Optional v2: vector recall — embed each turn, retrieve the top-k relevant past memories per message. Only add this if simple summaries feel thin; it's a real cost/complexity jump.

Personality "evolves" = the trait snapshot is recomputed when on-chain stats change. After a battle (you already emit events / have watchers like vrf-fulfill-watcher.ts), recompute lived-modifiers and let the next summary reflect it: "After my 10th win I've grown cocky." The evolution is anchored in real chain events, not random drift — which is exactly what makes it defensible.

Phased plan

Phase 0 — spike (½ day)

Pure function innatePersonality(pet) in @shared/core + a script that prints personalities for 5 random DNAs. Prove the determinism + that different DNA feels different. No UI, no LLM yet. This de-risks the whole idea.

Phase 1 — MVP chat (the demoable thing)

Add Postgres to backend; conversations + pet_memory tables.
POST /api/pets/:petId/chat with JWT + ownership check + Claude call.
useChatWithPet() in @shared/core; a chat panel in frontend/src/components/pet/ (you already have pet-interactions/ — add a chat-panel sibling to battle/breed/level-up).
System prompt from Layers 1–2. Working memory only.
Ship this. It already demos "same prompt, two pets, different souls."

Phase 2 — real memory

Episodic summarization every K turns.
Recompute lived-modifiers on battle/breed/level-up so personality shifts.
Prompt caching for cost.

Phase 3 — provable memory (the Web3 flex)

Hash pet_memory state; anchor the hash on-chain (a cheap event/PDA write) or sign it with the backend key. Show a "memory verified ✓ block #…" badge in the UI. This is the answer to "isn't this just a chatbot?"

Phase 4 (optional, high-wow) — pet-to-pet

Let two owned pets "talk" before a battle, or generate trash-talk based on both personalities. Cheap to add once Phase 1 exists, great for screenshots.

Cost & safety (don't skip — reviewers will ask)

Rate limit per wallet (e.g. N messages/day) — tied to the JWT address. Prevents your LLM bill from being a DoS vector.
Prompt-cache the persona block (static per conversation) → big savings.
Content safety: keep pets in-character; add a light system-prompt guard and the provider's moderation. It's a kids-friendly pet game in tone.
Cost ceiling: Haiku + caching + short context ≈ fractions of a cent per message. Budget is a non-issue at demo scale; the rate limit protects you at real scale.
Graceful degradation: if the LLM is down, fall back to a templated personality blurb derived from Layers 1–2 (which need no LLM at all).

What I would NOT do

❌ Don't store personality/memory only in a DB and call it Web3.
❌ Don't put chat memory on-chain directly (gas/rent insane, privacy bad) — anchor hashes, not content.
❌ Don't let the client call the LLM (key leakage, no ownership check).
❌ Don't build 5 AI features. Build this one until it's polished.
❌ Don't make personality random — it must be deterministic from DNA, or the whole "provable soul" story collapses.

Open decisions for you

DB choice: Supabase (fastest), Render Postgres (same host as backend), or SQLite (simplest, but weak for prod)? — I'd start Supabase or Render PG.
LLM tier for the demo: Haiku everywhere, or Opus for the "wow" recording? — Haiku in prod, record the demo with Opus.
Anchoring in scope for v1, or v2? — v2; ship chat first.
Both chains day one, or EVM-first? The personality math is chain-agnostic; only the ownership read differs. — EVM-first, Solana right after, since the shared hook makes it cheap.
Voice/TTS? Big "wow," but scope creep. — Skip for now.

Bottom line

Yes — make CryptoPets your flagship and make Living Pets its one strong feature. But win the argument that separates you from "ChatGPT with a JPEG": the pet's nature is computed from on-chain DNA, its growth is driven by on-chain battles, and its memory is tamper-evident. Lead with that. The chat is just the window you look through.

Start with Phase 0 (the deterministic personality function) — it's half a day and it tells you immediately whether the magic is there.

If you want, I can implement Phase 0 next: innatePersonality() in @shared/core plus a small script that prints souls for sample DNAs, so you can feel it before committing to the backend work.

Frontend / UX design

The feature must not feel bolted on. It slots into your existing dashboard interactions hub (pet-interactions/) as one more action — alongside Breeding Lab, Battle Arena, Level Up, Rename — and reuses your neon theme, Icon/Tones, PetContainer, and the usePet* shared-hook pattern.

Where it lives (matches your conventions)

Layer	New piece	Mirrors existing
Shared hook	`useChatWithPet()` in `@shared/core`	`useBattlePets`, `useBreedPets`
Panel	`frontend/src/components/pet/interactions/chat-panel/`	`battle-panel/`, `breed-panel/`
Hub card	a "Talk" card in `pet-interactions/index.tsx`	Breeding Lab / Battle Arena cards
Route	`CHAT_PATH = '/chat'` in `constants/interactionRoutes.ts`	`BATTLE_PATH`, `BREED_PATH`
Icon/Tone	a `ChatIcon` + e.g. `Tones.Cyan` or a new `Tones.Emerald`	`BattleIcon`/`EggIcon`

1. The hub card (entry point)

A new card in the interactions grid, same shape as your Breeding Lab / Battle Arena cards:

┌─────────────────────────────────┐
│ 💬  Talk to your Pet            │   ← Icon + title, neon header
├─────────────────────────────────┤
│  "Aria" is curious & loyal.     │   ← one-line personality teaser
│  Remembers your last 3 chats.   │   ← memory hint (Phase 2+)
│                                 │
│        [  Start chatting  ]     │   ← neon-button, disabled if no pets
└─────────────────────────────────┘

2. The chat panel (the main screen)

This is where the differentiator must be visible. The screen is split: the Soul Card on the left proves the personality comes from on-chain data, the conversation on the right is the experience. On mobile they stack (the soul card collapses to a header strip).

┌──────────────────────────────────────────────────────────────────────┐
│ 💬  Talk to Aria                                    [ Aria ▾ ]  ← pet  │  pet picker
├────────────────────────┬─────────────────────────────────────────────┤
│  THE SOUL CARD         │   CONVERSATION                              │
│  (left, ~280px)        │   (right, flex)                             │
│                        │                                             │
│   ┌──────────────┐     │   ┌───────────────────────────────────┐    │
│   │   🐾 sprite   │     │   │ Aria: Oh, you're back. Did you     │    │
│   │   (from DNA) │     │   │ finally beat pet #412? 😼          │    │
│   └──────────────┘     │   └───────────────────────────────────┘    │
│   Aria · Lv.7 · Rare   │              ┌──────────────────────────┐   │
│                        │              │ You: not yet, training... │   │
│   ── Innate (DNA) ──   │              └──────────────────────────┘   │
│   Curiosity   ▓▓▓▓▓▓▓░ │   ┌───────────────────────────────────┐    │
│   Aggression  ▓▓░░░░░░ │   │ Aria: typing…  ● ● ●               │    │
│   Loyalty     ▓▓▓▓▓▓░░ │   └───────────────────────────────────┘    │
│   Humor       ▓░░░░░░░ │                                             │
│                        │                                             │
│   ── Lived (chain) ──  │                                             │
│   ⚔ 12 wins / 3 losses │                                             │
│   → confident, teasing │                                             │
│                        │   ┌───────────────────────────────────┐    │
│   🔗 verify on-chain    │   │ Type a message…          [ Send ▶ ]│    │
│   ✓ memory block #…(P3)│   └───────────────────────────────────┘    │
└────────────────────────┴─────────────────────────────────────────────┘

Key UX intentions:

Trait bars are derived live from dna (the Phase 0 function). Hovering a bar shows "from DNA bits 0–7". This is the show-don't-tell of "provable soul."
"Lived (chain)" block reads winCount/lossCount/level and shows the one-line interpretation that's also fed to the LLM — the user sees why the pet talks the way it does.
"🔗 verify on-chain" links to the explorer / a contract read, hammering the Web3 point. "✓ memory block #…" badge appears only in Phase 3.
Pet picker (top-right) switches which owned pet you're talking to — each has a different soul, which is the wow moment.
Streaming: tokens stream in; reuse your transaction-status spinner / typing affordance for async feedback.

3. States to handle (reuse `StateCard`)

Same state matrix as your other interactions — don't reinvent:

Wallet disconnected → StateCard "Connect your wallet to talk to your pets."
No pets yet → StateCard "Create a pet first" → link to dashboard.
LLM/network error → graceful fallback: render the templated personality blurb from Layers 1–2 (no LLM needed) + a "Aria is napping, try again" message.
Rate-limited → "Aria needs a rest — N messages left today" (ties to the per-wallet limit).

4. Optional flourishes (cheap, high-impact for a demo)

"Compare two pets": two soul cards side by side, send the same prompt, show both replies — instantly proves personalities are real and distinct.
Pre-battle trash talk: a "Hype me up" button on the Battle Arena card that generates a one-liner in the pet's voice. Reuses the same backend endpoint.
Mood tint: color the chat accent by dominant trait (aggressive = magenta, curious = cyan) using your existing Tones.

Component sketch

// frontend/src/components/pet/interactions/chat-panel/index.tsx
const ChatPanel: React.FC<{ isStandaloneView: boolean }> = () => {
  const { pets } = usePetList();
  const [activeId, setActiveId] = useState(pets[0]?.id);
  const pet = pets.find(p => p.id === activeId);

  const soul = useMemo(() => innatePersonality(pet), [pet]);   // Phase 0, pure
  const { messages, send, isStreaming, error } = useChatWithPet(activeId);

  // left:  <SoulCard pet={pet} soul={soul} />   (trait bars + lived block + verify link)
  // right: <MessageList messages={messages} /> + <Composer onSend={send} disabled={isStreaming} />
};

Note innatePersonality(pet) needs no backend — the Soul Card renders instantly from on-chain data already loaded by usePetList. So you can build and ship the entire left half of this screen in Phase 0/1 before the LLM exists, which de-risks the UI work and gives you something visual immediately.

Uh oh!

AI integration

AI Integration — Design Decision: "Living Pets"

TL;DR — my verdict

On-chain vs off-chain — the definitive answer

The one rule

The headline you need to internalize

Per-field classification (every piece of data in this feature)

Why each category is where it is

Picture it

Why this is the right single bet

The honesty problem (and how we solve it)

How personality is derived (the core algorithm)

Step 1 — DNA → trait vector (deterministic, on-chain-grounded)

Step 2 — lived history → modifiers (on-chain facts)

Step 3 — compose the system prompt

Architecture (mapped onto what you already have)

Why these choices

Memory design (so "it remembers" is real, not hand-wavy)

Phased plan

Phase 0 — spike (½ day)

Phase 1 — MVP chat (the demoable thing)

Phase 2 — real memory

Phase 3 — provable memory (the Web3 flex)

Phase 4 (optional, high-wow) — pet-to-pet

Cost & safety (don't skip — reviewers will ask)

What I would NOT do

Open decisions for you

Bottom line

Frontend / UX design

Where it lives (matches your conventions)

1. The hub card (entry point)

2. The chat panel (the main screen)

3. States to handle (reuse StateCard)

4. Optional flourishes (cheap, high-impact for a demo)

Component sketch

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

3. States to handle (reuse `StateCard`)