Skip to content

feat: import Gemini conversation history (Google Takeout export) #152

Description

@ilblackdragon

Problem / Motivation

Google Gemini (formerly Bard) users can export their conversation history via Google Takeout. As Gemini is one of the three major AI assistants, supporting import from it ensures IronClaw can be a drop-in replacement for users coming from any major platform.

Proposed Solution

Implement the Importer trait for Google Takeout Gemini exports.

New file: src/cli/import_gemini.rs

Source format

Google Takeout exports a ZIP containing a Gemini Apps/ directory. Inside are individual conversation files, typically as JSON (or sometimes HTML). The JSON format contains:

{
  "conversations": [
    {
      "id": "gemini-conv-abc123",
      "title": "Help with async Rust",
      "create_time": "2025-06-15T14:30:00Z",
      "entries": [
        {
          "role": "user",
          "text": "How do I use tokio::spawn?",
          "create_time": "2025-06-15T14:30:05Z"
        },
        {
          "role": "model",
          "text": "tokio::spawn is used to...",
          "create_time": "2025-06-15T14:30:12Z"
        }
      ]
    }
  ]
}

Important: Google Takeout format varies over time and by export settings. The parser must handle multiple known variations:

  1. Single JSON file with conversations array
  2. Per-conversation JSON files in Gemini Apps/ directory (one file per conversation)
  3. HTML export — parse with basic HTML-to-text extraction (conversations in <div> blocks)

Implementation

pub struct GeminiImporter;

impl Importer for GeminiImporter {
    fn source_name(&self) -> &str { "Gemini" }

    fn parse(&self, path: &Path) -> Result<Vec<ImportedConversation>, ImportError> {
        // 1. Detect input type: ZIP file or directory
        // 2. If ZIP: open and look for "Gemini Apps/" directory entries
        // 3. Find conversation files (.json or .html)
        // 4. Parse each file:
        //    - JSON: direct deserialization
        //    - HTML: extract text content from structured divs
        // 5. Map roles: "user" → "user", "model" → "assistant"
        // 6. Return Vec<ImportedConversation>
    }
}

Role mapping

Gemini role IronClaw role
"user" "user"
"model" "assistant"
"system" "system"

Path auto-detection

When no path is specified:

  1. Check ~/Downloads/ for recent takeout-*.zip files
  2. Check ~/Downloads/Takeout/ for an extracted Takeout directory
  3. If found, look for Gemini Apps/ subdirectory inside

Architecture Notes

takeout-20250615.zip
└── Takeout/
    └── Gemini Apps/
        ├── conversation_abc.json    → ImportedConversation
        ├── conversation_def.json    → ImportedConversation
        └── conversation_ghi.html    → ImportedConversation (HTML parse)

OR (single-file variant):

takeout-20250615.zip
└── Takeout/
    └── Gemini Apps/
        └── conversations.json       → Vec<ImportedConversation>

The parser must handle both layouts. Check for a top-level conversations.json first, then fall back to per-file parsing.

Code Pointers

  • Core Importer trait — from the core import infrastructure issue
  • src/cli/import.rs — orchestrator
  • Google Takeout documentation for format details

Acceptance Criteria

  • GeminiImporter struct implements Importer trait
  • Handles ZIP file input (standard Takeout export)
  • Handles directory input (extracted Takeout)
  • Locates Gemini Apps/ directory inside ZIP/Takeout structure
  • Parses per-conversation JSON files
  • Parses single conversations.json file (if present)
  • Basic HTML-to-text fallback for HTML export format
  • Maps "model" role → "assistant"
  • Uses conversation id as source_id
  • Uses conversation title as title (falls back to first user message)
  • Handles missing fields with #[serde(default)]
  • Auto-detection: checks ~/Downloads/takeout-*.zip and ~/Downloads/Takeout/
  • Returns empty Vec (not error) when no Gemini data found in Takeout
  • Unit tests with JSON fixture (at least 2 conversations)
  • Unit test for HTML format parsing
  • Unit test for missing Gemini Apps/ directory in ZIP (clear error message)
  • Unit test for conversation with empty entries
  • No .unwrap() or .expect() in production code
  • crate:: imports only

Pitfalls & Landmines

  • Google Takeout format is not stable or documented. Google changes the export format without notice. The parser MUST be extremely lenient — use #[serde(default)] on every field, skip unknown fields with #[serde(flatten)] or deny_unknown_fields = false, and log warnings for unexpected structures instead of failing.
  • ZIP structure varies: Sometimes the Gemini Apps/ directory is at the root of the ZIP, sometimes nested under Takeout/. Check both paths.
  • HTML export: Some users export as HTML instead of JSON. The HTML format has conversations in structured <div> elements with class names. Use a simple regex or string-based parser — don't pull in a full HTML parsing crate. If HTML parsing is too complex, return an ImportError suggesting the user re-export as JSON.
  • Encoding: Takeout exports are UTF-8 but may contain BOM (byte order mark). Strip BOM before parsing.
  • Multi-language: Gemini conversations may be in any language. Don't assume ASCII — test with Unicode content.
  • "model" not "assistant": Gemini uses "model" as the role name. This is the most common mapping mistake.

Non-Goals

  • Importing Gemini via API (no public conversation history API)
  • Importing Gemini Gems/extensions configuration
  • Importing images or attachments from Gemini conversations
  • Supporting Bard-era export format (pre-Gemini rename)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions