diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md
new file mode 100644
index 00000000..2f6c22df
--- /dev/null
+++ b/DEVELOPMENT.md
@@ -0,0 +1,99 @@
+# Development Guide
+
+This guide covers the development setup and maintenance of the TiDB for AI documentation site.
+
+## Quick Start
+
+1. **Install UV** (fast dependency manager):
+
+ ```bash
+ curl -LsSf https://astral.sh/uv/install.sh | sh
+ ```
+
+2. **Install dependencies**:
+
+ ```bash
+ make install
+ ```
+
+3. **Start development server**:
+
+ ```bash
+ make serve
+ ```
+
+## Project Structure
+
+```
+├── demos.yml # Demo Gallery configuration
+├── scripts/generate_demos.py # Demo Gallery generation script
+├── src/
+│ ├── templates/ # Jinja2 templates
+│ └── ai/examples/ # Generated demo pages
+└── Makefile # Build commands
+```
+
+## Helpful Commands
+
+```bash
+# Dependencies
+make check # Check dependencies and setup
+make install # Install/update dependencies
+
+# Development workflow
+make serve # Start development server
+make build # Build documentation site
+make clean # Clean build artifacts
+
+# Demo management
+make generate-demos # Generate demo pages and gallery from demos.yml config
+make generate-demo-pages # Only generate demo pages.
+make generate-demo-gallery # Only generate demo gallery.
+
+# Other
+make help # Show all available commands
+```
+
+## Maintain the Demo Gallery
+
+The Demo Gallery showcases AI demos of TiDB and is configured via [demos.yml](demos.yml).
+
+To regenerate the demo gallery from configuration, run:
+
+```bash
+make generate-demos
+```
+
+### How to add a new demo
+
+You can follow the steps below to add a new demo:
+
+1. Add entry to `demos` array in `demos.yml` with unique `id`, title, description, and display properties
+
+ For example:
+
+ ```yaml
+ demos:
+ - id: "basic"
+ title: "Basic Usage"
+ description: "Learn fundamental PyTiDB operations"
+ icon: "⚙️"
+ background: "linear-gradient(135deg, #10b981, var(--brand-color))"
+ link: "basic-with-pytidb/"
+ doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/basic/README.md"
+ cover_image: null
+ ```
+
+2. Add the demo `id` to appropriate category's `demos` array
+
+ For example:
+
+ ```yaml
+ categories:
+ - id: "featured"
+ title: "Search"
+ demos: ["image-search"]
+ ```
+
+3. Run `make generate-demos` to regenerate
+4. Commit changes
diff --git a/Makefile b/Makefile
new file mode 100644
index 00000000..d4bee808
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,47 @@
+# TiDB for AI Documentation Site
+
+.PHONY: help install generate-demos generate-demo-pages generate-demo-gallery serve build clean check
+
+help:
+ @echo "TiDB for AI Documentation Site"
+ @echo "============================="
+ @echo "Available commands:"
+ @echo " install - Install dependencies using UV"
+ @echo " generate-demos - Generate demo pages and gallery from demos.yml config"
+ @echo " generate-demo-pages - Only generate demo pages."
+ @echo " generate-demo-gallery - Only generate demo gallery."
+ @echo " serve - Start the development server"
+ @echo " build - Build the documentation site"
+ @echo " clean - Clean build artifacts"
+ @echo " check - Check dependencies and project setup"
+ @echo ""
+ @echo "Demo gallery configuration:"
+ @echo " Edit demos.yml in the project root to manage gallery content"
+ @echo ""
+ @echo "Prerequisites:"
+ @echo " UV package manager - curl -LsSf https://astral.sh/uv/install.sh | sh"
+
+install:
+ uv pip install -e .
+
+generate-demos:
+ python scripts/generate_demos.py
+
+generate-demo-pages:
+ python scripts/generate_demos.py --skip-gallery
+
+generate-demo-gallery:
+ python scripts/generate_demos.py --skip-demos
+
+serve:
+ mkdocs serve
+
+build:
+ mkdocs build
+
+clean:
+ rm -rf site/
+ rm -rf .mkdocs_cache/
+
+check:
+ python scripts/check_dependencies.py
\ No newline at end of file
diff --git a/README.md b/README.md
index 38a9520b..a1469aef 100644
--- a/README.md
+++ b/README.md
@@ -14,6 +14,14 @@ To install the TiDB Python SDK, run the following command:
pip install pytidb
```
+Examples:
+
+- Getting Started: [Quickstart](https://pingcap.github.io/ai/examples/quickstart-with-pytidb/), [Basic Usage](https://pingcap.github.io/ai/examples/basic-with-pytidb/)
+- Search & Retrieval: [Vector Search](https://pingcap.github.io/ai/examples/vector-search-with-pytidb/), [Fulltext Search](https://pingcap.github.io/ai/examples/fulltext-search-with-pytidb/), [Hybrid Search](https://pingcap.github.io/ai/examples/hybrid-search-with-pytidb/)
+- AI Applications: [RAG](https://pingcap.github.io/ai/examples/rag-with-pytidb/), [Text2SQL](https://pingcap.github.io/ai/examples/text2sql-with-pytidb/), [Memory](https://pingcap.github.io/ai/examples/memory-with-pytidb/)
+- Advanced Features: [Auto Embedding](https://pingcap.github.io/ai/examples/auto-embedding-with-pytidb/), [Image Search](https://pingcap.github.io/ai/examples/image-search-with-pytidb/)
+
+
Integrations:
- AI Frameworks: [LlamaIndex](https://docs.pingcap.com/tidbcloud/vector-search-integrate-with-llamaindex/), [LangChain](https://docs.pingcap.com/tidbcloud/vector-search-integrate-with-langchain/)
@@ -21,6 +29,10 @@ Integrations:
- AI Services: [Bedrock](https://docs.pingcap.com/tidbcloud/vector-search-integrate-with-amazon-bedrock/)
- Embedding Models/Services: [JinaAI](https://docs.pingcap.com/tidbcloud/vector-search-integrate-with-jinaai-embedding/)
+## Contribute
+
+We welcome contributions to improve the TiDB for AI documentation! For development setup, maintenance scripts, and detailed contribution guidelines, please see [DEVELOPMENT.md](DEVELOPMENT.md).
+
## FAQ
### How can I get support?
diff --git a/demos.yml b/demos.yml
new file mode 100644
index 00000000..8489b90f
--- /dev/null
+++ b/demos.yml
@@ -0,0 +1,131 @@
+# Demo Gallery Configuration
+title: "Demo Gallery"
+description: |
+ Explore hands-on demos showcasing how TiDB empowers AI applications. E-commerce product search with full-text search
+ Get started quickly with TiDB Cloud Serverless to build your own AI-powered solutions.
+
+meta:
+ description: "Explore hands-on demos showcasing how TiDB empowers AI applications. Get started quickly with TiDB Cloud Serverless to build your own AI-powered solutions."
+
+categories:
+ - id: "featured"
+ title: "⭐ Featured"
+ demos: ["image-search", "rag", "memory"]
+
+ - id: "getting-started"
+ title: "🚀 Getting Started"
+ demos: ["basic", "auto-embedding"]
+
+ - id: "search"
+ title: "🔍 Search & Retrieval"
+ demos: ["vector-search", "fulltext-search", "hybrid-search", "image-search"]
+
+ - id: "ai-apps"
+ title: "🤖 AI Applications"
+ demos: ["rag", "memory", "text2sql"]
+
+demos:
+
+ - id: "image-search"
+ title: "Image Search"
+ description: "Build an image search application using multimodal embeddings for both text-to-image and image-to-image search."
+ category: "search"
+ icon: null
+ background: null
+ link: "image-search-with-pytidb/"
+ doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/image_search/README.md"
+ cover_image: "https://github.com/user-attachments/assets/7ba9733a-4d1f-4094-8edb-58731ebd08e9"
+
+ - id: "rag"
+ title: "RAG"
+ description: "Build a RAG application that combines document retrieval with language generation."
+ category: "ai-apps"
+ icon: null
+ background: null
+ link: "rag-with-pytidb/"
+ doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/rag/README.md"
+ cover_image: "https://github.com/user-attachments/assets/dfd85672-65ce-4a46-8dd2-9f77d826363e"
+
+ - id: "basic"
+ title: "Basic Usage"
+ description: "Learn fundamental PyTiDB operations including database connection, table creation, and data manipulation."
+ category: "getting-started"
+ icon: "⚙️"
+ background: "linear-gradient(135deg, #10b981, var(--brand-color))"
+ link: "basic-with-pytidb/"
+ doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/basic/README.md"
+ cover_image: null
+
+ - id: "auto-embedding"
+ title: "Auto Embedding"
+ description: "Automatically generate embeddings for your text data using built-in embedding models."
+ category: "getting-started"
+ icon: "🤖"
+ background: "radial-gradient(circle at center, #8b5cf6 0%, var(--brand-color) 100%)"
+ link: "auto-embedding-with-pytidb/"
+ doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/auto_embedding/README.md"
+ cover_image: null
+
+ - id: "vector-search"
+ title: "Vector Search"
+ description: "Implement semantic search using vector embeddings to find similar content."
+ category: "search"
+ icon: null
+ gradient_class: null
+ link: "vector-search-with-pytidb/"
+ doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/vector_search/README.md"
+ cover_image: "https://github.com/user-attachments/assets/6d7783a5-ce9c-4dcc-8b95-49d5f0ca735a"
+
+ - id: "fulltext-search"
+ title: "Fulltext Search"
+ description: "Perform traditional text search using MySQL fulltext search capabilities."
+ category: "search"
+ icon: null
+ gradient_class: null
+ link: "fulltext-search-with-pytidb/"
+ doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/fulltext_search/README.md"
+ cover_image: "https://github.com/user-attachments/assets/c81ddad4-f996-4b1f-85c0-5cbb55bc2a3a"
+
+ - id: "hybrid-search"
+ title: "Hybrid Search"
+ description: "Combine vector search and fulltext search for more comprehensive results."
+ category: "search"
+ icon: null
+ gradient_class: null
+ link: "hybrid-search-with-pytidb/"
+ doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/hybrid_search/README.md"
+ cover_image: "https://github.com/user-attachments/assets/6e1c639d-2160-44c8-86b4-958913b9eca5"
+
+ - id: "memory"
+ title: "Memory"
+ description: "Implement conversation memory for chatbots and conversational AI applications."
+ category: "ai-apps"
+ icon: "💭"
+ background: "linear-gradient(135deg, #8b5cf6, var(--brand-color))"
+ link: "memory-with-pytidb/"
+ doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/memory/README.md"
+ cover_image: null
+
+ - id: "text2sql"
+ title: "Text2SQL"
+ description: "Convert natural language queries into SQL statements using AI models."
+ category: "ai-apps"
+ icon: "💬"
+ background: "linear-gradient(135deg, #06b6d4, var(--brand-color))"
+ link: "text2sql-with-pytidb/"
+ doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/text2sql/README.md"
+ cover_image: null
+
+# CTA section configuration
+cta:
+ title: "Ready to build your AI application?"
+ description: "Start your AI journey with TiDB Cloud Serverless. Follow our quickstart guide to build your first AI-powered application in minutes, or explore specific examples for your use case."
+ buttons:
+ - text: "Try TiDB Cloud Serverless"
+ url: "https://tidbcloud.com/?utm_source=github&utm_medium=referral&utm_campaign=pytidb_readme"
+ type: "primary"
+ external: true
+ - text: "View Quickstart Guide"
+ url: "quickstart-with-pytidb/"
+ type: "secondary"
+ external: false
\ No newline at end of file
diff --git a/mkdocs.yml b/mkdocs.yml
index ecc9f6a0..991179d9 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -106,6 +106,17 @@ nav:
- Auto Embedding: ai/guides/auto-embedding.md
- Reranking: ai/guides/reranking.md
- Filtering: ai/guides/filtering.md
+ - 💡 Demos:
+ - Gallery: ai/examples/index.md
+ - Basic Usage: ai/examples/basic-with-pytidb.md
+ - Auto Embedding: ai/examples/auto-embedding-with-pytidb.md
+ - Vector Search: ai/examples/vector-search-with-pytidb.md
+ - Fulltext Search: ai/examples/fulltext-search-with-pytidb.md
+ - Hybrid Search: ai/examples/hybrid-search-with-pytidb.md
+ - Image Search: ai/examples/image-search-with-pytidb.md
+ - RAG: ai/examples/rag-with-pytidb.md
+ - Memory: ai/examples/memory-with-pytidb.md
+ - Text2SQL: ai/examples/text2sql-with-pytidb.md
- 🔌 Integrations:
- MCP Integration:
- TiDB MCP Server: ai/integrations/tidb-mcp-server.md
@@ -128,6 +139,17 @@ nav:
- Transaction: ai/guides/transaction.md
- Raw Queries: ai/guides/raw-queries.md
- Multiple Table Joins: ai/guides/joins.md
+ - Demos:
+ - Gallery: ai/examples/index.md
+ - Basic Usage: ai/examples/basic-with-pytidb.md
+ - Auto Embedding: ai/examples/auto-embedding-with-pytidb.md
+ - Vector Search: ai/examples/vector-search-with-pytidb.md
+ - Fulltext Search: ai/examples/fulltext-search-with-pytidb.md
+ - Hybrid Search: ai/examples/hybrid-search-with-pytidb.md
+ - Image Search: ai/examples/image-search-with-pytidb.md
+ - RAG: ai/examples/rag-with-pytidb.md
+ - Memory: ai/examples/memory-with-pytidb.md
+ - Text2SQL: ai/examples/text2sql-with-pytidb.md
- Integrations:
- MCP Integration:
- TiDB MCP Server: ai/integrations/tidb-mcp-server.md
@@ -136,7 +158,6 @@ nav:
- Claude Desktop: ai/integrations/tidb-mcp-claude-desktop.md
- LlamaIndex: ai/integrations/llamaindex.md
- LangChain: ai/integrations/langchain.md
- # - Examples:
extra:
diff --git a/pyproject.toml b/pyproject.toml
index c9fffc20..4e99023c 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,3 +1,8 @@
+# TiDB for AI Documentation
+# This project uses UV for fast dependency management
+# Install UV: curl -LsSf https://astral.sh/uv/install.sh | sh
+# Install deps: make install (which runs: uv pip install -e .)
+
[project]
name = "docs"
version = "0.1.0"
@@ -10,4 +15,8 @@ dependencies = [
"mkdocstrings[python]>=0.29.1",
"mkdocs>=1.6.1",
"mkdocs-redirects>=1.2.2",
+ "requests>=2.31.0",
+ "PyYAML>=6.0",
+ "Jinja2>=3.1.0",
+ "click>=8.0.0",
]
diff --git a/scripts/check_dependencies.py b/scripts/check_dependencies.py
new file mode 100644
index 00000000..025dab0f
--- /dev/null
+++ b/scripts/check_dependencies.py
@@ -0,0 +1,35 @@
+#!/usr/bin/env python3
+"""Simple dependency check for TiDB for AI documentation."""
+
+import sys
+import subprocess
+
+def main():
+ print("🚀 TiDB for AI Documentation - Quick Check")
+
+ # Check UV
+ try:
+ result = subprocess.run(["uv", "--version"], capture_output=True, text=True)
+ if result.returncode == 0:
+ print("✅ UV is available")
+ else:
+ print("❌ UV not working")
+ return 1
+ except FileNotFoundError:
+ print("❌ UV not found - Install: curl -LsSf https://astral.sh/uv/install.sh | sh")
+ return 1
+
+ # Check basic imports
+ try:
+ import mkdocs, requests, yaml
+ print("✅ Dependencies are installed")
+ except ImportError as e:
+ print(f"❌ Missing dependency: {e}")
+ print("💡 Run: make install")
+ return 1
+
+ print("🎉 Ready to go! Run 'make serve' to start.")
+ return 0
+
+if __name__ == "__main__":
+ sys.exit(main())
\ No newline at end of file
diff --git a/scripts/generate_demos.py b/scripts/generate_demos.py
new file mode 100755
index 00000000..441be5b4
--- /dev/null
+++ b/scripts/generate_demos.py
@@ -0,0 +1,279 @@
+#!/usr/bin/env python3
+"""
+Generate demo gallery and individual demo documentation pages.
+This script reads configuration from demos.yml and generates:
+1. Individual demo pages using demo_template.j2
+2. Gallery index page using gallery_template.j2
+"""
+
+import sys
+import requests
+import re
+import yaml
+from pathlib import Path
+from jinja2 import Environment, FileSystemLoader
+import click
+
+# Configuration
+LOCAL_EXAMPLES_DIR = Path("src/ai/examples")
+CONFIG_FILE = Path("demos.yml")
+DEMO_TEMPLATE_FILE = Path("src/templates/demo_page_template.j2")
+GALLERY_TEMPLATE_FILE = Path("src/templates/demo_gallery_template.j2")
+OUTPUT_FILE = LOCAL_EXAMPLES_DIR / "index.md"
+
+
+def load_config():
+ """Load configuration from YAML file."""
+ if not CONFIG_FILE.exists():
+ raise click.ClickException(f"Configuration file {CONFIG_FILE} not found.")
+
+ try:
+ with open(CONFIG_FILE, 'r', encoding='utf-8') as f:
+ config = yaml.safe_load(f)
+ return config
+ except yaml.YAMLError as e:
+ raise click.ClickException(f"Error parsing {CONFIG_FILE}: {e}")
+
+
+def fetch_readme_from_doc_link(doc_link):
+ """Fetch README.md content from the provided doc_link."""
+ # Convert GitHub tree URL to raw content URL
+ if "github.com" in doc_link and "/tree/" in doc_link:
+ raw_url = doc_link.replace("github.com", "raw.githubusercontent.com").replace("/tree/", "/")
+ elif doc_link.endswith("/README.md"):
+ raw_url = doc_link
+ else:
+ # Assume it's a directory link, append README.md
+ raw_url = doc_link.rstrip('/') + '/README.md'
+ if "github.com" in raw_url and "/tree/" in raw_url:
+ raw_url = raw_url.replace("github.com", "raw.githubusercontent.com").replace("/tree/", "/")
+
+ try:
+ response = requests.get(raw_url, timeout=30)
+ response.raise_for_status()
+ return response.text
+ except requests.RequestException as e:
+ click.echo(f"Error fetching README from {raw_url}: {e}", err=True)
+ return None
+
+
+def extract_repo_info_from_doc_link(doc_link):
+ """Extract repository and path information from doc_link."""
+ if "github.com" not in doc_link:
+ return None, None, None, None
+
+ # Parse URL to extract owner, repo, and path
+ parts = doc_link.replace("https://github.com/", "").split("/")
+ if len(parts) < 2:
+ return None, None, None, None
+
+ owner, repo = parts[0], parts[1]
+
+ # Extract path after /tree/branch/
+ if "/tree/" in doc_link:
+ try:
+ tree_index = parts.index("tree")
+ if len(parts) > tree_index + 2: # owner/repo/tree/branch/path...
+ branch = parts[tree_index + 1]
+ path_parts = parts[tree_index + 2:]
+ # Remove README.md if present
+ if path_parts and path_parts[-1] == "README.md":
+ path_parts = path_parts[:-1]
+ path = "/".join(path_parts)
+ return owner, repo, branch, path
+ except ValueError:
+ pass
+
+ return None, None, None, None
+
+
+def process_readme_content(content, demo_config):
+ """Process README content to adapt it for the documentation site."""
+ if not content:
+ return None
+
+ # Extract repository info
+ owner, repo, branch, example_path = extract_repo_info_from_doc_link(demo_config['doc_link'])
+
+ if not all([owner, repo, branch, example_path]):
+ click.echo(f"Warning: Could not extract repo info from {demo_config['doc_link']}", err=True)
+ return content
+
+ base_repo_url = f"https://github.com/{owner}/{repo}"
+ base_raw_url = f"https://raw.githubusercontent.com/{owner}/{repo}/{branch}"
+
+ # Fix relative links to point to the original repository
+ processed_content = re.sub(
+ r'\]\((?!https?://)(.*?)\)',
+ f']({base_repo_url}/tree/{branch}/{example_path}/\\1)',
+ content
+ )
+
+ # Fix relative image links
+ processed_content = re.sub(
+ r'!\[([^\]]*)\]\((?!https?://)(.*?)\)',
+ f'',
+ processed_content
+ )
+
+ return processed_content
+
+
+def create_demo_page(demo_config, content):
+ """Create a markdown file for a demo using the demo template."""
+ if not DEMO_TEMPLATE_FILE.exists():
+ raise click.ClickException(f"Demo template file {DEMO_TEMPLATE_FILE} not found.")
+
+ # Create the local examples directory if it doesn't exist
+ LOCAL_EXAMPLES_DIR.mkdir(parents=True, exist_ok=True)
+
+ try:
+ # Set up Jinja2 environment
+ env = Environment(loader=FileSystemLoader("src/templates"))
+ template = env.get_template('demo_page_template.j2')
+
+ # Render the template
+ rendered_content = template.render(
+ demo=demo_config,
+ content=content
+ )
+
+ # Create the markdown file
+ filename = f"{demo_config['id'].replace('_', '-')}-with-pytidb.md"
+ filepath = LOCAL_EXAMPLES_DIR / filename
+
+ with open(filepath, 'w', encoding='utf-8') as f:
+ f.write(rendered_content)
+
+ return filepath
+ except Exception as e:
+ raise click.ClickException(f"Error creating demo page for {demo_config['id']}: {e}")
+
+
+def generate_gallery_page(config):
+ """Generate the gallery page using Jinja2 template."""
+ if not GALLERY_TEMPLATE_FILE.exists():
+ raise click.ClickException(f"Gallery template file {GALLERY_TEMPLATE_FILE} not found.")
+
+ try:
+ # Set up Jinja2 environment
+ env = Environment(loader=FileSystemLoader("src/templates"))
+ template = env.get_template('demo_gallery_template.j2')
+
+ # Render the template
+ rendered_content = template.render(
+ config=config,
+ categories=config.get('categories', {}),
+ demos=config.get('demos', {})
+ )
+
+ # Write the rendered content to the output file
+ with open(OUTPUT_FILE, 'w', encoding='utf-8') as f:
+ f.write(rendered_content)
+
+ return True
+ except Exception as e:
+ raise click.ClickException(f"Error generating gallery page: {e}")
+
+
+def sync_demo_docs(config, fetch_from_remote=True):
+ """Sync demo documentation files based on demos.yml configuration."""
+ if not fetch_from_remote:
+ return []
+
+ created_files = []
+ demos_config = config.get('demos', [])
+
+ with click.progressbar(demos_config, label='Processing demos') as demos:
+ for demo_config in demos:
+ demo_id = demo_config['id']
+ doc_link = demo_config.get('doc_link')
+
+ if not doc_link:
+ click.echo(f"Warning: No doc_link found for demo '{demo_id}', skipping...", err=True)
+ continue
+
+ # Fetch README content
+ readme_content = fetch_readme_from_doc_link(doc_link)
+
+ if readme_content:
+ # Process content
+ processed_content = process_readme_content(readme_content, demo_config)
+
+ if processed_content:
+ # Create demo page
+ filepath = create_demo_page(demo_config, processed_content)
+ if filepath:
+ created_files.append((demo_id, filepath))
+ else:
+ click.echo(f"Failed to process content for {demo_id}", err=True)
+ else:
+ click.echo(f"Failed to fetch README for {demo_id}", err=True)
+
+ return created_files
+
+
+@click.command()
+@click.option('--skip-demos', is_flag=True,
+ help='Skip generating individual demo pages from remote repositories')
+@click.option('--skip-gallery', is_flag=True,
+ help='Skip generating the demo gallery index page')
+@click.option('--verbose', '-v', is_flag=True, help='Enable verbose output')
+def main(skip_demos, skip_gallery, verbose):
+ """Generate demo gallery and individual demo documentation pages.
+
+ This script reads configuration from demos.yml and generates:
+ 1. Individual demo pages using demo_template.j2
+ 2. Gallery index page using gallery_template.j2
+
+ By default, both demo pages and gallery are generated.
+ """
+ if verbose:
+ click.echo("Running in verbose mode...")
+
+ # Load configuration
+ try:
+ config = load_config()
+ except click.ClickException:
+ raise
+
+ created_files = []
+
+ # Generate demo pages (unless skipped)
+ if not skip_demos:
+ if verbose:
+ click.echo("Generating demo pages from remote repositories...")
+ created_files = sync_demo_docs(config, fetch_from_remote=True)
+
+ if created_files:
+ click.echo(f"\n✅ Generated {len(created_files)} demo pages:")
+ for demo_id, filepath in created_files:
+ click.echo(f" • {demo_id} → {filepath}")
+ elif verbose:
+ click.echo("No demo pages were generated.")
+ else:
+ if verbose:
+ click.echo("Skipping demo page generation...")
+
+ # Generate gallery page (unless skipped)
+ if not skip_gallery:
+ if verbose:
+ click.echo("Generating gallery page from template...")
+
+ if generate_gallery_page(config):
+ click.echo(f"✅ Gallery page generated: {OUTPUT_FILE}")
+ else:
+ raise click.ClickException("Failed to generate gallery page.")
+ else:
+ if verbose:
+ click.echo("Skipping gallery page generation...")
+
+ # Check if nothing was generated
+ if skip_demos and skip_gallery:
+ click.echo("⚠️ Both demos and gallery generation were skipped. Nothing to do.")
+ else:
+ click.echo("\n🎉 Done! You can now run 'mkdocs serve' to view the documentation site.")
+
+
+if __name__ == "__main__":
+ main()
\ No newline at end of file
diff --git a/src/ai/examples/auto-embedding-with-pytidb.md b/src/ai/examples/auto-embedding-with-pytidb.md
new file mode 100644
index 00000000..20759bf1
--- /dev/null
+++ b/src/ai/examples/auto-embedding-with-pytidb.md
@@ -0,0 +1,92 @@
+---
+title: Auto Embedding
+description: "Automatically generate embeddings for your text data using built-in embedding models."
+source_repo: "https://github.com/pingcap/pytidb/tree/main/examples/auto_embedding"
+---
+
+# Auto Embedding Demo
+
+This example showcases how to use the auto embedding feature with PyTiDB Client.
+
+* Connect to TiDB with PyTiDB Client
+* Define a table with a VectorField configured for automatic embedding
+* Insert plain text data, embeddings are populated automatically in the background
+* Run vector searches with natural language queries, embedding happens transparently
+
+## Prerequisites
+
+- **Python 3.10+**
+- **A TiDB Cloud Serverless cluster**: Create a free cluster here: [tidbcloud.com ↗️](https://tidbcloud.com/?utm_source=github&utm_medium=referral&utm_campaign=pytidb_readme)
+- **Jina AI API key**: Go to [Jina AI](https://jina.ai/embeddings/) to get your own API key
+
+## How to run
+
+**Step 1**: Clone the repository
+
+```bash
+git clone https://github.com/pingcap/pytidb.git
+cd pytidb/examples/auto_embedding/
+```
+
+**Step 2**: Install the required packages
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+pip install -r reqs.txt
+```
+
+**Step 3**: Set up environment to connect to database
+
+Go to [TiDB Cloud console](https://tidbcloud.com/clusters) to get the connection parameters and set up the environment variable like this:
+
+```bash
+cat > .env <
+
TiDB Hybrid Search Demo
+ + +## Prerequisites + +* Python 3.10+ +* TiDB database instance (👉 [Create a free TiDB Serverless Cluster](https://tidbcloud.com/free-trial)) +* OpenAI API key (Go to [OpenAI](https://platform.openai.com/api-keys) to get the API key) + +> **Note** +> +> Currently, full-text search is only available for the following product option and region: +> +> - TiDB Cloud Serverless: Frankfurt (eu-central-1), Singapore (ap-southeast-1) + +## How to run + +**Step 1**: Clone the repository + +```bash +git clone https://github.com/pingcap/pytidb.git +cd pytidb/examples/hybrid_search; +``` + +**Step 2**: Install the required packages and setup environment + +```bash +python -m venv .venv +source .venv/bin/activate +pip install -r reqs.txt +``` + +**Step 3**: Set up environment to connect to storage + +If you are using TiDB Cloud, you can find the connection parameters in the [TiDB Cloud console](https://tidbcloud.com/). + +```bash +cat > .env <
+
+
Pet image search via multimodal embeddings
+ + + +## Prerequisites + +- **Python 3.10+** +- **A TiDB Cloud Serverless cluster**: Create a free cluster here: [tidbcloud.com ↗️](https://tidbcloud.com/?utm_source=github&utm_medium=referral&utm_campaign=pytidb_readme) +- **Jina AI API Key**: Get your free API key at [jina.ai Embeddings ↗️](https://jina.ai/embeddings/) + +## How to run + +**Step 1**: Clone the repository to local + +```bash +git clone https://github.com/pingcap/pytidb.git +cd pytidb/examples/image_search/ +``` + +**Step 2**: Install the required packages + +```bash +python -m venv .venv +source .venv/bin/activate # Windows: .venv\Scripts\activate +pip install -r reqs.txt +``` + +**Step 3**: Set up environment variables + +Go to [TiDB Cloud console](https://tidbcloud.com/clusters) and get the connection parameters, then set up the environment variable like this: + +```bash +cat > .env <
+ Explore hands-on demos showcasing how TiDB empowers AI applications.
+Get started quickly with TiDB Cloud Serverless to build your own AI-powered solutions.
+
+
+ Build an image search application using multimodal embeddings for both text-to-image and image-to-image search. +
++ Build a RAG application that combines document retrieval with language generation. +
++ Implement conversation memory for chatbots and conversational AI applications. +
++ Implement semantic search using vector embeddings to find similar content. +
++ Perform traditional text search using MySQL fulltext search capabilities. +
++ Combine vector search and fulltext search for more comprehensive results. +
++ Build an image search application using multimodal embeddings for both text-to-image and image-to-image search. +
++ Build a RAG application that combines document retrieval with language generation. +
++ Implement conversation memory for chatbots and conversational AI applications. +
++ Convert natural language queries into SQL statements using AI models. +
++ Start your AI journey with TiDB Cloud Serverless. Follow our quickstart guide to build your first AI-powered application in minutes, or explore specific examples for your use case. +
+ +RAG application built with PyTiDB
+ + +## Prerequisites + +- **Python 3.10+** +- **A TiDB Cloud Serverless cluster**: Create a free cluster here: [tidbcloud.com ↗️](https://tidbcloud.com/?utm_source=github&utm_medium=referral&utm_campaign=pytidb_readme) +- **Ollama**: You can install it from [Ollama ↗️](https://ollama.com/download) + +## How to run + +**Step 1**: Prepare the inference API + +Pull the embedding and LLM model via ollama CLI: + +```bash +ollama pull mxbai-embed-large +ollama pull gemma3:4b +ollama run gemma3:4b +``` + +Test the `/embed` and `/generate` endpoints to make sure they are running: + +```bash +curl http://localhost:11434/api/embed -d '{ + "model": "mxbai-embed-large", + "input": "Llamas are members of the camelid family" +}' +``` + +```bash +curl http://localhost:11434/api/generate -d '{ + "model": "gemma3:4b", + "prompt": "Hello, Who are you?" +}' +``` + +**Step 2**: Clone the repository to local + +```bash +git clone https://github.com/pingcap/pytidb.git +cd pytidb/examples/rag/; +``` + +**Step 3**: Install the required packages and setup environment + +```bash +python -m venv .venv +source .venv/bin/activate +pip install -r reqs.txt +``` + +**Step 4**: Set up environment to connect to database + +Go to [TiDB Cloud console](https://tidbcloud.com/clusters) and get the connection parameters, then set up the environment variable like this: + +```bash +cat > .env <Semantic search with vector embeddings
+ + +## Prerequisites + +- **Python 3.10+** +- **A TiDB Cloud Serverless cluster**: Create a free cluster here: [tidbcloud.com ↗️](https://tidbcloud.com/?utm_source=github&utm_medium=referral&utm_campaign=pytidb_readme) +- **Ollama**: You can install it from [Ollama ↗️](https://ollama.com/download) + +## How to run + +**Step 1**: Start the embedding service with Ollama + +Pull the embedding model: + +```bash +ollama pull mxbai-embed-large +``` + +Test the embedding service to make sure it is running: + +```bash +curl http://localhost:11434/api/embed -d '{ + "model": "mxbai-embed-large", + "input": "Llamas are members of the camelid family" +}' +``` + +**Step 2**: Clone the repository to local + +```bash +git clone https://github.com/pingcap/pytidb.git +cd pytidb/examples/vector_search/ +``` + +**Step 3**: Install the required packages and set up the environment + +```bash +python -m venv .venv +source .venv/bin/activate +pip install -r reqs.txt +``` + +**Step 4**: Set up environment to connect to TiDB + +Go to [TiDB Cloud console](https://tidbcloud.com/clusters) and get the connection parameters, then set up the environment variable like this: + +```bash +cat > .env <+ {{ config.description | safe }} +
++ {{ demo.description }} +
++ {{ config.cta.description }} +
+ +