From 085ce337d350436804d91e75c46feece56c928d3 Mon Sep 17 00:00:00 2001 From: Mini256 Date: Mon, 14 Jul 2025 11:03:43 +0800 Subject: [PATCH 1/8] feat: migrate dependencies to pyproject.toml and add dependency checks - Consolidate all dependencies into pyproject.toml for better management - Remove separate scripts/requirements.txt file - Update Makefile to use 'pip install -e .' for installation - Add dependency check script (scripts/check_dependencies.py) - Update documentation to reflect new dependency management - Add 'make check' command to verify environment setup - Fix Examples ordering in navigation and sync script - All dependencies now managed in single pyproject.toml file --- Makefile | 31 +++ README.md | 31 +++ docs/example-gallery.md | 189 +++++++++++++++ mkdocs.yml | 23 +- pyproject.toml | 2 + scripts/check_dependencies.py | 158 +++++++++++++ scripts/sync_examples_from_pytidb.py | 215 ++++++++++++++++++ src/ai/examples/auto-embedding-with-pytidb.md | 85 +++++++ src/ai/examples/basic-with-pytidb.md | 98 ++++++++ .../examples/fulltext-search-with-pytidb.md | 64 ++++++ src/ai/examples/hybrid-search-with-pytidb.md | 124 ++++++++++ src/ai/examples/image-search-with-pytidb.md | 100 ++++++++ src/ai/examples/memory-with-pytidb.md | 93 ++++++++ src/ai/examples/quickstart-with-pytidb.md | 107 +++++++++ src/ai/examples/rag-with-pytidb.md | 101 ++++++++ src/ai/examples/text2sql-with-pytidb.md | 51 +++++ src/ai/examples/vector-search-with-pytidb.md | 82 +++++++ 17 files changed, 1553 insertions(+), 1 deletion(-) create mode 100644 Makefile create mode 100644 docs/example-gallery.md create mode 100644 scripts/check_dependencies.py create mode 100644 scripts/sync_examples_from_pytidb.py create mode 100644 src/ai/examples/auto-embedding-with-pytidb.md create mode 100644 src/ai/examples/basic-with-pytidb.md create mode 100644 src/ai/examples/fulltext-search-with-pytidb.md create mode 100644 src/ai/examples/hybrid-search-with-pytidb.md create mode 100644 src/ai/examples/image-search-with-pytidb.md create mode 100644 src/ai/examples/memory-with-pytidb.md create mode 100644 src/ai/examples/quickstart-with-pytidb.md create mode 100644 src/ai/examples/rag-with-pytidb.md create mode 100644 src/ai/examples/text2sql-with-pytidb.md create mode 100644 src/ai/examples/vector-search-with-pytidb.md diff --git a/Makefile b/Makefile new file mode 100644 index 00000000..312413b8 --- /dev/null +++ b/Makefile @@ -0,0 +1,31 @@ +# TiDB for AI Documentation Site + +.PHONY: help install sync-examples serve build clean check + +help: + @echo "Available commands:" + @echo " install - Install dependencies" + @echo " sync-examples - Sync examples from pytidb repository" + @echo " serve - Start the development server" + @echo " build - Build the documentation site" + @echo " clean - Clean build artifacts" + @echo " check - Check dependencies and project setup" + +install: + pip install -e . + +sync-examples: + python scripts/sync_examples_from_pytidb.py + +serve: + mkdocs serve + +build: + mkdocs build + +clean: + rm -rf site/ + rm -rf .mkdocs_cache/ + +check: + python scripts/check_dependencies.py \ No newline at end of file diff --git a/README.md b/README.md index 38a9520b..73447348 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,37 @@ TiDB is an open-source, distributed SQL database designed for modern AI applicat šŸš€ Follow our [Quickstart Guide](https://pingcap.github.io/ai/quickstart/) to begin building your first AI application with TiDB +## Example Gallery + +This documentation site includes an Example Gallery that showcases various AI applications built with TiDB. The examples are automatically synced from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples). + +### Available Examples + +- **Auto Embedding**: Automatic text embedding with TiDB +- **Basic Usage**: Basic operations with PyTiDB +- **Fulltext Search**: Full-text search capabilities +- **Hybrid Search**: Combining vector and text search +- **Image Search**: Image similarity search +- **Memory**: Agent memory storage +- **Quickstart**: Quick start guide +- **RAG**: Retrieval-Augmented Generation +- **Text2SQL**: Natural language to SQL conversion +- **Vector Search**: Vector similarity search + +### Updating Examples + +To sync the latest examples from the pytidb repository: + +```bash +make sync-examples +``` + +Or run the script directly: + +```bash +python scripts/sync_examples_from_pytidb.py +``` + ## PyTiDB TiDB provide a Python SDK and a series of integrations with popular AI frameworks to help developers build AI applications efficiently. diff --git a/docs/example-gallery.md b/docs/example-gallery.md new file mode 100644 index 00000000..23dd946b --- /dev/null +++ b/docs/example-gallery.md @@ -0,0 +1,189 @@ +# Example Gallery Implementation + +This document describes the implementation of the Example Gallery feature for the TiDB for AI documentation site. + +## Overview + +The Example Gallery automatically syncs examples from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples) and converts them into documentation pages, avoiding duplication of content between repositories. + +## Architecture + +### Files and Structure + +``` +scripts/ +ā””ā”€ā”€ sync_examples_from_pytidb.py # Main sync script + +src/ai/examples/ # Generated example files +ā”œā”€ā”€ auto-embedding-with-pytidb.md +ā”œā”€ā”€ basic-with-pytidb.md +ā”œā”€ā”€ fulltext-search-with-pytidb.md +ā”œā”€ā”€ hybrid-search-with-pytidb.md +ā”œā”€ā”€ image-search-with-pytidb.md +ā”œā”€ā”€ memory-with-pytidb.md +ā”œā”€ā”€ quickstart-with-pytidb.md +ā”œā”€ā”€ rag-with-pytidb.md +ā”œā”€ā”€ text2sql-with-pytidb.md +ā””ā”€ā”€ vector-search-with-pytidb.md + +mkdocs.yml # Updated with Examples section +Makefile # Build commands +README.md # Updated with Example Gallery info +pyproject.toml # All dependencies managed here +``` + +### Sync Process + +1. **Fetch Examples**: The script queries the GitHub API to get all example directories from the pytidb repository +2. **Download README**: For each example, it downloads the README.md file +3. **Process Content**: It adds metadata, fixes links, and formats the content for the documentation site +4. **Generate Files**: Creates local markdown files with proper naming convention +5. **Update Navigation**: Provides output to update the mkdocs.yml navigation + +### Content Processing + +Each example file is processed to: + +- Add YAML front matter with title, description, and source repository +- Include a source repository info box +- Fix relative links to point to the original repository +- Fix relative image links to use raw GitHub URLs +- Apply consistent naming and formatting + +## Usage + +### Initial Setup + +```bash +# Install dependencies (from pyproject.toml) +make install + +# Sync examples +make sync-examples +``` + +### Regular Updates + +To sync the latest examples from the pytidb repository: + +```bash +make sync-examples +``` + +Or run the script directly: + +```bash +python scripts/sync_examples_from_pytidb.py +``` + +### Development + +```bash +# Start development server +make serve + +# Build documentation +make build + +# Clean build artifacts +make clean +``` + +## Configuration + +### Dependency Management + +All dependencies are managed through `pyproject.toml` for consistency: + +```toml +dependencies = [ + "mkdocs-material>=9.6.12", + "mkdocs-jupyter>=0.25.1", + "mkdocstrings[python]>=0.29.1", + "mkdocs>=1.6.1", + "mkdocs-redirects>=1.2.2", + "requests>=2.31.0", # For sync script + "PyYAML>=6.0", # For sync script +] +``` + +### Example Display Names + +The sync script includes a mapping for display names: + +```python +EXAMPLE_DISPLAY_NAMES = { + "auto_embedding": "Auto Embedding", + "basic": "Basic Usage", + "fulltext_search": "Fulltext Search", + "hybrid_search": "Hybrid Search", + "image_search": "Image Search", + "memory": "Memory", + "quickstart": "Quickstart", + "rag": "RAG", + "text2sql": "Text2SQL", + "vector_search": "Vector Search", +} +``` + +### MkDocs Navigation + +The Examples section is configured in `mkdocs.yml`: + +```yaml +nav: + - Home: + # ... other sections ... + - šŸ’” Examples: + - Auto Embedding: ai/examples/auto-embedding-with-pytidb.md + - Basic Usage: ai/examples/basic-with-pytidb.md + # ... other examples ... + - Examples: + - Auto Embedding: ai/examples/auto-embedding-with-pytidb.md + - Basic Usage: ai/examples/basic-with-pytidb.md + # ... other examples ... +``` + +## Benefits + +1. **No Duplication**: Avoids maintaining the same content in multiple repositories +2. **Always Up-to-Date**: Examples are synced from the source repository +3. **Proper Attribution**: Each example clearly links back to the original source +4. **Consistent Formatting**: All examples follow the same documentation style +5. **Easy Maintenance**: Single command to update all examples + +## Maintenance + +### Adding New Examples + +When new examples are added to the pytidb repository: + +1. Run `make sync-examples` to fetch the new examples +2. Update the `EXAMPLE_DISPLAY_NAMES` mapping if needed +3. The mkdocs.yml navigation will need to be updated manually + +### Removing Examples + +If examples are removed from the pytidb repository: + +1. Run `make sync-examples` (it will skip missing examples) +2. Manually remove the corresponding files from `src/ai/examples/` +3. Update the mkdocs.yml navigation + +### Customization + +To customize the processing: + +1. Modify the `process_readme_content()` function in the sync script +2. Update the `EXAMPLE_DISPLAY_NAMES` mapping +3. Adjust the file naming convention in `create_example_file()` + +## Future Enhancements + +Potential improvements: + +1. **Automatic mkdocs.yml Updates**: Automatically update the navigation section +2. **Caching**: Cache downloaded content to avoid unnecessary API calls +3. **Validation**: Validate that all links and images are accessible +4. **Metadata Enhancement**: Extract more metadata from the examples +5. **Multi-language Support**: Support for multiple language versions \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index ecc9f6a0..a64b6d07 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -106,6 +106,17 @@ nav: - Auto Embedding: ai/guides/auto-embedding.md - Reranking: ai/guides/reranking.md - Filtering: ai/guides/filtering.md + - šŸ’” Examples: + - Quickstart: ai/examples/quickstart-with-pytidb.md + - Basic Usage: ai/examples/basic-with-pytidb.md + - Auto Embedding: ai/examples/auto-embedding-with-pytidb.md + - Vector Search: ai/examples/vector-search-with-pytidb.md + - Fulltext Search: ai/examples/fulltext-search-with-pytidb.md + - Hybrid Search: ai/examples/hybrid-search-with-pytidb.md + - Image Search: ai/examples/image-search-with-pytidb.md + - RAG: ai/examples/rag-with-pytidb.md + - Memory: ai/examples/memory-with-pytidb.md + - Text2SQL: ai/examples/text2sql-with-pytidb.md - šŸ”Œ Integrations: - MCP Integration: - TiDB MCP Server: ai/integrations/tidb-mcp-server.md @@ -128,6 +139,17 @@ nav: - Transaction: ai/guides/transaction.md - Raw Queries: ai/guides/raw-queries.md - Multiple Table Joins: ai/guides/joins.md + - Examples: + - Quickstart: ai/examples/quickstart-with-pytidb.md + - Basic Usage: ai/examples/basic-with-pytidb.md + - Auto Embedding: ai/examples/auto-embedding-with-pytidb.md + - Vector Search: ai/examples/vector-search-with-pytidb.md + - Fulltext Search: ai/examples/fulltext-search-with-pytidb.md + - Hybrid Search: ai/examples/hybrid-search-with-pytidb.md + - Image Search: ai/examples/image-search-with-pytidb.md + - RAG: ai/examples/rag-with-pytidb.md + - Memory: ai/examples/memory-with-pytidb.md + - Text2SQL: ai/examples/text2sql-with-pytidb.md - Integrations: - MCP Integration: - TiDB MCP Server: ai/integrations/tidb-mcp-server.md @@ -136,7 +158,6 @@ nav: - Claude Desktop: ai/integrations/tidb-mcp-claude-desktop.md - LlamaIndex: ai/integrations/llamaindex.md - LangChain: ai/integrations/langchain.md - # - Examples: extra: diff --git a/pyproject.toml b/pyproject.toml index c9fffc20..c2e97abb 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -10,4 +10,6 @@ dependencies = [ "mkdocstrings[python]>=0.29.1", "mkdocs>=1.6.1", "mkdocs-redirects>=1.2.2", + "requests>=2.31.0", + "PyYAML>=6.0", ] diff --git a/scripts/check_dependencies.py b/scripts/check_dependencies.py new file mode 100644 index 00000000..4dacbafb --- /dev/null +++ b/scripts/check_dependencies.py @@ -0,0 +1,158 @@ +#!/usr/bin/env python3 +""" +Check if all dependencies are properly installed and configured. +""" + +import sys +import importlib +import subprocess +from pathlib import Path + +def check_python_version(): + """Check if Python version is compatible.""" + min_version = (3, 12) + current_version = sys.version_info[:2] + + print("šŸ Python Version Check:") + print(f" Current: {'.'.join(map(str, current_version))}") + print(f" Required: {'.'.join(map(str, min_version))}+") + + if current_version >= min_version: + print(" āœ… Python version is compatible") + return True + else: + print(" āŒ Python version is too old") + return False + +def check_dependencies(): + """Check if all required dependencies are installed.""" + dependencies = [ + ("mkdocs", "MkDocs"), + ("material", "MkDocs Material"), + ("mkdocs_jupyter", "MkDocs Jupyter"), + ("mkdocstrings", "MkDocstrings"), + ("requests", "Requests"), + ("yaml", "PyYAML"), + ] + + print("\nšŸ“¦ Dependency Check:") + all_ok = True + + for module_name, display_name in dependencies: + try: + module = importlib.import_module(module_name) + version = getattr(module, '__version__', 'unknown') + print(f" āœ… {display_name}: {version}") + except ImportError: + print(f" āŒ {display_name}: not installed") + all_ok = False + + return all_ok + +def check_project_structure(): + """Check if project structure is correct.""" + required_files = [ + "pyproject.toml", + "mkdocs.yml", + "src/ai/quickstart.md", + "scripts/sync_examples_from_pytidb.py", + "Makefile", + ] + + print("\nšŸ“ Project Structure Check:") + all_ok = True + + for file_path in required_files: + path = Path(file_path) + if path.exists(): + print(f" āœ… {file_path}") + else: + print(f" āŒ {file_path}: missing") + all_ok = False + + return all_ok + +def check_examples_directory(): + """Check if examples directory exists and has content.""" + examples_dir = Path("src/ai/examples") + + print("\nšŸ“š Examples Directory Check:") + + if not examples_dir.exists(): + print(" āŒ Examples directory not found") + print(" šŸ’” Run 'make sync-examples' to create it") + return False + + example_files = list(examples_dir.glob("*-with-pytidb.md")) + + if not example_files: + print(" āŒ No example files found") + print(" šŸ’” Run 'make sync-examples' to sync examples") + return False + + print(f" āœ… Found {len(example_files)} example files") + for file in sorted(example_files): + print(f" - {file.name}") + + return True + +def check_makefile_commands(): + """Check if Makefile commands are available.""" + commands = ["install", "sync-examples", "serve", "build", "clean"] + + print("\nšŸ”§ Makefile Commands Check:") + + try: + result = subprocess.run( + ["make", "-n", "help"], + capture_output=True, + text=True, + cwd=Path.cwd() + ) + + if result.returncode == 0: + print(" āœ… Makefile is available") + for cmd in commands: + print(f" - make {cmd}") + else: + print(" āŒ Makefile not found or not working") + return False + except FileNotFoundError: + print(" āŒ 'make' command not found") + return False + + return True + +def main(): + """Main function to run all checks.""" + print("šŸš€ TiDB for AI Documentation - Dependency Check") + print("=" * 50) + + checks = [ + check_python_version(), + check_dependencies(), + check_project_structure(), + check_examples_directory(), + check_makefile_commands(), + ] + + print("\nšŸ“Š Summary:") + print("=" * 50) + + if all(checks): + print("āœ… All checks passed! Your environment is ready.") + print("\nšŸŽ‰ Next steps:") + print(" 1. Run 'make sync-examples' to sync latest examples") + print(" 2. Run 'make serve' to start the development server") + print(" 3. Visit http://localhost:8000 to view the documentation") + return 0 + else: + print("āŒ Some checks failed. Please fix the issues above.") + print("\nšŸ”§ Common solutions:") + print(" 1. Install dependencies: make install") + print(" 2. Sync examples: make sync-examples") + print(" 3. Check Python version: python --version") + return 1 + +if __name__ == "__main__": + sys.exit(main()) \ No newline at end of file diff --git a/scripts/sync_examples_from_pytidb.py b/scripts/sync_examples_from_pytidb.py new file mode 100644 index 00000000..a43f082d --- /dev/null +++ b/scripts/sync_examples_from_pytidb.py @@ -0,0 +1,215 @@ +#!/usr/bin/env python3 +""" +Sync examples from pytidb repository to this documentation site. +This script fetches README.md files from pytidb/examples and converts them +to be used in this documentation site. +""" + +import os +import sys +import requests +import re +import yaml +from pathlib import Path +from urllib.parse import urlparse + +# Configuration +PYTIDB_REPO = "pingcap/pytidb" +EXAMPLES_BASE_URL = f"https://raw.githubusercontent.com/{PYTIDB_REPO}/main/examples" +EXAMPLES_API_URL = f"https://api.github.com/repos/{PYTIDB_REPO}/contents/examples" +LOCAL_EXAMPLES_DIR = Path("src/ai/examples") +MKDOCS_CONFIG_FILE = Path("mkdocs.yml") + +# Example display names mapping +EXAMPLE_DISPLAY_NAMES = { + "auto_embedding": "Auto Embedding", + "basic": "Basic Usage", + "fulltext_search": "Fulltext Search", + "hybrid_search": "Hybrid Search", + "image_search": "Image Search", + "memory": "Memory", + "quickstart": "Quickstart", + "rag": "RAG", + "text2sql": "Text2SQL", + "vector_search": "Vector Search", +} + +# Fixed order for examples (as they should appear in navigation) +EXAMPLES_ORDER = [ + "quickstart", + "basic", + "auto_embedding", + "vector_search", + "fulltext_search", + "hybrid_search", + "image_search", + "rag", + "memory", + "text2sql" +] + +def fetch_github_directory_contents(url): + """Fetch directory contents from GitHub API.""" + try: + response = requests.get(url) + response.raise_for_status() + return response.json() + except requests.RequestException as e: + print(f"Error fetching directory contents: {e}") + return [] + +def fetch_readme_content(example_name): + """Fetch README.md content from a specific example directory.""" + readme_url = f"{EXAMPLES_BASE_URL}/{example_name}/README.md" + try: + response = requests.get(readme_url) + response.raise_for_status() + return response.text + except requests.RequestException as e: + print(f"Error fetching README for {example_name}: {e}") + return None + +def process_readme_content(content, example_name): + """Process README content to adapt it for the documentation site.""" + if not content: + return None + + display_name = EXAMPLE_DISPLAY_NAMES.get(example_name, example_name.replace('_', ' ').title()) + + # Add front matter with metadata + processed_content = f"""--- +title: {display_name} +description: "PyTiDB example: {example_name}" +source_repo: "https://github.com/pingcap/pytidb/tree/main/examples/{example_name}" +--- + +# {display_name} + +!!! info "Source Repository" + This example is from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples/{example_name}). + For the complete source code and latest updates, please visit the original repository. + +""" + + # Process the original content + processed_content += content + + # Fix relative links to point to the original repository + processed_content = re.sub( + r'\]\((?!https?://)(.*?)\)', + f'](https://github.com/pingcap/pytidb/tree/main/examples/{example_name}/\\1)', + processed_content + ) + + # Fix relative image links + processed_content = re.sub( + r'!\[([^\]]*)\]\((?!https?://)(.*?)\)', + f'![\\1](https://raw.githubusercontent.com/pingcap/pytidb/main/examples/{example_name}/\\2)', + processed_content + ) + + return processed_content + +def create_example_file(example_name, content): + """Create a markdown file for an example.""" + # Create the local examples directory if it doesn't exist + LOCAL_EXAMPLES_DIR.mkdir(parents=True, exist_ok=True) + + # Create the markdown file + filename = f"{example_name.replace('_', '-')}-with-pytidb.md" + filepath = LOCAL_EXAMPLES_DIR / filename + + with open(filepath, 'w', encoding='utf-8') as f: + f.write(content) + + print(f"Created: {filepath}") + return filepath + +def update_mkdocs_config(created_files): + """Update mkdocs.yml with the new examples.""" + if not MKDOCS_CONFIG_FILE.exists(): + print(f"Warning: {MKDOCS_CONFIG_FILE} not found. Skipping mkdocs.yml update.") + return + + try: + with open(MKDOCS_CONFIG_FILE, 'r', encoding='utf-8') as f: + content = f.read() + + # Create a mapping of example names to file paths + created_files_dict = {example_name: filepath for example_name, filepath in created_files} + + # Find the Examples section and update it in the specified order + examples_nav = [] + for example_name in EXAMPLES_ORDER: + if example_name in created_files_dict: + display_name = EXAMPLE_DISPLAY_NAMES.get(example_name, example_name.replace('_', ' ').title()) + relative_path = f"ai/examples/{example_name.replace('_', '-')}-with-pytidb.md" + examples_nav.append(f" - {display_name}: {relative_path}") + + examples_section = " - Examples:\n" + "\n".join(examples_nav) + + # Print the examples section for manual addition if needed + print(f"\nExamples section for mkdocs.yml:") + print(examples_section) + + print(f"\nNote: You may need to manually add or update the Examples section in mkdocs.yml") + + except Exception as e: + print(f"Error updating mkdocs.yml: {e}") + +def main(): + """Main function to sync examples from pytidb repository.""" + print("Syncing examples from pytidb repository...") + + # Fetch directory contents + contents = fetch_github_directory_contents(EXAMPLES_API_URL) + + if not contents: + print("No contents found or error occurred.") + return + + # Filter directories only and exclude assets + directories = [item for item in contents if item['type'] == 'dir' and item['name'] != 'assets'] + + if not directories: + print("No example directories found.") + return + + # Create a set of available directory names for quick lookup + available_directories = {directory['name'] for directory in directories} + + created_files = [] + + # Process examples in the specified order + for example_name in EXAMPLES_ORDER: + if example_name in available_directories: + print(f"Processing example: {example_name}") + + # Fetch README content + readme_content = fetch_readme_content(example_name) + + if readme_content: + # Process content + processed_content = process_readme_content(readme_content, example_name) + + if processed_content: + # Create local file + filepath = create_example_file(example_name, processed_content) + created_files.append((example_name, filepath)) + else: + print(f" No README.md found for {example_name}") + else: + print(f" Directory {example_name} not found in repository") + + print(f"\nSynced {len(created_files)} examples:") + for example_name, filepath in created_files: + print(f" - {example_name} -> {filepath}") + + # Update mkdocs.yml configuration + update_mkdocs_config(created_files) + + print("\nDone! The Example Gallery has been created.") + print("You can now run 'mkdocs serve' to view the documentation site.") + +if __name__ == "__main__": + main() \ No newline at end of file diff --git a/src/ai/examples/auto-embedding-with-pytidb.md b/src/ai/examples/auto-embedding-with-pytidb.md new file mode 100644 index 00000000..47912405 --- /dev/null +++ b/src/ai/examples/auto-embedding-with-pytidb.md @@ -0,0 +1,85 @@ +--- +title: Auto Embedding +description: "PyTiDB example: auto_embedding" +source_repo: "https://github.com/pingcap/pytidb/tree/main/examples/auto_embedding" +--- + +# Auto Embedding + +!!! info "Source Repository" + This example is from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples/auto_embedding). + For the complete source code and latest updates, please visit the original repository. + +# Auto Embedding Demo + +This example showcases how to use the auto embedding feature with PyTiDB Client. + +* Connect to TiDB with PyTiDB Client +* Define a table with a VectorField configured for automatic embedding +* Insert plain text data, embeddings are populated automatically in the background +* Run vector searches with natural language queries, embedding happens transparently + +## Prerequisites + +- **Python 3.10+** +- **A TiDB Cloud Serverless cluster**: Create a free cluster here: [tidbcloud.com ā†—ļø](https://tidbcloud.com/?utm_source=github&utm_medium=referral&utm_campaign=pytidb_readme) +- **Jina AI API key**: Go to [Jina AI](https://jina.ai/embeddings/) to get your own API key + +## How to run + +**Step 1**: Clone the repository + +```bash +git clone https://github.com/pingcap/pytidb.git +cd pytidb/examples/auto_embedding/ +``` + +**Step 2**: Install the required packages + +```bash +python -m venv .venv +source .venv/bin/activate +pip install -r reqs.txt +``` + +**Step 3**: Set up environment to connect to database + +Go to [TiDB Cloud console](https://tidbcloud.com/clusters) to get the connection parameters and set up the environment variable like this: + +```bash +cat > .env < .env < + E-commerce product search with full-text search +

E-commerce product search with full-text search

+

+ +## Prerequisites + +- **Python 3.10+** +- **A TiDB Cloud Serverless cluster**: Create a free cluster here: [tidbcloud.com ā†—ļø](https://tidbcloud.com/?utm_source=github&utm_medium=referral&utm_campaign=pytidb_readme) + +## How to run + +**Step 1**: Clone the repository to local + +```bash +git clone https://github.com/pingcap/pytidb.git +cd pytidb/examples/fulltext_search/; +``` + +**Step 2**: Install the required packages and setup environment + +```bash +python -m venv .venv +source .venv/bin/activate +pip install -r reqs.txt +``` + +**Step 3**: Set up environment to connect to database + +Go to the [TiDB Cloud console](https://tidbcloud.com/), create a new cluster if you don't have one, and then get the connection parameters on the connection dialog. + +```bash +cat > .env < + TiDB Hybrid Search Demo +

TiDB Hybrid Search Demo

+

+ +## Prerequisites + +* Python 3.10+ +* TiDB database instance (šŸ‘‰ [Create a free TiDB Serverless Cluster](https://tidbcloud.com/free-trial)) +* OpenAI API key (Go to [OpenAI](https://platform.openai.com/api-keys) to get the API key) + +> **Note** +> +> Currently, full-text search is only available for the following product option and region: +> +> - TiDB Cloud Serverless: Frankfurt (eu-central-1), Singapore (ap-southeast-1) + +## How to run + +**Step 1**: Clone the repository + +```bash +git clone https://github.com/pingcap/pytidb.git +cd pytidb/examples/hybrid_search; +``` + +**Step 2**: Install the required packages and setup environment + +```bash +python -m venv .venv +source .venv/bin/activate +pip install -r reqs.txt +``` + +**Step 3**: Set up environment to connect to storage + +If you are using TiDB Cloud, you can find the connection parameters in the [TiDB Cloud console](https://tidbcloud.com/). + +```bash +cat > .env < +EOF +``` + +**Step 4**: Run the demo + +**Option 1**: Run the Streamlit app + +If you want to check the demo with a web UI, you can run the following command: + +```bash +streamlit run app.py +``` + +Open the browser and visit `http://localhost:8501` + +**Option 2**: Run the demo script + +If you want to check the demo with a script, you can run the following command: + +```bash +python example.py +``` + +Expected output: + +``` +=== CONNECT TO TIDB === +Connected to TiDB. + +=== CREATE TABLE === +Table created. + +=== INSERT SAMPLE DATA === +Inserted 3 rows. + +=== PERFORM HYBRID SEARCH === +Search results: +[ + { + "_distance": 0.4740166257687124, + "_match_score": 1.6804268, + "_score": 0.03278688524590164, + "id": 60013, + "text": "TiDB is a distributed database that supports OLTP, OLAP, HTAP and AI workloads." + }, + { + "_distance": 0.6428459116216618, + "_match_score": 0.78427225, + "_score": 0.03200204813108039, + "id": 60015, + "text": "LlamaIndex is a Python library for building AI-powered applications." + }, + { + "_distance": 0.641581407158715, + "_match_score": null, + "_score": 0.016129032258064516, + "id": 60014, + "text": "PyTiDB is a Python library for developers to connect to TiDB." + } +] +``` + diff --git a/src/ai/examples/image-search-with-pytidb.md b/src/ai/examples/image-search-with-pytidb.md new file mode 100644 index 00000000..01ca0036 --- /dev/null +++ b/src/ai/examples/image-search-with-pytidb.md @@ -0,0 +1,100 @@ +--- +title: Image Search +description: "PyTiDB example: image_search" +source_repo: "https://github.com/pingcap/pytidb/tree/main/examples/image_search" +--- + +# Image Search + +!!! info "Source Repository" + This example is from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples/image_search). + For the complete source code and latest updates, please visit the original repository. + +# Pet Image Search Demo + +This example showcases how to build a powerful image search application by combining TiDB's vector search capabilities with multimodal embedding models. + +With just a few lines of code, you can create an intelligent search system that understands both text and images. + +- šŸ” **Text-to-Image Search**: Find the perfect pet photos by describing what you're looking for in natural language - from "fluffy orange cat" +- šŸ–¼ļø **Image-to-Image Search**: Upload a photo and instantly discover visually similar pets based on breed, color, pose and more + +

+ PyTiDB Image Search Demo +

Pet image search via multimodal embeddings

+

+ + +## Prerequisites + +- **Python 3.10+** +- **A TiDB Cloud Serverless cluster**: Create a free cluster here: [tidbcloud.com ā†—ļø](https://tidbcloud.com/?utm_source=github&utm_medium=referral&utm_campaign=pytidb_readme) +- **Jina AI API Key**: Get your free API key at [jina.ai Embeddings ā†—ļø](https://jina.ai/embeddings/) + +## How to run + +**Step 1**: Clone the repository to local + +```bash +git clone https://github.com/pingcap/pytidb.git +cd pytidb/examples/image_search/ +``` + +**Step 2**: Install the required packages + +```bash +python -m venv .venv +source .venv/bin/activate # Windows: .venv\Scripts\activate +pip install -r reqs.txt +``` + +**Step 3**: Set up environment variables + +Go to [TiDB Cloud console](https://tidbcloud.com/clusters) and get the connection parameters, then set up the environment variable like this: + +```bash +cat > .env < .env < .env < .env < + RAG application built with PyTiDB +

RAG application built with PyTiDB

+

+ +## Prerequisites + +- **Python 3.10+** +- **A TiDB Cloud Serverless cluster**: Create a free cluster here: [tidbcloud.com ā†—ļø](https://tidbcloud.com/?utm_source=github&utm_medium=referral&utm_campaign=pytidb_readme) +- **Ollama**: You can install it from [Ollama ā†—ļø](https://ollama.com/download) + +## How to run + +**Step 1**: Prepare the inference API + +Pull the embedding and LLM model via ollama CLI: + +```bash +ollama pull mxbai-embed-large +ollama pull gemma3:4b +ollama run gemma3:4b +``` + +Test the `/embed` and `/generate` endpoints to make sure they are running: + +```bash +curl http://localhost:11434/api/embed -d '{ + "model": "mxbai-embed-large", + "input": "Llamas are members of the camelid family" +}' +``` + +```bash +curl http://localhost:11434/api/generate -d '{ + "model": "gemma3:4b", + "prompt": "Hello, Who are you?" +}' +``` + +**Step 2**: Clone the repository to local + +```bash +git clone https://github.com/pingcap/pytidb.git +cd pytidb/examples/rag/; +``` + +**Step 3**: Install the required packages and setup environment + +```bash +python -m venv .venv +source .venv/bin/activate +pip install -r reqs.txt +``` + +**Step 4**: Set up environment to connect to database + +Go to [TiDB Cloud console](https://tidbcloud.com/clusters) and get the connection parameters, then set up the environment variable like this: + +```bash +cat > .env < + Semantic search with vector embeddings +

Semantic search with vector embeddings

+

+ +## Prerequisites + +- **Python 3.10+** +- **A TiDB Cloud Serverless cluster**: Create a free cluster here: [tidbcloud.com ā†—ļø](https://tidbcloud.com/?utm_source=github&utm_medium=referral&utm_campaign=pytidb_readme) +- **Ollama**: You can install it from [Ollama ā†—ļø](https://ollama.com/download) + +## How to run + +**Step 1**: Start the embedding service with Ollama + +Pull the embedding model: + +```bash +ollama pull mxbai-embed-large +``` + +Test the embedding service to make sure it is running: + +```bash +curl http://localhost:11434/api/embed -d '{ + "model": "mxbai-embed-large", + "input": "Llamas are members of the camelid family" +}' +``` + +**Step 2**: Clone the repository to local + +```bash +git clone https://github.com/pingcap/pytidb.git +cd pytidb/examples/vector_search/ +``` + +**Step 3**: Install the required packages and set up the environment + +```bash +python -m venv .venv +source .venv/bin/activate +pip install -r reqs.txt +``` + +**Step 4**: Set up environment to connect to TiDB + +Go to [TiDB Cloud console](https://tidbcloud.com/clusters) and get the connection parameters, then set up the environment variable like this: + +```bash +cat > .env < Date: Mon, 14 Jul 2025 11:05:57 +0800 Subject: [PATCH 2/8] feat: migrate to UV package manager for faster dependency management - Update Makefile to use 'uv pip install -e .' instead of pip - Add UV availability check in scripts/check_dependencies.py - Update documentation to include UV prerequisites - Add UV installation instructions in README.md and docs/ - Enhance Makefile help with UV information - Add comments in pyproject.toml about UV usage - UV provides significantly faster package installation and resolution UV installation: curl -LsSf https://astral.sh/uv/install.sh | sh --- Makefile | 9 +++++++-- README.md | 16 ++++++++++++++++ docs/example-gallery.md | 11 ++++++++++- pyproject.toml | 5 +++++ scripts/check_dependencies.py | 31 ++++++++++++++++++++++++++++--- 5 files changed, 66 insertions(+), 6 deletions(-) diff --git a/Makefile b/Makefile index 312413b8..34a7763a 100644 --- a/Makefile +++ b/Makefile @@ -3,16 +3,21 @@ .PHONY: help install sync-examples serve build clean check help: + @echo "TiDB for AI Documentation Site" + @echo "=============================" @echo "Available commands:" - @echo " install - Install dependencies" + @echo " install - Install dependencies using UV" @echo " sync-examples - Sync examples from pytidb repository" @echo " serve - Start the development server" @echo " build - Build the documentation site" @echo " clean - Clean build artifacts" @echo " check - Check dependencies and project setup" + @echo "" + @echo "Prerequisites:" + @echo " UV package manager - curl -LsSf https://astral.sh/uv/install.sh | sh" install: - pip install -e . + uv pip install -e . sync-examples: python scripts/sync_examples_from_pytidb.py diff --git a/README.md b/README.md index 73447348..cd4697aa 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,22 @@ This documentation site includes an Example Gallery that showcases various AI ap - **Text2SQL**: Natural language to SQL conversion - **Vector Search**: Vector similarity search +### Prerequisites + +This project uses [UV](https://docs.astral.sh/uv/) for fast dependency management. Install UV first: + +```bash +# Install UV +curl -LsSf https://astral.sh/uv/install.sh | sh +``` + +Then install the project dependencies: + +```bash +# Install all dependencies +make install +``` + ### Updating Examples To sync the latest examples from the pytidb repository: diff --git a/docs/example-gallery.md b/docs/example-gallery.md index 23dd946b..aa852bf8 100644 --- a/docs/example-gallery.md +++ b/docs/example-gallery.md @@ -54,8 +54,17 @@ Each example file is processed to: ### Initial Setup +First, make sure you have UV installed: + +```bash +# Install UV (if not already installed) +curl -LsSf https://astral.sh/uv/install.sh | sh +``` + +Then install dependencies: + ```bash -# Install dependencies (from pyproject.toml) +# Install dependencies (from pyproject.toml using UV) make install # Sync examples diff --git a/pyproject.toml b/pyproject.toml index c2e97abb..f3a1ffe3 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,3 +1,8 @@ +# TiDB for AI Documentation +# This project uses UV for fast dependency management +# Install UV: curl -LsSf https://astral.sh/uv/install.sh | sh +# Install deps: make install (which runs: uv pip install -e .) + [project] name = "docs" version = "0.1.0" diff --git a/scripts/check_dependencies.py b/scripts/check_dependencies.py index 4dacbafb..df3a5d85 100644 --- a/scripts/check_dependencies.py +++ b/scripts/check_dependencies.py @@ -96,6 +96,29 @@ def check_examples_directory(): return True +def check_uv_availability(): + """Check if uv is available.""" + print("\nāš” UV Package Manager Check:") + + try: + result = subprocess.run( + ["uv", "--version"], + capture_output=True, + text=True + ) + + if result.returncode == 0: + version = result.stdout.strip() + print(f" āœ… UV is available: {version}") + return True + else: + print(" āŒ UV not working properly") + return False + except FileNotFoundError: + print(" āŒ UV not found") + print(" šŸ’” Install UV: curl -LsSf https://astral.sh/uv/install.sh | sh") + return False + def check_makefile_commands(): """Check if Makefile commands are available.""" commands = ["install", "sync-examples", "serve", "build", "clean"] @@ -130,6 +153,7 @@ def main(): checks = [ check_python_version(), + check_uv_availability(), check_dependencies(), check_project_structure(), check_examples_directory(), @@ -149,9 +173,10 @@ def main(): else: print("āŒ Some checks failed. Please fix the issues above.") print("\nšŸ”§ Common solutions:") - print(" 1. Install dependencies: make install") - print(" 2. Sync examples: make sync-examples") - print(" 3. Check Python version: python --version") + print(" 1. Install UV: curl -LsSf https://astral.sh/uv/install.sh | sh") + print(" 2. Install dependencies: make install") + print(" 3. Sync examples: make sync-examples") + print(" 4. Check Python version: python --version") return 1 if __name__ == "__main__": From e03432260ef5e19dd9427a355627ddb61b9bdb7d Mon Sep 17 00:00:00 2001 From: Mini256 Date: Mon, 14 Jul 2025 11:07:06 +0800 Subject: [PATCH 3/8] simplify: streamline dependency check script - Reduce verbose checks to just essential ones - Only check UV availability and core imports - Much faster and cleaner output - Focus on what actually matters for getting started --- scripts/check_dependencies.py | 186 ++++------------------------------ 1 file changed, 19 insertions(+), 167 deletions(-) diff --git a/scripts/check_dependencies.py b/scripts/check_dependencies.py index df3a5d85..025dab0f 100644 --- a/scripts/check_dependencies.py +++ b/scripts/check_dependencies.py @@ -1,183 +1,35 @@ #!/usr/bin/env python3 -""" -Check if all dependencies are properly installed and configured. -""" +"""Simple dependency check for TiDB for AI documentation.""" import sys -import importlib import subprocess -from pathlib import Path -def check_python_version(): - """Check if Python version is compatible.""" - min_version = (3, 12) - current_version = sys.version_info[:2] - - print("šŸ Python Version Check:") - print(f" Current: {'.'.join(map(str, current_version))}") - print(f" Required: {'.'.join(map(str, min_version))}+") - - if current_version >= min_version: - print(" āœ… Python version is compatible") - return True - else: - print(" āŒ Python version is too old") - return False - -def check_dependencies(): - """Check if all required dependencies are installed.""" - dependencies = [ - ("mkdocs", "MkDocs"), - ("material", "MkDocs Material"), - ("mkdocs_jupyter", "MkDocs Jupyter"), - ("mkdocstrings", "MkDocstrings"), - ("requests", "Requests"), - ("yaml", "PyYAML"), - ] - - print("\nšŸ“¦ Dependency Check:") - all_ok = True - - for module_name, display_name in dependencies: - try: - module = importlib.import_module(module_name) - version = getattr(module, '__version__', 'unknown') - print(f" āœ… {display_name}: {version}") - except ImportError: - print(f" āŒ {display_name}: not installed") - all_ok = False - - return all_ok - -def check_project_structure(): - """Check if project structure is correct.""" - required_files = [ - "pyproject.toml", - "mkdocs.yml", - "src/ai/quickstart.md", - "scripts/sync_examples_from_pytidb.py", - "Makefile", - ] - - print("\nšŸ“ Project Structure Check:") - all_ok = True - - for file_path in required_files: - path = Path(file_path) - if path.exists(): - print(f" āœ… {file_path}") - else: - print(f" āŒ {file_path}: missing") - all_ok = False - - return all_ok - -def check_examples_directory(): - """Check if examples directory exists and has content.""" - examples_dir = Path("src/ai/examples") - - print("\nšŸ“š Examples Directory Check:") - - if not examples_dir.exists(): - print(" āŒ Examples directory not found") - print(" šŸ’” Run 'make sync-examples' to create it") - return False - - example_files = list(examples_dir.glob("*-with-pytidb.md")) - - if not example_files: - print(" āŒ No example files found") - print(" šŸ’” Run 'make sync-examples' to sync examples") - return False - - print(f" āœ… Found {len(example_files)} example files") - for file in sorted(example_files): - print(f" - {file.name}") - - return True - -def check_uv_availability(): - """Check if uv is available.""" - print("\nāš” UV Package Manager Check:") +def main(): + print("šŸš€ TiDB for AI Documentation - Quick Check") + # Check UV try: - result = subprocess.run( - ["uv", "--version"], - capture_output=True, - text=True - ) - + result = subprocess.run(["uv", "--version"], capture_output=True, text=True) if result.returncode == 0: - version = result.stdout.strip() - print(f" āœ… UV is available: {version}") - return True + print("āœ… UV is available") else: - print(" āŒ UV not working properly") - return False + print("āŒ UV not working") + return 1 except FileNotFoundError: - print(" āŒ UV not found") - print(" šŸ’” Install UV: curl -LsSf https://astral.sh/uv/install.sh | sh") - return False - -def check_makefile_commands(): - """Check if Makefile commands are available.""" - commands = ["install", "sync-examples", "serve", "build", "clean"] - - print("\nšŸ”§ Makefile Commands Check:") + print("āŒ UV not found - Install: curl -LsSf https://astral.sh/uv/install.sh | sh") + return 1 + # Check basic imports try: - result = subprocess.run( - ["make", "-n", "help"], - capture_output=True, - text=True, - cwd=Path.cwd() - ) - - if result.returncode == 0: - print(" āœ… Makefile is available") - for cmd in commands: - print(f" - make {cmd}") - else: - print(" āŒ Makefile not found or not working") - return False - except FileNotFoundError: - print(" āŒ 'make' command not found") - return False - - return True - -def main(): - """Main function to run all checks.""" - print("šŸš€ TiDB for AI Documentation - Dependency Check") - print("=" * 50) - - checks = [ - check_python_version(), - check_uv_availability(), - check_dependencies(), - check_project_structure(), - check_examples_directory(), - check_makefile_commands(), - ] - - print("\nšŸ“Š Summary:") - print("=" * 50) - - if all(checks): - print("āœ… All checks passed! Your environment is ready.") - print("\nšŸŽ‰ Next steps:") - print(" 1. Run 'make sync-examples' to sync latest examples") - print(" 2. Run 'make serve' to start the development server") - print(" 3. Visit http://localhost:8000 to view the documentation") - return 0 - else: - print("āŒ Some checks failed. Please fix the issues above.") - print("\nšŸ”§ Common solutions:") - print(" 1. Install UV: curl -LsSf https://astral.sh/uv/install.sh | sh") - print(" 2. Install dependencies: make install") - print(" 3. Sync examples: make sync-examples") - print(" 4. Check Python version: python --version") + import mkdocs, requests, yaml + print("āœ… Dependencies are installed") + except ImportError as e: + print(f"āŒ Missing dependency: {e}") + print("šŸ’” Run: make install") return 1 + + print("šŸŽ‰ Ready to go! Run 'make serve' to start.") + return 0 if __name__ == "__main__": sys.exit(main()) \ No newline at end of file From de30b06965d1a6af0b4c7e3268e3ed253bd21bcb Mon Sep 17 00:00:00 2001 From: Mini256 Date: Mon, 14 Jul 2025 11:24:42 +0800 Subject: [PATCH 4/8] fix --- DEVELOPMENT.md | 80 ++++++++++++++++ README.md | 59 +++--------- docs/example-gallery.md | 198 ---------------------------------------- 3 files changed, 92 insertions(+), 245 deletions(-) create mode 100644 DEVELOPMENT.md delete mode 100644 docs/example-gallery.md diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md new file mode 100644 index 00000000..ded416ed --- /dev/null +++ b/DEVELOPMENT.md @@ -0,0 +1,80 @@ +# Development Guide + +This guide covers the development setup and maintenance of the TiDB for AI documentation site. + +## Prerequisites + +This project uses [UV](https://docs.astral.sh/uv/) for fast dependency management. Install UV first: + +```bash +# Install UV +curl -LsSf https://astral.sh/uv/install.sh | sh +``` + +## Installation + +Install the project dependencies: + +```bash +# Install all dependencies +make install +``` + +Or use UV directly: + +```bash +uv pip install -e . +``` + +## Available Commands + +This project includes a Makefile with the following commands: + +```bash +# Show available commands +make help + +# Install dependencies using UV +make install + +# Sync examples from pytidb repository +make sync-examples + +# Start the development server +make serve + +# Build the documentation site +make build + +# Clean build artifacts +make clean + +# Check dependencies and project setup +make check +``` + +## Example Gallery Maintenance + +The Example Gallery automatically syncs content from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples). + +### Updating Examples + +To sync the latest examples from the pytidb repository: + +```bash +make sync-examples +``` + +Or run the script directly: + +```bash +python scripts/sync_examples_from_pytidb.py +``` + +### How It Works + +The sync script: +1. Fetches README.md files from the pytidb repository using GitHub API +2. Processes content by adding YAML frontmatter and fixing relative links +3. Creates local markdown files with naming convention: `{example_name}-with-pytidb.md` +4. Updates the navigation configuration automatically diff --git a/README.md b/README.md index cd4697aa..a1469aef 100644 --- a/README.md +++ b/README.md @@ -4,53 +4,6 @@ TiDB is an open-source, distributed SQL database designed for modern AI applicat šŸš€ Follow our [Quickstart Guide](https://pingcap.github.io/ai/quickstart/) to begin building your first AI application with TiDB -## Example Gallery - -This documentation site includes an Example Gallery that showcases various AI applications built with TiDB. The examples are automatically synced from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples). - -### Available Examples - -- **Auto Embedding**: Automatic text embedding with TiDB -- **Basic Usage**: Basic operations with PyTiDB -- **Fulltext Search**: Full-text search capabilities -- **Hybrid Search**: Combining vector and text search -- **Image Search**: Image similarity search -- **Memory**: Agent memory storage -- **Quickstart**: Quick start guide -- **RAG**: Retrieval-Augmented Generation -- **Text2SQL**: Natural language to SQL conversion -- **Vector Search**: Vector similarity search - -### Prerequisites - -This project uses [UV](https://docs.astral.sh/uv/) for fast dependency management. Install UV first: - -```bash -# Install UV -curl -LsSf https://astral.sh/uv/install.sh | sh -``` - -Then install the project dependencies: - -```bash -# Install all dependencies -make install -``` - -### Updating Examples - -To sync the latest examples from the pytidb repository: - -```bash -make sync-examples -``` - -Or run the script directly: - -```bash -python scripts/sync_examples_from_pytidb.py -``` - ## PyTiDB TiDB provide a Python SDK and a series of integrations with popular AI frameworks to help developers build AI applications efficiently. @@ -61,6 +14,14 @@ To install the TiDB Python SDK, run the following command: pip install pytidb ``` +Examples: + +- Getting Started: [Quickstart](https://pingcap.github.io/ai/examples/quickstart-with-pytidb/), [Basic Usage](https://pingcap.github.io/ai/examples/basic-with-pytidb/) +- Search & Retrieval: [Vector Search](https://pingcap.github.io/ai/examples/vector-search-with-pytidb/), [Fulltext Search](https://pingcap.github.io/ai/examples/fulltext-search-with-pytidb/), [Hybrid Search](https://pingcap.github.io/ai/examples/hybrid-search-with-pytidb/) +- AI Applications: [RAG](https://pingcap.github.io/ai/examples/rag-with-pytidb/), [Text2SQL](https://pingcap.github.io/ai/examples/text2sql-with-pytidb/), [Memory](https://pingcap.github.io/ai/examples/memory-with-pytidb/) +- Advanced Features: [Auto Embedding](https://pingcap.github.io/ai/examples/auto-embedding-with-pytidb/), [Image Search](https://pingcap.github.io/ai/examples/image-search-with-pytidb/) + + Integrations: - AI Frameworks: [LlamaIndex](https://docs.pingcap.com/tidbcloud/vector-search-integrate-with-llamaindex/), [LangChain](https://docs.pingcap.com/tidbcloud/vector-search-integrate-with-langchain/) @@ -68,6 +29,10 @@ Integrations: - AI Services: [Bedrock](https://docs.pingcap.com/tidbcloud/vector-search-integrate-with-amazon-bedrock/) - Embedding Models/Services: [JinaAI](https://docs.pingcap.com/tidbcloud/vector-search-integrate-with-jinaai-embedding/) +## Contribute + +We welcome contributions to improve the TiDB for AI documentation! For development setup, maintenance scripts, and detailed contribution guidelines, please see [DEVELOPMENT.md](DEVELOPMENT.md). + ## FAQ ### How can I get support? diff --git a/docs/example-gallery.md b/docs/example-gallery.md deleted file mode 100644 index aa852bf8..00000000 --- a/docs/example-gallery.md +++ /dev/null @@ -1,198 +0,0 @@ -# Example Gallery Implementation - -This document describes the implementation of the Example Gallery feature for the TiDB for AI documentation site. - -## Overview - -The Example Gallery automatically syncs examples from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples) and converts them into documentation pages, avoiding duplication of content between repositories. - -## Architecture - -### Files and Structure - -``` -scripts/ -ā””ā”€ā”€ sync_examples_from_pytidb.py # Main sync script - -src/ai/examples/ # Generated example files -ā”œā”€ā”€ auto-embedding-with-pytidb.md -ā”œā”€ā”€ basic-with-pytidb.md -ā”œā”€ā”€ fulltext-search-with-pytidb.md -ā”œā”€ā”€ hybrid-search-with-pytidb.md -ā”œā”€ā”€ image-search-with-pytidb.md -ā”œā”€ā”€ memory-with-pytidb.md -ā”œā”€ā”€ quickstart-with-pytidb.md -ā”œā”€ā”€ rag-with-pytidb.md -ā”œā”€ā”€ text2sql-with-pytidb.md -ā””ā”€ā”€ vector-search-with-pytidb.md - -mkdocs.yml # Updated with Examples section -Makefile # Build commands -README.md # Updated with Example Gallery info -pyproject.toml # All dependencies managed here -``` - -### Sync Process - -1. **Fetch Examples**: The script queries the GitHub API to get all example directories from the pytidb repository -2. **Download README**: For each example, it downloads the README.md file -3. **Process Content**: It adds metadata, fixes links, and formats the content for the documentation site -4. **Generate Files**: Creates local markdown files with proper naming convention -5. **Update Navigation**: Provides output to update the mkdocs.yml navigation - -### Content Processing - -Each example file is processed to: - -- Add YAML front matter with title, description, and source repository -- Include a source repository info box -- Fix relative links to point to the original repository -- Fix relative image links to use raw GitHub URLs -- Apply consistent naming and formatting - -## Usage - -### Initial Setup - -First, make sure you have UV installed: - -```bash -# Install UV (if not already installed) -curl -LsSf https://astral.sh/uv/install.sh | sh -``` - -Then install dependencies: - -```bash -# Install dependencies (from pyproject.toml using UV) -make install - -# Sync examples -make sync-examples -``` - -### Regular Updates - -To sync the latest examples from the pytidb repository: - -```bash -make sync-examples -``` - -Or run the script directly: - -```bash -python scripts/sync_examples_from_pytidb.py -``` - -### Development - -```bash -# Start development server -make serve - -# Build documentation -make build - -# Clean build artifacts -make clean -``` - -## Configuration - -### Dependency Management - -All dependencies are managed through `pyproject.toml` for consistency: - -```toml -dependencies = [ - "mkdocs-material>=9.6.12", - "mkdocs-jupyter>=0.25.1", - "mkdocstrings[python]>=0.29.1", - "mkdocs>=1.6.1", - "mkdocs-redirects>=1.2.2", - "requests>=2.31.0", # For sync script - "PyYAML>=6.0", # For sync script -] -``` - -### Example Display Names - -The sync script includes a mapping for display names: - -```python -EXAMPLE_DISPLAY_NAMES = { - "auto_embedding": "Auto Embedding", - "basic": "Basic Usage", - "fulltext_search": "Fulltext Search", - "hybrid_search": "Hybrid Search", - "image_search": "Image Search", - "memory": "Memory", - "quickstart": "Quickstart", - "rag": "RAG", - "text2sql": "Text2SQL", - "vector_search": "Vector Search", -} -``` - -### MkDocs Navigation - -The Examples section is configured in `mkdocs.yml`: - -```yaml -nav: - - Home: - # ... other sections ... - - šŸ’” Examples: - - Auto Embedding: ai/examples/auto-embedding-with-pytidb.md - - Basic Usage: ai/examples/basic-with-pytidb.md - # ... other examples ... - - Examples: - - Auto Embedding: ai/examples/auto-embedding-with-pytidb.md - - Basic Usage: ai/examples/basic-with-pytidb.md - # ... other examples ... -``` - -## Benefits - -1. **No Duplication**: Avoids maintaining the same content in multiple repositories -2. **Always Up-to-Date**: Examples are synced from the source repository -3. **Proper Attribution**: Each example clearly links back to the original source -4. **Consistent Formatting**: All examples follow the same documentation style -5. **Easy Maintenance**: Single command to update all examples - -## Maintenance - -### Adding New Examples - -When new examples are added to the pytidb repository: - -1. Run `make sync-examples` to fetch the new examples -2. Update the `EXAMPLE_DISPLAY_NAMES` mapping if needed -3. The mkdocs.yml navigation will need to be updated manually - -### Removing Examples - -If examples are removed from the pytidb repository: - -1. Run `make sync-examples` (it will skip missing examples) -2. Manually remove the corresponding files from `src/ai/examples/` -3. Update the mkdocs.yml navigation - -### Customization - -To customize the processing: - -1. Modify the `process_readme_content()` function in the sync script -2. Update the `EXAMPLE_DISPLAY_NAMES` mapping -3. Adjust the file naming convention in `create_example_file()` - -## Future Enhancements - -Potential improvements: - -1. **Automatic mkdocs.yml Updates**: Automatically update the navigation section -2. **Caching**: Cache downloaded content to avoid unnecessary API calls -3. **Validation**: Validate that all links and images are accessible -4. **Metadata Enhancement**: Extract more metadata from the examples -5. **Multi-language Support**: Support for multiple language versions \ No newline at end of file From 3d221bc1311050b3c5ddc2e015251f05ebbda12f Mon Sep 17 00:00:00 2001 From: Mini256 Date: Mon, 14 Jul 2025 17:10:38 +0800 Subject: [PATCH 5/8] feat: add example gallery and enhance example documentation --- mkdocs.yml | 2 + scripts/sync_examples_from_pytidb.py | 184 +++++++++- src/ai/examples/index.md | 500 +++++++++++++++++++++++++++ src/styles/extra.css | 3 +- 4 files changed, 681 insertions(+), 8 deletions(-) create mode 100644 src/ai/examples/index.md diff --git a/mkdocs.yml b/mkdocs.yml index a64b6d07..8d41c164 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -107,6 +107,7 @@ nav: - Reranking: ai/guides/reranking.md - Filtering: ai/guides/filtering.md - šŸ’” Examples: + - Gallery: ai/examples/index.md - Quickstart: ai/examples/quickstart-with-pytidb.md - Basic Usage: ai/examples/basic-with-pytidb.md - Auto Embedding: ai/examples/auto-embedding-with-pytidb.md @@ -140,6 +141,7 @@ nav: - Raw Queries: ai/guides/raw-queries.md - Multiple Table Joins: ai/guides/joins.md - Examples: + - Gallery: ai/examples/index.md - Quickstart: ai/examples/quickstart-with-pytidb.md - Basic Usage: ai/examples/basic-with-pytidb.md - Auto Embedding: ai/examples/auto-embedding-with-pytidb.md diff --git a/scripts/sync_examples_from_pytidb.py b/scripts/sync_examples_from_pytidb.py index a43f082d..21ca2ad0 100644 --- a/scripts/sync_examples_from_pytidb.py +++ b/scripts/sync_examples_from_pytidb.py @@ -34,6 +34,48 @@ "vector_search": "Vector Search", } +# Example descriptions for gallery +EXAMPLE_DESCRIPTIONS = { + "quickstart": "Get started with PyTiDB in minutes. Learn the basics of connecting to TiDB, creating tables, and performing vector search.", + "basic": "Learn fundamental PyTiDB operations including database connection, table creation, and data manipulation.", + "auto_embedding": "Automatically generate embeddings for your text data using built-in embedding models.", + "vector_search": "Implement semantic search using vector embeddings to find similar content.", + "fulltext_search": "Perform traditional text search using MySQL fulltext search capabilities.", + "hybrid_search": "Combine vector search and fulltext search for more comprehensive results.", + "image_search": "Build an image search application using multimodal embeddings for both text-to-image and image-to-image search.", + "rag": "Build a RAG application that combines document retrieval with language generation.", + "memory": "Implement conversation memory for chatbots and conversational AI applications.", + "text2sql": "Convert natural language queries into SQL statements using AI models.", +} + +# Example icons for gallery +EXAMPLE_ICONS = { + "quickstart": ":rocket:{ .lg .middle }", + "basic": ":material-cog:{ .lg .middle }", + "auto_embedding": ":material-auto-fix:{ .lg .middle }", + "vector_search": ":material-vector-triangle:{ .lg .middle }", + "fulltext_search": ":material-file-search:{ .lg .middle }", + "hybrid_search": ":material-merge:{ .lg .middle }", + "image_search": ":material-image-search:{ .lg .middle }", + "rag": ":material-message-question:{ .lg .middle }", + "memory": ":material-memory:{ .lg .middle }", + "text2sql": ":material-code-braces:{ .lg .middle }", +} + +# Example key features for table +EXAMPLE_FEATURES = { + "quickstart": "Database connection, table creation, vector search", + "basic": "CRUD operations, data manipulation", + "auto_embedding": "Built-in embedding models, text processing", + "vector_search": "Similarity search, embedding-based retrieval", + "fulltext_search": "MySQL fulltext search, keyword matching", + "hybrid_search": "Vector + fulltext search, result ranking", + "image_search": "Text-to-image, image-to-image search", + "rag": "Document retrieval, AI generation", + "memory": "Chatbot memory, context preservation", + "text2sql": "NL query conversion, SQL generation", +} + # Fixed order for examples (as they should appear in navigation) EXAMPLES_ORDER = [ "quickstart", @@ -69,13 +111,38 @@ def fetch_readme_content(example_name): print(f"Error fetching README for {example_name}: {e}") return None +def extract_cover_image(content): + """Extract the first image from content to use as cover image.""" + if not content: + return None + + # Look for image patterns + image_patterns = [ + r'!\[([^\]]*)\]\((https://[^\)]+)\)', # Already absolute URLs + r'!\[([^\]]*)\]\(([^\)]+)\)', # Relative URLs + r']+src="([^"]+)"[^>]*>', # HTML img tags + ] + + for pattern in image_patterns: + matches = re.findall(pattern, content) + if matches: + if len(matches[0]) == 2: # Markdown format + return matches[0][1] + else: # HTML format + return matches[0] + + return None + def process_readme_content(content, example_name): """Process README content to adapt it for the documentation site.""" if not content: - return None + return None, None display_name = EXAMPLE_DISPLAY_NAMES.get(example_name, example_name.replace('_', ' ').title()) + # Extract cover image before processing + cover_image = extract_cover_image(content) + # Add front matter with metadata processed_content = f"""--- title: {display_name} @@ -108,7 +175,11 @@ def process_readme_content(content, example_name): processed_content ) - return processed_content + # If we have a relative cover image, make it absolute + if cover_image and not cover_image.startswith('http'): + cover_image = f"https://raw.githubusercontent.com/pingcap/pytidb/main/examples/{example_name}/{cover_image}" + + return processed_content, cover_image def create_example_file(example_name, content): """Create a markdown file for an example.""" @@ -125,6 +196,87 @@ def create_example_file(example_name, content): print(f"Created: {filepath}") return filepath +def create_gallery_page(example_data): + """Create the Example Gallery page with grids layout.""" + + # Start with the frontmatter and header + gallery_content = """--- +title: Example Gallery +description: "Explore all PyTiDB examples with cover images and descriptions" +hide: + - navigation + - toc +--- + +# Example Gallery + +Explore these hands-on examples to learn how to build AI applications with TiDB and PyTiDB. + +
+ +""" + + # Add grid cards for each example + for example_name in EXAMPLES_ORDER: + if example_name in example_data: + display_name = EXAMPLE_DISPLAY_NAMES.get(example_name, example_name.replace('_', ' ').title()) + description = EXAMPLE_DESCRIPTIONS.get(example_name, f"Learn about {display_name}") + icon = EXAMPLE_ICONS.get(example_name, ":material-cog:{ .lg .middle }") + cover_image = example_data[example_name].get('cover_image') + + # Create the card + card_content = f"""- {icon} **{display_name}** + + --- + + {description} + +""" + + # Add cover image if available + if cover_image: + if example_name == "image_search": + # Special handling for image search with specific styling + card_content += f' ![{display_name} Demo]({cover_image}){{ width="300" }}\n\n' + else: + card_content += f' ![{display_name} Demo]({cover_image}){{ width="250" }}\n\n' + + # Add link + card_content += f" [:octicons-arrow-right-24: {display_name}]({example_name.replace('_', '-')}-with-pytidb.md)\n\n" + + gallery_content += card_content + + # Close the grid + gallery_content += """
+ +## All Examples + +| Example | Description | Key Features | +|---------|-------------|--------------| +""" + + # Add table rows + for example_name in EXAMPLES_ORDER: + if example_name in example_data: + display_name = EXAMPLE_DISPLAY_NAMES.get(example_name, example_name.replace('_', ' ').title()) + description = EXAMPLE_DESCRIPTIONS.get(example_name, f"Learn about {display_name}") + features = EXAMPLE_FEATURES.get(example_name, "Various features") + + gallery_content += f"| [{display_name}]({example_name.replace('_', '-')}-with-pytidb.md) | {description} | {features} |\n" + + # Add footer + gallery_content += """ +--- + +!!! tip "Getting Started" + New to PyTiDB? Start with the [Quickstart](quickstart-with-pytidb.md) example to learn the basics, then explore other examples based on your use case. + +!!! info "Source Code" + All examples are available in the [PyTiDB repository](https://github.com/pingcap/pytidb/tree/main/examples). Each example includes complete source code and detailed instructions. +""" + + return gallery_content + def update_mkdocs_config(created_files): """Update mkdocs.yml with the new examples.""" if not MKDOCS_CONFIG_FILE.exists(): @@ -136,7 +288,7 @@ def update_mkdocs_config(created_files): content = f.read() # Create a mapping of example names to file paths - created_files_dict = {example_name: filepath for example_name, filepath in created_files} + created_files_dict = {example_name: filepath for example_name, filepath, _ in created_files} # Find the Examples section and update it in the specified order examples_nav = [] @@ -179,6 +331,7 @@ def main(): available_directories = {directory['name'] for directory in directories} created_files = [] + example_data = {} # Process examples in the specified order for example_name in EXAMPLES_ORDER: @@ -190,20 +343,37 @@ def main(): if readme_content: # Process content - processed_content = process_readme_content(readme_content, example_name) + processed_content, cover_image = process_readme_content(readme_content, example_name) if processed_content: # Create local file filepath = create_example_file(example_name, processed_content) - created_files.append((example_name, filepath)) + created_files.append((example_name, filepath, cover_image)) + + # Store example data for gallery generation + example_data[example_name] = { + 'cover_image': cover_image, + 'filepath': filepath + } else: print(f" No README.md found for {example_name}") else: print(f" Directory {example_name} not found in repository") print(f"\nSynced {len(created_files)} examples:") - for example_name, filepath in created_files: - print(f" - {example_name} -> {filepath}") + for example_name, filepath, cover_image in created_files: + cover_status = "with cover image" if cover_image else "no cover image" + print(f" - {example_name} -> {filepath} ({cover_status})") + + # Generate Example Gallery page + print("\nGenerating Example Gallery page...") + gallery_content = create_gallery_page(example_data) + gallery_filepath = LOCAL_EXAMPLES_DIR / "index.md" + + with open(gallery_filepath, 'w', encoding='utf-8') as f: + f.write(gallery_content) + + print(f"Created gallery page: {gallery_filepath}") # Update mkdocs.yml configuration update_mkdocs_config(created_files) diff --git a/src/ai/examples/index.md b/src/ai/examples/index.md new file mode 100644 index 00000000..81295433 --- /dev/null +++ b/src/ai/examples/index.md @@ -0,0 +1,500 @@ +--- +title: Demo Gallery +description: Explore hands-on demos showing how TiDB powers AI applications. +hide: + - navigation + - toc + - pageTitle + - editButton +--- + + + + + +
+
+

Demo Gallery

+

+ Explore hands-on demos showcasing how TiDB empowers AI applications. Get started quickly with TiDB Cloud Serverless to build your own AI-powered solutions. +

+
+ +
+
+
+
Categories
+ +
+
+
+ + + + +
+

Ready to build your AI application?

+

+ Start your AI journey with TiDB Cloud Serverless. Follow our quickstart guide to build your first AI-powered application in minutes, or explore specific examples for your use case. +

+
+ Try TiDB Cloud Serverless + View Quickstart Guide +
+
+
+
+
diff --git a/src/styles/extra.css b/src/styles/extra.css index 6578b58e..2c9a270b 100644 --- a/src/styles/extra.css +++ b/src/styles/extra.css @@ -179,4 +179,5 @@ .md-header__title { margin-left: 0 !important; -} \ No newline at end of file +} + From c4f7ae4c5f9a6819654f0190ad9a97b573d71115 Mon Sep 17 00:00:00 2001 From: Mini256 Date: Mon, 14 Jul 2025 19:14:37 +0800 Subject: [PATCH 6/8] refine generate-demos script --- DEVELOPMENT.md | 115 ++--- Makefile | 21 +- demos.yml | 126 ++++++ mkdocs.yml | 6 +- pyproject.toml | 2 + scripts/generate_demos.py | 279 ++++++++++++ scripts/sync_examples_from_pytidb.py | 385 ----------------- src/ai/examples/auto-embedding-with-pytidb.md | 21 +- src/ai/examples/basic-with-pytidb.md | 22 +- .../examples/fulltext-search-with-pytidb.md | 21 +- src/ai/examples/hybrid-search-with-pytidb.md | 21 +- src/ai/examples/image-search-with-pytidb.md | 21 +- src/ai/examples/index.md | 88 ++-- src/ai/examples/memory-with-pytidb.md | 21 +- src/ai/examples/quickstart-with-pytidb.md | 107 ----- src/ai/examples/rag-with-pytidb.md | 21 +- src/ai/examples/text2sql-with-pytidb.md | 22 +- src/ai/examples/vector-search-with-pytidb.md | 21 +- src/templates/demo_gallery_template.j2 | 396 ++++++++++++++++++ src/templates/demo_page_template.j2 | 19 + 20 files changed, 1080 insertions(+), 655 deletions(-) create mode 100644 demos.yml create mode 100755 scripts/generate_demos.py delete mode 100644 scripts/sync_examples_from_pytidb.py delete mode 100644 src/ai/examples/quickstart-with-pytidb.md create mode 100644 src/templates/demo_gallery_template.j2 create mode 100644 src/templates/demo_page_template.j2 diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index ded416ed..ad884fc4 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -2,79 +2,96 @@ This guide covers the development setup and maintenance of the TiDB for AI documentation site. -## Prerequisites +## Quick Start -This project uses [UV](https://docs.astral.sh/uv/) for fast dependency management. Install UV first: +1. **Install UV** (fast dependency manager): -```bash -# Install UV -curl -LsSf https://astral.sh/uv/install.sh | sh -``` + ```bash + curl -LsSf https://astral.sh/uv/install.sh | sh + ``` -## Installation +2. **Install dependencies**: -Install the project dependencies: + ```bash + make install + ``` -```bash -# Install all dependencies -make install -``` +3. **Start development server**: -Or use UV directly: + ```bash + make serve + ``` -```bash -uv pip install -e . -``` +## Project Structure -## Available Commands +``` +ā”œā”€ā”€ demos.yml # Demo Gallery configuration +ā”œā”€ā”€ scripts/generate_demos.py # Demo Gallery generation script +ā”œā”€ā”€ src/ +ā”‚ ā”œā”€ā”€ templates/ # Jinja2 templates +ā”‚ ā””ā”€ā”€ ai/examples/ # Generated demo pages +ā””ā”€ā”€ Makefile # Build commands +``` -This project includes a Makefile with the following commands: +## Helpful Commands ```bash -# Show available commands -make help +# Dependencies +make check # Check dependencies and setup +make install # Install/update dependencies -# Install dependencies using UV -make install +# Development workflow +make serve # Start development server +make build # Build documentation site +make clean # Clean build artifacts -# Sync examples from pytidb repository -make sync-examples +# Demo management +make generate-demos # Generate demo pages and gallery from demos.yml config -# Start the development server -make serve +# Other +make help # Show all available commands +``` + +## Maintain the Demo Gallery -# Build the documentation site -make build +The Demo Gallery showcases AI demos of TiDB and is configured via [demos.yml](demos.yml). -# Clean build artifacts -make clean +To regenerate the demo gallery from configuration, run: -# Check dependencies and project setup -make check +```bash +make generate-demos ``` -## Example Gallery Maintenance +### How to add a new demo -The Example Gallery automatically syncs content from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples). +You can follow the steps below to add a new demo: -### Updating Examples +1. Add entry to `demos` array in `demos.yml` with unique `id`, title, description, and display properties -To sync the latest examples from the pytidb repository: + For example: -```bash -make sync-examples -``` + ```yaml + demos: + - id: "basic" + title: "Basic Usage" + description: "Learn fundamental PyTiDB operations" + icon: "āš™ļø" + background: "linear-gradient(135deg, #10b981, var(--brand-color))" + link: "basic-with-pytidb/" + doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/basic/README.md" + cover_image: null + ``` -Or run the script directly: +2. Add the demo `id` to appropriate category's `demos` array -```bash -python scripts/sync_examples_from_pytidb.py -``` + For example: -### How It Works + ```yaml + categories: + - id: "featured" + title: "Search" + demos: ["image-search"] + ``` -The sync script: -1. Fetches README.md files from the pytidb repository using GitHub API -2. Processes content by adding YAML frontmatter and fixing relative links -3. Creates local markdown files with naming convention: `{example_name}-with-pytidb.md` -4. Updates the navigation configuration automatically +3. Run `make generate-demos` to regenerate +4. Commit changes diff --git a/Makefile b/Makefile index 34a7763a..e4c5c0d0 100644 --- a/Makefile +++ b/Makefile @@ -1,17 +1,20 @@ # TiDB for AI Documentation Site -.PHONY: help install sync-examples serve build clean check +.PHONY: help install generate-demos serve build clean check help: @echo "TiDB for AI Documentation Site" @echo "=============================" @echo "Available commands:" - @echo " install - Install dependencies using UV" - @echo " sync-examples - Sync examples from pytidb repository" - @echo " serve - Start the development server" - @echo " build - Build the documentation site" - @echo " clean - Clean build artifacts" - @echo " check - Check dependencies and project setup" + @echo " install - Install dependencies using UV" + @echo " generate-demos - Generate demo pages and gallery from demos.yml config" + @echo " serve - Start the development server" + @echo " build - Build the documentation site" + @echo " clean - Clean build artifacts" + @echo " check - Check dependencies and project setup" + @echo "" + @echo "Demo gallery configuration:" + @echo " Edit demos.yml in the project root to manage gallery content" @echo "" @echo "Prerequisites:" @echo " UV package manager - curl -LsSf https://astral.sh/uv/install.sh | sh" @@ -19,8 +22,8 @@ help: install: uv pip install -e . -sync-examples: - python scripts/sync_examples_from_pytidb.py +generate-demos: + python scripts/generate_demos.py serve: mkdocs serve diff --git a/demos.yml b/demos.yml new file mode 100644 index 00000000..09c09961 --- /dev/null +++ b/demos.yml @@ -0,0 +1,126 @@ +# Demo Gallery Configuration +title: "Demo Gallery" +description: "Explore hands-on demos showcasing how TiDB empowers AI applications. Get started quickly with TiDB Cloud Serverless to build your own AI-powered solutions." + +categories: + - id: "featured" + title: "ā­ Featured" + demos: ["image-search", "rag", "memory"] + + - id: "getting-started" + title: "šŸš€ Getting Started" + demos: ["basic", "auto-embedding"] + + - id: "search" + title: "šŸ” Search & Retrieval" + demos: ["vector-search", "fulltext-search", "hybrid-search", "image-search"] + + - id: "ai-apps" + title: "šŸ¤– AI Applications" + demos: ["rag", "memory", "text2sql"] + +demos: + + - id: "image-search" + title: "Image Search" + description: "Build an image search application using multimodal embeddings for both text-to-image and image-to-image search." + category: "search" + icon: null + background: null + link: "image-search-with-pytidb/" + doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/image_search/README.md" + cover_image: "https://github.com/user-attachments/assets/7ba9733a-4d1f-4094-8edb-58731ebd08e9" + + - id: "rag" + title: "RAG" + description: "Build a RAG application that combines document retrieval with language generation." + category: "ai-apps" + icon: null + background: null + link: "rag-with-pytidb/" + doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/rag/README.md" + cover_image: "https://github.com/user-attachments/assets/dfd85672-65ce-4a46-8dd2-9f77d826363e" + + - id: "basic" + title: "Basic Usage" + description: "Learn fundamental PyTiDB operations including database connection, table creation, and data manipulation." + category: "getting-started" + icon: "āš™ļø" + background: "linear-gradient(135deg, #10b981, var(--brand-color))" + link: "basic-with-pytidb/" + doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/basic/README.md" + cover_image: null + + - id: "auto-embedding" + title: "Auto Embedding" + description: "Automatically generate embeddings for your text data using built-in embedding models." + category: "getting-started" + icon: "šŸ¤–" + background: "radial-gradient(circle at center, #8b5cf6 0%, var(--brand-color) 100%)" + link: "auto-embedding-with-pytidb/" + doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/auto_embedding/README.md" + cover_image: null + + - id: "vector-search" + title: "Vector Search" + description: "Implement semantic search using vector embeddings to find similar content." + category: "search" + icon: null + gradient_class: null + link: "vector-search-with-pytidb/" + doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/vector_search/README.md" + cover_image: "https://github.com/user-attachments/assets/6d7783a5-ce9c-4dcc-8b95-49d5f0ca735a" + + - id: "fulltext-search" + title: "Fulltext Search" + description: "Perform traditional text search using MySQL fulltext search capabilities." + category: "search" + icon: null + gradient_class: null + link: "fulltext-search-with-pytidb/" + doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/fulltext_search/README.md" + cover_image: "https://github.com/user-attachments/assets/c81ddad4-f996-4b1f-85c0-5cbb55bc2a3a" + + - id: "hybrid-search" + title: "Hybrid Search" + description: "Combine vector search and fulltext search for more comprehensive results." + category: "search" + icon: null + gradient_class: null + link: "hybrid-search-with-pytidb/" + doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/hybrid_search/README.md" + cover_image: "https://github.com/user-attachments/assets/6e1c639d-2160-44c8-86b4-958913b9eca5" + + - id: "memory" + title: "Memory" + description: "Implement conversation memory for chatbots and conversational AI applications." + category: "ai-apps" + icon: "šŸ’­" + background: "linear-gradient(135deg, #8b5cf6, var(--brand-color))" + link: "memory-with-pytidb/" + doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/memory/README.md" + cover_image: null + + - id: "text2sql" + title: "Text2SQL" + description: "Convert natural language queries into SQL statements using AI models." + category: "ai-apps" + icon: "šŸ’¬" + background: "linear-gradient(135deg, #06b6d4, var(--brand-color))" + link: "text2sql-with-pytidb/" + doc_link: "https://github.com/pingcap/pytidb/tree/main/examples/text2sql/README.md" + cover_image: null + +# CTA section configuration +cta: + title: "Ready to build your AI application?" + description: "Start your AI journey with TiDB Cloud Serverless. Follow our quickstart guide to build your first AI-powered application in minutes, or explore specific examples for your use case." + buttons: + - text: "Try TiDB Cloud Serverless" + url: "https://tidbcloud.com/serverless" + type: "primary" + external: true + - text: "View Quickstart Guide" + url: "quickstart-with-pytidb/" + type: "secondary" + external: false \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 8d41c164..991179d9 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -106,9 +106,8 @@ nav: - Auto Embedding: ai/guides/auto-embedding.md - Reranking: ai/guides/reranking.md - Filtering: ai/guides/filtering.md - - šŸ’” Examples: + - šŸ’” Demos: - Gallery: ai/examples/index.md - - Quickstart: ai/examples/quickstart-with-pytidb.md - Basic Usage: ai/examples/basic-with-pytidb.md - Auto Embedding: ai/examples/auto-embedding-with-pytidb.md - Vector Search: ai/examples/vector-search-with-pytidb.md @@ -140,9 +139,8 @@ nav: - Transaction: ai/guides/transaction.md - Raw Queries: ai/guides/raw-queries.md - Multiple Table Joins: ai/guides/joins.md - - Examples: + - Demos: - Gallery: ai/examples/index.md - - Quickstart: ai/examples/quickstart-with-pytidb.md - Basic Usage: ai/examples/basic-with-pytidb.md - Auto Embedding: ai/examples/auto-embedding-with-pytidb.md - Vector Search: ai/examples/vector-search-with-pytidb.md diff --git a/pyproject.toml b/pyproject.toml index f3a1ffe3..4e99023c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -17,4 +17,6 @@ dependencies = [ "mkdocs-redirects>=1.2.2", "requests>=2.31.0", "PyYAML>=6.0", + "Jinja2>=3.1.0", + "click>=8.0.0", ] diff --git a/scripts/generate_demos.py b/scripts/generate_demos.py new file mode 100755 index 00000000..441be5b4 --- /dev/null +++ b/scripts/generate_demos.py @@ -0,0 +1,279 @@ +#!/usr/bin/env python3 +""" +Generate demo gallery and individual demo documentation pages. +This script reads configuration from demos.yml and generates: +1. Individual demo pages using demo_template.j2 +2. Gallery index page using gallery_template.j2 +""" + +import sys +import requests +import re +import yaml +from pathlib import Path +from jinja2 import Environment, FileSystemLoader +import click + +# Configuration +LOCAL_EXAMPLES_DIR = Path("src/ai/examples") +CONFIG_FILE = Path("demos.yml") +DEMO_TEMPLATE_FILE = Path("src/templates/demo_page_template.j2") +GALLERY_TEMPLATE_FILE = Path("src/templates/demo_gallery_template.j2") +OUTPUT_FILE = LOCAL_EXAMPLES_DIR / "index.md" + + +def load_config(): + """Load configuration from YAML file.""" + if not CONFIG_FILE.exists(): + raise click.ClickException(f"Configuration file {CONFIG_FILE} not found.") + + try: + with open(CONFIG_FILE, 'r', encoding='utf-8') as f: + config = yaml.safe_load(f) + return config + except yaml.YAMLError as e: + raise click.ClickException(f"Error parsing {CONFIG_FILE}: {e}") + + +def fetch_readme_from_doc_link(doc_link): + """Fetch README.md content from the provided doc_link.""" + # Convert GitHub tree URL to raw content URL + if "github.com" in doc_link and "/tree/" in doc_link: + raw_url = doc_link.replace("github.com", "raw.githubusercontent.com").replace("/tree/", "/") + elif doc_link.endswith("/README.md"): + raw_url = doc_link + else: + # Assume it's a directory link, append README.md + raw_url = doc_link.rstrip('/') + '/README.md' + if "github.com" in raw_url and "/tree/" in raw_url: + raw_url = raw_url.replace("github.com", "raw.githubusercontent.com").replace("/tree/", "/") + + try: + response = requests.get(raw_url, timeout=30) + response.raise_for_status() + return response.text + except requests.RequestException as e: + click.echo(f"Error fetching README from {raw_url}: {e}", err=True) + return None + + +def extract_repo_info_from_doc_link(doc_link): + """Extract repository and path information from doc_link.""" + if "github.com" not in doc_link: + return None, None, None, None + + # Parse URL to extract owner, repo, and path + parts = doc_link.replace("https://github.com/", "").split("/") + if len(parts) < 2: + return None, None, None, None + + owner, repo = parts[0], parts[1] + + # Extract path after /tree/branch/ + if "/tree/" in doc_link: + try: + tree_index = parts.index("tree") + if len(parts) > tree_index + 2: # owner/repo/tree/branch/path... + branch = parts[tree_index + 1] + path_parts = parts[tree_index + 2:] + # Remove README.md if present + if path_parts and path_parts[-1] == "README.md": + path_parts = path_parts[:-1] + path = "/".join(path_parts) + return owner, repo, branch, path + except ValueError: + pass + + return None, None, None, None + + +def process_readme_content(content, demo_config): + """Process README content to adapt it for the documentation site.""" + if not content: + return None + + # Extract repository info + owner, repo, branch, example_path = extract_repo_info_from_doc_link(demo_config['doc_link']) + + if not all([owner, repo, branch, example_path]): + click.echo(f"Warning: Could not extract repo info from {demo_config['doc_link']}", err=True) + return content + + base_repo_url = f"https://github.com/{owner}/{repo}" + base_raw_url = f"https://raw.githubusercontent.com/{owner}/{repo}/{branch}" + + # Fix relative links to point to the original repository + processed_content = re.sub( + r'\]\((?!https?://)(.*?)\)', + f']({base_repo_url}/tree/{branch}/{example_path}/\\1)', + content + ) + + # Fix relative image links + processed_content = re.sub( + r'!\[([^\]]*)\]\((?!https?://)(.*?)\)', + f'![\\1]({base_raw_url}/{example_path}/\\2)', + processed_content + ) + + return processed_content + + +def create_demo_page(demo_config, content): + """Create a markdown file for a demo using the demo template.""" + if not DEMO_TEMPLATE_FILE.exists(): + raise click.ClickException(f"Demo template file {DEMO_TEMPLATE_FILE} not found.") + + # Create the local examples directory if it doesn't exist + LOCAL_EXAMPLES_DIR.mkdir(parents=True, exist_ok=True) + + try: + # Set up Jinja2 environment + env = Environment(loader=FileSystemLoader("src/templates")) + template = env.get_template('demo_page_template.j2') + + # Render the template + rendered_content = template.render( + demo=demo_config, + content=content + ) + + # Create the markdown file + filename = f"{demo_config['id'].replace('_', '-')}-with-pytidb.md" + filepath = LOCAL_EXAMPLES_DIR / filename + + with open(filepath, 'w', encoding='utf-8') as f: + f.write(rendered_content) + + return filepath + except Exception as e: + raise click.ClickException(f"Error creating demo page for {demo_config['id']}: {e}") + + +def generate_gallery_page(config): + """Generate the gallery page using Jinja2 template.""" + if not GALLERY_TEMPLATE_FILE.exists(): + raise click.ClickException(f"Gallery template file {GALLERY_TEMPLATE_FILE} not found.") + + try: + # Set up Jinja2 environment + env = Environment(loader=FileSystemLoader("src/templates")) + template = env.get_template('demo_gallery_template.j2') + + # Render the template + rendered_content = template.render( + config=config, + categories=config.get('categories', {}), + demos=config.get('demos', {}) + ) + + # Write the rendered content to the output file + with open(OUTPUT_FILE, 'w', encoding='utf-8') as f: + f.write(rendered_content) + + return True + except Exception as e: + raise click.ClickException(f"Error generating gallery page: {e}") + + +def sync_demo_docs(config, fetch_from_remote=True): + """Sync demo documentation files based on demos.yml configuration.""" + if not fetch_from_remote: + return [] + + created_files = [] + demos_config = config.get('demos', []) + + with click.progressbar(demos_config, label='Processing demos') as demos: + for demo_config in demos: + demo_id = demo_config['id'] + doc_link = demo_config.get('doc_link') + + if not doc_link: + click.echo(f"Warning: No doc_link found for demo '{demo_id}', skipping...", err=True) + continue + + # Fetch README content + readme_content = fetch_readme_from_doc_link(doc_link) + + if readme_content: + # Process content + processed_content = process_readme_content(readme_content, demo_config) + + if processed_content: + # Create demo page + filepath = create_demo_page(demo_config, processed_content) + if filepath: + created_files.append((demo_id, filepath)) + else: + click.echo(f"Failed to process content for {demo_id}", err=True) + else: + click.echo(f"Failed to fetch README for {demo_id}", err=True) + + return created_files + + +@click.command() +@click.option('--skip-demos', is_flag=True, + help='Skip generating individual demo pages from remote repositories') +@click.option('--skip-gallery', is_flag=True, + help='Skip generating the demo gallery index page') +@click.option('--verbose', '-v', is_flag=True, help='Enable verbose output') +def main(skip_demos, skip_gallery, verbose): + """Generate demo gallery and individual demo documentation pages. + + This script reads configuration from demos.yml and generates: + 1. Individual demo pages using demo_template.j2 + 2. Gallery index page using gallery_template.j2 + + By default, both demo pages and gallery are generated. + """ + if verbose: + click.echo("Running in verbose mode...") + + # Load configuration + try: + config = load_config() + except click.ClickException: + raise + + created_files = [] + + # Generate demo pages (unless skipped) + if not skip_demos: + if verbose: + click.echo("Generating demo pages from remote repositories...") + created_files = sync_demo_docs(config, fetch_from_remote=True) + + if created_files: + click.echo(f"\nāœ… Generated {len(created_files)} demo pages:") + for demo_id, filepath in created_files: + click.echo(f" ā€¢ {demo_id} ā†’ {filepath}") + elif verbose: + click.echo("No demo pages were generated.") + else: + if verbose: + click.echo("Skipping demo page generation...") + + # Generate gallery page (unless skipped) + if not skip_gallery: + if verbose: + click.echo("Generating gallery page from template...") + + if generate_gallery_page(config): + click.echo(f"āœ… Gallery page generated: {OUTPUT_FILE}") + else: + raise click.ClickException("Failed to generate gallery page.") + else: + if verbose: + click.echo("Skipping gallery page generation...") + + # Check if nothing was generated + if skip_demos and skip_gallery: + click.echo("āš ļø Both demos and gallery generation were skipped. Nothing to do.") + else: + click.echo("\nšŸŽ‰ Done! You can now run 'mkdocs serve' to view the documentation site.") + + +if __name__ == "__main__": + main() \ No newline at end of file diff --git a/scripts/sync_examples_from_pytidb.py b/scripts/sync_examples_from_pytidb.py deleted file mode 100644 index 21ca2ad0..00000000 --- a/scripts/sync_examples_from_pytidb.py +++ /dev/null @@ -1,385 +0,0 @@ -#!/usr/bin/env python3 -""" -Sync examples from pytidb repository to this documentation site. -This script fetches README.md files from pytidb/examples and converts them -to be used in this documentation site. -""" - -import os -import sys -import requests -import re -import yaml -from pathlib import Path -from urllib.parse import urlparse - -# Configuration -PYTIDB_REPO = "pingcap/pytidb" -EXAMPLES_BASE_URL = f"https://raw.githubusercontent.com/{PYTIDB_REPO}/main/examples" -EXAMPLES_API_URL = f"https://api.github.com/repos/{PYTIDB_REPO}/contents/examples" -LOCAL_EXAMPLES_DIR = Path("src/ai/examples") -MKDOCS_CONFIG_FILE = Path("mkdocs.yml") - -# Example display names mapping -EXAMPLE_DISPLAY_NAMES = { - "auto_embedding": "Auto Embedding", - "basic": "Basic Usage", - "fulltext_search": "Fulltext Search", - "hybrid_search": "Hybrid Search", - "image_search": "Image Search", - "memory": "Memory", - "quickstart": "Quickstart", - "rag": "RAG", - "text2sql": "Text2SQL", - "vector_search": "Vector Search", -} - -# Example descriptions for gallery -EXAMPLE_DESCRIPTIONS = { - "quickstart": "Get started with PyTiDB in minutes. Learn the basics of connecting to TiDB, creating tables, and performing vector search.", - "basic": "Learn fundamental PyTiDB operations including database connection, table creation, and data manipulation.", - "auto_embedding": "Automatically generate embeddings for your text data using built-in embedding models.", - "vector_search": "Implement semantic search using vector embeddings to find similar content.", - "fulltext_search": "Perform traditional text search using MySQL fulltext search capabilities.", - "hybrid_search": "Combine vector search and fulltext search for more comprehensive results.", - "image_search": "Build an image search application using multimodal embeddings for both text-to-image and image-to-image search.", - "rag": "Build a RAG application that combines document retrieval with language generation.", - "memory": "Implement conversation memory for chatbots and conversational AI applications.", - "text2sql": "Convert natural language queries into SQL statements using AI models.", -} - -# Example icons for gallery -EXAMPLE_ICONS = { - "quickstart": ":rocket:{ .lg .middle }", - "basic": ":material-cog:{ .lg .middle }", - "auto_embedding": ":material-auto-fix:{ .lg .middle }", - "vector_search": ":material-vector-triangle:{ .lg .middle }", - "fulltext_search": ":material-file-search:{ .lg .middle }", - "hybrid_search": ":material-merge:{ .lg .middle }", - "image_search": ":material-image-search:{ .lg .middle }", - "rag": ":material-message-question:{ .lg .middle }", - "memory": ":material-memory:{ .lg .middle }", - "text2sql": ":material-code-braces:{ .lg .middle }", -} - -# Example key features for table -EXAMPLE_FEATURES = { - "quickstart": "Database connection, table creation, vector search", - "basic": "CRUD operations, data manipulation", - "auto_embedding": "Built-in embedding models, text processing", - "vector_search": "Similarity search, embedding-based retrieval", - "fulltext_search": "MySQL fulltext search, keyword matching", - "hybrid_search": "Vector + fulltext search, result ranking", - "image_search": "Text-to-image, image-to-image search", - "rag": "Document retrieval, AI generation", - "memory": "Chatbot memory, context preservation", - "text2sql": "NL query conversion, SQL generation", -} - -# Fixed order for examples (as they should appear in navigation) -EXAMPLES_ORDER = [ - "quickstart", - "basic", - "auto_embedding", - "vector_search", - "fulltext_search", - "hybrid_search", - "image_search", - "rag", - "memory", - "text2sql" -] - -def fetch_github_directory_contents(url): - """Fetch directory contents from GitHub API.""" - try: - response = requests.get(url) - response.raise_for_status() - return response.json() - except requests.RequestException as e: - print(f"Error fetching directory contents: {e}") - return [] - -def fetch_readme_content(example_name): - """Fetch README.md content from a specific example directory.""" - readme_url = f"{EXAMPLES_BASE_URL}/{example_name}/README.md" - try: - response = requests.get(readme_url) - response.raise_for_status() - return response.text - except requests.RequestException as e: - print(f"Error fetching README for {example_name}: {e}") - return None - -def extract_cover_image(content): - """Extract the first image from content to use as cover image.""" - if not content: - return None - - # Look for image patterns - image_patterns = [ - r'!\[([^\]]*)\]\((https://[^\)]+)\)', # Already absolute URLs - r'!\[([^\]]*)\]\(([^\)]+)\)', # Relative URLs - r']+src="([^"]+)"[^>]*>', # HTML img tags - ] - - for pattern in image_patterns: - matches = re.findall(pattern, content) - if matches: - if len(matches[0]) == 2: # Markdown format - return matches[0][1] - else: # HTML format - return matches[0] - - return None - -def process_readme_content(content, example_name): - """Process README content to adapt it for the documentation site.""" - if not content: - return None, None - - display_name = EXAMPLE_DISPLAY_NAMES.get(example_name, example_name.replace('_', ' ').title()) - - # Extract cover image before processing - cover_image = extract_cover_image(content) - - # Add front matter with metadata - processed_content = f"""--- -title: {display_name} -description: "PyTiDB example: {example_name}" -source_repo: "https://github.com/pingcap/pytidb/tree/main/examples/{example_name}" ---- - -# {display_name} - -!!! info "Source Repository" - This example is from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples/{example_name}). - For the complete source code and latest updates, please visit the original repository. - -""" - - # Process the original content - processed_content += content - - # Fix relative links to point to the original repository - processed_content = re.sub( - r'\]\((?!https?://)(.*?)\)', - f'](https://github.com/pingcap/pytidb/tree/main/examples/{example_name}/\\1)', - processed_content - ) - - # Fix relative image links - processed_content = re.sub( - r'!\[([^\]]*)\]\((?!https?://)(.*?)\)', - f'![\\1](https://raw.githubusercontent.com/pingcap/pytidb/main/examples/{example_name}/\\2)', - processed_content - ) - - # If we have a relative cover image, make it absolute - if cover_image and not cover_image.startswith('http'): - cover_image = f"https://raw.githubusercontent.com/pingcap/pytidb/main/examples/{example_name}/{cover_image}" - - return processed_content, cover_image - -def create_example_file(example_name, content): - """Create a markdown file for an example.""" - # Create the local examples directory if it doesn't exist - LOCAL_EXAMPLES_DIR.mkdir(parents=True, exist_ok=True) - - # Create the markdown file - filename = f"{example_name.replace('_', '-')}-with-pytidb.md" - filepath = LOCAL_EXAMPLES_DIR / filename - - with open(filepath, 'w', encoding='utf-8') as f: - f.write(content) - - print(f"Created: {filepath}") - return filepath - -def create_gallery_page(example_data): - """Create the Example Gallery page with grids layout.""" - - # Start with the frontmatter and header - gallery_content = """--- -title: Example Gallery -description: "Explore all PyTiDB examples with cover images and descriptions" -hide: - - navigation - - toc ---- - -# Example Gallery - -Explore these hands-on examples to learn how to build AI applications with TiDB and PyTiDB. - -
- -""" - - # Add grid cards for each example - for example_name in EXAMPLES_ORDER: - if example_name in example_data: - display_name = EXAMPLE_DISPLAY_NAMES.get(example_name, example_name.replace('_', ' ').title()) - description = EXAMPLE_DESCRIPTIONS.get(example_name, f"Learn about {display_name}") - icon = EXAMPLE_ICONS.get(example_name, ":material-cog:{ .lg .middle }") - cover_image = example_data[example_name].get('cover_image') - - # Create the card - card_content = f"""- {icon} **{display_name}** - - --- - - {description} - -""" - - # Add cover image if available - if cover_image: - if example_name == "image_search": - # Special handling for image search with specific styling - card_content += f' ![{display_name} Demo]({cover_image}){{ width="300" }}\n\n' - else: - card_content += f' ![{display_name} Demo]({cover_image}){{ width="250" }}\n\n' - - # Add link - card_content += f" [:octicons-arrow-right-24: {display_name}]({example_name.replace('_', '-')}-with-pytidb.md)\n\n" - - gallery_content += card_content - - # Close the grid - gallery_content += """
- -## All Examples - -| Example | Description | Key Features | -|---------|-------------|--------------| -""" - - # Add table rows - for example_name in EXAMPLES_ORDER: - if example_name in example_data: - display_name = EXAMPLE_DISPLAY_NAMES.get(example_name, example_name.replace('_', ' ').title()) - description = EXAMPLE_DESCRIPTIONS.get(example_name, f"Learn about {display_name}") - features = EXAMPLE_FEATURES.get(example_name, "Various features") - - gallery_content += f"| [{display_name}]({example_name.replace('_', '-')}-with-pytidb.md) | {description} | {features} |\n" - - # Add footer - gallery_content += """ ---- - -!!! tip "Getting Started" - New to PyTiDB? Start with the [Quickstart](quickstart-with-pytidb.md) example to learn the basics, then explore other examples based on your use case. - -!!! info "Source Code" - All examples are available in the [PyTiDB repository](https://github.com/pingcap/pytidb/tree/main/examples). Each example includes complete source code and detailed instructions. -""" - - return gallery_content - -def update_mkdocs_config(created_files): - """Update mkdocs.yml with the new examples.""" - if not MKDOCS_CONFIG_FILE.exists(): - print(f"Warning: {MKDOCS_CONFIG_FILE} not found. Skipping mkdocs.yml update.") - return - - try: - with open(MKDOCS_CONFIG_FILE, 'r', encoding='utf-8') as f: - content = f.read() - - # Create a mapping of example names to file paths - created_files_dict = {example_name: filepath for example_name, filepath, _ in created_files} - - # Find the Examples section and update it in the specified order - examples_nav = [] - for example_name in EXAMPLES_ORDER: - if example_name in created_files_dict: - display_name = EXAMPLE_DISPLAY_NAMES.get(example_name, example_name.replace('_', ' ').title()) - relative_path = f"ai/examples/{example_name.replace('_', '-')}-with-pytidb.md" - examples_nav.append(f" - {display_name}: {relative_path}") - - examples_section = " - Examples:\n" + "\n".join(examples_nav) - - # Print the examples section for manual addition if needed - print(f"\nExamples section for mkdocs.yml:") - print(examples_section) - - print(f"\nNote: You may need to manually add or update the Examples section in mkdocs.yml") - - except Exception as e: - print(f"Error updating mkdocs.yml: {e}") - -def main(): - """Main function to sync examples from pytidb repository.""" - print("Syncing examples from pytidb repository...") - - # Fetch directory contents - contents = fetch_github_directory_contents(EXAMPLES_API_URL) - - if not contents: - print("No contents found or error occurred.") - return - - # Filter directories only and exclude assets - directories = [item for item in contents if item['type'] == 'dir' and item['name'] != 'assets'] - - if not directories: - print("No example directories found.") - return - - # Create a set of available directory names for quick lookup - available_directories = {directory['name'] for directory in directories} - - created_files = [] - example_data = {} - - # Process examples in the specified order - for example_name in EXAMPLES_ORDER: - if example_name in available_directories: - print(f"Processing example: {example_name}") - - # Fetch README content - readme_content = fetch_readme_content(example_name) - - if readme_content: - # Process content - processed_content, cover_image = process_readme_content(readme_content, example_name) - - if processed_content: - # Create local file - filepath = create_example_file(example_name, processed_content) - created_files.append((example_name, filepath, cover_image)) - - # Store example data for gallery generation - example_data[example_name] = { - 'cover_image': cover_image, - 'filepath': filepath - } - else: - print(f" No README.md found for {example_name}") - else: - print(f" Directory {example_name} not found in repository") - - print(f"\nSynced {len(created_files)} examples:") - for example_name, filepath, cover_image in created_files: - cover_status = "with cover image" if cover_image else "no cover image" - print(f" - {example_name} -> {filepath} ({cover_status})") - - # Generate Example Gallery page - print("\nGenerating Example Gallery page...") - gallery_content = create_gallery_page(example_data) - gallery_filepath = LOCAL_EXAMPLES_DIR / "index.md" - - with open(gallery_filepath, 'w', encoding='utf-8') as f: - f.write(gallery_content) - - print(f"Created gallery page: {gallery_filepath}") - - # Update mkdocs.yml configuration - update_mkdocs_config(created_files) - - print("\nDone! The Example Gallery has been created.") - print("You can now run 'mkdocs serve' to view the documentation site.") - -if __name__ == "__main__": - main() \ No newline at end of file diff --git a/src/ai/examples/auto-embedding-with-pytidb.md b/src/ai/examples/auto-embedding-with-pytidb.md index 47912405..20759bf1 100644 --- a/src/ai/examples/auto-embedding-with-pytidb.md +++ b/src/ai/examples/auto-embedding-with-pytidb.md @@ -1,15 +1,9 @@ --- title: Auto Embedding -description: "PyTiDB example: auto_embedding" +description: "Automatically generate embeddings for your text data using built-in embedding models." source_repo: "https://github.com/pingcap/pytidb/tree/main/examples/auto_embedding" --- -# Auto Embedding - -!!! info "Source Repository" - This example is from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples/auto_embedding). - For the complete source code and latest updates, please visit the original repository. - # Auto Embedding Demo This example showcases how to use the auto embedding feature with PyTiDB Client. @@ -83,3 +77,16 @@ id: 1, text: TiDB is a distributed database that supports OLTP, OLAP, HTAP and A id: 2, text: PyTiDB is a Python library for developers to connect to TiDB., distance: 0.422506501973434 id: 3, text: LlamaIndex is a Python library for building AI-powered applications., distance: 0.5267239638442787 ``` + + +--- + +## Related Resources + +- **Source Code**: [View on GitHub](https://github.com/pingcap/pytidb/tree/main/examples/auto_embedding) +- **Category**: Getting-Started + +- **Description**: Automatically generate embeddings for your text data using built-in embedding models. + + +[šŸ  Back to Demo Gallery](../index.md){ .md-button .md-button--primary } \ No newline at end of file diff --git a/src/ai/examples/basic-with-pytidb.md b/src/ai/examples/basic-with-pytidb.md index be328884..66b8c050 100644 --- a/src/ai/examples/basic-with-pytidb.md +++ b/src/ai/examples/basic-with-pytidb.md @@ -1,15 +1,9 @@ --- title: Basic Usage -description: "PyTiDB example: basic" +description: "Learn fundamental PyTiDB operations including database connection, table creation, and data manipulation." source_repo: "https://github.com/pingcap/pytidb/tree/main/examples/basic" --- -# Basic Usage - -!!! info "Source Repository" - This example is from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples/basic). - For the complete source code and latest updates, please visit the original repository. - # Basic CRUD Demo This example demonstrates basic CRUD (Create, Read, Update, Delete) operations with PyTiDB. @@ -95,4 +89,16 @@ Number of rows: 2 Table dropped Basic CRUD operations completed! -``` \ No newline at end of file +``` + +--- + +## Related Resources + +- **Source Code**: [View on GitHub](https://github.com/pingcap/pytidb/tree/main/examples/basic) +- **Category**: Getting-Started + +- **Description**: Learn fundamental PyTiDB operations including database connection, table creation, and data manipulation. + + +[šŸ  Back to Demo Gallery](../index.md){ .md-button .md-button--primary } \ No newline at end of file diff --git a/src/ai/examples/fulltext-search-with-pytidb.md b/src/ai/examples/fulltext-search-with-pytidb.md index 41352936..84ec35c3 100644 --- a/src/ai/examples/fulltext-search-with-pytidb.md +++ b/src/ai/examples/fulltext-search-with-pytidb.md @@ -1,15 +1,9 @@ --- title: Fulltext Search -description: "PyTiDB example: fulltext_search" +description: "Perform traditional text search using MySQL fulltext search capabilities." source_repo: "https://github.com/pingcap/pytidb/tree/main/examples/fulltext_search" --- -# Fulltext Search - -!!! info "Source Repository" - This example is from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples/fulltext_search). - For the complete source code and latest updates, please visit the original repository. - # Fulltext Search Example This example demonstrates how to build a E-commerce product search application using TiDB's full-text search feature with multilingual support. Users can search for products by keywords in their preferred language. @@ -62,3 +56,16 @@ streamlit run app.py ``` **Step 5**: open the browser and visit `http://localhost:8501` + + +--- + +## Related Resources + +- **Source Code**: [View on GitHub](https://github.com/pingcap/pytidb/tree/main/examples/fulltext_search) +- **Category**: Search + +- **Description**: Perform traditional text search using MySQL fulltext search capabilities. + + +[šŸ  Back to Demo Gallery](../index.md){ .md-button .md-button--primary } \ No newline at end of file diff --git a/src/ai/examples/hybrid-search-with-pytidb.md b/src/ai/examples/hybrid-search-with-pytidb.md index 6ef594bb..9969175d 100644 --- a/src/ai/examples/hybrid-search-with-pytidb.md +++ b/src/ai/examples/hybrid-search-with-pytidb.md @@ -1,15 +1,9 @@ --- title: Hybrid Search -description: "PyTiDB example: hybrid_search" +description: "Combine vector search and fulltext search for more comprehensive results." source_repo: "https://github.com/pingcap/pytidb/tree/main/examples/hybrid_search" --- -# Hybrid Search - -!!! info "Source Repository" - This example is from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples/hybrid_search). - For the complete source code and latest updates, please visit the original repository. - # Hybrid Search Demo In this demo, we will show you how to use hybrid search to combine vector search and full-text search on a set of documents. @@ -122,3 +116,16 @@ Search results: ] ``` + + +--- + +## Related Resources + +- **Source Code**: [View on GitHub](https://github.com/pingcap/pytidb/tree/main/examples/hybrid_search) +- **Category**: Search + +- **Description**: Combine vector search and fulltext search for more comprehensive results. + + +[šŸ  Back to Demo Gallery](../index.md){ .md-button .md-button--primary } \ No newline at end of file diff --git a/src/ai/examples/image-search-with-pytidb.md b/src/ai/examples/image-search-with-pytidb.md index 01ca0036..75fd7983 100644 --- a/src/ai/examples/image-search-with-pytidb.md +++ b/src/ai/examples/image-search-with-pytidb.md @@ -1,15 +1,9 @@ --- title: Image Search -description: "PyTiDB example: image_search" +description: "Build an image search application using multimodal embeddings for both text-to-image and image-to-image search." source_repo: "https://github.com/pingcap/pytidb/tree/main/examples/image_search" --- -# Image Search - -!!! info "Source Repository" - This example is from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples/image_search). - For the complete source code and latest updates, please visit the original repository. - # Pet Image Search Demo This example showcases how to build a powerful image search application by combining TiDB's vector search capabilities with multimodal embedding models. @@ -98,3 +92,16 @@ Or if you want to load all the data in the Oxford Pets dataset, click the **Load 1. Select the **Search type** in the sidebar 2. Input a text description of the pet you're looking for, or upload a photo of a dog or cat 3. Click the **Search** button + + +--- + +## Related Resources + +- **Source Code**: [View on GitHub](https://github.com/pingcap/pytidb/tree/main/examples/image_search) +- **Category**: Search + +- **Description**: Build an image search application using multimodal embeddings for both text-to-image and image-to-image search. + + +[šŸ  Back to Demo Gallery](../index.md){ .md-button .md-button--primary } \ No newline at end of file diff --git a/src/ai/examples/index.md b/src/ai/examples/index.md index 81295433..e9a545ff 100644 --- a/src/ai/examples/index.md +++ b/src/ai/examples/index.md @@ -1,6 +1,6 @@ --- title: Demo Gallery -description: Explore hands-on demos showing how TiDB powers AI applications. +description: Explore hands-on demos showcasing how TiDB empowers AI applications. Get started quickly with TiDB Cloud Serverless to build your own AI-powered solutions. hide: - navigation - toc @@ -188,12 +188,6 @@ html { font-size: 2.5rem; } -.card-gradient.blue { background: linear-gradient(135deg, var(--brand-color), #6366f1); } -.card-gradient.green { background: linear-gradient(135deg, #10b981, var(--brand-color)); } -.card-gradient.purple { background: linear-gradient(135deg, #8b5cf6, var(--brand-color)); } -.card-gradient.indigo { background: linear-gradient(135deg, #6366f1, var(--brand-color)); } -.card-gradient.cyan { background: linear-gradient(135deg, #06b6d4, var(--brand-color)); } - .card-badge { position: absolute; top: 0.5rem; @@ -338,7 +332,7 @@ html {

Demo Gallery

- Explore hands-on demos showcasing how TiDB empowers AI applications. Get started quickly with TiDB Cloud Serverless to build your own AI-powered solutions. + Explore hands-on demos showcasing how TiDB empowers AI applications. Get started quickly with TiDB Cloud Serverless to build your own AI-powered solutions.

@@ -346,29 +340,18 @@ html {
Categories
-
-
- -
\ No newline at end of file diff --git a/src/ai/examples/memory-with-pytidb.md b/src/ai/examples/memory-with-pytidb.md index 13dd9c33..15a7a4db 100644 --- a/src/ai/examples/memory-with-pytidb.md +++ b/src/ai/examples/memory-with-pytidb.md @@ -1,15 +1,9 @@ --- title: Memory -description: "PyTiDB example: memory" +description: "Implement conversation memory for chatbots and conversational AI applications." source_repo: "https://github.com/pingcap/pytidb/tree/main/examples/memory" --- -# Memory - -!!! info "Source Repository" - This example is from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples/memory). - For the complete source code and latest updates, please visit the original repository. - # Agent Memory Examples * Use `pytidb` to connect to TiDB @@ -91,3 +85,16 @@ Goodbye! ``` As you can see, the AI assistant remembers you! + + +--- + +## Related Resources + +- **Source Code**: [View on GitHub](https://github.com/pingcap/pytidb/tree/main/examples/memory) +- **Category**: Ai-Apps + +- **Description**: Implement conversation memory for chatbots and conversational AI applications. + + +[šŸ  Back to Demo Gallery](../index.md){ .md-button .md-button--primary } \ No newline at end of file diff --git a/src/ai/examples/quickstart-with-pytidb.md b/src/ai/examples/quickstart-with-pytidb.md deleted file mode 100644 index 39e23456..00000000 --- a/src/ai/examples/quickstart-with-pytidb.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -title: Quickstart -description: "PyTiDB example: quickstart" -source_repo: "https://github.com/pingcap/pytidb/tree/main/examples/quickstart" ---- - -# Quickstart - -!!! info "Source Repository" - This example is from the [pytidb repository](https://github.com/pingcap/pytidb/tree/main/examples/quickstart). - For the complete source code and latest updates, please visit the original repository. - -# Quickstart Demo - -This example demonstrates how to use the PyTiDB Python SDK to connect to a TiDB database, create a table, insert data, perform semantic search, and clean up resources. - -## Prerequisites - -* Python 3.8+ -* TiDB server connection string (local or TiDB Cloud) -* OpenAI API key (for embedding) - -## How to run - -**Step1**: Clone the repository - -```bash -git clone https://github.com/pingcap/pytidb.git -cd pytidb/examples/quickstart/ -``` - -**Step2**: Install the required packages - -```bash -python -m venv .venv -source .venv/bin/activate -pip install -r reqs.txt -``` - -**Step3**: Set up environment to connect to storage - -If you are using TiDB Cloud, you can find the connection parameters in the [TiDB Cloud console](https://tidbcloud.com/) and set up the environment variable like this: - -```bash -cat > .env < .env < + +/* CSS Variables */ +:root { + --brand-color: #de243d; + --brand-hover: #b71e34; + --border-radius-sm: 0.5rem; + --border-radius-md: 0.75rem; + --border-radius-lg: 1rem; + --spacing-sm: 1rem; + --spacing-md: 2rem; + --spacing-lg: 3rem; + --transition-fast: 0.2s; + --transition-normal: 0.3s; + --dark-overlay: rgba(255, 255, 255, 0.08); + --dark-border: rgba(255, 255, 255, 0.1); + --dark-bg-subtle: rgba(255, 255, 255, 0.05); +} + +/* Smooth scrolling for the entire page */ +html { + scroll-behavior: smooth; +} + +/* Gallery Container */ +.gallery-container { + max-width: 1280px; + margin: 0 auto; + padding: var(--spacing-md) var(--spacing-sm); +} + +/* Header */ +.gallery-header { + text-align: center; + margin-bottom: var(--spacing-lg); +} + +.gallery-title { + font-size: 72px !important; + font-weight: 800 !important; + margin-bottom: 8px !important; + line-height: 1 !important; + color: var(--md-default-fg-color) !important; +} + +.gallery-description { + font-size: 22px !important; + color: var(--md-default-fg-color--light) !important; + padding: 0 120px; + margin-bottom: 5rem !important; +} + +/* Layout */ +.gallery-layout { + display: flex; + gap: var(--spacing-sm) !important; +} + +/* Sidebar */ +.gallery-sidebar { + width: 8rem; + flex-shrink: 0; +} + +.sidebar-nav { + position: sticky; + top: 140px; +} + +.sidebar-title { + font-size: 14px !important; + font-weight: 400 !important; + color: var(--md-default-fg-color--light) !important; + text-transform: uppercase; + letter-spacing: 0.05em; + margin: 0 0 0.5rem 0 !important; +} + +.sidebar-links { + display: flex; + flex-direction: column; + gap: 2px; + margin-left: -12px; +} + +.sidebar-link { + display: block; + padding: 8px 12px; + border-radius: var(--border-radius-sm); + font-size: 14px; + font-weight: 400 !important; + color: var(--md-default-fg-color--light) !important; + text-decoration: none !important; + transition: all var(--transition-fast) ease; + text-align: left; + cursor: pointer; +} + +.sidebar-link:hover { + background-color: var(--md-default-fg-color--lightest) !important; + color: var(--md-default-fg-color) !important; + font-weight: 500 !important; + transform: translateX(2px); +} + +.sidebar-link:focus-visible { + outline: 2px solid var(--brand-color); + outline-offset: 2px; +} + +/* Content */ +.gallery-content { + flex: 1; + padding: 0 var(--spacing-lg); +} + +.gallery-section { + margin-bottom: var(--spacing-lg); + scroll-margin-top: 120px; +} + +.section-title { + font-size: 24px !important; + font-weight: 700 !important; + color: var(--md-default-fg-color) !important; + margin: 0 0 1.5rem 0 !important; +} + +.cards-grid { + display: grid; + grid-template-columns: repeat(3, 1fr); + gap: var(--spacing-sm) !important; +} + +/* Cards */ +.gallery-card { + display: block; + background: var(--md-default-bg-color); + border: 1px solid var(--md-default-fg-color--lightest); + border-radius: var(--border-radius-md); + overflow: hidden; + transition: all var(--transition-normal) ease; + text-decoration: none !important; + color: inherit; +} + +.gallery-card:hover { + transform: translateY(-2px); + box-shadow: var(--md-shadow-z2); + text-decoration: none !important; + outline: 2px solid var(--brand-color); + outline-offset: 2px; +} + + +.gallery-card:hover .card-title { + color: var(--brand-color) !important; +} + +.card-image { + height: 8rem; + position: relative; + overflow: hidden; + background-color: var(--md-default-fg-color--lightest); + border-bottom: 1px solid var(--md-default-fg-color--lightest); +} + +.card-image img { + width: 100%; + height: 100%; + object-fit: cover; +} + +.card-gradient { + display: flex; + align-items: center; + justify-content: center; + font-size: 2.5rem; +} + +.card-badge { + position: absolute; + top: 0.5rem; + left: 0.5rem; + background: rgba(0, 0, 0, 0.7); + color: white; + padding: 0.2rem 0.4rem; + border-radius: 0.2rem; + font-size: 0.5rem !important; + font-weight: 500 !important; +} + +.card-content { + padding: 12px; +} + +.card-title { + font-size: 0.75rem !important; + font-weight: 600 !important; + line-height: 1.5 !important; + margin: 0 !important; + color: var(--md-default-fg-color) !important; +} + +.card-description { + color: var(--md-default-fg-color--light) !important; + font-size: 0.65rem !important; + line-height: 1.5; + display: -webkit-box; + margin: 0; + -webkit-line-clamp: 3; + -webkit-box-orient: vertical; + overflow: hidden; +} + +/* CTA */ +.gallery-cta { + background: linear-gradient(135deg, rgba(222, 36, 61, 0.08) 0%, rgba(99, 102, 241, 0.08) 100%); + border: 1px solid rgba(222, 36, 61, 0.1); + border-radius: var(--border-radius-lg); + padding: var(--spacing-md); + text-align: center; + margin-top: var(--spacing-lg); +} + +.cta-title { + font-size: 1.5rem; + font-weight: 700; + color: var(--md-default-fg-color); + margin-bottom: var(--spacing-sm); +} + +.cta-description { + color: var(--md-default-fg-color--light); + margin: 0 auto var(--spacing-md); + max-width: 42rem; +} + +.cta-buttons { + display: flex; + justify-content: center; + gap: var(--spacing-sm); + flex-wrap: wrap; +} + +/* Button shared styles */ +.btn-primary, +.btn-secondary { + padding: 0.75rem 1.5rem; + border-radius: var(--border-radius-sm); + font-weight: 500; + text-decoration: none !important; + transition: all var(--transition-fast); +} + +.btn-primary { + background-color: var(--brand-color); + color: white !important; +} + +.btn-primary:hover { + background-color: var(--brand-hover); + color: white !important; +} + +.btn-secondary { + border: 1px solid var(--md-default-fg-color--lighter); + background-color: var(--md-default-bg-color); + color: var(--md-default-fg-color) !important; +} + +.btn-secondary:hover { + background-color: var(--md-default-fg-color--lightest); + color: var(--md-default-fg-color) !important; +} + +/* Dark mode styles */ +[data-md-color-scheme="tidb-dark"] .sidebar-link:hover { + background-color: var(--dark-overlay) !important; +} + +[data-md-color-scheme="tidb-dark"] .gallery-card { + border-color: var(--dark-border); +} + +[data-md-color-scheme="tidb-dark"] .card-image { + background-color: var(--dark-bg-subtle); + border-bottom-color: var(--dark-border); +} + +[data-md-color-scheme="tidb-dark"] .gallery-cta { + background: linear-gradient(135deg, rgba(222, 36, 61, 0.12) 0%, rgba(99, 102, 241, 0.12) 100%); + border-color: rgba(222, 36, 61, 0.2); +} + +[data-md-color-scheme="tidb-dark"] .btn-secondary:hover { + background-color: rgba(255, 255, 255, 0.1); +} + +/* Responsive adjustments */ +@media (max-width: 768px) { + .gallery-layout { flex-direction: column; } + .gallery-sidebar { width: 100%; } + .sidebar-nav { position: static; } + .sidebar-links { flex-direction: row; gap: 0.5rem; flex-wrap: wrap; } + .sidebar-link { padding: 12px 16px !important; min-height: 44px; display: flex; align-items: center; } + .gallery-content { padding: 0; } + .gallery-description { padding: 0 20px; } + .cards-grid { grid-template-columns: 1fr; } + .gallery-title { font-size: 48px !important; } +} + +@media (max-width: 1024px) and (min-width: 769px) { + .cards-grid { grid-template-columns: repeat(2, 1fr); } +} + + + + + +
+
+

{{ config.title }}

+

+ {{ config.description | safe }} +

+
+ +
+
+
+
Categories
+ +
+
+
+ {%- for category in categories %} + {%- if category.demos %} + {%- set category_demos = [] %} + {%- for demo_id in category.demos %} + {%- set demo = demos | selectattr('id', 'equalto', demo_id) | first %} + {%- if demo %}{% set _ = category_demos.append(demo) %}{% endif %} + {%- endfor %} + + {%- endif %} + {%- endfor %} +
+

{{ config.cta.title }}

+

+ {{ config.cta.description }} +

+
+ {%- for button in config.cta.buttons %} + {{ button.text }} + {%- endfor %} +
+
+
+
+
\ No newline at end of file diff --git a/src/templates/demo_page_template.j2 b/src/templates/demo_page_template.j2 new file mode 100644 index 00000000..fd772e66 --- /dev/null +++ b/src/templates/demo_page_template.j2 @@ -0,0 +1,19 @@ +--- +title: {{ demo.title }} +description: "{{ demo.description }}" +source_repo: "{{ demo.doc_link | replace('/README.md', '') }}" +--- + +{{ content | safe }} + +--- + +## Related Resources + +- **Source Code**: [View on GitHub]({{ demo.doc_link | replace('/README.md', '') }}) +- **Category**: {{ demo.category | title }} +{% if demo.description %} +- **Description**: {{ demo.description }} +{% endif %} + +[šŸ  Back to Demo Gallery](../index.md){ .md-button .md-button--primary } \ No newline at end of file From c20b6764dd2013e82aaced9fff44e4ff3c1aef05 Mon Sep 17 00:00:00 2001 From: Mini256 Date: Mon, 14 Jul 2025 19:15:06 +0800 Subject: [PATCH 7/8] update deps --- uv.lock | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/uv.lock b/uv.lock index 835ed14a..9b1ae93a 100644 --- a/uv.lock +++ b/uv.lock @@ -231,20 +231,28 @@ name = "docs" version = "0.1.0" source = { virtual = "." } dependencies = [ + { name = "click" }, + { name = "jinja2" }, { name = "mkdocs" }, { name = "mkdocs-jupyter" }, { name = "mkdocs-material" }, { name = "mkdocs-redirects" }, { name = "mkdocstrings", extra = ["python"] }, + { name = "pyyaml" }, + { name = "requests" }, ] [package.metadata] requires-dist = [ + { name = "click", specifier = ">=8.0.0" }, + { name = "jinja2", specifier = ">=3.1.0" }, { name = "mkdocs", specifier = ">=1.6.1" }, { name = "mkdocs-jupyter", specifier = ">=0.25.1" }, { name = "mkdocs-material", specifier = ">=9.6.12" }, { name = "mkdocs-redirects", specifier = ">=1.2.2" }, { name = "mkdocstrings", extras = ["python"], specifier = ">=0.29.1" }, + { name = "pyyaml", specifier = ">=6.0" }, + { name = "requests", specifier = ">=2.31.0" }, ] [[package]] From 52190ea8177b199126d1ef248df2662fba0ab25c Mon Sep 17 00:00:00 2001 From: Mini256 Date: Mon, 14 Jul 2025 19:39:50 +0800 Subject: [PATCH 8/8] refine --- DEVELOPMENT.md | 4 +++- Makefile | 10 +++++++++- demos.yml | 9 +++++++-- src/ai/examples/index.md | 18 ++++++++++++++++-- src/styles/extra.css | 1 - src/templates/demo_gallery_template.j2 | 17 ++++++++++++++--- 6 files changed, 49 insertions(+), 10 deletions(-) diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index ad884fc4..2f6c22df 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -46,7 +46,9 @@ make build # Build documentation site make clean # Clean build artifacts # Demo management -make generate-demos # Generate demo pages and gallery from demos.yml config +make generate-demos # Generate demo pages and gallery from demos.yml config +make generate-demo-pages # Only generate demo pages. +make generate-demo-gallery # Only generate demo gallery. # Other make help # Show all available commands diff --git a/Makefile b/Makefile index e4c5c0d0..d4bee808 100644 --- a/Makefile +++ b/Makefile @@ -1,6 +1,6 @@ # TiDB for AI Documentation Site -.PHONY: help install generate-demos serve build clean check +.PHONY: help install generate-demos generate-demo-pages generate-demo-gallery serve build clean check help: @echo "TiDB for AI Documentation Site" @@ -8,6 +8,8 @@ help: @echo "Available commands:" @echo " install - Install dependencies using UV" @echo " generate-demos - Generate demo pages and gallery from demos.yml config" + @echo " generate-demo-pages - Only generate demo pages." + @echo " generate-demo-gallery - Only generate demo gallery." @echo " serve - Start the development server" @echo " build - Build the documentation site" @echo " clean - Clean build artifacts" @@ -25,6 +27,12 @@ install: generate-demos: python scripts/generate_demos.py +generate-demo-pages: + python scripts/generate_demos.py --skip-gallery + +generate-demo-gallery: + python scripts/generate_demos.py --skip-demos + serve: mkdocs serve diff --git a/demos.yml b/demos.yml index 09c09961..8489b90f 100644 --- a/demos.yml +++ b/demos.yml @@ -1,6 +1,11 @@ # Demo Gallery Configuration title: "Demo Gallery" -description: "Explore hands-on demos showcasing how TiDB empowers AI applications. Get started quickly with TiDB Cloud Serverless to build your own AI-powered solutions." +description: | + Explore hands-on demos showcasing how TiDB empowers AI applications.
+ Get started quickly with TiDB Cloud Serverless to build your own AI-powered solutions. + +meta: + description: "Explore hands-on demos showcasing how TiDB empowers AI applications. Get started quickly with TiDB Cloud Serverless to build your own AI-powered solutions." categories: - id: "featured" @@ -117,7 +122,7 @@ cta: description: "Start your AI journey with TiDB Cloud Serverless. Follow our quickstart guide to build your first AI-powered application in minutes, or explore specific examples for your use case." buttons: - text: "Try TiDB Cloud Serverless" - url: "https://tidbcloud.com/serverless" + url: "https://tidbcloud.com/?utm_source=github&utm_medium=referral&utm_campaign=pytidb_readme" type: "primary" external: true - text: "View Quickstart Guide" diff --git a/src/ai/examples/index.md b/src/ai/examples/index.md index e9a545ff..ce5cc8db 100644 --- a/src/ai/examples/index.md +++ b/src/ai/examples/index.md @@ -60,6 +60,18 @@ html { margin-bottom: 5rem !important; } +/* Gallery CTA link styles */ +.gallery-cta-link { + position: relative; + text-decoration: none; + transition: all 0.2s ease-in-out; +} + +.gallery-cta-link:hover { + border-bottom: 3px solid var(--brand-color); +} + + /* Layout */ .gallery-layout { display: flex; @@ -332,7 +344,9 @@ html {

Demo Gallery

- Explore hands-on demos showcasing how TiDB empowers AI applications. Get started quickly with TiDB Cloud Serverless to build your own AI-powered solutions. + Explore hands-on demos showcasing how TiDB empowers AI applications.
+Get started quickly with TiDB Cloud Serverless to build your own AI-powered solutions. +

@@ -507,7 +521,7 @@ html { Start your AI journey with TiDB Cloud Serverless. Follow our quickstart guide to build your first AI-powered application in minutes, or explore specific examples for your use case.

- Try TiDB Cloud Serverless + Try TiDB Cloud Serverless View Quickstart Guide
diff --git a/src/styles/extra.css b/src/styles/extra.css index 2c9a270b..b2e3a3cf 100644 --- a/src/styles/extra.css +++ b/src/styles/extra.css @@ -180,4 +180,3 @@ .md-header__title { margin-left: 0 !important; } - diff --git a/src/templates/demo_gallery_template.j2 b/src/templates/demo_gallery_template.j2 index dab6735e..0d7876b8 100644 --- a/src/templates/demo_gallery_template.j2 +++ b/src/templates/demo_gallery_template.j2 @@ -1,6 +1,6 @@ --- title: {{ config.title }} -description: {{ config.description }} +description: {{ config.meta.description }} hide: - navigation - toc @@ -60,6 +60,18 @@ html { margin-bottom: 5rem !important; } +/* Gallery CTA link styles */ +.gallery-cta-link { + position: relative; + text-decoration: none; + transition: all 0.3s ease-in-out; +} + +.gallery-cta-link:hover { + border-bottom: 3px solid var(--brand-color); +} + + /* Layout */ .gallery-layout { display: flex; @@ -341,8 +353,7 @@ html {
Categories