VTK Python Documentation Enhancement

A comprehensive toolkit for enhancing VTK Python stub files with rich documentation and generating beautiful markdown API documentation.

🚀 Features

• VTK Documentation Extraction: Extract comprehensive documentation from installed VTK library using Python introspection
• Enhanced Python Stubs: Enhance VTK's official Python stub files with rich docstrings for superior IDE IntelliSense
• Rich Markdown Documentation: Generate beautiful, web-ready markdown documentation organized by modules and classes
• Parallel Processing: High-performance parallel processing with 12 worker threads
• Modular Architecture: Memory-efficient per-module processing for optimal performance
• Comprehensive Verification: Built-in tools to verify pipeline integrity and completeness

📋 Requirements

• Python 3.7+
• VTK Python package installed
• Standard Python libraries: json, re, ast, pathlib, concurrent.futures, subprocess, typing

🛠️ Installation & Setup

Option 1: Automated Setup (Recommended)

1. Clone this repository:

git clone <repository-url>
cd vtk-python-docs

2. Run the automated setup script:

./setup_venv.sh

This will create a virtual environment, install VTK, and set up the project.

Option 2: Manual Setup

1. Clone this repository:

git clone <repository-url>
cd vtk-python-docs

2. Create and activate a virtual environment:

python3 -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

3. Install requirements:

pip install -r requirements.txt
python setup.py

📖 Usage

Quick Start - Complete Build from Scratch

# Clean and build everything (recommended for testing)
python build.py

# Or step by step:
python clean.py                    # Clean previous builds
python scripts/extract_all_vtk_docs.py     # Extract VTK documentation
python scripts/generate_official_stubs.py  # Generate official VTK stubs
python scripts/enhance_all_vtk_stubs.py    # Enhance stubs with documentation
python scripts/generate_markdown_docs.py   # Generate markdown docs
python tools/verify_vtk_docs.py --all  # Verify integrity

Individual Pipeline Steps

# 1. Extract VTK documentation (creates docs/vtk-docs/)
python scripts/extract_all_vtk_docs.py

# 2. Generate official VTK stubs (creates docs/python-stubs-official/)
python scripts/generate_official_stubs.py

# 3. Enhance Python stubs with documentation (creates docs/python-stubs-enhanced/)
python scripts/enhance_all_vtk_stubs.py

# 4. Generate markdown documentation (creates docs/python-api/)
python scripts/generate_markdown_docs.py

# 5. Verify everything works correctly
python tools/verify_vtk_docs.py --all

Individual Components

VTK Documentation Extraction

# Extract all modules in parallel (recommended)
python scripts/extract_all_vtk_docs.py

# Extract single module for testing
python scripts/extract_vtk_docs.py --module vtkCommonCore --output docs/vtk-docs/

Official VTK Stub Generation

# Generate official VTK stubs using VTK's built-in generator
python scripts/generate_official_stubs.py

# Or specify custom output directory
python scripts/generate_official_stubs.py --output docs/python-stubs-official

Python Stub Enhancement

# Enhance all stubs in parallel (recommended)
python scripts/enhance_all_vtk_stubs.py

# Enhance single module for testing
python scripts/enhance_vtk_stubs.py --module vtkCommonCore --docs-dir docs/vtk-docs/ --output docs/python-stubs-enhanced/

Markdown Documentation Generation

# Generate complete markdown documentation
python scripts/generate_markdown_docs.py

IDE Integration

Configure your IDE to use the enhanced stubs by pointing to docs/python-stubs-enhanced/. The included pyrightconfig.json is pre-configured for this.

📁 Output Structure

docs/
├── vtk-docs/                    # Extracted VTK documentation (JSON)
│   ├── vtkCommonCore.json
│   ├── vtkCommonDataModel.json
│   └── ... (150+ modules)
├── python-stubs-official/       # Official VTK stubs (generated by VTK)
│   ├── vtkCommonCore.pyi
│   ├── vtkCommonDataModel.pyi
│   └── ... (150+ modules)
├── python-stubs-enhanced/       # Enhanced Python stubs with documentation
│   └── vtkmodules/
│       ├── vtkCommonCore.pyi
│       ├── vtkCommonDataModel.pyi
│       └── ... (150+ modules)
└── python-api/                  # Rich markdown documentation
    ├── index.md                 # Main documentation index
    ├── classes.md               # All classes sorted by 4th character
    ├── modules.md               # All classes organized by module
    ├── vtkCommonCore/           # Module-specific documentation
    │   ├── index.md
    │   ├── vtkObject.md
    │   └── ... (class files)
    └── ... (150+ module directories)

🔧 Architecture

Modular Design

• Per-module processing: Each VTK module is processed independently for optimal memory usage
• Parallel execution: 12 worker threads for maximum performance
• Clean separation: Documentation extraction, stub enhancement, and markdown generation are separate stages

Key Components

Core Scripts

• scripts/extract_all_vtk_docs.py - Parallel VTK documentation extraction
• scripts/enhance_all_vtk_stubs.py - Parallel Python stub enhancement
• scripts/generate_markdown_docs.py - Rich markdown documentation generation

Single-Module Scripts

• scripts/extract_vtk_docs.py - Extract documentation for one module
• scripts/enhance_vtk_stubs.py - Enhance stubs for one module

Verification Tools

• tools/verify_vtk_docs.py - Comprehensive pipeline verification

📊 Performance

• Documentation Extraction: ~6 seconds for 2,942 classes across 151 modules
• Stub Enhancement: ~15 seconds for 151 stub files
• Markdown Generation: ~1.5 seconds for complete web documentation
• Total Pipeline: Under 30 seconds for complete enhancement

🎯 Key Features

Documentation Enhancement

• Extracts comprehensive class and method documentation from VTK library
• Converts C++ Doxygen tags to Python format (@param → :param:, etc.)
• Handles @overload signatures correctly (docstring only on final signature)
• Adds placeholder docstrings only when no meaningful documentation exists

Markdown Documentation

• Beautiful, web-ready markdown with rich formatting
• Organized by VTK database sections for logical grouping
• Multiple navigation options (by module, by 4th character)
• Cross-references and search-friendly structure
• Automatic VTK version detection and tagging

IDE Integration

• Enhanced IntelliSense with comprehensive VTK documentation
• Proper typing support from VTK's official stub generation
• Clean, Pythonic documentation format
• Compatible with PyRight, Pylance, and other language servers

🧪 Verification

Verify the integrity of your documentation pipeline:

# Verify all components
python tools/verify_vtk_docs.py --all

# Verify specific components
python tools/verify_vtk_docs.py --docs      # JSON documentation
python tools/verify_vtk_docs.py --stubs     # Enhanced stubs
python tools/verify_vtk_docs.py --markdown  # Markdown docs

📈 Statistics

Based on VTK 9.5.0:

• 151 VTK modules processed
• 2,942 VTK classes documented
• 99.3% success rate (150/151 modules enhanced)
• Comprehensive coverage of VTK Python API

🤝 Contributing

This project provides a complete, production-ready VTK documentation enhancement pipeline. Contributions for improvements, bug fixes, or support for newer VTK versions are welcome.

📄 License

This project enhances the official VTK Python bindings and documentation. Please refer to VTK's licensing terms for the underlying VTK library.

---