vit-captioner v0.1.2

ViT Captioner v0.1.2 Release Notes

Performance & Stability Improvements

This release includes significant optimizations to improve stability and performance:

• Thread Safety Enhancements: Fixed threading issues with matplotlib by implementing a non-interactive backend, preventing segmentation faults during parallel processing
• Fallback Mechanisms: Added robust error handling that preserves output even when visualization fails
• Resource Management: Implemented better memory management with proper cleanup of resources
• Single Model Loading: Optimized to load the ViT-GPT2 model only once for multiple frames, reducing memory usage and improving speed

Code Quality Improvements

• Unified Version Management: Simplified version tracking across the package
• Progress Tracking: Enhanced progress bars for better visibility of processing status
• Verbose Mode: Added proper support for verbose output flag throughout the application

Documentation Updates

• Updated README with new features and optimizations
• Expanded examples to show proper usage of the verbose flag
• Added details about performance considerations

Under the Hood

• Fixed matplotlib threading issues that were causing segmentation faults
• Suppressed redundant transformer model warnings for cleaner output
• Limited the number of concurrent workers to prevent memory overload
• Improved error handling throughout the codebase

Thank you for using ViT Captioner! This release makes the tool more robust for processing videos of all sizes.

vit-captioner v0.1.1

ViT Captioner v0.1.1

The initial release of ViT Captioner, a Python package that extracts keyframes from videos and generates captions using the Vision Transformer (ViT-GPT2) model.

Features

• Extracts keyframes from videos using Katna library or uniform sampling
• Generates captions for images using the ViT-GPT2 model
• Matches keyframes with precise timestamps in videos
• Converts videos to SRT subtitle files with AI-generated captions
• Visualizes keyframes and timeline data with corresponding metadata
• Provides both command-line interface and Python API

Installation

# Install from PyPI
pip install vit-captioner

# Or install from source
git clone https://github.com/lachlanchen/VideoCaptionerWithVit.git
cd VideoCaptionerWithVit
pip install -e .

Usage Examples

Command Line

# Extract 10 keyframes from a video with visualization
vit-captioner extract -V /path/to/video.mp4 -N 10 -v

# Generate caption for an image
vit-captioner caption-image -I /path/to/image.jpg

# Convert video to captions (generates SRT and JSON)
vit-captioner caption-video -V /path/to/video.mp4 -N 10

# Find matching timestamps for keyframes
vit-captioner find-timestamps -V /path/to/video.mp4 -K /path/to/keyframes_folder -v

Python API

from vit_captioner.keyframes.extractor import KeyFrameExtractor
from vit_captioner.captioning.image import ImageCaptioner
from vit_captioner.captioning.video import VideoToCaption

# Extract keyframes
extractor = KeyFrameExtractor("/path/to/video.mp4")
frames_dir = extractor.extract_key_frames("/path/to/video.mp4", 10)

# Generate caption for an image
captioner = ImageCaptioner()
caption = captioner.predict_caption("/path/to/image.jpg")

# Convert video to captions
converter = VideoToCaption("/path/to/video.mp4", num_frames=10)
converter.convert()

Requirements

• Python 3.6+
• PyTorch
• Transformers
• OpenCV
• Katna (for keyframe extraction)
• Matplotlib
• tqdm (for progress bars)

All outputs include datetime suffixes for versioning, and visualization data is automatically saved alongside figures.

vit-capitoner v0.1.0

ViT Captioner v0.1.0

The initial release of ViT Captioner, a Python package that extracts keyframes from videos and generates captions using the Vision Transformer (ViT-GPT2) model.

Features

• Extracts keyframes from videos using Katna library or uniform sampling
• Generates captions for images using the ViT-GPT2 model
• Matches keyframes with precise timestamps in videos
• Converts videos to SRT subtitle files with AI-generated captions
• Visualizes keyframes and timeline data with corresponding metadata
• Provides both command-line interface and Python API

Installation

# Install from PyPI
pip install vit-captioner

# Or install from source
git clone https://github.com/lachlanchen/VideoCaptionerWithVit.git
cd VideoCaptionerWithVit
pip install -e .

Usage Examples

Command Line

# Extract 10 keyframes from a video with visualization
vit-captioner extract -V /path/to/video.mp4 -N 10 -v

# Generate caption for an image
vit-captioner caption-image -I /path/to/image.jpg

# Convert video to captions (generates SRT and JSON)
vit-captioner caption-video -V /path/to/video.mp4 -N 10

# Find matching timestamps for keyframes
vit-captioner find-timestamps -V /path/to/video.mp4 -K /path/to/keyframes_folder -v

Python API

from vit_captioner.keyframes.extractor import KeyFrameExtractor
from vit_captioner.captioning.image import ImageCaptioner
from vit_captioner.captioning.video import VideoToCaption

# Extract keyframes
extractor = KeyFrameExtractor("/path/to/video.mp4")
frames_dir = extractor.extract_key_frames("/path/to/video.mp4", 10)

# Generate caption for an image
captioner = ImageCaptioner()
caption = captioner.predict_caption("/path/to/image.jpg")

# Convert video to captions
converter = VideoToCaption("/path/to/video.mp4", num_frames=10)
converter.convert()

Requirements

• Python 3.6+
• PyTorch
• Transformers
• OpenCV
• Katna (for keyframe extraction)
• Matplotlib
• tqdm (for progress bars)

All outputs include datetime suffixes for versioning, and visualization data is automatically saved alongside figures.