MicroPL Python System

A comprehensive Python translation of the MATLAB-based MicroPL microscopy system for controlling scientific hardware and running automated experiments.

Overview

This project translates legacy MATLAB drivers for a home-built epifluorescence microscope system into modern, well-tested Python code. The system provides complete control over a multi-instrument scientific setup with 85% of core drivers now implemented.

✅ Fully Implemented Hardware

🎥 Camera Systems
- PointGrey/FLIR Cameras - Complete PySpin integration with all modes
- Thorlabs Cameras - UC480 .NET SDK support with simulation fallback
🎛️ Motion Control
- SPiiplus XY/Z Stages - Full 3-axis motion control with TCP/IP communication
- Elliptec Rotation Stages - ND filter positioning with serial communication
🔬 Spectroscopy
- Ocean Optics Spectrometers - SeaBreeze integration with wavelength calibration
- Horiba JY Dual-Detector System - Complete monochromator and CCD control
🏗️ Infrastructure
- Base Driver Framework - Thread-safe abstract base classes
- Configuration Management - Type-safe Pydantic configuration
- Logging System - Hardware-aware structured logging
- Testing Suite - Comprehensive pytest-based testing with mocks

🔄 Remaining Hardware (15%)

Measurement Electronics - 5 instruments for specialized measurements:

Data Translation ADC (voltage acquisition)
HF2LI Lock-in Amplifier (phase/amplitude)
Keithley Source Meter (precision I-V)
Quantum Composers (timing/pulses)
LeCroy Oscilloscope (waveform analysis)

Key Features

🔧 Production-Ready Drivers: Complete implementations with simulation modes
🧪 Experiment Framework: Standardized experiment lifecycle with progress tracking
📊 Comprehensive Testing: Unit tests, integration tests, and hardware mocks
📋 Type-Safe Configuration: Pydantic-based configuration with validation
📝 Advanced Logging: Structured logging with hardware context
🛡️ Robust Error Handling: Exception handling and recovery mechanisms
🔄 Thread Safety: Concurrent hardware operations with proper locking
🎯 Hardware Simulation: Full simulation modes for development without hardware
🔌 Cross-Platform: Supports Windows (production) and macOS/Linux (development)

Project Structure

MicroPL_Python/
├── drivers/                     # Hardware drivers (85% complete)
│   ├── base_driver.py          # Abstract base class with thread safety
│   ├── cameras/                # Camera drivers (COMPLETE)
│   │   ├── pointgrey_camera.py    # PointGrey/FLIR (PySpin SDK)
│   │   ├── thorlabs_camera.py     # Thorlabs UC480 (.NET SDK)
│   │   └── camera_types.py        # Type-safe data structures
│   ├── stages/                 # Motion control (COMPLETE)
│   │   ├── spiiplus_driver.py     # XY/Z stages (TCP/IP)
│   │   ├── elliptec_driver.py     # Rotation stages (Serial)
│   │   └── nd_slider.py           # ND filter positioning
│   └── spectrometers/          # Spectroscopy (COMPLETE)
│       ├── oceanoptics_driver.py  # Ocean Optics (SeaBreeze)
│       ├── horiba_driver.py       # Horiba JY dual-detector
│       └── base_spectrometer.py   # Unified interface
├── experiments/                # Experiment framework
│   └── base_experiment.py      # Base experiment class
├── utils/                      # Utility modules
│   ├── logging_config.py       # Hardware-aware logging
│   └── config_manager.py       # Pydantic configuration
├── tests/                      # Comprehensive test suite
├── config/                     # Configuration files
└── requirements.txt            # All dependencies with platform markers

Quick Start

1. Environment Setup

# Create Python 3.13 virtual environment
python3.13 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Quick Start

1. Environment Setup

# Create Python 3.13 virtual environment
python3.13 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

2. Hardware Demos

Comprehensive demonstration scripts are available for all implemented drivers:

# Camera systems
python thorlabs_demo.py       # Thorlabs camera with UC480 SDK
python pointgrey_demo.py      # PointGrey/FLIR camera demo (requires PySpin)

# Motion control  
python spiiplus_demo.py       # XY/Z stage control
python elliptec_demo.py       # Rotation stages and ND filters

# Spectroscopy
python oceanoptics_demo.py    # Ocean Optics spectrometer
python horiba_demo.py         # Horiba JY dual-detector system
python spectrometer_demo.py   # Unified spectrometer interface

# All demos support simulation mode for development without hardware

3. Multi-System Example

from drivers.cameras.thorlabs_camera import ThorlabsCamera
from drivers.stages.spiiplus_driver import SPiiplusDriver  
from drivers.spectrometers.oceanoptics_driver import OceanOpticsDriver
from utils.logging_config import setup_logging

# Set up logging
setup_logging(log_level="INFO")

# Connect to multiple instruments
with ThorlabsCamera(simulate=True) as camera, \
     SPiiplusDriver(simulate=True) as stage, \
     OceanOpticsDriver(simulate=True) as spectrometer:
    
    # Move to measurement position
    stage.position = [10.0, 5.0, 2.0]  # X, Y, Z in mm
    stage.wait_for_completion()
    
    # Configure camera
    camera.exposure_time = 50.0  # milliseconds
    
    # Acquire synchronized data
    image = camera.capture_image()
    spectrum = spectrometer.acquire_spectrum(integration_time=0.1)
    
    print(f"Image shape: {image.data.shape}")
    print(f"Spectrum points: {len(spectrum.wavelengths)}")

4. Experiment Framework Example

from experiments.base_experiment import SimpleImageAcquisition

# Create experiment
experiment = SimpleImageAcquisition(
    camera_device=camera,
    exposure_times=[1.0, 5.0, 10.0, 20.0],
    num_averages=5
)

# Add progress tracking
experiment.add_progress_callback(
    lambda p: print(f"Progress: {p.completion_percentage:.1f}%")
)

# Run experiment
experiment.start()
print(f"Status: {experiment.status}")

# Save results
results_file = experiment.save_results()

Hardware Support Status

✅ Cameras (COMPLETE)

PointGrey/FLIR Cameras

Connection: USB via PySpin SDK
Modes: 0 (full res, 16-bit), 5 (binned, 16-bit), 7 (full res, 12-bit)
Features: Exposure/gain control, image averaging, HDR acquisition
Fallback: OpenCV support for development without PySpin

Thorlabs Cameras

Connection: UC480 .NET SDK integration
Features: Software trigger, RGBA8 pixel format, exposure control
Simulation: Full simulation mode for development
Requirements: Thorlabs Scientific Imaging software (Windows)

✅ Motion Control (COMPLETE)

SPiiplus Stage Controllers

Connection: TCP/IP Ethernet (10.0.0.100:701)
Features: 3-axis coordinated motion, position limits, velocity control
Safety: Emergency stop, soft limits, position monitoring
Advanced: Named positions, LCM support, simulation mode

Elliptec Rotation Stages

Connection: Serial (USB/RS-232)
Features: ND filter positioning, angular control, multi-device support
Calibration: Multiple filter positions with position feedback

✅ Spectroscopy (COMPLETE)

Ocean Optics Spectrometers

Connection: USB via SeaBreeze library
Features: Universal Ocean Optics support, wavelength calibration
Processing: Dark spectrum subtraction, non-linearity correction
Control: Integration time, temperature monitoring

Horiba JY Dual-Detector System

Connection: USB/TCP
Detectors: VIS CCD (1024px), IR detector (512px)
Control: Grating selection (150/300/1200 lines/mm), slit width
Features: Wavelength scale calculation, response correction, mirror positioning

🔄 Measurement Electronics (TODO - 5 instruments)

High Priority

Data Translation ADC (DT9816): Voltage acquisition (.NET integration)
HF2LI Lock-in Amplifier: Phase/amplitude measurements (ziDAQ library)
Keithley Source Meter: Precision I-V measurements (SCPI commands)

Medium Priority

Quantum Composers: Pulse generation and timing control
LeCroy Oscilloscope: Waveform acquisition and analysis

Testing

The project includes comprehensive testing with simulation support:

# Run all tests (simulation mode - no hardware required)
pytest

# Run with coverage report  
pytest --cov=drivers --cov=experiments --cov=utils --cov-report=html

# Run integration tests (requires hardware - use carefully)
pytest -m integration

# Run specific driver tests
pytest tests/drivers/cameras/test_pointgrey_camera.py
pytest tests/drivers/stages/test_spiiplus_driver.py  
pytest tests/drivers/spectrometers/test_oceanoptics_driver.py

# Run performance benchmarks
pytest -m benchmark --benchmark-sort=mean

Test Categories

🧪 Unit Tests: Individual components with mocked hardware (safe for continuous testing)
🔗 Integration Tests: Real hardware tests (marked @pytest.mark.integration)
📊 Property Tests: Hypothesis-based property testing for edge cases
⚡ Benchmarks: Performance characterization and optimization
🎭 Mock Tests: Complete simulation of hardware behavior

Configuration

Configuration is managed through Pydantic models with validation:

from utils.config_manager import load_config, save_config

# Load configuration (supports JSON/YAML + environment variables)
config = load_config("config/my_config.json")

# Access nested configuration
print(config.camera.default_exposure)
print(config.system.log_level)

# Environment variable override
# MICROPL_CAMERA__DEFAULT_GAIN=10.0

Logging

Advanced logging with hardware context and performance tracking:

from utils.logging_config import setup_logging, LoggingContext

# Set up logging
logger = setup_logging(
    log_level="INFO",
    log_dir="./logs",
    structured_logging=True  # JSON format
)

# Hardware context logging
with LoggingContext(device="camera1", operation="acquire"):
    logger.info("Starting image acquisition")

Contributing

🛠️ Development Guidelines

Fork the repository and create a feature branch
Test First: Add comprehensive tests for new functionality
Simulation Mode: Ensure all drivers work without hardware
Type Safety: Use full type annotations throughout
Documentation: Update docstrings and README as needed
Platform Compatibility: Test on Windows (production) and macOS/Linux (development)

git checkout -b feature/new-driver
# Add your changes with tests
pytest  # Ensure all tests pass
git commit -m 'Add new driver with simulation support'  
git push origin feature/new-driver
# Open Pull Request

🔬 Hardware Translation Guidelines

When implementing remaining measurement electronics:

Mark Uncertainties: Use # TODO: comments for unclear MATLAB sections
Preserve Functionality: Maintain original behavior where possible
Add Safety: Implement bounds checking and validation
Simulation First: Create mock hardware before real implementation
Test Thoroughly: Both simulation and integration tests

Development Status & Roadmap

🎉 Current Status: 85% Complete Core Drivers

✅ Production-Ready Components:

All Primary Hardware: Cameras, stages, spectrometers fully implemented
Infrastructure: Base classes, configuration, logging, testing complete
Documentation: Comprehensive README, inline docs, demo scripts
Cross-Platform: Full Windows/macOS/Linux development support

🚀 Phase 2: Integration & Optimization (Next)

Multi-Instrument Experiments: Coordinated measurement sequences
GUI Development: PySide6 interface for complete system control
Performance Optimization: Real-time acquisition and processing
Advanced Features: HDR imaging, automatic exposure, data export

🔬 Phase 3: Measurement Electronics (As Needed)

Implementation of the remaining 5 specialized instruments based on experimental requirements:

Data Translation ADC - Voltage measurements
HF2LI Lock-in Amplifier - Phase/amplitude analysis
Keithley Source Meter - Precision I-V characterization
Quantum Composers - Timing and pulse generation
LeCroy Oscilloscope - Waveform acquisition

Hardware Dependencies

✅ Implemented Hardware Dependencies

Windows (Production Environment)

PySpin SDK: PointGrey/FLIR camera support (Windows only)
UC480 .NET SDK: Thorlabs camera support (via pythonnet)
SeaBreeze Library: Universal Ocean Optics support
Serial Interface: Elliptec rotation stages (pyserial)
TCP/IP: SPiiplus motion controllers (built-in socket)

Cross-Platform (Development)

OpenCV: Alternative camera interface for development
Hardware Simulation: Complete mock hardware for all drivers
Platform Markers: Automatic dependency management per platform

🔄 TODO Hardware Dependencies

OLClassLib.dll: Data Translation ADC (.NET library)
ziDAQ: Zurich Instruments HF2LI (LabOne software)
PyVISA: SCPI communication for Keithley, Quantum Composers
RS232 Adapters: Legacy LeCroy oscilloscope communication

📦 Installation Modes

# Full installation (Windows with hardware)
pip install -r requirements.txt

# Development installation (any platform)  
pip install -r requirements.txt --skip-platform-specific

# Minimal installation (core functionality only)
pip install numpy pandas pyserial pydantic pytest

Support & Troubleshooting

🆘 Quick Help

Hardware Connection Issues:

# Test hardware simulation (no hardware required)
python thorlabs_demo.py       # Should work on any platform
python spiiplus_demo.py       # Test motion control simulation  
python oceanoptics_demo.py    # Test spectrometer simulation

Dependency Problems:

# Check installed packages
pip list | grep -E "(PySpin|pythonnet|opencv)"

# Platform-specific installation
pip install -r requirements.txt  # Windows: includes PySpin, pythonnet
pip install opencv-python         # Cross-platform fallback

Development vs Production:

Development: All drivers work in simulation mode on any platform
Production: Hardware-specific SDKs required (Windows recommended)
Testing: Use simulation mode for safe development and CI/CD

📚 Documentation Hierarchy

README.md (this file) - Project overview and quick start
Demo Scripts - Hands-on examples for each driver
Inline Docstrings - Detailed API documentation
Strategy.md - Implementation roadmap and status
Original MATLAB - Reference for hardware behavior

🐛 Issue Reporting

For issues related to:

Hardware Setup: Check original MATLAB documentation and demo scripts
Python Implementation: Create GitHub issues with error logs and system info
Dependencies: Verify requirements.txt installation and platform compatibility
Performance: Use simulation mode to isolate hardware vs software issues

This is a research tool for scientific instrumentation. Test thoroughly with simulation mode before connecting expensive hardware.

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
.github		.github
config		config
drivers		drivers
experiments		experiments
tests		tests
utils		utils
.gitignore		.gitignore
README.md		README.md
requirements.txt		requirements.txt

p-parkinson/MicroPL_Python

Folders and files

Latest commit

History

Repository files navigation

MicroPL Python System

Overview

✅ Fully Implemented Hardware

🔄 Remaining Hardware (15%)

Key Features

Project Structure

Quick Start

1. Environment Setup

Quick Start

1. Environment Setup

2. Hardware Demos

3. Multi-System Example

4. Experiment Framework Example

Hardware Support Status

✅ Cameras (COMPLETE)

✅ Motion Control (COMPLETE)

✅ Spectroscopy (COMPLETE)

🔄 Measurement Electronics (TODO - 5 instruments)

Testing

Test Categories

Configuration

Logging

Contributing

🛠️ Development Guidelines

🔬 Hardware Translation Guidelines

Development Status & Roadmap

🎉 Current Status: 85% Complete Core Drivers

🚀 Phase 2: Integration & Optimization (Next)

🔬 Phase 3: Measurement Electronics (As Needed)

Hardware Dependencies

✅ Implemented Hardware Dependencies

🔄 TODO Hardware Dependencies

📦 Installation Modes

Support & Troubleshooting

🆘 Quick Help

📚 Documentation Hierarchy

🐛 Issue Reporting

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages