SBS Data Processing and Visualization Pipeline

Overview

This repository contains a Python pipeline for processing, analyzing, and visualizing Stimulated Brillouin Scattering data, provide both backend data processing and a graphical user interface (GUI) for interactive exploration.

The main functionalities include:

• Data Reading: Reads raw data from CSV files.
• Preprocessing: Performs interpolation and reshaping of the raw data.
• Peak Fitting: Fits single or double Lorentzian peaks to the data using GPU-accelerated fitting via pygpufit.
• Visualization: Displays 2D heatmaps, frequency plots, and histograms for various parameters.
• Interactive GUI: Provides an intuitive interface for users to interact with processed data through plots, histograms, and tables.

---

Features

1. Data Preprocessing:

   • Reshapes raw data into structured arrays.
   • Applies interpolation to align data points.

2. Single and Double Peak Fitting:

   • Utilizes GPU-based least squares fitting (pygpufit) for high-performance computation.
   • Supports fitting of both single and double Lorentzian peaks with constraints.

3. Visualization:

   • Generates 2D heatmaps for amplitude, shift, width, offset, and error.
   • Displays frequency plots for individual pixels.
   • Produces histograms for statistical analysis of fitted parameters.

4. Export Options:

   • Saves processed data as TIFF files.
   • Generates colorbars for visualization consistency.

5. Interactive GUI:

   • Allows users to navigate through planes, adjust visualization parameters, and view detailed information for specific pixels.

---

Environment Setup

Prerequisites

Ensure the following dependencies are installed in your Python environment:

1. Python 3.x: Ensure Python 3.x is installed on your system.

2. Required Libraries:

   • numpy
   • pandas
   • matplotlib
   • pyqt5
   • scipy
   • joblib
   • pyarrow
   • tifffile

   Install these libraries using pip:

   pip install numpy pandas matplotlib pyqt5 scipy joblib pyarrow tifffile

3. pygpufit:

   • A library for GPU-accelerated Lorentzian fitting.
   • Homepage: github.com/gpufit/Gpufit
   • Customization: We have extended the functionality of gpufit by adding support for Lorentzian line shapes, enabling more accurate fitting for our specific use case.
   • Installation:
     • Download the appropriate .whl file from the GitHub Releases page.
     • Install using pip:

       pip install path_to_wheel_file/pygpufit-*.whl

   • Compatibility: The library has been tested and optimized for NVIDIA GPUs in the RTX series, including RTX20**, RTX30**, and RTX40**. These GPUs provide excellent performance for GPU-accelerated computations.

4. UI File:

   • Ensure the UI file (Ui_dou3.py) generated by PyQt5's pyuic tool is available in the project directory.

5. Data Files:

   • Place the required data files test2.txt, test2_append.txt in the specified directory (D:\workspace\01\20240809\test2).

---

Usage Instructions

Step 1: Adjust Parameters

Before running the application, you may need to adjust certain parameters in the script or configuration files. Key parameters include:

• File Paths: Ensure the paths to your data files (test2.txt, test2_append.txt) are correctly set.
• Experiment Settings: Modify the experiment_settings class in main.py to match your experimental setup, such as samples_ignored, shift, x_pixel_to_um, y_pixel_to_um, etc.
• Constraints: Adjust the constraints for peak fitting in pipeline.py if necessary.

Step 2: Launch the Application

Run the main script using the following command:

python main.py

Step 3: Load Data

The application automatically loads data from the predefined file paths. Ensure the data files exist in the specified directories.

Step 4: Navigate Through Planes

• Use the slider (zplane) to select the plane you want to visualize.
• Adjust the minimum and maximum frequency values (minfreq, maxfreq) to control the color scale of the heatmap.

Step 5: Select Parameters for Visualization

• Choose a parameter from the dropdown menus (comboBox, comboBox_2) to display corresponding heatmaps and histograms.
• Parameters include shift, amplitude, width, offset, error, and double peak-specific parameters like l1_shift, l1_amplitude, etc.

Step 6: View Frequency Plots

• Click on any point in the heatmap to view the frequency plot for that pixel.
• The frequency plot displays raw data, single fit, and double fit (if enabled) for the selected pixel.

Step 7: Enable Double Peak Fitting (Optional)

• Check the double_peak checkbox to enable double peak fitting.
• Additional parameters for double peaks will appear in the dropdown menus.

Step 8: Save Results

• The application automatically saves the processed data as TIFF files in the tiff_export subfolder.

Step 9: Exit the Application

Close the application window to exit.

---

Performance Metrics

The application calculates and displays the total processing time, including:

• Time taken to read files.
• Time for preprocessing.
• Time for fitting.
• Total execution time.

Example output:

读文件：0.1234s
预处理：0.5678s
拟合：1.2345s
总计：2.3456s

---

Notes

• Ensure all required files and libraries are correctly placed and installed before running the application.
• Modify file paths in the script if your data files are stored in different directories.
• For advanced customization, refer to the pipeline module and adjust preprocessing and fitting functions as needed.

If you are interested in more details, please refer to our paper "Stimulated Brillouin scattering microscopy with a high-peak-power 780 nm pulsed laser system"

License

This project is licensed under the MIT License - see the LICENSE file for details.