Merged
Conversation
…s Generator, Spindle, and Hill Muscle Model
…network connections
…larity and error handling
…abe simulation - Introduced `continuous_saver.py` to manage memory-efficient saving of simulation data in chunks, preventing out-of-memory errors during long simulations. - Created `10b_load_and_analyze.py` for loading and analyzing saved chunks, including membrane potentials and spike data. - Updated `10b_watanabe_compute_force.py` and `10c_watanabe_visualize.py` to reference the new NEO Block format for results. - Added `README_CONTINUOUS_SAVING.md` to document the continuous saving approach, usage, and configuration options. - Implemented `convert_existing_spikes.py` to convert existing spike data into the new chunks format for compatibility.
…tions for improved simulation accuracy
…ing utilities for improved simulation performance
…upport Add comprehensive optimization framework for motor neuron descending drive parameters with support for gamma point process spike generation. New features: - Multi-objective Optuna-based optimization for DD parameters - Gamma process spike generation with configurable shape parameter - ISI and CV analysis tools for validating firing patterns - Muscle-specific optimized configurations (FDI, TA, VLVM) - Force-based optimization using Watanabe (2015) model - Automated bash scripts for parallel optimization runs Key additions: - examples/finetune/optimize_dd_for_target_firing_rate.py * Optimize DD neurons, connection probability, drive frequency, gamma shape * Target firing rate mean and standard deviation * Pareto front multi-objective optimization - examples/finetune/optimize_dd_for_target_force.py * Force-based parameter optimization - examples/finetune/extract_isi_and_cv_per_ramps.py * ISI statistics extraction and CV analysis - bash_scripts/run_dd_optimization.sh * Parallel optimization for multiple muscles * Configurable gamma shape ranges Core improvements: - Add DescendingDrive__Pool with gamma process support - DD_Gamma cell class with shape-controlled spike regularity (CV = 1/sqrt(shape)) - Enhanced documentation for gamma process parameters - Optimized alpha MN configs for FDI, TA, VLVM muscles Technical details: - Gamma shape parameter controls temporal regularity of DD inputs - Lower shape (0.5-1.5) produces high CV irregular firing - Higher shape (2-10) produces more regular cortical-like patterns - Supports both Poisson (shape=1) and gamma processes Files modified: - myogen/simulator/neuron/populations/descending_drive.py - myogen/simulator/neuron/cells.py (DD_Gamma class) - Core NMODL files (kdrRL, mAHP, napp) - Examples updated for new DD interface Removed: - debug_napp_attributes.py (obsolete debug file)
Add comprehensive support for optimizing motor neuron membrane noise (Gfluctdv) alongside descending drive parameters, plus automated full pipeline script with proper gamma shape propagation. New features: - Gfluctdv (fluctuating conductance) optimization in DD framework - Full pipeline automation script (run_full_pipeline.sh) - Automatic gamma shape value propagation through muscle names - Skip flags for faster pipeline reruns Key additions: 1. Gfluctdv Optimization Support (optimize_dd_for_target_firing_rate.py) - New --enable-gfluctdv flag to enable membrane noise optimization - Optimize gfluctdv_noise_amplitude (5e-6 to 3e-5 S/cm² range) - Apply Gfluctdv to all motor neuron dendrites during simulation - Save Gfluctdv parameters in optimization results JSON - Study naming includes "gfluctdv_" marker when enabled 2. Full Pipeline Automation (bash_scripts/run_full_pipeline.sh) - Automatically runs all 5 pipeline steps in sequence - Accepts --gamma-shape-min and --gamma-shape-max parameters - Correctly propagates gamma suffix to all muscle names - Supports --enable-gfluctdv for combined variability studies - Skip flags: --skip-dd-optimization, --skip-force-computation, etc. - Custom output format: --output-format (jpg, png, pdf, svg) - Progress tracking with step counters and status messages 3. Enhanced DD Optimization Script (bash_scripts/run_dd_optimization.sh) - New --enable-gfluctdv flag passes through to Python script - New --gfluctdv-noise-min and --gfluctdv-noise-max options - Study prefix includes "gfluctdv_" when enabled - Updated help documentation with Gfluctdv examples 4. Documentation Updates - Updated bash_scripts/README.md with full pipeline quick start - Added sth.txt with comparison study examples - Test script for quick Gfluctdv validation Use cases: - Study 1: DD input variability only (high CV gamma shape) ./bash_scripts/run_full_pipeline.sh --gamma-shape-min 0.5 --gamma-shape-max 0.75 - Study 2: Combined variability (DD + Gfluctdv membrane noise) ./bash_scripts/run_full_pipeline.sh --gamma-shape-min 0.5 --gamma-shape-max 0.75 --enable-gfluctdv - Study 3: Baseline (regular CV) ./bash_scripts/run_full_pipeline.sh --gamma-shape-min 2.0 --gamma-shape-max 3.0 Technical details: - Gfluctdv implements Ornstein-Uhlenbeck stochastic process - Adds realistic membrane fluctuations (~2-3 mV RMS baseline) - Noise amplitude optimization range: biological 5e-6 to 3e-5 S/cm² - Default correlation time: τ = 20 ms (not optimized) - When enabled, adds 5th optimization parameter to Optuna search Files modified: - examples/finetune/optimize_dd_for_target_firing_rate.py - bash_scripts/run_dd_optimization.sh - bash_scripts/README.md - bash_scripts/sth.txt Files added: - bash_scripts/run_full_pipeline.sh (main automation script) - test_gfluctdv_optimization.sh (quick validation)
CRITICAL BUG FIX: Gfluctdv membrane noise was applied during DD optimization but NOT in subsequent pipeline steps, creating inconsistent simulations. Changes: -------- 1. **compute_force_from_optimized_dd.py** (lines 69-101): - Load gfluctdv_enabled and gfluctdv_noise_amplitude from DD results JSON - Apply Gfluctdv to motor neurons if it was enabled during optimization - Print Gfluctdv status in diagnostic output 2. **optimize_dd_for_target_force.py** (lines 136-142, 544-551, 629-644, 506-508, 516): - Declare GFLUCTDV_ENABLED and GFLUCTDV_NOISE_AMPLITUDE as global variables - Load Gfluctdv settings from baseline DD optimization results - Apply Gfluctdv in run_simulation_and_compute_force() function - Save Gfluctdv parameters in exported results JSON - Print Gfluctdv status in configuration output 3. **extract_isi_and_cv_per_ramps.py** (lines 141-158, 202-211): - Load gfluctdv_enabled and gfluctdv_noise_amplitude from optimization results - Replace commented-out Gfluctdv code with conditional application - Apply optimized Gfluctdv parameters if enabled - Print clear diagnostic messages about Gfluctdv status Impact: ------- When running the full pipeline with --enable-gfluctdv flag: - DD optimization: Optimizes gfluctdv_noise_amplitude parameter - Force computation: Uses the same gfluctdv_noise_amplitude - Force optimization: Uses the same gfluctdv_noise_amplitude - ISI extraction: Uses the same gfluctdv_noise_amplitude This ensures all pipeline steps use identical Gfluctdv settings for consistent and reproducible results. Technical Details: ------------------ - All three scripts now load gfluctdv settings from DD optimization JSON - Gfluctdv applied to motor neuron dendrites using optimized amplitude - Backward compatible: scripts work correctly with/without Gfluctdv - Clear diagnostic output shows Gfluctdv status at each step
The script calls Python directly for plotting (Step 5) but wasn't activating the virtual environment, causing failures if run outside venv. Added venv activation at start of script: - Checks if VIRTUAL_ENV is already set - Activates .venv/bin/activate if needed - Ensures all Python calls have required dependencies This makes the pipeline work correctly when run from any context.
…ookup
BUG: Plots were grayscale because muscle names with gamma suffixes
(e.g., 'THIRTY_gamma0.5-0.75') didn't match colormap dictionary keys
(which only contain short names like 'THIRTY').
When colormap lookup failed, it defaulted to 'Greys' → grayscale plot.
Fix:
----
- Extract short muscle name before colormap lookup (line 378)
Example: 'THIRTY_gfluctdv_gamma0.5-0.75' → 'THIRTY'
- Use short names in legend for cleaner display (line 418)
Now plots correctly use:
- THIRTY → Red ('Reds' colormap)
- TWENTYFIVE → Blue ('Blues' colormap)
- TWENTY → Green ('Greens' colormap)
- FIFTEEN → Purple ('Purples' colormap)
- TEN → Orange ('Oranges' colormap)
- FIVE → Brown ('YlOrBr' colormap)
This works for all muscle naming patterns:
- 'THIRTY'
- 'THIRTY_gamma2.0-3.0'
- 'THIRTY_gfluctdv_gamma0.5-0.75'
- etc.
Extended force optimization to include higher force levels (75% and 90% MVC) in addition to the existing levels (5%, 15%, 30%, 50%). Changes: -------- 1. run_full_pipeline.sh (line 123): - Updated FORCE_LEVELS=(5 15 30 50 75 90) - Pipeline now automatically processes all 6 force levels 2. README.md (line 26): - Updated documentation to reflect new force levels - Step 3 now lists: 5%, 15%, 30%, 50%, 75%, 90% MVC 3. sth.txt (lines 11, 13): - Updated manual command examples with new force levels Impact: ------- When running full pipeline, Steps 3 and 4 will now process: - Step 3 (Force optimization): 6 × 6 = 36 jobs (6 muscles × 6 force levels) - Step 4 (ISI extraction): 6 × 6 = 36 jobs (6 muscles × 6 force levels) Total increase: 12 additional optimization jobs + 12 additional ISI extraction jobs This provides better coverage of the full force range from low (5%) to near-maximal (90%) MVC.
- Created `simulate_iemg.sh` for intramuscular EMG simulation with customizable parameters and output structure. - Created `simulate_semg.sh` for surface EMG simulation with similar customization options. - Enhanced `IntramuscularEMG` class to initialize motor units more efficiently and handle empty motor units. - Updated noise addition methods in both `IntramuscularEMG` and `SurfaceEMG` classes to apply SNR independently across electrode channels. - Improved handling of motor unit processing in `SurfaceEMG` to ensure consistent normalization and output for all motor units. - Added `noise` dependency to `pyproject.toml` and `uv.lock` for noise generation in simulations.
…core plot - Implemented `14_compare_iemg_semg_decomposition.py` to visualize decomposition performance metrics including F1 Score, PNR, match rates, and precision vs sensitivity. - Created `15_plot_f1_comparison_styled.py` for a standalone, publication-ready F1 score comparison plot with enhanced aesthetics and styling.
- Gfluctdv.mod: Cleaned up parameter formatting and removed unnecessary blank lines. - L_Ca_inact.mod: Adjusted spacing for parameters and comments for clarity. - gammapointprocess.mod: Standardized spacing in parameter definitions. - gh.mod: Reformatted code for better readability and added missing newline at end of file. - izap.mod: Improved parameter alignment and spacing for consistency. - kdrRL.mod: Reformatted parameter definitions for better readability. - mAHP.mod: Cleaned up spacing and alignment in parameter and assigned sections. - muscle_unit.mod: Adjusted spacing in parameter definitions. - muscle_unit_calcium.mod: Removed unnecessary blank lines for cleaner code. - na3rp.mod: Standardized spacing and alignment in parameter definitions. - napp.mod: Improved formatting of parameters for better readability. - naps.mod: Cleaned up spacing and alignment in parameter definitions. - nsloc.mod: Adjusted spacing for parameters and comments for clarity. - vecevent.mod: Standardized spacing and formatting in procedures and destructor.
…tests - Introduced a new Cython module for simulate_fiber_v2, significantly improving performance (5-10x speedup). - Updated _spindle.pyx to expose integrate method with a return type. - Fixed a typo in mAHP.mod regarding the tau variable. - Created profile_fiber.py for profiling the Python version of simulate_fiber_v2. - Added test_cython_fiber.py to validate the outputs of the Cython implementation against the Python version.
…idation, and enhance plotting functionality - Updated current generation functions to use nanoamperes (nA) instead of microvolts (uV). - Introduced a new helper function `_broadcast_and_validate_float` for validating float parameters. - Adjusted parameter types and documentation in `create_sinusoidal_current`, `create_sawtooth_current`, `create_step_current`, `create_ramp_current`, and `create_trapezoid_current` functions. - Enhanced plotting functions in `force.py` to highlight specific motor units (MUs) with default matplotlib colors. - Removed profiling and validation test scripts for Cython implementation as they are no longer needed.
- Updated `GolgiTendonOrganModel` and `SpindleModel` to accept `Quantity__ms` for simulation time and time step parameters. - Adjusted internal calculations to use the magnitude of Quantity types where necessary. - Removed unnecessary float type hints and improved type safety. - Cleaned up code formatting and removed redundant lines for better readability. - Deleted `cortical_inputs.py` as it was no longer needed. - Minor adjustments in `mAHP.mod` and `config_loader.py` for consistency and clarity.
…lities - Updated comments and variable names to use 'um' instead of 'μm' for consistency across muscle and neuron models. - Adjusted synaptic weight and threshold parameters to use quantities for better unit handling in the neuron network. - Introduced ContinuousSaver class for efficient data saving during long simulations, reducing memory usage by saving data in chunks. - Added functions to load and combine saved chunks into a single dataset, and convert chunk data to NEO Block format for compatibility with existing analysis tools.
…ysis - Deleted `10f_input_vs_output_spectrum.py` which contained the input vs output spectrum analysis. - Removed `README_CONTINUOUS_SAVING.md` that provided instructions for continuous saving during long simulations. - Eliminated `convert_existing_spikes.py` which was used to convert spike data to the new chunks format. - Removed `01_multiprocessing_spike_trains.py` demonstrating multiprocessing for motor unit simulations.
…thon to 3.2.2, narwhals to 2.13.0, and pandas-stubs to 2.3.3.251201
- Updated the README for the Watanabe spinal network model to include a more concise reference and key findings. - Added detailed descriptions for new parameters in the fiber simulation functions to improve clarity and usability. - Enhanced the SurfaceEMG class to support customizable IAP kernel lengths for more accurate MUAP duration calculations. - Implemented a fallback mechanism in the Cython fiber simulation to handle unsupported parameters gracefully. - Improved logging messages for better user feedback during the search for the mknrndll.bat file. - Cleaned up plotting functions for better readability and consistency in titles. - Added type annotations and documentation for various utility types to enhance code maintainability. - Updated project dependencies in pyproject.toml and uv.lock to include sphinx-hoverxref for improved documentation capabilities.
…parameters - Introduced a new example script `10_extract_data_from_neo_blocks.py` demonstrating data extraction from various Neo Block types including SPIKE_TRAIN, SURFACE_MUAP, SURFACE_EMG, INTRAMUSCULAR_MUAP, and INTRAMUSCULAR_EMG. - Updated `alpha_mn_default.yaml` to include additional parameters for soma and dendrite configurations, allowing for more flexible model tuning. - Corrected documentation in `muscle.py` to clarify units for muscle length, velocity, and acceleration. - Enhanced `motor_neurons.py` to support negative interpolation for recruitment thresholds, improving the model's adaptability to different neuron configurations.
… are consistent in spike train simulation example
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.