Config Structuring

Configuration Structure Guide

Overview

Neuraldecoding uses a hierarchical configuration system using hydra with YAML files organized in a modular structure.

Directory Structure

The structure is similar to this. Each config starts in the experiment_name folder (the folder that feeds into config parser). N.B. submodules can be removed as experiment choice, e.g. you can delete dataset and trainer and keep decoder to just decode data.

configs/
├── experiment_name/
│   ├── config.yaml                 # Main configuration file
│   ├── dataset/
│   │   └── [Waiting for dataset to finalize, todo]
│   ├── decoder/
│   │   └── [decoder_name].yaml
│   └── trainer/
│       └── [trainer_name].yaml

Example:

configs/
├── experiment_name/
│   ├── config.yaml                 # Main configuration file
│   ├── dataset/
│   │   ├── zstruct_monkey.yaml
│   │   ├── nwb
|   |   |   └── monkey_brain.yaml
│   │   ├── xpc
|   |   |   └── monkey_brain.yaml
│   ├── decoder/
│   │   └── decoder.yaml
│   └── trainer/
│       └── trainer.yaml

Main Configuration File

config.yaml

The main configuration file serves as the entry point and used for hydra to compose all configs.

Example:

With directory structured as before, the main config would be

defaults:
  - dataset/zstruct_monkey
  - dataset/nwb/monkey_brain
  - dataset/xpc/monkey_brain
  - decoder/decoder
  - trainer/trainer
  - _self_

dataset:
  date: 2023-04-03
  runs: [3]
  subject: Joker
  nwb:
    save: true

Dataset Configuration File

dataset/stuff

Please ask luis about the configuration. [TODO]

Decoder Configuration File

decoder/[decoder_name].yaml

The decoder configuration yaml file defines the model architecture, stabilization parameters, and file path to the model.

Structure Overview

The decoder configuration consists of three main components:

• model: Defines the model and parameters
• stabilization: Configures stabilization [NOT IMPLEMENTED SO ITS JUST PLACEHOLDER]
• fpath: Specifies the file path for saved models

Configuration Parameters

model:

• name: String identifier for the model type (e.g., "lstm", "transformer", "linear")
• parameters: Dictionary containing model-specific hyperparameters
• input_shape: List defining the input tensor dimensions
• output_shape: List defining the output tensor dimensions

stabilization:

• name: String identifier for the stabilization method
• parameters: Dictionary containing stabilization-specific parameters
• date_0: String timestamp for the base date
• date_k: String timestamp for the target date

fpath:

• fpath: String path where decoder model is saved

Example Configuration

model:
    name: "LSTM" # Names defined in model module
    parameters:
      input_size: 1
      num_outputs: 10
      hidden_size: 64
      num_layers: 2  
      rnn_type: lstm  
      device: cpu  
      hidden_noise_std: 0.0  
      dropout_input: False 
      drop_prob: 0.2  
    input_shape: [1] # input shape in list
    output_shape: [10] # output shape in list
    
stabilization:
  name: "LatentSpaceAlignment"
  parameters:
    dim_red_method: "FactorAnalysis"
    alignment_method: "ProcrustesAlignment"
    ndims: 10
  date_0: "2024-08-23"
  date_k: "2024-11-11"

fpath: "tests/decoder/models/linear_regression_model.pkl" # directory to saved model

Trainer Configuration File

trainer/[trainer_name].yaml

The trainer configuration file defines the trainer including model architecture, optimization settings, scheduling, loss functions, training parameters, and data paths.

Structure Overview

The trainer configuration consists of six main components:

• model: Defines the neural network architecture and model-specific parameters
• optimizer: Configures the optimizor
• scheduler: Sets up learning rate scheduling
• loss_func: Specifies the loss function and its configuration
• training: Contains training loop parameters and settings
• data: Defines paths to training and validation datasets

Configuration Parameters

model:

• type: String identifier for the model architecture
• parameters: Dictionary containing model-specific hyperparameters

optimizer:

• type: String identifier for the optimizer
• params: Dictionary containing optimizer parameters like learning rate and weight decay

scheduler:

• type: String identifier for the learning rate scheduler
• params: Dictionary containing scheduler-specific parameters

loss_func

• type: String identifier for the loss function
• params: Dictionary containing loss function parameters

training

• num_epochs: Integer number of training epochs
• batch_size: Integer batch size for training
• device: String specifying computation device
• print_results: Boolean flag for result printing
• print_every: Integer frequency for printing results

data

• train_data: Dictionary with training data path string
• valid_data: Dictionary with validation data path string

Example Configuration

model:
  type: LSTM  
  parameters:
    input_size: 1
    num_outputs: 10
    hidden_size: 64
    num_layers: 2  
    rnn_type: lstm  
    device: cpu  
    hidden_noise_std: 0.0  
    dropout_input: False 
    drop_prob: 0.2  

optimizer:
  type: Adam
  params:
    lr: 0.001
    weight_decay: 0.0001

scheduler:
  type: StepLR
  params:
    step_size: 10
    gamma: 0.5

loss_func:
  type: MSELoss
  params: {}

training:
  num_epochs: 2
  batch_size: 32
  device: cpu # cuda , if avail 
  print_results: True
  print_every: 5

data:
  train_data:
    path: "tests/trainder/data/train.npz"
  valid_data:
    path: "tests/trainder/data/train.npz"