Configuration and Experiments

Configuration and experiments

Hydra composes the root configuration from focused groups under src/spiking_rl_lab/configs/.

Configuration groups

Group	Purpose
agent/	Algorithm and its policy/network parameters
env/	Environment backend and Gymnasium settings
networks/	Reusable node sequences
runner/	Mode, seed, paths, DagsHub repository
trainer/	skrl trainer settings
optuna/	Planned search space
experiment/	Complete configurations retained as reproducible baselines

config.yaml should remain a composition root. Put reusable defaults in their group and complete configurations worth retaining in experiment/.

New agents, environments, networks, runners, trainers, and experiment baselines should be added as focused files in the corresponding group.

Override rules

Select a group with its group name:

uv run spiking-rl-lab env=cartpole_v1

Override a leaf with its full dotted path:

uv run spiking-rl-lab \
  env.params.n_envs=8 \
  agent.params.rollouts=128 \
  trainer.params.timesteps=100000

Select an experiment first, then add machine-specific or temporary overrides:

uv run spiking-rl-lab \
  experiment=reinforce_cartpole_ann \
  agent.params.device=cpu \
  runner.seed=7

Use --cfg job to verify composition before a long run.

Root configuration

The typed BaseConfig has five sections:

• env: registered backend plus backend parameters
• agent: registered algorithm plus all algorithm, network, and policy parameters
• runner: execution mode, reproducibility, output, checkpoint, and tracking
• trainer: trainer selection and skrl trainer parameters
• optuna: planned optimization settings.

Unknown or missing constructor parameters are rejected when a configured component is built.

Network and policy configuration

A policy network is an ordered list of registered nodes:

policy_network:
  nodes:
    - name: linear
      params:
        out_features: 64
    - name: torch_activation
      params:
        activation: relu
    - name: linear
      params:
        out_features: 2

The last dimension must equal flatdim(action_space). For CartPole that is two categorical logits. A minimal spiking hidden layer can be expressed as:

policy_network:
  nodes:
    - name: linear
      params:
        out_features: 64
    - name: lif
      params: {}
    - name: linear
      params:
        out_features: 2
policy:
  name: categorical
  params: {}

reinforce_cartpole_snn is the retained CartPole SNN baseline. It applies population_code to each normalized observation feature. Population coding requires inputs centered around zero with a scale of roughly one, so this preset uses RunningStandardScaler. The number of Gaussian-tuned neurons per feature and their width are configured by neurons_per_feature and sigma. The remaining rollout, truncated-BPTT, entropy, preprocessing, environment-parallelism, and training-budget settings are kept explicitly in the preset for reproducibility.

Available standard activations are relu, silu, sigmoid, and tanh. See Algorithms for policy/action-space compatibility and spiking state behavior.

Environments

The Gymnasium backend accepts:

name: gymnasium
params:
  id: CartPole-v1
  n_envs: 4
  render: false

n_envs: 1 creates a normal Gymnasium environment. Higher values create a synchronous vector environment. The result is wrapped for skrl in both cases.

An environment preset only describes the environment. It does not automatically change the agent's policy type or network output size.

Runner modes

Mode	Status	Behavior
train	Implemented	Train, log artifacts, and evaluate the best checkpoint if present
evaluate	Implemented	Load an optional checkpoint and run for eval_timesteps
optimize	Not implemented	Currently raises NotImplementedError

trainer.use_parallel selects skrl's ParallelTrainer. Otherwise the project uses SequentialTrainer.

Reproducibility checklist

For results that must be repeatable:

1. add a complete file under configs/experiment/
2. set runner.deterministic: true and retain the seed
3. keep environment count, policy, network, and preprocessing unchanged
4. retain uv.lock
5. record the Git commit and any working-tree diff
6. use the composed .hydra/config.yaml when loading a checkpoint.

The runner automates items 5 and 6 for normal training runs.