Configuration and Experiments

Configuration and experiments

Hydra composes the root configuration from focused groups under src/spiking_rl_lab/configs/.

Configuration groups

Group	Purpose	Current entries
agent/	Algorithm and its policy/network parameters	reinforce
env/	Environment backend and Gymnasium settings	cartpole_v1, pendulum_v1, ant_v5
networks/	Reusable node sequences	reinforce_mlp
runner/	Mode, seed, paths, DagsHub repository	base
trainer/	skrl trainer settings	base
optuna/	Planned search space	base
experiment/	Complete configurations retained as reproducible baselines	reinforce_cartpole_ann

config.yaml should remain a composition root. Put reusable defaults in their group and complete configurations worth retaining in experiment/.

The entries above are the presets currently present in the repository, not a fixed set. New agents, environments, networks, runners, trainers, and experiment baselines should be added as focused files in the corresponding group.

Override rules

Select a group with its group name:

uv run spiking-rl-lab env=cartpole_v1

Override a leaf with its full dotted path:

uv run spiking-rl-lab \
  env.params.n_envs=8 \
  agent.params.rollouts=128 \
  trainer.params.timesteps=100000

Select an experiment first, then add machine-specific or temporary overrides:

uv run spiking-rl-lab \
  experiment=reinforce_cartpole_ann \
  agent.params.device=cpu \
  runner.seed=7

Use --cfg job to verify composition before a long run.

Root configuration

The typed BaseConfig has five sections:

• env: registered backend plus backend parameters
• agent: registered algorithm plus all algorithm, network, and policy parameters
• runner: execution mode, reproducibility, output, checkpoint, and tracking
• trainer: trainer selection and skrl trainer parameters
• optuna: planned optimization settings.

Unknown or missing constructor parameters are rejected when a configured component is built.

Network and policy configuration

A policy network is an ordered list of registered nodes:

policy_network:
  nodes:
    - name: linear
      params:
        out_features: 64
    - name: torch_activation
      params:
        activation: relu
    - name: linear
      params:
        out_features: 2

The last dimension must equal flatdim(action_space). For CartPole that is two categorical logits. A minimal spiking hidden layer can be expressed as:

policy_network:
  nodes:
    - name: linear
      params:
        out_features: 64
    - name: lif
      params: {}
    - name: linear
      params:
        out_features: 2
policy:
  name: categorical
  params: {}

Available standard activations are relu, silu, sigmoid, and tanh. See Algorithms for policy/action-space compatibility and spiking state behavior.

Environments

The Gymnasium backend accepts:

name: gymnasium
params:
  id: CartPole-v1
  n_envs: 4
  render: false

n_envs: 1 creates a normal Gymnasium environment. Higher values create a synchronous vector environment. The result is wrapped for skrl in both cases.

An environment preset only describes the environment. It does not automatically change the agent's policy type or network output size.

Runner modes

Mode	Status	Behavior
train	Implemented	Train, log artifacts, and evaluate the best checkpoint if present
evaluate	Implemented	Load an optional checkpoint and run for eval_timesteps
optimize	Not implemented	Currently raises NotImplementedError

trainer.use_parallel selects skrl's ParallelTrainer. Otherwise the project uses SequentialTrainer.

Reproducibility checklist

For results that must be repeatable:

1. add a complete file under configs/experiment/
2. set runner.deterministic: true and retain the seed
3. keep environment count, policy, network, and preprocessing unchanged
4. retain uv.lock
5. record the Git commit and any working-tree diff
6. use the composed .hydra/config.yaml when loading a checkpoint.

The runner automates items 5 and 6 for normal training runs.