IQL-PyTorch for Stretchable Neuromorphic Array

The environment is described in environment.yml, and you should also clone and install the custom D4RL fork from this repository.

Training

• Train a single model
  To train a model for the umaze environment, run:

  python main.py --env-name antmaze-umaze-v2 --act relu

• Train multiple models
  To train all three models, use:

  ./train_checkpoints.sh

Pre-trained Models

Pre-trained models are available in the ref_ckpts_0411/ directory.

Evaluation

1. Evaluate pre-trained models
   Assuming the checkpoints are in the ref_ckpts_0411/ folder, evaluate the ant agents with:

   ./run_computer_exps.sh

2. Simulate synapse weights
   Generate simulated synapse weights under various conditions by running:

   python generate_simulated_synapse.py

3. Run stretchable ant experiments
   For experiments with stretchable ant agents, execute:

   ./run_stretch_exps.sh