Skip to content
Actor-critic controller for plasma medicine
Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data/ac_plasma_PHYS_Discrete12_noise0.11_m4_dev0.0LINpB_ep100b10000_l2s64_rtg0.99_lr0.005_PlasmaModel_26-02-2019_06-27-27/1
LICENSE
PyPolySample.py
README.md
execute_ac_f18_plasma.py
logz.py
model_files2.py
plasma.py
plot.py
plot_control_seq.py
plot_ensemble_training.py
train_ac_f18_plasma.py

README.md

PlasmaRL

Actor-critic controller of thermal effects of an atomospheric pressure plasma jet

Supporting Code for: Witman, M; Gidon, D.; Graves, D.B.; Smit, B.; Mesbah, A. "Sim-to-real transfer reinforcement learning for control of thermal effects of an atomospheric pressure plasma jet"

  1. Train the RLC: python train_ac_f18_plasma.py PlasmaModel -ep 100 -epext 1 --discount 0.99 -n 3000 -e 1 -l 2 -s 64 -b 10000 -lr 0.005 -ntu 10 -ngsptu 10 --exp_name test

    PlasmaModel : use the plasma model dynamics files
    -ep 100 : number of episode/roll-outs per training epoch
    -epext 1 : number of timesteps at the end which are NOT included
    --discount 0.99 : the discount factor
    -n 3000 : number of training epochs
    -e 1 : number of agents to train starting from different random seeds
    -l 2 : number of hidden layers in actor/critic networks
    -s 64 : nodes per hidden layer
    -b 10000 : number of timesteps to be included in the batch of training data in each epoch
    -lr 0.005 : learning rate
    -ntu 10 : number of target updates (ntu), num. times the critic networks targets are updated per actor update
    -ngsptu 10 : number of gradient steps per target update (ngsptu)
    --exp_name test : data is saved in "./data/ac_test_PlasmaModel_timestamp/

  2. Validate the RLC on the physics model over randomized dynamics swith invivo=True to use the RLC to send messages to a socket): python train_ac_f18_plasma.py PlasmaModel --execute data/ac_test_PlasmaModel_timestamp/1/

  3. If you want to validate the RLC on a live device, set invivo=True in train_ac_f18_plasma.py and modify the destination for your message

  4. The ./data folder ships with the E-RLC in the above publication

The code is quite specific to the application of the paper above since not much focus has yet been placed to generalize it for different physics based models of plasma dynamics. Feel free to contact authors for advice/assistance using the code with different models.

You can’t perform that action at this time.