DroneControl

Branches

• main: This branch. Modular extention of https://github.com/BhaskarJoshi-01/DroneControl to environments with moving obstacles.
• varying-bias: Modular extention of https://github.com/BhaskarJoshi-01/DroneControl to environments with gaussian noise with time varying bias.

Component Interactions

flowchart LR

%% Define all node components
subgraph NoiseWrapper
subgraph Environment
step["step()"]
stateProcessor["_computeProcessedState()"]
end
noiseGenerator["Noise Generator"]
denoiseEngine["Denoise Engine"]
end

policy["Policy"]

%% Define all connections
step -->|Raw State|noiseGenerator
noiseGenerator -->|Noisy Raw State|denoiseEngine
denoiseEngine -->|Denoised Raw State|stateProcessor
stateProcessor -->|Observation| policy
step -->|Reward| policy
policy -->|Action| denoiseEngine
policy -->|Action| step

Loading

Making an environment config

To make an environment configuration, create a JSON in the configs directory of the following shape:

{
    "xmin": float,
    "xmax": float,
    "ymin": float,
    "ymax": float,
    "zmin": float,
    "zmax": float,
    "provideFixedObstacles": bool,
    "obstacles": List[List[3]] or null,
    "minObstacles": int,
    "maxObstacles": int,
    "dynamicObstacls": bool,
    "randomizeObstaclesEveryEpisode": bool,
    "fixedAltitude": bool,
    "episodeLength": int,
    "showDebugLines": bool,
    "randomizeDronePosition": bool,
    "simFreq": int,
    "controlFreq": int,
    "noiseParameters": {
        "mu": float,
        "sigma": float,
        "denoiseEngine": DenoiseEngineData
    }
}

DenoiseEngineData

The following Denoise methods exist and the ways to add them into the experiment configuration:

1. No Denoiser

To use no denoiser, just set "denoiseEngine" in the config JSON to null.

2. Low Pass Filter

To use a Low Pass Filter as the Denoise Engine, set "denoiseEngine" to the following:

{
    ...
    "denoiseEngine": {
        "method": "lpf",
        "parameters": {
            "order": int,
            "criticalFreq": float,
            "ftype": string
        }
    }
}

• order: Order of the filter. Can be thought of as the window size to consider while denoising the current observation.

• criticalFreq: Critical frequency of the underlying data. If you don't know what it means, just set it to 2.

• ftype: Type of IIR to design. Options are "bessel" for Bessel/Thomson and "butter", for Butterworth.

3. Kalman Filter

To use a Kalman Filter as the Denoise Engine, set "denoiseEngine" to the following:

{
    ...
    "denoiseEngine": {
        "method": "kf",
        "parameters": {
            "processNoise": float
        }
    }
}

• processNoise: Standard Deviation of the process noise.

Training a Model

To train a model, you first need an environment in which it will be trained. For that, make the environment config file as shown above and place it in the configs directory. Then make a Training Configuration File of the following shape:

{
    "taskName": str,
    "envConfigFile": str (path to file relative to configs/),
    "outputModelName": str (model name relative to SBAgent/models/)
}

Check out trainConfigs/trainConfig1.json to see an example.

Once the train configuration file is prepared, use the TrainDispatcher.py script to train the model (usage explained below).

The Scripts

TrainDispatcher.py

Used to train a model based on the given train config file.

python TrainDispatcher.py [-h] [-s STEPS] [--local] [-d DYNAMIC] [-o OBSTACLES] trainConfigPath

• trainConfigPath is the path to the train configuration file.
• -s to specify the number of timesteps to train for. Defaults to 2,000,000.
• Add the --local flag to run the training on your local machine. If this flag is omitted, the script generates a batch job configuration and dispatches it on one of the nodes on IIIT-H's HPC.
• -o to specify the number of obstacles
• -d to specify if the obstacles are dynamic or not

SBAgent/EvaluateModel.py

Used to evaluate an existing model on an environment template. The template allows the user to set the noise parameters and denoiser to be used as arguments instead of having to create a new file for every possible combination of the three. To run it, run:

cd SBAgent
python EvaluateModel.py [-h] [-t TRIALS] [--gui] [--no-gui] [-d DYNAMIC] [-o OBSTACLES] modelPath mu sigma {none,kf,lpf}

• modelPath is the relative path of the model to be evaluated.
• mu is the mean of the noise to be injected into the state space.
• sigma is the standard deviation of the noise to be injected into the state space.
• {none,kf,lpf} are the choices for the denoiser to use.
• -t to specify the number of episodes to evaluate the model for. Defaults to 10.
• --gui to launch the PyBullet window to visualize the agent behavior.
• -d to specify if obstacles are dynamic or not
• -o to specify the number of obstacles.

SBAgent/ParallelEvaluationPipeline.py

Used to perform a series of model evaluations in a single call. Defined in the code is a set of values of $\mu$, $\sigma$ and which denoiser to use, and the file calls EvaluateModel.py on every combination of the three variables, and prints the results of the evaluation. To run it:

cd SBAgent
python ParallelEvaluationPipelineDispatcher.py [-h] [-t TRIALS] [--local] [-o OBSTACLES] [-d DYNAMIC] [-mu MU] [-si SIGMA] modelPath

• modelPath is path to the model to be evaluated.
• -t is the number of episodes each evaluation is done for.
• add the --local flag to run the script locally on your system
• -mu to specify if evaluation should iterate over different values of mean.
• -si to specify if evaluation should iterate over different values of the standard deviation.

Though this script makes a call to EvaluateModel, it supresses all the print statements from that file to get a clean output. Only outputs are of the final evaluation results in table form.

Models

Check out base/ModelLogs.md for the description of every model trained from scratch.

Check out finetuned/ModelLogs.md for the description of every model finetuned from an existing model.