heli-gym 🚁

OpenAI GYM environment for 6-DOF Helicopter simulation.

This is GYM environment package for reinforcement learning for helicopter flight tasks using minimum complexity helicopter model.

---

Environment Details

• Pure Gym environment
• Realistic Dynamic Model based on Minimum Complexity Helicopter Model (Heffley and Mnich) In addition, inflow dynamics are added and model is adjusted so that it covers multiple flight conditions.
• Rendering is done by OpenGL.

---

Action Space

Num	Act	Unit	Min	Max
0	Collective		-1	1
1	Lon. Cyclic		-1	1
2	Lat. Cyclic		-1	1
3	Pedal		-1	1

---

Observation Space

Num	Obs	Unit	Min	Max
0	power	hp	0	∞
1	lon. air speed	ft/s	-∞	∞
2	lat. air speed	ft/s	-∞	∞
3	down air speed	ft/s	-∞	∞
4	north velocity	ft/s	-∞	∞
5	east velocity	ft/s	-∞	∞
6	descend rate	ft/s	-∞	∞
7	roll angle	rad	-π	π
8	pitch angle	rad	-π	π
9	yaw angle	rad	-π	π
10	roll rate (body)	rad/s	-∞	∞
11	pitch rate (body)	rad/s	-∞	∞
12	yaw rate (body)	rad/s	-∞	∞
13	x loc (earth)	ft	-∞	∞
14	y loc (earth)	ft	-∞	∞
15	sea altitude	ft	0	∞
16	ground altitude	ft	0	∞

---

Tasks

For now only one task is available. However, rewarding is not tested well yet.

Environment	Details
HeliHover-v0	Hover
HeliForwardFlight-v0	Forward Flight
HeliObliqueFlight-v0	Oblique Flight

---

Tested OS

Environment tested on these OSs. If you have any problem, probably shared libraries for rendering make it, please look at renderer page.

Environment should be run at least 100 FPS to simulate helicopter precisely. Receiving max FPS with NVIDIA 1070-TI with Intel i7-8700K given in the table. Limiting factor of FPS is CPU because of calculation of dynamics of helicopter in Python. If system cannot reach 100 FPS, please make sure that GPU driver be installed and make sure that CPU can handle with the Python.

Tested OS	Max FPS
Windows 10	350
Ubuntu 18.04	470
Ubuntu 16.04	500

For the user who has lower than the version of Ubuntu 16.04 (like Ubuntu 14.04 etc.) or other Linux distros, please re-compile dependent libraries. Re-compile instruction can be found at renderer page.

---

Setup

Run following command.

pip install -e .

---

Usage

Create environment by either,

from heligym import HeliHover
env = HeliHover()

or

import gym
import heligym
env = gym.make("HeliHover-v0")

The rest is usual as of any GYM environment !

TODO List

To do list, ordered according to importance order.

• Fix rotor rotation rendering problem
• Make terrain height map input generic
• Place all helicopter related resources into single folder
• Add turbulence model (probably Dryden Turbulence Model)
• Convert all dynamics related scripts into C++ code with Python api.

To do list for Reinforcement Learning tasks.

• Read http://avstop.com/ac/basichelicopterhandbook/ch11.html
• Add hover task
• Add forward flight task
• Add oblique flight task
• Add turning flight task
• Add ground task
• Add slalom maneuver task