Spaceship rendezvous
with Reinforcement Learning

Description: The agent, a spaceship orbiting the Earth, wants to make a rendezvous with a target spaceship. The agent has limited fuel. It will try to use it up to get its orbit as close as possible to the target's orbit.

The Environment

Made with PyGame and Poliastro.

Steps of 1 minute
Action in each step is a change in the velocity vector $dv=[x,y,z]$ where $x,y,z\in[-10,10]$.
The game ends when:

• The agent runs out of fuel
• The agent crashed into the earth
• The agent orbit is perfectly(with certain error margin) alinged to the targets one.

An orbit can be described by the following six parameters:

• Semi-major axis, $a$
• Eccentricity, $e$
• Inclination, $i$
• Argument of perigee, $\omega$
• Ascending node, $\Omega$
• True anomaly, $\upsilon$

The orbits difference is measured as the sum of the distances between apogees and perigees.

The reward system gives one point when the orbit gets closer to the target's orbit and minus one when it gets further from it.

Proximal Policy Optimization algorithm (PPO)

Before and after training: