Neuro-Orbital Engine: Continuous-Control Spiking Neural Networks in a Custom N-Body Physics Environment

Overview

This project is a custom-built, continuous-control simulation engine written entirely in C. It integrates a Spiking Neural Network (SNN) with an N-Body gravitational physics simulator to train a satellite to achieve and maintain a stable orbit around a dynamic central star.

I intentionally avoided high-level game engines and machine learning frameworks. By building the physics and neural architectures from scratch, this project serves as a rigorous exercise in low-level state management, numerical integration, and the implementation of unsupervised reinforcement learning algorithms.

System Architecture

The engine is decoupled into three primary, continuous loops:

1. The Physics Integrator

Instead of relying on standard Euler or Runge-Kutta methods, which tend to leak orbital energy over time, I implemented a 4th-Order Yoshida symplectic integrator. This algorithm evaluates the state of 100 interacting gravitational bodies across multiple sub-steps per frame, preserving the Hamiltonian energy of the system over extended time scales.

2. The Neural Reservoir

The satellite is piloted by a custom Leaky Integrate-and-Fire (LIF) neural network. The network takes continuous 1D sensory inputs (radial distance error and velocity) and encodes them into spike trains. I implemented a custom Reward-Modulated Hebbian Gaussian Spike-Timing-Dependent Plasticity (HG-STDP) algorithm. This allows the network to dynamically update its synaptic weights in real-time, learning orbital control through continuous environmental feedback rather than pre-calculated backpropagation.

3. The Renderer

A lightweight visualizer utilizing Raylib renders the multi-body system and overlays real-time telemetry, including control effort, orbital error, and spiking frequencies, directly from the memory buffers.

---

Engineering Challenges and System Realities

The most significant engineering work in this project occurred at the intersection of orbital mechanics and machine learning. Strapping a raw neural network to a highly sensitive dynamical system exposed several complex edge cases. Documenting these failures and their solutions was the core focus of the development process. there are a lot of issues in this right now but i hope i can improve them.

---

The Custom HG-STDP Algorithm: Breaking Symmetry in 3D Space

Standard Spike-Timing-Dependent Plasticity (STDP) is fundamentally designed for discrete, associative learning, making it highly unstable for continuous-control problems like orbital mechanics. To bridge the gap between spiking logic and continuous 3D physics, I implemented a custom Reward-Modulated Hebbian Gaussian STDP (HG-STDP).

This required two major mathematical departures from traditional SNN architecture:

1. Gaussian Continuous Reward Signal

Initial implementations attempted to reward the network by minimizing the temporal derivative of the system's Hamiltonian energy ($\frac{dH}{dt}$). This inadvertently trained the network to act as a continuous brake, bleeding energy until the satellite crashed into the central star.

To correct this, I designed a spatially driven Gaussian reward function that provides a smooth, continuous gradient peaking exactly at the target orbital radius ($R_{target}$):

$$R_{raw} = \exp\left( - \frac{(r - R_{target})^2}{2\sigma^2} \right) - 0.5$$

This raw reward is then normalized and passed through a low-pass temporal filter to create a stabilized reward trace ($\bar{R}$), preventing the network from chasing high-frequency orbital jitter:

$$\tau_R \frac{d\bar{R}}{dt} = -\bar{R} + \tanh\left(\frac{R_{raw}}{R_{scale}}\right)$$

2. Stochastic Eligibility Traces (Symmetry Breaking)

The most critical mathematical challenge was the "Diagonal Lock" phenomenon. In a standard continuous STDP model, a firing neuron updates its downstream synapses uniformly. Because the satellite has three output thrusters ($X, Y, Z$), a uniform weight update mathematically locked the thrust vectors onto a perfect diagonal ($F_x = F_y = F_z$), completely stripping the satellite of its ability to steer.

To solve this, I decoupled the eligibility trace updates by introducing a uniform stochastic exploration variable, $\eta_j \in [-1, 1]$, independently generated for each specific output dimension $j$. The modified eligibility trace equation becomes:

$$\tau_e \frac{de_{ij}}{dt} = -e_{ij} + (x_i^+ - x_i^-) \cdot S_i \cdot \eta_j$$

Where $S_i \in {0,1}$ is the spike event of the presynaptic neuron, and $x_i^+$ and $x_i^-$ are the respective pre- and post-synaptic exponential traces.

By modulating the trace with dimension-specific noise, the mathematical symmetry was broken. The network was forced to test independent directional thrusts, evaluating their success against the temporally delayed Gaussian reward trace. The final synaptic weight update is then governed by:

$$\Delta w_{ij} = \alpha \cdot \bar{R} \cdot e_{ij} - \lambda \cdot w_{ij}$$

Where $\alpha$ is the learning rate and $\lambda$ acts as homeostatic weight decay, ensuring the thrusters power down when the satellite reaches the stable target orbit.

Technical Implementation Details

• Language: C (C99)
• Libraries: Raylib, Math.h
• Core Concepts Applied: Manual memory management, matrix and vector mathematics, dynamical systems theory, unsupervised learning, and hardware-agnostic computation.

Build Instructions

Ensure you have the GCC compiler and Raylib installed and configured on your system.

gcc main.c physics.c neural.c encoder.c reservoir.c -o neuro_orbital -O3 -lraylib -lm 
./neuro_orbital