Evolutionary Tank Game : RL & Genetic Programming Sandbox

A top-down 2D combat simulator (Python, Pygame) built as a testbed for comparing three agent architectures: DQN-style neural networks, genetic-programming decision trees, and random baselines. A separate genetic algorithm evolves each agent's physical morphology (speed, size, awareness radius, fire cooldown, bullet speed) across generations.

Originally built for ECE 848 (Evolutionary Computation). Full methodology and results write-up: Final_Project_Report.pdf.

40 agents spawn per generation (15 neural network, 15 genetic programming, 10 random), fight it out across 3 hand-built maps, and are scored on kills, survival, and ammo pickups. The top performers in each category are bred into the next generation's population, so both behavior and body evolve over time.

How it works

Perception & actions: Each agent perceives the world within a genetically-tuned awareness_radius and, every frame, chooses one of 5 actions: move forward/backward, rotate left/right, or shoot.

Behavior is learned two different ways:

• Neural network agents: (agent_brain.py) Use DQN feed-forward neural networks (model.py) to adapt within a generation, from moment to moment.
• Genetic programming agents: encode decisions as expression trees that only change between generations via subtree mutation (no crossover).
• Random agents: serve as a control for both of the above.

Morphology evolves independently via a genetic algorithm (genome.py): after every 100 deaths across the population, the top 2 performers in each of 3 fitness categories (kills, survival, pickups) are selected per algorithm family, paired up, and bred with configurable crossover/mutation probabilities into 30 children, topped up with 10 fresh random agents to keep genetic diversity flowing in.

Results overview

Across 30 generations:

• Kills: neural networks led early (they can adapt mid-generation), genetic programs caught up and briefly overtook them by the midpoint, and NNs reclaimed the lead once they adapted to the improved GP opponents.
• Deaths: GP agents were consistently better at survival than NN agents, but neither showed strong improvement over generations, pointing to a design limitation (bullet speed vs. agent reaction time, or map crowding) rather than a learning failure.
• Pickups: NN agents learned to collect ammo dramatically faster than GP agents.
• Outcome: survival correlated strongly with morphology (small + fast builds died less), while kill count was driven far more by behavior than by body. By generation 29, the population had split into two distinct morphological niches: small/fast vs. large/aware. This is directly visible in the size-gene distribution.

Full charts, correlation tables, and discussion are in the project report.

Setup

pip install -r requirements.txt
python main.py

Press Tab during a run to toggle the awareness-radius debug overlay. Per-generation telemetry (kills, deaths, pickups, gene values) is written automatically to logs/<timestamp>/gen_N.csv.

Project structure

main.py            # Game loop, rendering, physics, collision, GA selection/breeding, map loading
agent_brain.py      # DQN agent: replay buffer, epsilon-greedy policy, train/target network sync
model.py            # MLP policy/target network used by agent_brain.py
genome.py           # MorphologyGenome: crossover, mutation, clamping for physical traits
map1.txt/2/3        # ASCII-encoded level layouts (# = wall, space = boundary gap)
logs/<run>/          # Per-generation CSV telemetry (kills, deaths, pickups, gene values)
scores_log.csv       # Aggregated run history
requirements.txt     # Python dependencies

Known issues

• Deaths don't improve much over generations despite morphology being fairly predictive of survival. It is worth investigating whether bullet speed is outpacing reaction time, or maps are too crowded for evasive movement to matter.

Fixed

• Simulation throughput degrading with generation count: This was an issue I had when I wrote the original report, training slowed enough by ~generation 25 that a planned 100-generation run had to be cut short, but this has now been resolved. Wall collision/awareness checks (for w in walls: ...) were scanning every wall on the map for every bullet and every agent, every frame so O(agents × walls) and O(bullets × walls) per frame. Walls now live in a tile-indexed spatial grid (wall_grid / nearby_walls() in main.py), so each check only looks at the handful of grid cells actually near the query rect instead of the whole map. Separately, get_observation() was re-scanning the full tank/bullet/pack lists and redoing the same awareness-radius filtering that update_awareness() had already computed moments earlier in the same frame. That duplicate O(N) pass per agent is gone; it now reuses the cached visibility set. Also dropped a policy_net.eval() call that was re-running on every single NN inference (every agent, every frame) despite the network having no Dropout/BatchNorm layers to make it necessary.