🔍 Implement multiplayer micro-stack

[Terkwood]

Rationale

In the current multiplayer implementation, the first person to visit the site in a given period has to endure an impossibly long wait time (75 seconds+) while the bigger box with Kafka starts up. Having a tiny little implementation dependent on Redis Streams can solve this.

RAM footprint of the total system will decrease by about 100x.

Our goal is to sunset the Kafka implementation.

Write the following

• micro-judge (⚖ Experimental: redis/rust judge #133)
• micro-changelog (Implement micro-changelog #175)
• micro-game-lobby (Add micro game lobby #184)
• micro-color-chooser (Add micro-color-chooser #334)
• micro-history-provider (also required by 📮Reconnect with AI play  #238, PR Add micro history provider #331 )
• micro-sync Implement micro-sync #333
• simplify backend extension for gateway (also see Gateway: send commands to both backends #205)
• Gateway: Implement fake PROVIDLE flow #368 fake the providle / jvm-startup interaction
• 🏞 Gateway: coordinate multi-player events using redis streams #332 gateway needs to coordinate events using redis streams,...
• potentially using shared libs defining common types (✂ Deal with duplicated model definitions  #339)

Testing and verification

• Multiplayer micro-stack integration: functional testing #362 functional testing
• Verify make move payload in micro-sync  #352 check make move payload across all boundaries

Found work

• fix botlink serialization woes caused by this effort Fix botlink serialization #405
• carefully check updated DC.yml against tiny
• fix gateway r2d2 extension error gateway: fix r2d2 extension error #401
• fix botlink regression Fix regression in botlink #400
• fix read error in micro-color-chooser Fix read error in micro-color-chooser #365
• check into session abandonment in game lobby Verify session abandonment in game lobby  #370
• make sure colorschosen event propagates back to client Fix propagation of ColorsChosen #371
• ensure game state created correctly and visible to judge Ensure game state created correctly and visible to judge #378
• fix private game rejection message Fix private game rejection message #391

Admin cleanup

• Post-Kafka admin cleanup, buildkit support  #374 clean up admin scripts
• set gateway, micro-judge, micro-changelog, botlink release versions

Deferred work

We don't need the participation service ported immediately, as it was never actually finished for the old JVM stack. Here's the ticket related to that work: #325

We can delete Kafka and all related apps, once this ticket is complete. See #229

Memory usage tracking

With micro-judge v0.1.1

 docker stats --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}"

NAME                           CPU %               MEM USAGE / LIMIT          
bugout_reverse-proxy_1         0.00%               5.852MiB / 62.77GiB
bugout_micro-judge_1           0.00%               4.223MiB / 62.77GiB
bugout_micro-color-chooser_1   0.00%               4.156MiB / 62.77GiB
bugout_micro-changelog_1       0.00%               4.859MiB / 62.77GiB
bugout_micro-sync_1            0.00%               4.297MiB / 62.77GiB
bugout_gateway_1               0.00%               4.5MiB / 62.77GiB
bugout_micro-game-lobby_1      0.00%               4.543MiB / 62.77GiB
bugout_redis_1                 0.05%               9.043MiB / 62.77GiB