Battlesnake engine hosted in other regions

[TheApX]

Even though FAQ says that the game engine runs in AWS US-WEST-2 and everyone can host nearby, having engine in only one region is not inclusive. Developers in North America can host their snakes on their own hardware at home and get reasonable latency (order of 100 ms). But developers from other countries experience much higher latency when hosting at home, up to the point where there is no time left to do any heavy computations.

Having game engine replicas in other regions (such as Europe and Asia) would reduce latency for developers outside North America.

I understand that it is might be challenging to run a distributed engine that allows for inclusive games with fair latency for all regions. But I have a simple idea that might solve the problem.

The engine can still run in a single region, but delegate latency calculation to workers in different region. My (limited) understanding lets me assume that engine currently sends requests directly to battlesnakes, and measures latency. I suggest to run smart proxy servers in different regions that measure and report latency. So instead of sending requests directly, the engine will send requests to proxies closest to battlesnakes, those proxies will send request to battlesnake, measure latency, and report latency back to the engine along with the battlesnake response. I'm not sure if it is possible with zero coding using open source software, but it is definitely simple to implement and wouldn't need much computation resources. The closest proxy can be chosen manually in battlesnake settings or automatically based on ping when the battlesnake is created/refreshed.

[xtagon]

Makes sense, I think people have proposed this before.

One note, the proposed solution could be exploited by some players, because battlesnakes don't have to be stateless, they can use the time in between turns to continue computing.

[TheApX]

It will be the same if the engine waits for all responses in between turns. And it has to, otherwise how would it update the game board in between turns?

Let's consider the following situation:

• A game with two snakes:
  • Snake C(lose) is running in AWS US-WEST-2
  • Snake D(distant) is running at a house in Perth, Australia (the opposite side of the globe)
• Time needed for a packed to reach from engine to
  • Snake C is 0 ms
  • Snake D is 300 ms
• Time needed for a packed to reach engine from
  • Snake C is 0 ms
  • Snake D is 200 ms
• So ping is
  • 0 ms for Snake C
  • 500 ms for Snake D

The engine initiates Move 1 at moment T. It reaches

• Snake C at moment T+0ms
• Snake D at moment T+300ms

Timeouts happen at

• T+500ms for Snake C
• T+800ms for Snake D

The engine now waits for all responses. It will get response

• from Snake C at T+500ms
• from Snake D at T+1000ms

The engine initiates Move 2 immediately after the last response, i.e. at T+1000ms. It reaches

• Snake C at T+1000ms.
• Snake D at T+1000ms+300ms = T+1300ms.

Both snakes observe a full cycle now.

• Snake C started computations at T and got the second move at T+1000ms = 1000ms later.
• Snake D started computations at T+300ms and got the second more at T+1300ms = 1000ms later.

I.e. both snakes had the same 1000ms for calculations based on information from Move 1. Despite getting this information at a different moment they still have the same amount of time for calculations.

This does slow down the whole game, giving all snakes more time for calculations. If both snakes were running in AWS US-WEST-2, full cycle would take only 500ms and both snakes would have only 500ms total per turn. But it's not like one snake gets 1000ms, and another one gets only 500ms.

[xtagon]

Most snakes will be stateless and those who use stateful in-between-turn computations would be at a greater advantage. It's technically possible to do that now, but it's not a problem because the advantage is very small. IMO if it was on the order of another few hundred ms, it would change the incentives for snake request/response architecture.

[TheApX]

Yes, it is an advantage, but it's vague. Most snakes don't even use 500ms they get now, and snakes that do are at a greater advantage already. There will always be different snakes, some won't use much, some will use every possible chance to improve their behavior. That's why we have tiers, so that entry-level snakes don't compete with advanced snakes by experienced developers.

IMHO as long as everybody gets the same opportunities, it is fair. Whether to use these opportunities is up to everyone's skill and will. My suggestion is to give everyone the same opportunity to host their snakes wherever they want and get as close to 500ms timeout as possible. But I admit that my understanding of "fair" may be different from yours, and I'm open to discussion.

[xtagon]

@TheApX After re-reading your description, yes it does sound more fair. Thank you for explaining step by step

[robbles]

I think for a snake to exploit the extra time, they would need to not be using one of the proxies themselves, but successfully detect that another snake in the same game is using one. That seems pretty complex and risky as a starting point, and then on top of that they'd need to coordinate the use of the extra time between turns successfully. If someone can pull that off without making their snake unreliable, I think they deserve the advantage.

That said, I think one thing at a minimum that should probably be controlled for is the presence of a proxy related delay in a given game. Even if we're OK with some people taking advantage of it, the ability to exploit it shouldn't depend on which snakes they're facing. If the additional delay is in any game, it should probably be in all games.

[bvanvugt]

Yeah, we love this idea! And as @xtagon suggests it's come up before.

We used to believe that, because anyone can deploy in us-west, it wasn't really an issue. That assumption turned out to be false once EU and APAC devs started using local snakes for dev and finding latencies were impossibly high.

The proxy idea you suggest is actually one of our leading solutions atm, among a few other trickier ideas. Tagging @robbles too, who might have more thoughts.

I'm going to flag this as will fix, although there's no timeline yet.

[TheApX]

Sorry for duplicate, I've looked through the list of discussions and haven't found a similar one.

The assumption is also not correct because AWS seems to offer only AWS Lambda as an Always Free Compute resource. It is available to everyone, but it's a bit too AWS-specific, compared to generic VMs (EC2), which is much more flexible. It's much easier to run a snake on a VM/Desktop/RaspberryPi/etc than AWS Lambda (more flexibility, lower learning curve), so people may choose to self-host. One can host for free on AWS EC2 for 12 months, but it's time-limited and will force to relocate or pay later.

[robbles]

We've shipped this feature! Check out out the blog post and the documentation for more info.

[TheApX]

This is awesome! Congratulations!

I've already tried it, and it's great! Apparently, the default region works best for my snakes, but I'm glad that we can get more international competition.

[robbles]

Thanks! Just out of curiosity, where are your snakes hosted?

[TheApX]

Oracle Cloud, region us-phoenix-1