Dynamic connections manager

[povilasb]

ETD + review: 2 days

• Currently Crust employs a simple HashMap based connection pool which accommodates as many connections as you ask.

• This is not feasible since file descriptor limits can be exhausted in the system. So instead have a connection pool with maximum connection limit.

• Make default limit 200 but allow to configure this via Crust config.

• When connection pool is full and we want to make a new connection, one has to be evicted. Eviction policy depends on what kind of connections we have: hole punched, TCP, UDP, etc. We will strive for the right balance of mostly used and the hardest to established connections. Currently only TCP direct connections are supported, but keep in mind TCP hole punched connections as well, cause that is soon to come.
  During first iteration eviction policy should be kept simple:

  1. hold on to hole punched connections for as long as possible. Meaning keep removing direct connections from the pool until there are no such;
  2. when direct connection is evicted from the pool, cache its connection in memory since it's relatively easy to reestablish such connection when needed;
  3. when connection pool contains only hole punched connections, remove the least recently used one.

• When establishing connection, retry up to 3 times until it succeeds. The timeout is 120 seconds.