New database manager

[ChihChengLiang]

What was wrong?

Currently, Trinity client creates an extra process for each database consumer, this cause overheads and performance degrade.

How was it fixed?

Write a DB manager that handles all the communication to the database and a DB client that talks to the manager.

Benchmark shows the new DB manager-client has 1.5x to 2x performance comparing to the existing one.

$ python scripts/benchmark/db_managers/bench_db_client.py 
Client 1: 6412.4640859929395 get-set per second
Client 0: 6294.6387453949055 get-set per second
Client 2: 5925.177664645094 get-set per second
Client 1: 6466.534301796517 get-set per second
Client 0: 6419.005003848052 get-set per second
Client 2: 6012.5870262597755 get-set per second
Client 1: 6271.011569056251 get-set per second
Client 0: 6263.727016479054 get-set per second
Client 2: 6715.727595295333 get-set per second
$ python scripts/benchmark/db_managers/bench_old_manager.py 
Client 0: 4235.169438244839 get-set per second
Client 1: 4207.37306063274 get-set per second
Client 2: 4158.8369075974715 get-set per second
Client 0: 3686.812208983483 get-set per second
Client 1: 3679.808669849572 get-set per second
Client 2: 3590.928601338237 get-set per second
Client 0: 4002.483401664311 get-set per second
Client 1: 4020.7755111135643 get-set per second
Client 2: 4116.866458079233 get-set per second

To-Do

Cute Animal Picture

[ChihChengLiang]

Feel free to give rename suggestions and tear them apart.

[pipermerriam]

Here's some preliminary review. If you don't mind, I might take this and re-work it a little to see whether I can find a nice middle ground. The schema.py approach seems like it may add more complexity than is strictly necessary and make the actual server and client implementations a little harder to understand. (probably not something I'll do right away).

[ChihChengLiang]

@pipermerriam Thank you for the review. Addressed the comments.

If you don't mind, I might take this and re-work it a little to see whether I can find a nice middle ground.

Feel free to do so.

[cburgdorf]

Closing as this has landed with #859