Where will we host our blockchain database?!?!?!

[zomglings]

For the launch, in the interests of getting things up and running as soon as possible, we will store our blockchain crawl data on a Postgres instance we set up on Google Cloud Platform.

Pricing for AWS RDS and GCP Cloud SQL instances will come it at more than $400 per month with 4 TB of storage. Since this is quite expensive, we will set up and manage the database on our own Compute Engine instance. We do not need high availability here (for now).

The expected cost of the GCE instance is ~$200 per month. We will use an API instance powered by this database to benchmark alternative storage solutions.

The very first alternative we will test (after Moonstream launch) is to use S3 as the storage layer instead of a live Postgres database. This is something Quickwit has been doing to implement an Elasticsearch-like search engine with S3 as the storage mechanism.

We will see if we can get reasonable API performance using Activeloop's Hub to store and retrieve data from S3.

I will create a separate issue for implementing and benchmarking a Hub-powered API against a Postgres-powered API.