Query CAR database according to epoch

[lemmih]

Issue summary

There are two kinds of data stores in Forest: Read-only CAR files and writeable ParityDB databases. Since all values are uniquely determined by their key, it does not matter for correctness which data store we query first. It does matter for performance, though, and we want to query as few data stores as possible.

We first query the CAR data stores in the order they were added. If a key isn't present in the CAR data stores, we then query the ParityDB database. However, in regular operation, we're significantly more likely to query new data than old data. As such, querying the data stores with the latest data first is a better option. The ParityDB database contains current data and is, therefore, the newest and should be queried first. The CAR data stores should be sorted from highest to lowest by epoch.

• Query ParityDB first.
• Sort CAR data stores by the epoch of the heaviest tipset.
• (Optional) Trim unusable CAR data stores if we know we'll never access their data. This happens if we're evaluating tipset 1000, and we have CAR stores for epochs 0-1500 and 1500-3000. The second CAR store will not be used for evaluating this tipset and can be removed from the list.

Other information and links