Trim Old Blocks

[BooDoo]

Addresses #59

When a new BlockBatch is fetched in updateBlocks, trim BlockBatches in DB to only include 2 most recent for this BtUser.
Deleting BlockBatch items will clean up Blocks table automatically via cascade.

[jsha]

destroyOldBlocks should only get called when we are finished with a batch and set the 'complete' flag. Otherwise we can wind up with the most recently collected batches being incomplete ones.

[jsha]

BTW, this has to wait on submission until #102 is in, or the delete cascades won't actually work.

Note: If you're running locally you may already have these foreign key constraints from sequelize.sync(). The prod instance doesn't because the associations between models were added after initial DB creation.

[BooDoo]

Moved destroyOldBlocks call to success callback for new BlockBatch being finalized/saved; will wait on review and acceptance of 102 for rebase.

[BooDoo]

Rebased on master (foreign key constraints, store size on BlockBatch, &c)

[jsha]

I would simplify this by leaving out the BtUser.find. Just pass the source_uid to destroyOldBlocks, and it can do a

BlockBatch.findAll({
  where: { source_uid: source_uid }
}

[BooDoo]

For some reason I thought we might want access to the BtUser record in destroyOldBlocks but there's no good case for that. Changed.

[jsha]

One more small thing: Could you check if the offset and order options work for .destroy()? I.e. if you replaced the outermost BlockBatch.findAll with a BlockBatch.destroy(...offset: 2, order: 'id DESC'), would that work?

Also, can you double check the SQL output? By my understanding, the destroy command you have is missing a where clause. But I might misunderstand the destroy() API.

[BooDoo]

Destroy's first argument is a WHERE object, see here

You can't provide offset in LIMIT clause of mySQL DELETE query; best you can do is hack around it using subquery a la:

DELETE FROM `BlockBatches` 
WHERE `id` IN (
  SELECT `id` FROM (
    SELECT `id`
    FROM `BlockBatches`
    WHERE `source_uid` = 99999999
    ORDER BY `id` DESC
    LIMIT 2, 50000
  ) AS ids);

which isn't performant or legible!

[jsha]

Excellent, thanks for looking into it! Merging.