Skip to content
This repository has been archived by the owner on Jul 13, 2023. It is now read-only.

Persistent dynamodb garbage collector #124

Closed
bbangert opened this issue Jul 29, 2015 · 1 comment
Closed

Persistent dynamodb garbage collector #124

bbangert opened this issue Jul 29, 2015 · 1 comment

Comments

@bbangert
Copy link
Member

Create a script that runs persistently on a machine to sweep old/dead data out of DynamoDB.

This script should be runnable by cron, save its place (using dynamodb last-key/start-key for table scans), and watch the dynamodb capacity usage to avoid using up available capacity.

Rough outline of run logic:

  • Get prior start location in router table (perhaps saving the location each run in Dynamodb as well)
  • Router Table scan for N UAID's
  • If the user hasn't connected in M days, check storage/message table and clear old data
  • If the user hasn't connected in Z days, clear out user/storage/message entirely
  • Save last-key from router table scan
@bbangert bbangert added this to the 1.4.0 milestone Aug 4, 2015
@bbangert bbangert self-assigned this Aug 17, 2015
@bbangert bbangert changed the title Create GC script to clear out old messages Persistent dynamodb garbage collector Aug 27, 2015
@bbangert bbangert modified the milestones: 1.4.0, 1.6.0 Aug 27, 2015
@bbangert bbangert removed this from the 1.6.0 milestone Sep 10, 2015
@bbangert
Copy link
Member Author

Dumping in favor of table rotation.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant