Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
How to Age-Out (Purge Old) Data? #45
I am looking at integrating OpenTAXII as a source of threat intel data for the default deployment of Apache Metron. In most cases, I envision Metron users leveraging their own existing aggregator platform that would be used instead. Although if none exists, I am hoping that OpenTAXII can fill that role.
I have some logic that will likely be added to Metron that deploys OpenTAXII and can sync with some external 3rd party feeds. If I setup a mechanism that continually syncs external threat intel feeds, I need to ensure that the data store size doesn't grow without bound. That's what I need.
Are you planning of running this using the SQLite database? Then all you can do is implement a cleanup routine inside the persistence layer (and possible in the manager), which runs periodically any cleanup task. As @traut mentioned, this is currently not the case.