Join GitHub today
GitHub is home to over 20 million developers working together to host and review code, manage projects, and build software together.
Pruning Issues #202
I had pruning set at one million events, and recently I noticed that I had about 1.6 that were showing up. The daily jobs seem to run without issue (I was getting reports, etc). I forced the daily cache job to run manually and it proceeded to delete 600k event records. I took a look at the database tables and it looks like the tables data, tcphdr, and iphdr don't get cleared out. I had about 34641618 rows in my data table. I manually ran the following SQL commands to clean up the database:
DELETE FROM data USING data LEFT OUTER JOIN event USING (sid,cid) WHERE event.sid IS NULL;
I tried looking for the piece of code that had the SQL commands for pruning, but I couldn't find it. It comes down to two issues, pruning didn't seem to be running, and when run manually it left data in the data, iphdr, tcphdr tables. It might leave data in the udphdr and icmphdr tables as well, I'm not monitoring that traffic with these sensors so I can't say for certain.
This comment has been minimized.
This comment has been minimized.Show comment Hide comment
Snorby is keeping the events at 500k now, so it looks like the issue with the daily cache job is gone. It looks like the data in the data, iphdr, and tcphdr tables still remains even though the associated event is deleted. I've been looking for the code that handles this but I haven't been able to find it, can you point me in the right direction? I'd be glad to help where I can.