I had pruning set at one million events, and recently I noticed that I had about 1.6 that were showing up. The daily jobs seem to run without issue (I was getting reports, etc). I forced the daily cache job to run manually and it proceeded to delete 600k event records. I took a look at the database tables and it looks like the tables data, tcphdr, and iphdr don't get cleared out. I had about 34641618 rows in my data table. I manually ran the following SQL commands to clean up the database:
DELETE FROM data USING data LEFT OUTER JOIN event USING (sid,cid) WHERE event.sid IS NULL;
DELETE FROM iphdr USING iphdr LEFT OUTER JOIN event USING (sid,cid) WHERE event.sid IS NULL;
DELETE FROM tcphdr USING tcphdr LEFT OUTER JOIN event USING (sid,cid) WHERE event.sid IS NULL;
I tried looking for the piece of code that had the SQL commands for pruning, but I couldn't find it. It comes down to two issues, pruning didn't seem to be running, and when run manually it left data in the data, iphdr, tcphdr tables. It might leave data in the udphdr and icmphdr tables as well, I'm not monitoring that traffic with these sensors so I can't say for certain.
Snorby is keeping the events at 500k now, so it looks like the issue with the daily cache job is gone. It looks like the data in the data, iphdr, and tcphdr tables still remains even though the associated event is deleted. I've been looking for the code that handles this but I haven't been able to find it, can you point me in the right direction? I'd be glad to help where I can.