Added authlog purge + improved efficiency of clearing out large syslog tables #301

Merged
merged 3 commits into from Oct 18, 2014

Projects

None yet

2 participants

@laf
Member
laf commented Oct 2, 2014

I've found that with large volumes of syslog data, a delete based on the datetime alone is not enough as MySQL can't complete the transaction in time so it fails. This has led to the build up of over 9m records in my test box alone.

This change now removes data in batches of 1000 without issue. It ran through 250,000 entries in 1 hour so should be fine now.

It also adds the clearing of authlog entries (logs web interface access).

@paulgear
Member
paulgear commented Oct 7, 2014

A couple of issues:

  • Should we be thinking about purging more frequently (e.g. poller or discovery) rather than trying to do this daily? I'm not really sure of the pros & cons.
  • I'm sure there's a much more PHP-literate way of controlling that loop. At the very least we should be using break rather than $loop=0.
@laf
Member
laf commented Oct 7, 2014

I don't think we should do the purging from poller or disco just in case it locks them up.

I'll change the loop = 0 to break and test it.

@laf
Member
laf commented Oct 7, 2014

Updated and tested against local install. Still loops through correctly and removes data when it needs to.

@laf laf added the enhancement label Oct 9, 2014
@paulgear
Member

How does that loop work when $loop is not defined anywhere? Just use while (TRUE) ...

@laf
Member
laf commented Oct 11, 2014

Updated.

@paulgear paulgear merged commit 71a3eb1 into librenms:master Oct 18, 2014

1 check passed

ci/scrutinizer Scrutinizer: 6 new issues
Details
@laf laf deleted the unknown repository branch Apr 30, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment