Skip to content

Commit

Permalink
Add database log expiry back to housekeeping.
Browse files Browse the repository at this point in the history
Clean up database log entries with a daily housekeeping task.  For now,
we're keeping 2 weeks worth of log entries, but we also have a max
number of rows allowed (10K/day = 140K rows).  While this sounds like a
lot, it shouldn't have a major impact on database size--probably adding
up to about 25MiB to the DB size (and about 8MiB for indices).

These values may be adjusted in the future, as we get a better feel for
log message representation within these limits.  Also, we may modify the
expiry algorithm to preferentially expire logging from certain
"less-important" applications (i.e. adding a 1wk expiry for
mythcommflag/mythpreviewgen logging or similar).  However, this
implementation is a good start and will prevent unbounded growth of DB
logs.
  • Loading branch information
sphery committed May 10, 2011
1 parent 0b9625e commit 6b660a1
Show file tree
Hide file tree
Showing 2 changed files with 56 additions and 0 deletions.
55 changes: 55 additions & 0 deletions mythtv/programs/mythbackend/housekeeper.cpp
Expand Up @@ -188,6 +188,14 @@ void HouseKeeper::RunHouseKeeping(void)
// These tasks are only done from the master backend
if (isMaster)
{
// Clean out old database logging entries
if (wantToRun("LogClean", 1, 0, 24))
{
VERBOSE(VB_GENERAL, "Running LogClean");
flushDBLogs();
updateLastrun("LogClean");
}

// Run mythfilldatabase to grab the TV listings
if (gCoreContext->GetNumSetting("MythFillEnabled", 1))
{
Expand Down Expand Up @@ -291,6 +299,53 @@ void HouseKeeper::RunHouseKeeping(void)
}
}

void HouseKeeper::flushDBLogs()
{
int numdays = 14;
uint64_t maxrows = 10000 * numdays; // likely high enough to keep numdays

QDateTime days = QDateTime::currentDateTime();
days = days.addDays(0 - numdays);

MSqlQuery query(MSqlQuery::InitCon());
if (query.isConnected())
{
QString sql = "DELETE FROM logging WHERE msgtime < :DAYS ;";
query.prepare(sql);
query.bindValue(":DAYS", days);
VERBOSE(VB_GENERAL|VB_EXTRA, QString("Deleting database log entries "
"from before %1.").arg(days.toString()));
if (!query.exec())
MythDB::DBError("Delete old log entries", query);

sql = "SELECT COUNT(0) FROM logging;";
query.prepare(sql);
if (query.exec())
{
uint64_t totalrows = 0;
while (query.next())
{
totalrows = query.value(0).toLongLong();
VERBOSE(VB_GENERAL|VB_EXTRA, QString("Database has %1 log "
"entries.").arg(totalrows));
}
if (totalrows > maxrows)
{
sql = "DELETE FROM logging ORDER BY msgtime LIMIT :ROWS;";
query.prepare(sql);
quint64 extrarows = totalrows - maxrows;
query.bindValue(":ROWS", extrarows);
VERBOSE(VB_GENERAL|VB_EXTRA, QString("Deleting oldest %1 "
"database log entries.").arg(extrarows));
if (!query.exec())
MythDB::DBError("Delete excess log entries", query);
}
}
else
MythDB::DBError("Query logging table size", query);
}
}

void MFDThread::run(void)
{
if (!m_parent)
Expand Down
1 change: 1 addition & 0 deletions mythtv/programs/mythbackend/housekeeper.h
Expand Up @@ -50,6 +50,7 @@ class HouseKeeper
bool nowIfPossible = false);
void updateLastrun(const QString &dbTag);
QDateTime getLastRun(const QString &dbTag);
void flushDBLogs();
void runFillDatabase();
void CleanupMyOldRecordings(void);
void CleanupAllOldInUsePrograms(void);
Expand Down

0 comments on commit 6b660a1

Please sign in to comment.