Skip to content

Commit cbb8eb1

Browse files
committed
Add duplicate checking and limited matching optimizations and other
scheduler related changes. The three major changes are as follows. Split the checks against the oldrecorded and recorded tables for previous recordings away from the "place" phase of scheduling and call it the "check" phase. This makes it easier to report how much time is spent doing this and leads to the next major change. Drastically reduce the number of checks against the oldrecorded and recorded tables for previous recordings. Historically, this has been the most significant contributor to long scheduler runs. Not every unnecessary check is eliminated, but the vast majority of them are. Trying to remove the few remaining ones would probably take longer than simply rechecking them. Allow for checking a subset of the program table when new guide data is available. This is primarily aimed at EIT scanning where typically only a few hours of guide data for specific channels is updated in an orderly fashion. Not rechecking the unchanged guide data greatly reduces the amount of work the scheduler has to do. Please note the EIT scanner has not been updated yet to take advantage of this change. The other changes are as follows. Use more expressive reschedule requests. This is primarily to support the changes listed above. It also makes it easier to understand why reschedules occurred when reading the logs and might lead to the elimination of unnecessary reschedules in the future. MATCH reschedule requests should be used when the guide data or a specific recording rule is changed. The syntax is as follows. RESCHEDULE_RECORDINGS MATCH <recordid> <sourceid> <mplexid> <maxstarttime> <reason> maxstarttime should be in ISO format. If recordid, sourceid or mplexid are non-zero or maxstarttime is a valid time, the scheduler will restrict itself to recording rules and guide data matching those parameters. reason is purely for purposes of logging. CHECK reschedule requests should be used when the status of a specific episode is affected such as when "never record" or "allow re-record" are selected or a recording finishes or is deleted. The syntax is as follows. RESCHEDULE_RECORDINGS CHECK <recstatus> <recordid> <findid> <reason> <title> <subtitle> <description> <programid> recordid should be the parent recordid, when applicable, otherwise, the normal recordid. Setting programid to '**any**" and title to "" is a special case used to emulate an old reschedule request for recordid 0. Setting programid to "**any**" and title to other than "" is another special case used when all entries for a title are deleted from oldrecorded. recstatus and reason are purely for purposes of logging. PLACE reschedule request should be used in all other cases. The syntax is as follows. RESCHEDULE_RECORDINGS PLACE <reason> reason is purely for purposes of logging. Update the PHP and Python bindings to use the new reschedule requests. Please note the bindings API has not changed to make full use of the new requests. I'm leaving it to the bindings maintainers to decide how best to do that. Clear record.duplicate when a recording is queued for deletion. It should have been this way all along and avoids a redundant reschedule when the file is actually unlinked. Lower the very detailed scheduler placement logging to LOG_DEBUG. This makes VB_SCHEDULE/LOG_INFO more useful for less voluminous purposes. Add findid to the recordmatch table. This avoids having to calculate it in multiple places. Tighten the window for rescheduling long running programs from "24 hours ago" to "~8 hours ago". Avoid creating unnecessary RecordingInfo objects that are going to get deleted later anyway. Fixes #10533
1 parent 4621e13 commit cbb8eb1

25 files changed

+707
-388
lines changed

mythtv/bindings/perl/MythTV.pm

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -106,15 +106,15 @@ package MythTV;
106106
# Note: as of July 21, 2010, this is actually a string, to account for proto
107107
# versions of the form "58a". This will get used if protocol versions are
108108
# changed on a fixes branch ongoing.
109-
our $PROTO_VERSION = "72";
110-
our $PROTO_TOKEN = "D78EFD6F";
109+
our $PROTO_VERSION = "73";
110+
our $PROTO_TOKEN = "D7FE8D6F";
111111

112112
# currentDatabaseVersion is defined in libmythtv in
113113
# mythtv/libs/libmythtv/dbcheck.cpp and should be the current MythTV core
114114
# schema version supported in the main code. We need to check that the schema
115115
# version in the database is as expected by the bindings, which are expected
116116
# to be kept in sync with the main code.
117-
our $SCHEMA_VERSION = "1299";
117+
our $SCHEMA_VERSION = "1300";
118118

119119
# NUMPROGRAMLINES is defined in mythtv/libs/libmythtv/programinfo.h and is
120120
# the number of items in a ProgramInfo QStringList group used by

mythtv/bindings/php/MythBackend.php

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,8 @@ class MythBackend {
1111

1212
// MYTH_PROTO_VERSION is defined in libmyth in mythtv/libs/libmyth/mythcontext.h
1313
// and should be the current MythTV protocol version.
14-
static $protocol_version = '72';
15-
static $protocol_token = 'D78EFD6F';
14+
static $protocol_version = '73';
15+
static $protocol_token = 'D7FE8D6F';
1616

1717
// The character string used by the backend to separate records
1818
static $backend_separator = '[]:[]';
@@ -205,7 +205,17 @@ public function queryProgramRows($query = null, $offset = 1) {
205205
* need to indicate every record rule is affected, then use -1.
206206
/**/
207207
public function rescheduleRecording($recordid = -1) {
208-
$this->sendCommand('RESCHEDULE_RECORDINGS '.$recordid);
208+
if ($recordid == 0) {
209+
$this->sendCommand(array('RESCHEDULE_RECORDINGS ',
210+
'CHECK 0 0 0 PHP',
211+
'', '', '', '**any**'));
212+
}
213+
else {
214+
if ($recordid == -1)
215+
$recordid = 0;
216+
$this->sendCommand(array('RESCHEDULE_RECORDINGS ',
217+
'MATCH '.$recordid.' 0 0 - PHP'));
218+
}
209219
Cache::clear();
210220
if ($this->listenForEvent('SCHEDULE_CHANGE'))
211221
return true;

mythtv/bindings/python/MythTV/mythproto.py

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -670,7 +670,17 @@ def reschedule(self, recordid=-1, wait=False):
670670
['BACKEND_MESSAGE',
671671
'SCHEDULE_CHANGE',
672672
'empty']))
673-
self.backendCommand('RESCHEDULE_RECORDINGS '+str(recordid))
673+
if recordid == 0:
674+
self.backendCommand(BACKEND_SEP.join(\
675+
['RESCHEDULE_RECORDINGS',
676+
'CHECK 0 0 0 Python',
677+
'', '', '', '**any**']))
678+
else:
679+
if recordid == -1:
680+
recordid = 0
681+
self.backendCommand(BACKEND_SEP.join(\
682+
['RESCHEDULE_RECORDINGS',
683+
'MATCH ' + str(recordid) + ' 0 0 - Python']))
674684
if wait:
675685
eventlock.wait()
676686

mythtv/bindings/python/MythTV/static.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,11 +5,11 @@
55
"""
66

77
OWN_VERSION = (0,25,-1,3)
8-
SCHEMA_VERSION = 1299
8+
SCHEMA_VERSION = 1300
99
NVSCHEMA_VERSION = 1007
1010
MUSICSCHEMA_VERSION = 1018
11-
PROTO_VERSION = '72'
12-
PROTO_TOKEN = 'D78EFD6F'
11+
PROTO_VERSION = '73'
12+
PROTO_TOKEN = 'D7FE8D6F'
1313
BACKEND_SEP = '[]:[]'
1414
INSTALL_PREFIX = '/usr/local'
1515

mythtv/libs/libmyth/programinfo.cpp

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2678,7 +2678,8 @@ void ProgramInfo::SaveDeletePendingFlag(bool deleteFlag)
26782678
MSqlQuery query(MSqlQuery::InitCon());
26792679

26802680
query.prepare("UPDATE recorded"
2681-
" SET deletepending = :DELETEFLAG"
2681+
" SET deletepending = :DELETEFLAG, "
2682+
" duplicate = 0 "
26822683
" WHERE chanid = :CHANID"
26832684
" AND starttime = :STARTTIME ;");
26842685
query.bindValue(":CHANID", chanid);

mythtv/libs/libmythbase/mythversion.h

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
/// Update this whenever the plug-in API changes.
1313
/// Including changes in the libmythbase, libmyth, libmythtv, libmythav* and
1414
/// libmythui class methods used by plug-ins.
15-
#define MYTH_BINARY_VERSION "0.25.20120408-1"
15+
#define MYTH_BINARY_VERSION "0.26.20120411-1"
1616

1717
/** \brief Increment this whenever the MythTV network protocol changes.
1818
*
@@ -35,8 +35,8 @@
3535
* mythtv/bindings/python/MythTV/static.py (version number)
3636
* mythtv/bindings/python/MythTV/mythproto.py (layout)
3737
*/
38-
#define MYTH_PROTO_VERSION "72"
39-
#define MYTH_PROTO_TOKEN "D78EFD6F"
38+
#define MYTH_PROTO_VERSION "73"
39+
#define MYTH_PROTO_TOKEN "D7FE8D6F"
4040

4141
/** \brief Increment this whenever the MythTV core database schema changes.
4242
*
@@ -57,7 +57,7 @@
5757
* mythtv/bindings/php/MythBackend.php
5858
#endif
5959

60-
#define MYTH_DATABASE_VERSION "1299"
60+
#define MYTH_DATABASE_VERSION "1300"
6161

6262

6363
MBASE_PUBLIC const char *GetMythSourceVersion();

mythtv/libs/libmythtv/dbcheck.cpp

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1948,6 +1948,18 @@ NULL
19481948
return false;
19491949
}
19501950

1951+
if (dbver == "1299")
1952+
{
1953+
const char *updates[] = {
1954+
"ALTER TABLE recordmatch ADD COLUMN findid INT NOT NULL DEFAULT 0",
1955+
"ALTER TABLE recordmatch ADD INDEX (recordid, findid)",
1956+
NULL
1957+
};
1958+
1959+
if (!performActualUpdate(updates, "1300", dbver))
1960+
return false;
1961+
}
1962+
19511963
return true;
19521964
}
19531965

mythtv/libs/libmythtv/eitscanner.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -177,7 +177,7 @@ void EITScanner::RescheduleRecordings(void)
177177
QDateTime::currentDateTime().addSecs(kMinRescheduleInterval);
178178
resched_lock.unlock();
179179

180-
ScheduledRecording::signalChange(-1);
180+
ScheduledRecording::RescheduleMatch(0, 0, 0, QDateTime(), "EITScanner");
181181
}
182182

183183
/** \fn EITScanner::StartPassiveScan(ChannelBase*, EITSource*, bool)

mythtv/libs/libmythtv/recordinginfo.cpp

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1155,7 +1155,7 @@ void RecordingInfo::ReactivateRecording(void)
11551155
if (!result.exec())
11561156
MythDB::DBError("ReactivateRecording", result);
11571157

1158-
ScheduledRecording::signalChange(0);
1158+
ScheduledRecording::ReschedulePlace("Reactivate");
11591159
}
11601160

11611161
/**
@@ -1223,7 +1223,7 @@ void RecordingInfo::AddHistory(bool resched, bool forcedup, bool future)
12231223
// The adding of an entry to oldrecorded may affect near-future
12241224
// scheduling decisions, so recalculate if told
12251225
if (resched)
1226-
ScheduledRecording::signalChange(0);
1226+
ScheduledRecording::RescheduleCheck(*this, "AddHistory");
12271227
}
12281228

12291229
/** \fn RecordingInfo::DeleteHistory(void)
@@ -1257,7 +1257,7 @@ void RecordingInfo::DeleteHistory(void)
12571257

12581258
// The removal of an entry from oldrecorded may affect near-future
12591259
// scheduling decisions, so recalculate
1260-
ScheduledRecording::signalChange(0);
1260+
ScheduledRecording::RescheduleCheck(*this, "DeleteHistory");
12611261
}
12621262

12631263
/** \fn RecordingInfo::ForgetHistory(void)
@@ -1321,7 +1321,7 @@ void RecordingInfo::ForgetHistory(void)
13211321

13221322
// The removal of an entry from oldrecorded may affect near-future
13231323
// scheduling decisions, so recalculate
1324-
ScheduledRecording::signalChange(0);
1324+
ScheduledRecording::RescheduleCheck(*this, "ForgetHistory");
13251325
}
13261326

13271327
/** \fn RecordingInfo::SetDupHistory(void)
@@ -1347,7 +1347,7 @@ void RecordingInfo::SetDupHistory(void)
13471347
if (!result.exec())
13481348
MythDB::DBError("setDupHistory", result);
13491349

1350-
ScheduledRecording::signalChange(0);
1350+
ScheduledRecording::RescheduleCheck(*this, "SetHistory");
13511351
}
13521352

13531353
/**

mythtv/libs/libmythtv/recordinginfo.h

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -228,8 +228,6 @@ class MTV_PUBLIC RecordingInfo : public ProgramInfo
228228
void ApplyTranscoderProfileChange(const QString &profile) const;//pi
229229
void ApplyTranscoderProfileChangeById(int);
230230

231-
static void signalChange(int recordid);
232-
233231
RecStatusType oldrecstatus;
234232
RecStatusType savedrecstatus;
235233
bool future;

mythtv/libs/libmythtv/recordingrule.cpp

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
#include "mythcorecontext.h"
99

1010
// libmythtv
11-
#include "scheduledrecording.h" // For signalChange()
11+
#include "scheduledrecording.h" // For RescheduleMatch()
1212
#include "playgroup.h" // For GetInitialName()
1313
#include "recordingprofile.h" // For constants
1414
#include "mythmiscutil.h"
@@ -380,7 +380,8 @@ bool RecordingRule::Save(bool sendSig)
380380
m_recordID = query.lastInsertId().toInt();
381381

382382
if (sendSig)
383-
ScheduledRecording::signalChange(m_recordID);
383+
ScheduledRecording::RescheduleMatch(m_recordID, 0, 0, QDateTime(),
384+
QString("SaveRule %1").arg(m_title));
384385

385386
return true;
386387
}
@@ -408,7 +409,8 @@ bool RecordingRule::Delete(bool sendSig)
408409
}
409410

410411
if (sendSig)
411-
ScheduledRecording::signalChange(m_recordID);
412+
ScheduledRecording::RescheduleMatch(m_recordID, 0, 0, QDateTime(),
413+
QString("DeleteRule %1").arg(m_title));
412414

413415
// Set m_recordID to zero, the rule is no longer in the database so it's
414416
// not valid. Should you want, this allows a rule to be removed from the

mythtv/libs/libmythtv/scheduledrecording.cpp

Lines changed: 39 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,22 +9,56 @@ ScheduledRecording::~ScheduledRecording()
99
{
1010
}
1111

12-
void ScheduledRecording::signalChange(int recordid)
12+
void ScheduledRecording::SendReschedule(const QStringList &request)
1313
{
1414
if (gCoreContext->IsBackend())
1515
{
16-
MythEvent me(QString("RESCHEDULE_RECORDINGS %1").arg(recordid));
16+
MythEvent me(QString("RESCHEDULE_RECORDINGS"), request);
1717
gCoreContext->dispatch(me);
1818
}
1919
else
2020
{
2121
QStringList slist;
22-
slist << QString("RESCHEDULE_RECORDINGS %1").arg(recordid);
22+
slist << QString("RESCHEDULE_RECORDINGS");
23+
slist << request;
2324
if (!gCoreContext->SendReceiveStringList(slist))
2425
LOG(VB_GENERAL, LOG_ERR,
25-
QString("Error rescheduling id %1 in "
26-
"ScheduledRecording::signalChange") .arg(recordid));
26+
QString("Error rescheduling %1 in "
27+
"ScheduledRecording::SendReschedule").arg(request[0]));
2728
}
2829
}
2930

31+
QStringList ScheduledRecording::BuildMatchRequest(uint recordid,
32+
uint sourceid, uint mplexid, const QDateTime &maxstarttime,
33+
const QString &why)
34+
{
35+
return QStringList(QString("MATCH %1 %2 %3 %4 %5")
36+
.arg(recordid).arg(sourceid).arg(mplexid)
37+
.arg(maxstarttime.isValid() ?
38+
maxstarttime.toString(Qt::ISODate) :
39+
"-")
40+
.arg(why));
41+
};
42+
43+
QStringList ScheduledRecording::BuildCheckRequest(const RecordingInfo &recinfo,
44+
const QString &why)
45+
{
46+
return QStringList(QString("CHECK %1 %2 %3 %4")
47+
.arg(recinfo.GetRecordingStatus())
48+
.arg(recinfo.GetParentRecordingRuleID() ?
49+
recinfo.GetParentRecordingRuleID() :
50+
recinfo.GetRecordingRuleID())
51+
.arg(recinfo.GetFindID())
52+
.arg(why))
53+
<< recinfo.GetTitle()
54+
<< recinfo.GetSubtitle()
55+
<< recinfo.GetDescription()
56+
<< recinfo.GetProgramID();
57+
};
58+
59+
QStringList ScheduledRecording::BuildPlaceRequest(const QString &why)
60+
{
61+
return QStringList(QString("PLACE %1").arg(why));
62+
};
63+
3064
/* vim: set expandtab tabstop=4 shiftwidth=4: */

mythtv/libs/libmythtv/scheduledrecording.h

Lines changed: 30 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,18 +2,43 @@
22
#define SCHEDULEDRECORDING_H
33

44
#include "mythtvexp.h"
5+
#include "qdatetime.h"
6+
#include "recordinginfo.h"
57

68
class MTV_PUBLIC ScheduledRecording
79
{
10+
friend class Scheduler;
11+
812
public:
13+
// Use when a recording rule or program data changes. Use 0 for
14+
// recordid when all recordids are potentially affected, Use
15+
// invalid starttime and 0 for chanids when not time nor channel
16+
// specific.
17+
static void RescheduleMatch(uint recordid, uint sourceid, uint mplexid,
18+
const QDateTime &maxstarttime, const QString &why)
19+
{ SendReschedule(BuildMatchRequest(recordid, sourceid, mplexid,
20+
maxstarttime, why)); };
21+
22+
// Use when previous or current recorded duplicate status changes.
23+
static void RescheduleCheck(const RecordingInfo &recinfo,
24+
const QString &why)
25+
{ SendReschedule(BuildCheckRequest(recinfo, why)); };
26+
27+
// Use when none of recording rule, program data or duplicate
28+
// status changes.
29+
static void ReschedulePlace(const QString &why)
30+
{ SendReschedule(BuildPlaceRequest(why)); };
31+
32+
private:
933
ScheduledRecording();
1034
~ScheduledRecording();
1135

12-
static void signalChange(int recordid);
13-
// Use -1 for recordid when all recordids are potentially
14-
// affected, such as when the program table is updated.
15-
// Use 0 for recordid when a reschdule isn't specific to a single
16-
// recordid, such as when a recording type priority is changed.
36+
static void SendReschedule(const QStringList &request);
37+
static QStringList BuildMatchRequest(uint recordid, uint sourceid,
38+
uint mplexid, const QDateTime &maxstarttime, const QString &why);
39+
static QStringList BuildCheckRequest(const RecordingInfo &recinfo,
40+
const QString &why);
41+
static QStringList BuildPlaceRequest(const QString &why);
1742
};
1843

1944
#endif

mythtv/libs/libmythtv/tv_rec.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2668,7 +2668,7 @@ void TVRec::NotifySchedulerOfRecording(RecordingInfo *rec)
26682668
rec->AddHistory(false);
26692669

26702670
// + save RecordingRule so that we get a recordid
2671-
// (don't allow signalChange(), avoiding unneeded reschedule)
2671+
// (don't allow RescheduleMatch(), avoiding unneeded reschedule)
26722672
rec->GetRecordingRule()->Save(false);
26732673

26742674
// + save recordid to recorded entry

mythtv/programs/mythbackend/main_helpers.cpp

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -370,7 +370,10 @@ int handle_command(const MythBackendCommandLineParser &cmdline)
370370
}
371371

372372
verboseMask |= VB_SCHEDULE;
373+
LogLevel_t oldLogLevel = logLevel;
374+
logLevel = LOG_DEBUG;
373375
sched->PrintList(true);
376+
logLevel = oldLogLevel;
374377
delete sched;
375378
return GENERIC_EXIT_OK;
376379
}
@@ -381,7 +384,8 @@ int handle_command(const MythBackendCommandLineParser &cmdline)
381384
if (gCoreContext->ConnectToMasterServer())
382385
{
383386
LOG(VB_GENERAL, LOG_INFO, "Connected to master for reschedule");
384-
ScheduledRecording::signalChange(-1);
387+
ScheduledRecording::RescheduleMatch(0, 0, 0, QDateTime(),
388+
"MythBackendCommand");
385389
ok = true;
386390
}
387391
else

0 commit comments

Comments
 (0)