Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
Fix inefficiency in marking first/last showings without programids.
For users without programids (where listings provider doesn't specify a
programid and doesn't specify both season and episode number),
mythfilldatabase marks first/last showings by comparing title, subtitle,
and description to find unique episodes.  If the listings provider
provides extremely long descriptions (>>255 characters), this results in
mysqld creating on-disk temp tables, and can create extremely long run
times for this specific query (with one user reporting times on the
order of 30min using an AMD Athlon II 5050e with 2GB RAM).

This change limits the query to use only the first 1024 characters of
description for the SELECT and the GROUP BY.  This is equivalent to what
we were doing before ccb759a changed the description from TEXT to
VARCHAR, since MySQL automatically uses max_sort_length (defaults to
1024) for sorting text and blob fields (including GROUP BY).  Even in
the event of 2 shows that differ only after the first 1024 characters of
the description, but that air at the same time, the logic is sound.  In
the event 2 shows with different start times that differ only after the
first 1024 characters of the description exist in the listings, one
won't be properly marked as first/last showing, but I'd hope that
listings providers would provide some indication of differences before
the end of the novel (but, again, this is the same behavior we would
have seen before the change to use VARCHAR for description).

Thanks to Jan Schneider for noticing the issue and performing a bunch of
test queries to help me find a way to work around it.
(cherry picked from commit 8b0daaf)
  • Loading branch information
sphery committed Aug 29, 2011
1 parent 4942639 commit 1de0431
Showing 1 changed file with 10 additions and 8 deletions.
18 changes: 10 additions & 8 deletions mythtv/programs/mythfilldatabase/main.cpp
Expand Up @@ -885,9 +885,10 @@ int main(int argc, char *argv[])
}
}
int found = query.size();
query.prepare("SELECT MIN(starttime),title,subtitle,description "
query.prepare("SELECT MIN(starttime),title,subtitle,"
" LEFT(description, 1024) AS partdesc "
"FROM program WHERE programid = '' "
"GROUP BY title,subtitle,description;");
"GROUP BY title,subtitle,partdesc;");
if (query.exec())
{
while(query.next())
Expand All @@ -896,11 +897,11 @@ int main(int argc, char *argv[])
"WHERE starttime = :STARTTIME "
" AND title = :TITLE "
" AND subtitle = :SUBTITLE "
" AND description = :DESCRIPTION");
" AND LEFT(description, 1024) = :PARTDESC");
updt.bindValue(":STARTTIME", query.value(0).toDateTime());
updt.bindValue(":TITLE", query.value(1).toString());
updt.bindValue(":SUBTITLE", query.value(2).toString());
updt.bindValue(":DESCRIPTION", query.value(3).toString());
updt.bindValue(":PARTDESC", query.value(3).toString());
if (!updt.exec())
MythDB::DBError("Marking first showings", updt);
}
Expand All @@ -925,9 +926,10 @@ int main(int argc, char *argv[])
}
}
found = query.size();
query.prepare("SELECT MAX(starttime),title,subtitle,description "
query.prepare("SELECT MAX(starttime),title,subtitle,"
" LEFT(description, 1024) AS partdesc "
"FROM program WHERE programid = '' "
"GROUP BY title,subtitle,description;");
"GROUP BY title,subtitle,partdesc;");
if (query.exec())
{
while(query.next())
Expand All @@ -936,11 +938,11 @@ int main(int argc, char *argv[])
"WHERE starttime = :STARTTIME "
" AND title = :TITLE "
" AND subtitle = :SUBTITLE "
" AND description = :DESCRIPTION");
" AND LEFT(description, 1024) = :PARTDESC");
updt.bindValue(":STARTTIME", query.value(0).toDateTime());
updt.bindValue(":TITLE", query.value(1).toString());
updt.bindValue(":SUBTITLE", query.value(2).toString());
updt.bindValue(":DESCRIPTION", query.value(3).toString());
updt.bindValue(":PARTDESC", query.value(3).toString());
if (!updt.exec())
MythDB::DBError("Marking last showings", updt);
}
Expand Down

0 comments on commit 1de0431

Please sign in to comment.