Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix potential SQL constraint integrity violation in the PVR database #10544

Closed
wants to merge 3 commits into from

Conversation

Projects
None yet
10 participants
@Jalle19
Copy link
Member

commented Sep 23, 2016

Supersedes #8953
Fixes trac #16031
Closes kodi-pvr/pvr.hts#172

This fixes the problem without bumping the database version so it should be more possibly to get it merged for V17.

Basically the code that persists channels has been made smarter. Instead of just blindly looking at the primary key stored in the object we query the database to check if a channel with the unique ID and client ID already exists. If it does, we update that channel, if not we create a new channel. There is more background information in the referenced PRs.

This fix also has the added side-effect that it increases database performance. While a new SELECT statement has been added, we now do an UPDATE instead of a REPLACE INTO which prevents indexes from having to be re-calculated (REPLACE INTO basically does a DELETE followed by an INSERT). This also means we don't need to worry about the channel ID changing after the channel has been updated (which was a side effect of using REPLACE INTO).

@mention-bot

This comment has been minimized.

Copy link

commented Sep 23, 2016

@Jalle19, thank you for improving Kodi! According to the last 5 commits, we found the potential reviewers: @opdenkamp, @xhaggi and @MartijnKaijser . Final approval needs to be given by the componenent maintainer.

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 23, 2016

@ksooo the const you requested seems to have been lost while rebasing, I added it here now.

@MilhouseVH

This comment has been minimized.

Copy link
Contributor

commented Sep 23, 2016

Will do, thanks.

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 23, 2016

@MilhouseVH thanks!

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 24, 2016

Think I found the actual problem now, I'll push more commits soon. I had no idea this database stuff would be so tricky.

@Jalle19 Jalle19 force-pushed the Jalle19:fix-pvr-database-access branch from 3c95521 to 9c336df Sep 24, 2016

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 24, 2016

jenkins build this please

@ksooo can you review?

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 24, 2016

I've runtime tested every possible combination now, including startup, reset PVR data, change channels between radio/TV etc. I'm not seeing any SQL errors in the log anymore (the problems mentioned on the forum was most likely due to the missing quotes around strings in the UPDATE statement). I did notice another bug though and that was that some channels were being persisted with idChannel = -1 which is obviously wrong, hence I changed ChannelExists() to GetChannelId().

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 24, 2016

@MilhouseVH if you could include this in your next build that would be awesome, I'm fairly certain this works now.

@bkuhls

This comment has been minimized.

Copy link
Contributor

commented Sep 24, 2016

Your latest patch shows an SQL error during startup:

11:28:07 T:2726878112   DEBUG: PVRManager - Process - entering main loop
11:28:07 T:3004607392    INFO: PVRChannelGroupsContainer - SearchMissingChannelIcons - starting channel icon search
11:28:07 T:2710100896   DEBUG: Thread EPGUpdater start, auto delete: false
11:28:07 T:2821708704  NOTICE: Start - EPG thread started
11:28:07 T:2726878112   DEBUG: CPVRChannelGroup - Persist - persisting channel group 'Alle Kanäle' with 96 channels
11:28:07 T:3021457152   DEBUG: ------ Window Init (DialogExtendedProgressBar.xml) ------
11:28:07 T:3021457152   DEBUG: Window DialogExtendedProgressBar.xml was already loaded
11:28:07 T:3021457152   DEBUG: Alloc resources: 0.00ms
11:28:07 T:2726878112   ERROR: SQL: Field not found: idChannel
11:28:07 T:2726878112   ERROR: PVRManager - Process - an error occured while trying to execute the last update job, trying to recover
11:28:07 T:2726878112  NOTICE: PVRManager - Process - no add-ons enabled anymore. restarting the pvrmanager
11:28:07 T:2726878112   DEBUG: Thread PVRManager 2726878112 terminating

Using latest Kodi master, pvr-hts master, tvheadend git master.
EPG data is set not to be stored in Kodi databases, channel order and channel numbers are configured to be used from the backend.

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 24, 2016

I'm starting to hate this database layer more and more. I don't have time to look at this until tomorrow.

@FernetMenta

This comment has been minimized.

Copy link
Member

commented Sep 24, 2016

mind if we revert d2650c6 until problems are sorted?

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 24, 2016

If you insist.

@@ -21,6 +21,7 @@
#include "PVRDatabase.h"

#include <utility>
#include <xbmc/pvr/channels/PVRChannelGroup.h>

This comment has been minimized.

Copy link
@xhaggi

xhaggi Sep 24, 2016

Member

should the brackets be removed?

This comment has been minimized.

Copy link
@Jalle19

Jalle19 Sep 25, 2016

Author Member

Sure, this is something that CLion generated automatically. Will change to standard quotes.

@FernetMenta

This comment has been minimized.

Copy link
Member

commented Sep 24, 2016

@Jalle19 I don't want to be the bad guy but a revert seems the most appropriate solution here.

#10507 is not really a bug fix for a severe issue. It has always been like this. Now fixing the "fix" turns out to be much more complicated and may turn into something more complex. I see no reason why we should have pvr broken for the users of nightlies.


/* Update the channel ID */
int newChannelId = GetChannelId(channel.ClientID(), channel.UniqueID());
channel.SetChannelID(newChannelId);

This comment has been minimized.

Copy link
@xhaggi

xhaggi Sep 24, 2016

Member

you don't need to query the db again to get the id of the row being inserted. should be replaced by m_pDS->lastinsertid(). don't forget to cast to int.

if (!ChannelExists(channel.m_iClientId, channel.m_iUniqueId))
int existingChannelId = GetChannelId(channel.ClientID(), channel.UniqueID());

if (existingChannelId == -1)

This comment has been minimized.

Copy link
@xhaggi

xhaggi Sep 24, 2016

Member

in case GetChannelId will return a row with an invalid primary id (everything lower 1) we should do the insert.

@xhaggi

This comment has been minimized.

Copy link
Member

commented Sep 24, 2016

@bkuhls this PR is not part of the master.

@bkuhls

This comment has been minimized.

Copy link
Contributor

commented Sep 24, 2016

@xhaggi: I know, I added it on top of Kodi git master for testing purposes.

@xhaggi

This comment has been minimized.

Copy link
Member

commented Sep 24, 2016

@bkuhls sorry, but it was not clear to me.

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 25, 2016

@bkuhls can you try again with the lastest version of this PR?
@xhaggi fixed your issues

jenkins build this please

@xhaggi

This comment has been minimized.

Copy link
Member

commented Sep 25, 2016

looks good thanks

@xhaggi

This comment has been minimized.

Copy link
Member

commented Sep 25, 2016

could you rebase your branch to solve the conflicts

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 25, 2016

Trying to, got a bit complicated due to the revert.

@Jalle19 Jalle19 force-pushed the Jalle19:fix-pvr-database-access branch from c6476e3 to 27013cb Sep 25, 2016

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 25, 2016

@xhaggi rebased

jenkins build this please

@FernetMenta

This comment has been minimized.

Copy link
Member

commented Sep 25, 2016

note that the scope of this pr is different now. It does not fix a crash as the title pretends. in order to avoid confusion I suggest to close this one and resubmit with a description of what it actually fixes.

@MartijnKaijser

This comment has been minimized.

Copy link
Member

commented Sep 25, 2016

You can also just edit the PR description

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 26, 2016

@xhaggi there should be a try/catch block further up in the call stack.

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 26, 2016

Actually switching to %u wouldn't make any sense since both iUniqueId and iClientId are signed integers.

@Jalle19 Jalle19 force-pushed the Jalle19:fix-pvr-database-access branch from 7622095 to 026354a Sep 26, 2016

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 26, 2016

@FernetMenta I dropped the const keyword

Does anyone have any idea on why this thing keeps crashing? Based on the three last crash logs they are either running out of memory (crashing on new or malloc) or they're crashing when destructing a field_value which makes no sense. I am not able to reproduce these crashes in any way.

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 26, 2016

Based on the gathered crash logs so far everyone with issues is running ARM 32-bit with uclibc.

@Mike-KL

This comment has been minimized.

Copy link

commented Sep 26, 2016

@xhaggi @Jalle19 Attached is a copy of my tv29.db

TV29.zip

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 26, 2016

@bkuhls can you try with the latest commit?

@Mike-KL I used your database file but everything started up nicely.

@Mike-KL

This comment has been minimized.

Copy link

commented Sep 26, 2016

@Milhouse @Jalle19 @xhaggi Been some other crashlogs whilst at work today, dont know if they provide any related useful info (Not noticed crashes whilst idle in earlier Milhouse builds)

kodi_crashlog.zip

@bkuhls

This comment has been minimized.

Copy link
Contributor

commented Sep 26, 2016

@Jalle19 Yes, I am using uClibc, but not on arm:

19:30:35 T:3030193920  NOTICE: Kodi compiled Sep 26 2016 by GCC 4.9.2 for Linux x86 32-bit version 3.14.25 (200217)

With your latest patches, including the "stab in the dark" one, Kodi does not crash anymore on startup, it produces these error messages instead:

19:30:42 T:2748742560   DEBUG: Thread PVRGUIInfo start, auto delete: false
19:30:42 T:2775096224   DEBUG: PVRManager - Process - entering main loop
19:30:42 T:2830445472    INFO: PVRChannelGroupsContainer - SearchMissingChannelIcons - starting channel icon search
19:30:42 T:2720791456   DEBUG: Thread EPGUpdater start, auto delete: false
19:30:42 T:3013344160  NOTICE: Start - EPG thread started
19:30:42 T:2775096224   DEBUG: CPVRChannelGroup - Persist - persisting channel group 'Alle Kanäle' with 97 channels
19:30:42 T:3030193920   DEBUG: ------ Window Init (DialogExtendedProgressBar.xml) ------
19:30:42 T:3030193920   DEBUG: Window DialogExtendedProgressBar.xml was already loaded
19:30:42 T:3030193920   DEBUG: Alloc resources: 0.00ms
19:30:42 T:2830445472   ERROR: SQL: Field index not found: 0
19:30:42 T:2775096224   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: UPDATE channels SET bIsRadio = 0, bIsHidden = 0, bIsUserSetIcon = 0, bIsUserSetName = 0, bIsLocked = 0, sIconPath = '/data/Data/kodi/Senderlogos/zdf_neo hd.png', sChannelName = 'zdf_neo HD', bIsVirtual = 0, bEPGEnabled = 1, sEPGScraper = 'client', iLastWatched = 1474754171, idChannel = 15, idEpg = 14 WHERE iUniqueId = 175787782 AND iClientId = 2146250725
19:30:42 T:2830445472   ERROR: GetSingleValue - failed on query 'SELECT idChannel FROM channels WHERE iClientId = 2146250725 AND iUniqueId = 213267705'
19:30:42 T:2775096224   ERROR: ExecuteQuery - failed to execute query 'UPDATE channels SET bIsRadio = 0, bIsHidden = 0, bIsUserSetIcon = 0, bIsUserSetName = 0, bIsLocked = 0, sIconPath = '/data/Data/kodi/Senderlogos/zdf_neo hd.png', sChannelName = 'zdf_neo HD', bIsVirtual = 0, bEPGEnabled = 1, sEPGScraper = 'client', iLastWatched = 1474754171, idChannel = 15, idEpg = 14 WHERE iUniqueId = 175787782 AND iClientId = 2146250725'
19:30:42 T:2830445472   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (213267705, 0, 0, 0, 0, 0, '', 'TVGA', 0, 1, 'client', 0, 2146250725, 15)
19:30:42 T:2830445472   ERROR: ExecuteQuery - failed to execute query 'INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (213267705, 0, 0, 0, 0, 0, '', 'TVGA', 0, 1, 'client', 0, 2146250725, 15)'
19:30:42 T:2830445472   ERROR: SQL: Field index not found: 0
19:30:42 T:2830445472   ERROR: GetSingleValue - failed on query 'SELECT idChannel FROM channels WHERE iClientId = 2146250725 AND iUniqueId = 407020287'
19:30:43 T:2830445472   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (407020287, 0, 0, 0, 0, 0, '', 'ARAGON TV', 0, 1, 'client', 0, 2146250725, 22)
19:30:43 T:2830445472   ERROR: ExecuteQuery - failed to execute query 'INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (407020287, 0, 0, 0, 0, 0, '', 'ARAGON TV', 0, 1, 'client', 0, 2146250725, 22)'
19:30:43 T:2775096224   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: UPDATE channels SET bIsRadio = 0, bIsHidden = 0, bIsUserSetIcon = 0, bIsUserSetName = 0, bIsLocked = 0, sIconPath = '/data/Data/kodi/Senderlogos/tagesschau24 hd.png', sChannelName = 'tagesschau24 HD', bIsVirtual = 0, bEPGEnabled = 1, sEPGScraper = 'client', iLastWatched = 1474843836, idChannel = 23, idEpg = 18 WHERE iUniqueId = 315091849 AND iClientId = 2146250725
19:30:43 T:2775096224   ERROR: ExecuteQuery - failed to execute query 'UPDATE channels SET bIsRadio = 0, bIsHidden = 0, bIsUserSetIcon = 0, bIsUserSetName = 0, bIsLocked = 0, sIconPath = '/data/Data/kodi/Senderlogos/tagesschau24 hd.png', sChannelName = 'tagesschau24 HD', bIsVirtual = 0, bEPGEnabled = 1, sEPGScraper = 'client', iLastWatched = 1474843836, idChannel = 23, idEpg = 18 WHERE iUniqueId = 315091849 AND iClientId = 2146250725'
19:30:43 T:2830445472   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (438504961, 0, 0, 0, 0, 0, '', 'N24 DOKU', 0, 1, 'client', 0, 2146250725, 24)
19:30:43 T:2830445472   ERROR: ExecuteQuery - failed to execute query 'INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (438504961, 0, 0, 0, 0, 0, '', 'N24 DOKU', 0, 1, 'client', 0, 2146250725, 24)'
19:30:43 T:2775096224   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (397159228, 0, 0, 0, 0, 0, '/data/Data/kodi/Senderlogos/einsplus hd.png', 'EinsPlus HD', 0, 1, 'client', 1474798540, 2146250725, 21)
19:30:43 T:2775096224   ERROR: ExecuteQuery - failed to execute query 'INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (397159228, 0, 0, 0, 0, 0, '/data/Data/kodi/Senderlogos/einsplus hd.png', 'EinsPlus HD', 0, 1, 'client', 1474798540, 2146250725, 21)'
19:30:43 T:2830445472   ERROR: SQL: Field index not found: 0
19:30:43 T:2830445472   ERROR: GetSingleValue - failed on query 'SELECT idChannel FROM channels WHERE iClientId = 2146250725 AND iUniqueId = 1135085647'
19:30:43 T:2830445472   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (1135085647, 0, 0, 0, 0, 0, '', 'TBN Espana', 0, 1, 'client', 0, 2146250725, 53)
19:30:43 T:2830445472   ERROR: ExecuteQuery - failed to execute query 'INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (1135085647, 0, 0, 0, 0, 0, '', 'TBN Espana', 0, 1, 'client', 0, 2146250725, 53)'
19:30:43 T:2775096224   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (497943573, 0, 0, 0, 0, 0, '/data/Data/kodi/Senderlogos/zdf hd.png', 'ZDF HD', 0, 1, 'client', 1474766274, 2146250725, 26)
19:30:43 T:2775096224   ERROR: ExecuteQuery - failed to execute query 'INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (497943573, 0, 0, 0, 0, 0, '/data/Data/kodi/Senderlogos/zdf hd.png', 'ZDF HD', 0, 1, 'client', 1474766274, 2146250725, 26)'
19:30:43 T:2830445472   ERROR: SQL: Field index not found: 0
19:30:43 T:2830445472   ERROR: GetSingleValue - failed on query 'SELECT idChannel FROM channels WHERE iClientId = 2146250725 AND iUniqueId = 1549433231'
19:30:43 T:2830445472   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (1549433231, 0, 0, 0, 0, 0, '', 'France 24 (en Français)', 0, 1, 'client', 0, 2146250725, 73)
19:30:43 T:2830445472   ERROR: ExecuteQuery - failed to execute query 'INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (1549433231, 0, 0, 0, 0, 0, '', 'France 24 (en Français)', 0, 1, 'client', 0, 2146250725, 73)'
19:30:43 T:2775096224   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: UPDATE channels SET bIsRadio = 0, bIsHidden = 0, bIsUserSetIcon = 0, bIsUserSetName = 0, bIsLocked = 0, sIconPath = '/data/Data/kodi/Senderlogos/tlc.png', sChannelName = 'TLC', bIsVirtual = 0, bEPGEnabled = 1, sEPGScraper = 'client', iLastWatched = 0, idChannel = 73, idEpg = 27 WHERE iUniqueId = 516412587 AND iClientId = 2146250725
19:30:43 T:2775096224   ERROR: ExecuteQuery - failed to execute query 'UPDATE channels SET bIsRadio = 0, bIsHidden = 0, bIsUserSetIcon = 0, bIsUserSetName = 0, bIsLocked = 0, sIconPath = '/data/Data/kodi/Senderlogos/tlc.png', sChannelName = 'TLC', bIsVirtual = 0, bEPGEnabled = 1, sEPGScraper = 'client', iLastWatched = 0, idChannel = 73, idEpg = 27 WHERE iUniqueId = 516412587 AND iClientId = 2146250725'
19:30:43 T:2830445472   ERROR: SQL: Field index not found: 0
19:30:43 T:2830445472   ERROR: GetSingleValue - failed on query 'SELECT idChannel FROM channels WHERE iClientId = 2146250725 AND iUniqueId = 1668302497'
19:30:43 T:2775096224   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (534915409, 0, 0, 0, 0, 0, '/data/Data/kodi/Senderlogos/tv5 monde.png', 'TV5 MONDE', 0, 1, 'client', 0, 2146250725, 28)
19:30:43 T:2775096224   ERROR: ExecuteQuery - failed to execute query 'INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (534915409, 0, 0, 0, 0, 0, '/data/Data/kodi/Senderlogos/tv5 monde.png', 'TV5 MONDE', 0, 1, 'client', 0, 2146250725, 28)'
19:30:43 T:2830445472   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (1668302497, 0, 0, 0, 0, 0, '', 'R1', 0, 1, 'client', 0, 2146250725, 82)
19:30:43 T:2830445472   ERROR: ExecuteQuery - failed to execute query 'INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (1668302497, 0, 0, 0, 0, 0, '', 'R1', 0, 1, 'client', 0, 2146250725, 82)'
19:30:43 T:2830445472   ERROR: SQL: Field index not found: 0
19:30:43 T:2830445472   ERROR: GetSingleValue - failed on query 'SELECT idChannel FROM channels WHERE iClientId = 2146250725 AND iUniqueId = 1954260872'
19:30:43 T:2775096224   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: UPDATE channels SET bIsRadio = 0, bIsHidden = 0, bIsUserSetIcon = 0, bIsUserSetName = 0, bIsLocked = 0, sIconPath = '/data/Data/kodi/Senderlogos/3sat hd.png', sChannelName = '3sat HD', bIsVirtual = 0, bEPGEnabled = 1, sEPGScraper = 'client', iLastWatched = 1474835514, idChannel = 134, idEpg = 31 WHERE iUniqueId = 631883453 AND iClientId = 2146250725
19:30:43 T:2775096224   ERROR: ExecuteQuery - failed to execute query 'UPDATE channels SET bIsRadio = 0, bIsHidden = 0, bIsUserSetIcon = 0, bIsUserSetName = 0, bIsLocked = 0, sIconPath = '/data/Data/kodi/Senderlogos/3sat hd.png', sChannelName = '3sat HD', bIsVirtual = 0, bEPGEnabled = 1, sEPGScraper = 'client', iLastWatched = 1474835514, idChannel = 134, idEpg = 31 WHERE iUniqueId = 631883453 AND iClientId = 2146250725'
19:30:43 T:2830445472   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (1954260872, 0, 0, 0, 0, 0, '', 'oe24.TV', 0, 1, 'client', 1474843725, 2146250725, 90)
19:30:43 T:2830445472   ERROR: ExecuteQuery - failed to execute query 'INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (1954260872, 0, 0, 0, 0, 0, '', 'oe24.TV', 0, 1, 'client', 1474843725, 2146250725, 90)'
19:30:43 T:2775096224   ERROR: SQL: [TV29.db[Abort due to constraint violation
                                            Query: INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (679943908, 0, 0, 0, 0, 0, '/data/Data/kodi/Senderlogos/disney channel.png', 'Disney Channel', 0, 1, 'client', 1474796131, 2146250725, 32)
19:30:43 T:2775096224   ERROR: ExecuteQuery - failed to execute query 'INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (679943908, 0, 0, 0, 0, 0, '/data/Data/kodi/Senderlogos/disney channel.png', 'Disney Channel', 0, 1, 'client', 1474796131, 2146250725, 32)'
19:30:43 T:3030193920   DEBUG: ------ Window Init (DialogExtendedProgressBar.xml) ------
19:30:43 T:3030193920   DEBUG: Window DialogExtendedProgressBar.xml was already loaded
19:30:43 T:3030193920   DEBUG: Alloc resources: 0.00ms
19:30:44 T:3030193920   DEBUG: ------ Window Deinit (DialogExtendedProgressBar.xml) ------
19:30:44 T:3030193920   DEBUG: ------ Window Deinit (DialogNotification.xml) ------
19:30:45 T:2775096224   DEBUG: CPVRChannelGroup - Persist - persisting channel group 'Alle Kanäle' with 37 channels
19:30:47 T:2775096224   DEBUG: CPVRTimers - Update - updating timers

Deleting TV29.db to let Kodi re-create it produces the same SQL-related error messages.
All channels and their EPG data are present and a quick test showed no obvious problems.

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 26, 2016

Okay so the patch definitely fixes some weird shit, but we still have the constraint violation stuff. I don't understand how an UPDATE can cause that.

Anyone have any ideas?

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 26, 2016

@bkuhls let me guess, if you execute one of those failing UPDATE statement manually you get no error, right?

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 26, 2016

Furthermore the Field index not found: 0 error makes zero sense, the code that triggers that looks like this:

if (ds->query(query) && ds->num_rows() > 0)
      ret = ds->fv(0).get_asString();

Basically it would mean that more than zero rows are returned, but that/those rows don't contain any columns.

Something is seriously wrong with our SQLite database abstraction, or SQLite on uclibc is completely broken.

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Sep 26, 2016

@bkuhls can you test this on any other device, preferably an x86?

@bkuhls

This comment has been minimized.

Copy link
Contributor

commented Sep 26, 2016

@Jalle19: I am using Kodi on x86, not on arm.
The SQL query

UPDATE channels SET bIsRadio = 0, bIsHidden = 0, bIsUserSetIcon = 0, bIsUserSetName = 0, bIsLocked = 0, sIconPath = '/data/Data/kodi/Senderlogos/zdf_neo hd.png', sChannelName = 'zdf_neo HD', bIsVirtual = 0, bEPGEnabled = 1, sEPGScraper = 'client', iLastWatched = 1474754171, idChannel = 15, idEpg = 14 WHERE iUniqueId = 175787782 AND iClientId = 2146250725

produces this error in Adminer:

UNIQUE constraint failed: channels.idChannel 

Searching for idChannel=15 by

SELECT *, rowid FROM "channels" WHERE "idChannel" = '15'

results in one row where iUniqueId=213267705 and sChannelName='TVGA', which is a totally different channel...

When querying for iUniqueId=175787782 (the one where Kodi fails) results in one row with idChannel=14.

Maybe Kodi somehow confuses the channels? Are other backends than tvheadend also affected?

Please keep in mind that I am using some non-default settings:
#10544 (comment)
"EPG data is set not to be stored in Kodi databases, channel order and channel numbers are configured to be used from the backend."

Here is the result of

SELECT *, rowid FROM "channels" WHERE "idChannel" >= '14' AND "idChannel" <= '15'

as CSV

idChannel,iUniqueId,bIsRadio,bIsHidden,bIsUserSetIcon,bIsUserSetName,bIsLocked,sIconPath,sChannelName,bIsVirtual,bEPGEnabled,sEPGScraper,iLastWatched,iClientId,idEpg
14,175787782,0,0,0,0,0,/data/Data/kodi/Senderlogos/zdf_neo hd.png,zdf_neo HD,0,1,client,1474754171,2146250725,-1
15,213267705,0,0,0,0,0,"",TVGA,0,1,client,0,2146250725,15
"idChannel = %i, idEpg = %i WHERE iUniqueId = %i AND iClientId = %i",
(channel.IsRadio() ? 1 :0), (channel.IsHidden() ? 1 : 0), (channel.IsUserSetIcon() ? 1 : 0), (channel.IsUserSetName() ? 1 : 0), (channel.IsLocked() ? 1 : 0),
channel.IconPath().c_str(), channel.ChannelName().c_str(), 0, (channel.EPGEnabled() ? 1 : 0), channel.EPGScraper().c_str(), channel.LastWatched(),
existingChannelId, channel.EpgID(), channel.UniqueID(), channel.ClientID());

This comment has been minimized.

Copy link
@xhaggi

xhaggi Sep 26, 2016

Member

we should update the channel by its primary key not by clientid and uniqueid. the pk is fetched before and should be used here.you can safely left the clientid and uniqueid out of the query.

@Mike-KL

This comment has been minimized.

Copy link

commented Sep 26, 2016

Guys if it's relevant I personally don't allow sort of channels via back end but via kodi and reason relates to my original idea to use two backends and frontends and arrange channels as wanted/needed within kodi krypton.

I was reminded by popcornmix & Milhouse that two backends not officially supported so switched one off. But still been getting occasional kodi crashes at boot but then generally stable after that...

@Mike-KL

This comment has been minimized.

Copy link

commented Oct 6, 2016

@Jalle19 just an update that in my usage case after initial removal of #10544 from Milhouse Krypton Nightly Builds on Rpi2 I am not experience kodi crashes.

http://forum.kodi.tv/showthread.php?tid=269814&pid=2426293#pid2426293
http://forum.kodi.tv/showthread.php?tid=269814&pid=2426818#pid2426818
http://forum.kodi.tv/showthread.php?tid=269814&pid=2427027#pid2427027

After addition of #10605 to nightly build, kodi crashes and then clean kodi starts again on builds afterwards.

http://forum.kodi.tv/showthread.php?tid=269814&pid=2428397#pid2428397
http://forum.kodi.tv/showthread.php?tid=269814&pid=2428397#pid2428397
http://forum.kodi.tv/showthread.php?tid=269814&pid=2431320#pid2431320

If its useful and there are any particular user tests you would like me to perform on my setup using tvheadend front and backends don't hesitate to advise.

@Mike-KL

This comment has been minimized.

Copy link

commented Oct 15, 2016

@Jalle19 @xhaggi Ref my previous comments, had another crash after installing build #1013 which is 1st kodi crash of this type since build #1002

http://forum.kodi.tv/showthread.php?tid=269814&pid=2436392#pid2436392

@itsdaveit

This comment has been minimized.

Copy link

commented Dec 5, 2016

Hi, I did a fresh install of KodiSetup-20161129-fe8b16f-master on my Win10 x64 some days ago an ran into this error. Im using VDR 2.2 with the vnsi pvr backend. Kodi doesnt crash, but fails to load dthe PVR client.

kodi_log.txt

When I try to run the insert against the DB i get:

UNIQUE constraint failed: channels.iClientId, channels.iUniqueId: INSERT INTO channels (iUniqueId, bIsRadio, bIsHidden, bIsUserSetIcon, bIsUserSetName, bIsLocked, sIconPath, sChannelName, bIsVirtual, bEPGEnabled, sEPGScraper, iLastWatched, iClientId, idEpg) VALUES (71820975, 0, 0, 0, 0, 0, '', 'gotv', 0, 1, 'client', 0, 1254006891, -1)

...because this Channel is allready in the db, but with blsRadio=1

So i guess when VDR finds some changes, there i a chance that Kodi fails to update the TB29.db at the next start

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Dec 6, 2016

Yeah, I need to find the time to finish this PR.

@MartijnKaijser

This comment has been minimized.

Copy link
Member

commented Jul 29, 2017

@Jalle19 you still want to work on tjis one if you have time?

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Jul 29, 2017

@MartijnKaijser yeah, leave it open for now

@jenkins4kodi

This comment has been minimized.

Copy link
Contributor

commented Nov 19, 2017

@Jalle19 this needs a rebase

@Jalle19

This comment has been minimized.

Copy link
Member Author

commented Nov 19, 2017

I think @ksooo incidentally fixed this in another PR, closing this one

@Jalle19 Jalle19 closed this Nov 19, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.