Repeated crash with UK external XMLTV file - buffer overrun? #59
Comments
|
Can you upload a copy of your external XMLTV file somewhere? That should be enough to be able to reproduce it. If you happen to have a Kodi crashlog then please upload that as well. |
|
Thanks Jalle19 I have put it on Dropbox - https://www.dropbox.com/sh/xfqki8fyu6xcclz/AAAKTLbhqzV2pkNC9HmXr86-a?dl=0 Please let me know when you have it downloaded so I can close the link but actually you don't need it - just use the examples I give below: However, please note that after spending hours editing xmltv.xml files, I've already pinpointed the problem as I detailed above - I'm pretty sure you could make your own xmltv.xml file with more than 995 characters in the This xmltv.xml file: ` `
doesn't crash This one: ` `
does - every single boot. Nothing in the xmltv.xml file matters except the character length of the |
|
Thanks, you can delete the Dropbox link now. I'll try to look into this tomorrow. |
|
Unfortunately I'm not able to reproduce this. What platform are you using? |
|
Raspberry Pi model B - which is inherently memory limited I guess - OpenElec 15 - all builds, same behaviour - currently Milhouse build #603 but the 5.95.1 official beta has same behaviour, so I think it is more to do with RPi |
|
Can you check if you have any crash logs stored on your Pi? The should be in your home directory. |
|
OK - will do this evening - thanks for looking into this |
|
ok - log uploaded to same dropbox |
|
Was the log helpful? Anything else I can do to help? - I read through your xmltv code and nothing jumps out at me - I suspect a problem in tinyxml2 on the pi but that's just a hunch - it seems clear that the problem is one of memory allocation somehow because I can reliably trigger the crash by adding a single character to the length of the description string in the 'desc' xml tag - despite trawling through the header files in tinyxml2, I can't see anything wrong with the way it's coded - it's a pity you cannot reproduce the bug - if you're prepared to give pointers to get me up and running with building your plugin/kodi on OS X for the RPi, I would be willing to have a crack at debugging it for you but, as I've not attempted cross-compiling for the RPi or buildling Kodi before, I'd probably get stuck without some help. |
|
I tried reproducing this on a Raspberry Pi using a separate program but I couldn't reproduce it. Do you mind running the test program for me to see if it fails on your device? Here's how: It should just print the description. |
|
I certainly wouldn't mind - I'd be very happy to help in fact - but I cannot compile this on my RPi as I switched from a full OS to OpenELEC recently to improve stability / performance. As you know, this is a limited distribution that lacks compilers etc. I only have one working SD card at the moment as my other one is broken but I will go and buy one and get a version of debian up and running to do as you ask - I wonder if this is another piece of the puzzle - perhaps it is something to do with Kodi on OpenElec, rather than Kodi on debian? By the way, in case you don't read my reply on the Kodi forum, I apologise if my post came over as critical of your addon - that was not the intention - I wanted to warn other buyers of vbox that they might have problems. Just thought - how easy would that be to cross-compile on my mac then run from the Pi? |
|
Try this one instead: http://werket.tlk.fi/~negge/test-long-description.tar.gz, it's precompiled so you only need to untar the archive and run the |
|
Well sorry - you're right - it works fine - no crash, so I guess that rules out tinyxml2, right? - thanks for compiling it (I spent about 5 hours last night trying to get a cross-compiler working on OS X - it's a nightmare - I think I will pursue doing this so that I can start learning how to compile for kodi but I might bail on OS X and try running linux in a VM) - the bug seems quite odd - I am starting to suspect Kodi itself but then how would the simple IPTV plugin not cause a crash? I have updated my firmware recently to a more recent build of openelec, so I will check to see if your add on still crashes - perhaps something in the RPi kernel or elsewhere has been fixed - will report back soon |
|
Maybe Simple IPTV trims the long program information? |
|
I was wondering the same thing - I will investigate |
|
I'll update the version of tinyxml2 that is used (the current one is very old), that should hopefully solve the issue. @Mhir it doesn't, but it does use a different XML parsing library. |
|
Thanks Jalle19 - I did some more investigating - I will report my findings just in case it's helpful. Firstly, Simple IPTV doesn't care what is in the Secondly, I found something a little bit interesting - pvr.vbox doesn't care about the actual size of the contents of I hope this info is helpful. Jalle19 - What I don't understand is how your test code for tinyxml2 ran without crashing as surely it replicates the function within pvr.vbox that is responsible for parsing the XMLTV file? |
|
The addon uses system libraries on Linux, which means the version is not exactly the same as on Windows or OS X. I've changed it in my work-in-progress branch to use a local copy of the latest version so we can guarantee that the same code is being used on all platforms. I'll most likely make a new release tomorrow, stay tuned. |
|
OK - thank you very much |
|
Oh - I think I just realised something else that I wasn't completely clear about - I don't know if it makes a difference to diagnosis but it isn't a proper 'boot loop' that you get but rather that Kodi crashes and then auto-restarts, so the underlying OS doesn't crash and my ssh link stays intact - could this explain why your standalone code ran ok? |
|
That just means Kodi crashed, not your operating systems. |
|
sure - i just wanted you to know that in case it helped you to track down the bug |
|
Hi Jalle19 - I'm sorry to be the bearer of bad tidings but unfortunately the bug persists :( On the plus side, it crashes much faster, so whatever you have done with tinyxml2 has improved performance! (and more seriously on the plus side - the icon loading works nicely, which is a big improvement - thanks for implementing this - I wasn't able to load a full xmltv file so I wasn't able to check if the graphics in the EPG for each individual show also load like they do on simple IPTV but I will test this when the main bug is eventually found and fixed). BW sunstealer PS - thanks again for all the hard work on this and please let me know what I can do to help you fix it |
|
Can you post a new crashlog? |
|
sure - will do shortly can I ask please - where is EPG_TAG defined? Is it in the kodi headers? |
|
ok - find the logs in the same dropbox - I just threw the lot in there - obviously the most recent one is probably the most useful but they are all probably the same as I let it crash and restart kodi quite a few times while I was doing something else in the background. Please let me know what you're thinking - I am trying to set up kodi compilation in a virtualbox vm so that I can help but if you're already figuring it out, then it would be helpful to know |
|
I really have no idea what's going on with your setup. Can you try to replicate it with a complete fresh profile? You can do that by logging in over SSH and move |
|
I'm sad to say that once again I'm not able to reproduce this issue on my Windows machine (with the file you recently posted). |
|
@sunstealer the xmltv file I am using have your 996 aaa string as well as other long desc.
The log files are on the same link - http://tinyurl.com/nue6yso |
|
So the parsing only fails when the file is on a network share? |
|
Yes I think so and only with RPi, |
|
@Mhir - sorry I just got back home - this is getting weirder and weirder! I cannot begin to fathom how the combination of a network file and 996 vs 995 characters on a single line of text within a specific part of an xml file causes a crash! It's totally bizarre! I am amazed that your test was positive only if the file is on a network drive and will try locating my files on the RPi to see if it works - will report back shortly! Bearing in mind that the same xmltv file works with a different plugin when stored on the same network share, so the fundamental nfs code within linux must surely be alright - I'm totally baffled by this and I expect Jalle19 is too but maybe he's having a moment of genius as we speak! Thanks so much for taking this further - maybe now we can get to the root of the problem. @Jalle19 - thanks for trying again with the other file - this issue is clearly turning out to be a lot more complex than I ever imagined it would be - I'm not entirely certain now that your addon is actually to blame but maybe your addon is demanding something differently from the OS or from Kodi itself than the other plugins do? I have started looking at nightik's code to see if I can understand how it differs - if I turn up anything potentially interesting, I will let you know but don't hold your breath!! ;-) |
|
@Mhir - you're only bloomin' right mate! That's bizarre - I am shocked! I have it working now with an external xmltv.xml file, stored locally on the RPi. This isn't a long-term solution for me because I don't have the atlas grabber compiled for the RPi - it runs on my NAS on a cron job @Jalle19 - any ideas? This is really very strange. One thing I did notice that is also very odd is that the time (top right) flickers between the current time and one hour before (we are in British Summer Time right now - it's almost as if the time is being reset to GMT then GMT+1 then back very rapidly - back and forth - each time something loads into the guide) - this happens only when the addon is parsing the xmltv file then it goes back to normal when it finishes. All the entries in the guide flicker in the same way between the current program and the one an hour previously until it finishes the parsing. This might be an unrelated issue - would you like me to start a new bug @Jalle19 for this? Oh - incidentally, I had to downgrade to the same 5.95.2 build to get things to work - as @Mhir did - the more recent builds ignore the external xmltv file. |
|
@sunstealer super 👍 , maybe until it get's resolved, you can run the grabber process on the NAS and save the file remotely on your RPi? |
|
@Mhir - yeh - thanks mate - in the end I went back to Milhouse builds of openelec because they solve other problems that are in 5.95.2, so now I have a higher build of pvr.vbox that doesn't work on the RPi, so I am back on simple iptv client for now as it seems more robust generally at the moment - will keep trying with this though, as and when @Jalle19 makes progress with the bugs. |
|
What's the issue with the latest release? |
|
A combination of #76, which seems to be a new bug, unrelated to the problem that @Mhir found the solution to in 5.95.2 and the other weird bug that causes flickering time / EPG during parsing of the external xmltv, which happens on 5.95.2 & the latest builds - this is very strange and sometimes seems to cause crashes if kodi tries to do other things while the parsing is going on, like download updates for addons, for example. I haven't had time to dig into this in any detail I'm afraid but I can't get a stable / fully functioning TV service with pvr.vbox yet unfortunately. |
…o work with all underlying file implementations (hopefully fixes kodi-pvr#59, hopefully fixes kodi-pvr#76)
|
Sorry - I've been away so wasn't able to check this until now - just tried latest version of pvr.vbox and it still crashes when parsing the XMLTV file as before - this should be reopened I'm afraid |
|
Do you have a new crash log? |
|
yes - can do if you think it will help - I never had the impression you found the previous ones helpful but perhaps I was wrong? I am at work right now but will send you another one when I can. Do you need me to enable anything in particular other than the usual debug level? |
|
The code has changed (which is why I closed this) so it could turn up something new. You can enable debug logging, nothing more is required. |
|
Sorry for necro-ing this, but as the issue is still flagged as open I thought it would be ok to comment. I recently purchased a Vbox XTi-3442 and am seeing the same issue as originally described. If I use the OTA EPG from Freeview then there is no issue. If however I switch the Vbox to use an XMLTV file then the add-on crashes Kodi during the EPG parsing phase and Kodi ends up in a crash-restart loop. If I also specify the XMLTV file directly in the add-on as an External EPG source it causes the same issue, so it would seem to be the parsing of the data within the add-on causing the issue. The Vbox itself appears to have no problem with the XMLTV file. I'm currently running LibreELEC 8.2 on an x86 machine. I'm not at my machine right now, but is there anything I can supply that might be of help in tracking this issue down? |
|
Just wondered if anybody had had a chance to look into this? I've updated to LibreELEC 8.2.5, running on an x86 machine, and still see the issue. As before, the Vbox has no problem with the external XMLTV file, but the PVR.Vbox add-on crashes Kodi when opening the file either directly (as an external file) or directly from the Vbox. |
|
I had a similar issue to this on LibreElec with pvr.vuplus. It was a race condition ok loading the initial EPG. I wonder if it could be the same issue here. |
|
This shouldn't be an issue, we hold a lock in the update thread and stall Kodi's initial EPG request until the lock is released. Could be that there's a bug in that logic though. |
|
I did the same with vuplus. The issue was caused by two channel updates for EPG at the same time. The problem only ever occurred in *Elec. |
|
Strange |
|
Is there someone on this thread who can try this out solution if I implement it? |
This tag:
<desc lang="en">With Catriona Young. Mozart: Overture: Singspiel in three acts: Die Entfuhrung aus dem Serail, K384. Balakirev: Oriental Fantasy orch Lyapunov: Islamey. Holst: Oriental Suite: Beni Mora, Op 29 No 1. Borusan Istanbul Philharmonic Orchestra, conductor Sascha Goetzel. 1.04 Handel: Solomon - oratorio, Act 3; Sinfonia (Arrival of the Queen of Sheba). 1.08 Respighi: Suite: Belkis, regina di Saba (Belkis, Queen of Sheba). 1.33 Erkin: Kocekce. 1.43 Mozart: Quintet for piano, oboe, clarinet, bassoon and horn in E flat, K452. 2.08 Purcell: Timon of Athens, the Man-Hater, Z632. 2.31 Bartok: Violin Concerto No 1 Op Post, Sz36. 2.53 Martinu: Symphony No 1. 3.30 Dvorak: Slavonic Dance No 10 in E minor, Op 72 No 2. 3.36 Mendelssohn: Venetian Boat Song, Op 30 No 6 (Songs Without Words, Book II). 3.40 Vivaldi: Concerto in F for violin, two oboes, two horns, bassoon and cello, Rv569. 3.53 Part: The Woman with the Alabaster box for chorus. 4.00 Faure: Piano Nocturne No 6 in D flat, Op 63. 4.09 Borgs</desc>is ok and loads without crashing during the initial population of the EPG after booting
This tag:
<desc lang="en">With Catriona Young. Mozart: Overture: Singspiel in three acts: Die Entfuhrung aus dem Serail, K384. Balakirev: Oriental Fantasy orch Lyapunov: Islamey. Holst: Oriental Suite: Beni Mora, Op 29 No 1. Borusan Istanbul Philharmonic Orchestra, conductor Sascha Goetzel. 1.04 Handel: Solomon - oratorio, Act 3; Sinfonia (Arrival of the Queen of Sheba). 1.08 Respighi: Suite: Belkis, regina di Saba (Belkis, Queen of Sheba). 1.33 Erkin: Kocekce. 1.43 Mozart: Quintet for piano, oboe, clarinet, bassoon and horn in E flat, K452. 2.08 Purcell: Timon of Athens, the Man-Hater, Z632. 2.31 Bartok: Violin Concerto No 1 Op Post, Sz36. 2.53 Martinu: Symphony No 1. 3.30 Dvorak: Slavonic Dance No 10 in E minor, Op 72 No 2. 3.36 Mendelssohn: Venetian Boat Song, Op 30 No 6 (Songs Without Words, Book II). 3.40 Vivaldi: Concerto in F for violin, two oboes, two horns, bassoon and cello, Rv569. 3.53 Part: The Woman with the Alabaster box for chorus. 4.00 Faure: Piano Nocturne No 6 in D flat, Op 63. 4.09 Borgst</desc>reliably crashes every time - with a Kodi reboot loop effectively being created as it can't get past the EPG upload without a crash.
The difference is 996 characters vs. 995 characters. Presumably, there is some memory being allocated somewhere with a fixed size?
This is a problem in the UK where program descriptions can be even longer than this.
For example, the full tag in this particular program is:
<desc lang="en">With Catriona Young. Mozart: Overture: Singspiel in three acts: Die Entfuhrung aus dem Serail, K384. Balakirev: Oriental Fantasy orch Lyapunov: Islamey. Holst: Oriental Suite: Beni Mora, Op 29 No 1. Borusan Istanbul Philharmonic Orchestra, conductor Sascha Goetzel. 1.04 Handel: Solomon - oratorio, Act 3; Sinfonia (Arrival of the Queen of Sheba). 1.08 Respighi: Suite: Belkis, regina di Saba (Belkis, Queen of Sheba). 1.33 Erkin: Kocekce. 1.43 Mozart: Quintet for piano, oboe, clarinet, bassoon and horn in E flat, K452. 2.08 Purcell: Timon of Athens, the Man-Hater, Z632. 2.31 Bartok: Violin Concerto No 1 Op Post, Sz36. 2.53 Martinu: Symphony No 1. 3.30 Dvorak: Slavonic Dance No 10 in E minor, Op 72 No 2. 3.36 Mendelssohn: Venetian Boat Song, Op 30 No 6 (Songs Without Words, Book II). 3.40 Vivaldi: Concerto in F for violin, two oboes, two horns, bassoon and cello, Rv569. 3.53 Part: The Woman with the Alabaster box for chorus. 4.00 Faure: Piano Nocturne No 6 in D flat, Op 63. 4.09 Borgstrom: Music to Johan Gabriel Borkman. 4.21 Haydn: Overture: Sinfonia (L' Isola disabitata). 4.31 Mozart: Overture: Der Schauspieldirektor, K486. 4.35 Schubert: Impromptu No 2 in E flat, D899. 4.41 Delibes: Suite: Sylvia. 4.59 Reutter: Ecce quomodo moritur justus. 5.06 Mendelssohn: String Quartet No 2 in A minor, Op 13. 5.36 Bach: Ricercare a 6 (Musical Offering, BWV1079). 5.45 Kutev: Symphonic Dances. 5.56 Noskowski: The Highlander's Fantasy, Op 17. 6.05 Reinecke: Ballade for flute and orchestra. 6.13 Bach: Quintet No 4 in E flat for flute, oboe, violin, viola and double bass, Op 11.</desc>This renders the add-on effectively unusable in the UK because without external XMLTV, the EPG is too lousy to use and with external XMLTV, it crashes. It is worth noting that the Simple IPTV PVR plugin parses the same XMLTV file perfectly and also loads the channel icons and the graphics for each program, which vbox.pvr doesn't seem to be able to do.
I hope this is enough detail to track down the problem - please let me know - I tried to find the bug in your source but didn't find it yet, so please let me know where it is when you find it - thanks
The text was updated successfully, but these errors were encountered: