HDHomeRunDVR: Difficulties with audio-only streams (II) #65
Comments
|
Sorry for the delay in responding; I have to read this a couple times and absorb it, but one thing I can point out now is that I just pass along the MIME type reported by the HDHomeRun system (DVR or tuner). I haven't personally encountered a true audio-only stream, mine here are also MPEG video streams. But, internally to Kodi if it doesn't find a valid video stream it may be triggering an audio-only scenario. It takes the MIME type I provide as a "hint" of sorts, it still does things on it's own to try and figure out how to best handle things. Changes I've made can deal with a pure audio stream by not assuming certain things, and if I tell Kodi it's "audio/mpeg" it will pop open the music UI instead of the video UI. Let me read this through a few times to see if there is anything I might be able to take an action on for you. |
|
Hi Mike, First up, it's not just Milkdrop / Milkdrop 2. I ran a test with the "Vortex" visualization, and the exact same behaviour ocurrs. I therefore expect any visualizers that come bundled with Kodi by default to illicit the same results (as yet untested by me). Secondly, Considering we're streaming from the RECORD engine, it seems to only output an MPEG2 transport stream, as when initiating recordings, the nature of the TV station doesn't matter. Be it TV or radio, a recorded section will result in a "*.mpg" file on the NAS. The Radio channels, are shown as "MP2" in the visualization screen for the skin i use, which is essentially correct. The audio "portion" of the stream, on European DAB radio is MPEG1 - Layer 2 (MP2). That however does not rule out the presence of a video portion coming from the HDHomeRun. I just cannot be sure of it's presence at the Kodi side of the link (yet), and / or whether it is coming from OTA data, or perhaps pseudo-generated in post-processing by the HDHomeRun to satisfy the MPEG2 standard. Neither can I be absolutely sure that what I see on my TV is "video" data per-se or some other kind of stream data displayed as video on-screen. What I can be sure of, is that the other HDHomeRun PVR addon (without DVR functionality), when using the "VIEW" portion of the HDHomeRun device, does not cause any issues, whereas what we're doing via the "RECORD" engine is causing mayhem. I will attempt another recording over the weekend, and analyze it with the MediaInfo tool. Maybe that will yeild more info as to how standards-compliant, the mpeg2 transport stream the HDHomeRun RECORD engine outputs, really is. Dan / Gib. |
|
There is definitely a difference in what the RECORD engine throws out as opposed to the tuner on it's own. I can prove it rewrites some of the TS information, but I'm not nearly astute enough to know exactly why or if that can be problematic. If I had to guess, I would point at the extra packets you get at the head of a RECORD engine stream. There aren't many, and they are standards-compliant, but it could be screwing up ffmpeg somehow. The observation that the built-in HDHomeRun addon works OK is a good one. It works by just sending the URL to the tuner to Kodi and let's Kodi do all the work. Not that it's a bad thing, just a difference. (Kodi's implementation can't handle seeking on the RECORD streams, which is the primary reason I had to make my own). Can you get me a recorded .MPG file that exhibits this problem again? I'm about out of TODOs on the PVR right now with the new release pending, I have time to play. I can even cook you up a custom version of the PVR to help assist you in your diagnoses, we can do things like force the media type a certain way or try screwing around with the PIDs in the stream. If this can be narrowed down to a TS concern, and if it doesn't require writing an entire demuxer to solve, there is always the possibility the data can be screwed with enough as it's being read to trick Kodi/ffmpeg into doing our bidding. I don't think I have the skillset to do much more than maybe screw with the stream metadata, but if the condition can be detected easily enough, maybe!! |
|
Hi Mike, Will do m8. I will PM you a link via one of the forums to whatever I can conjure up. The curious thing is that it doesn't always fail. I have had random (albeit limited) occasions where the two do play together, but this is a rarity rather than the norm, which leads me to believing that it's something about the stream and / or how it's delivered from the HDHomeRun box that causes the issue. It's probably also the case that my kludge of a network setup is also contributing to the issue / confusion. Unfortunately, there's no way I can work around this part of the problem. I do feel that a solution is within touching distance though. I have the DVR subscription, and I'd like to be able to use it, without having to go back to the days of using 3rd-party middleware like NextPVR like i did when i had my old TV Tuner card. Watch this space. Dan / Gib. |
|
I took a sample, but the filesize is a bit too big to upload. Needless to say, what I've seen via MediaInfo is not what was expected... General Audio Menu I've obfuscated the NAS name, but everything else is untouched. I intend to look for another radio channel that may not have the "Menu" element, like what was displayed in the screenshots for comparision. The Spectrum visualizer seemed to work without issue when tuning up Radio 5 Live today, but will need to do this again after a sizable break to determine it's not conflicting. I will do some more testing and post back once that's done. It would seem, at this stage that since there is no video element present, that my initial supposition was incorrect. But, the "Menu" element, might be related to the "Red Button" data that gets broadcast alongside the channel, that on TV stations allows for modern TV-text and red-button services (for BBC channels), and that may be causing an issue. I also need to get a long ethernet cable to connect the switch the HDHomeRun is connected to, directly to the router and rule out the Powerline adapters, but that may be a while down the road, due to lack of available funds at the moment. EDIT: I also plan to attempt playback of the pre-recorded stream, and see if there's any difference. I've tried altering buffer sizes before, without much difference, but if all-else fails will try again. Currently, all buffering settings are at defaults. Dan / Gib. |
|
No worries. Since it's an MPEG-TS file you could also just crop it at a
reasonable size, it won't affect the outcome (use a multiple of 188 bytes).
Ideally I would like to help compare a recorded stream from that channel
against one straight from the tuner. You can grab a stream from the tuner
with something like cURL or even wget pretty easily.
Based on your descriptions I think we may find a key difference in the
transport stream metadata packets.
…On Sun, Nov 3, 2019, 4:23 PM Dan Morgan ***@***.***> wrote:
I took a sample, but the filesize is a bit too big to upload. Needless to
say, what I've seen via MediaInfo is not what was expected...
*General ID : 4161 (0x1041) Complete name : \**********\HDHomeRun\5 Live
Sport\5 Live Sport 20060515 [20191102-1200].mpg
Format : MPEG-TS
File size : 1.38 GiB
Duration : 3 h 5 min
Overall bit rate mode : Variable
Overall bit rate : 1 064 kb/s
FileExtension_Invalid : ts m2t m2s m4t m4s tmf ts tp trp ty
Audio
ID : 1402 (0x57A)
Menu ID : 5632 (0x1600)
Format : MPEG Audio
Format version : Version 1
Format profile : Layer 2
Codec ID : 3
Duration : 3 h 5 min
Bit rate mode : Constant
Bit rate : 96.0 kb/s
Channel(s) : 1 channel
Sampling rate : 48.0 kHz
Frame rate : 41.667 FPS (1152 SPF)
Compression mode : Lossy
Stream size : 128 MiB (9%)
Language : English
descriptor_tag_extension : 6
Menu
ID : 1400 (0x578)
Menu ID : 5632 (0x1600)
Duration : 3 h 5 min
List : 1402 (0x57A) (MPEG Audio, English) / 7210 (0x1C2A) () / 7201
(0x1C21) () / 7270 (0x1C66) ()
Language : English*
I've obfuscated the NAS name, but everything else is untouched. I intend
to look for another radio channel that may not have the "Menu" element,
like what was displayed in the screenshots for comparision. The Spectrum
visualizer seemed to work without issue when tuning up radio 5 Live today,
but will need to do this again after a sizable break to determine it's not
conflicting.
I will do some more testing and post back once that's done. It would seem,
at this stage that since there is no video element present, that my initial
supposition was incorrect. But, the "Menu" element, might be related to the
"Red Button" data that gets broadcast alongside the channel, that on TV
stations allows for modern TV-text and red-button services (for BBC
channels), and that *may* be causing an issue.
I also need to get a long ethernet cable to connect the switch the
HDHomeRun is connected to, directly to the router and rule out the
Powerline adapters, but that may be a while down the road, due to lack of
available funds at the moment.
Dan / Gib.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#65?email_source=notifications&email_token=AAFMMB7ISDPKFIWTKWZPM5DQR46MTA5CNFSM4JFM43WKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEC55FJY#issuecomment-549180071>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAFMMB6HRBKZSKUGLSMGSNTQR46MTANCNFSM4JFM43WA>
.
|
|
Well I feel like an idiot Mike. I should of tried this before, but here goes: Tried playing back the recorded stream via Video -> Files section: Success. Just to be sure, tried another internet-based audio stream (Digitally Imported via the AudioAddict Plugin): Success. So I decided to do a last ditch change... I modified both buffers for the Zuki HDHomeRun plugin to maximum permissable values (Read buffer: 32Kb, Ring Buffer: 16MB), and guess what? Success! I did a restart of Kodi as I've had false starts like this before, whereby It's OK on one run, not on the next. Again, success. Early days yet, but based on what I'm seeing, Milkdrop must be running out of audio to process with default settings, thus causing a stream freeze at the beginning. Another thing I've noticed is that at default buffer levels, starting the stream with no visualizer enabled, then enabling Milkdrop 2 via the OSD after the fact also works. Milkdrop does cause a micro-pause on start as it loads any and all presets from it's presets folder prior to selecting which one to start with, but at this stage it looks like a simple buffering issue. I will monitor it over the coming days, but since "maxing out" the buffer settings in the addon doesn't seem to cause any ill-effects elsewhere (yet), I'm inclined to leave it as is. Dan / Gib. |
|
Awesome! Are you on Krypton or Leia? The setting should be called "DVR stream read chunk size" on Leia. A little background for these settings to assist you:
So here's what I think happened when you bumped it up to 32K (again, kudos!). Instead of the default 4K of data being sent back to Kodi for the initial stream read, it got back 32K. There was probably enough TS data in the 32K for it to actually figure out what it was doing, as opposed to making a bad guess. There is a very very short timeout Kodi uses during this "what is it" phase, so I'm guessing the timer doesn't start until after the first packet is delivered, so viola - you fixed it! I really didn't expect that setting to be of any value to anyone, but I'm very glad I left it there. I'm thinking it may be a good idea to sacrifice a few milliseconds of "start stream" performance to ensure that more than 4K of data is ready for Kodi anytime a new stream is started. I'll do some math, but I think we can go notably higher than 32K, perhaps 128K or 256K. That's not a lot of data and I don't think anyone would notice. Such a thing may also help with the momentary audio drop that happens when starting a stream (it's not just you, I get that a lot too). LMK how things keep going on your end, I will play with this on my end. My preference would be to apply some effective change so that people can leave the defaults in place and still have things work right. YOU DA MAN!!! |
|
Did some tinkering, you may have found the best way to solve this, it doesn't look like a code change will help. Kodi only asks me for the chunk size once when starting a stream so I can't do something dynamic like make it bigger the first time through. That said, I can force the stream to buffer a certain amount before indicating that the stream is "started", which really isn't doing much of anything. Regardless of how much data I have in there, Kodi still only asks for the "chunk size" as specified in the settings. Based on my experimentation, there is no measurable benefit to buffering this additional data during startup, the total time to start a stream and complete the initial read is the same -- it just moves those milliseconds from one place to another. The key seems to be what you have uncovered -- making sure Kodi has enough data up front to make a "good choice" when it analyzes the stream. If you don't mind, can you see if you can still resolve the problem with increasingly smaller chunk size values down to 8K? Obviously 4K isn't enough, and I think 32K is too high to be a default for most purposes, but if 8K solves it I can change the default. If you find 8K does not, but 16K does, I would like to send you a version that allows for a 12K value to see if that works. My reasoning for a 12K test is that a 96kbps audio stream would take 1 second to stream that much data, which seems like a good breaking point to hit. I don't think Video streams would be affected at all, the bitrate is plenty high enough that a change from 4K to 12K shouldn't be noticeable for most people. Thank you again Dan for all your research. It's very hard to help without having direct access to the streams you do, and I am very thankful for the time and effort you have put into helping make this work as well as it possibly can within the confines of the Kodi world! |
|
Hi Mike, I've only briefly scanned your two replies at this point, as I'm off to work shortly, but one thing I wanted to say, from my experiences with Milkdrop / 2 in both Winamp and Kodi, is that it displays it's visualization slightly in retrospect. i.e: It needs a decent sized "back-buffer" of audio data to process to display itself. This was my thinking after seeing other forms of playback (as mentioned above) work without issue. I realised that all other playback attempts would probably be using Kodi's own buffering system to play, whereby there would be enough data for Milkdrop to work. Vortex doesn't work at all for me, on any audio sources. It doesn't crash, but there's simply nothing displayed. Since it was ported from XBOX, there may be an issue there, but that's not your addon's fault. Secondly, I don't use it anyway, so I can live without that. I will read your replies properly, tonight, when I get home, but I would suggest leaving this issue open until I can post a proper response, and maybe up till the weekend, so I can do some more real-world testing. There are a couple of shows I plan to listen to over the next couple of days, where I can see if this is resolved properly. I've had false dawns before, so don't want to announce a glorious success only to be back at square one again. I'll post another reply later. I'm on Kodi Leia 18.4, but the settings are still named as they always were for me. I might have to reset / remove the settings file and set it up again to see the settings name changes you mention. Dan / Gib. |
|
I did think of something I can do to help in code; I can strip out that useless metadata the RECORD engine always sends when a stream starts. I never bothered since it was innocuous, but it should be pretty easy overall. This would send more valid data to Kodi on the first read, which would increase it's chances of getting the packets it wants. I forget exactly how big this metadata is, but I think it's around 1K, which is pretty significant -- 25% of the first chunk (using the 4K default) is garbage. Oh, and the strings for the settings are actually in a "strings.po" file, the settings.xml has the internal names only, replacing that would just cause you to lose your settings. Cosmetic regardless, the setting still does the right things no matter what it says it's name is ... |
|
Hi Mike, I did as you suggested and reset the "DVR Ring Buffer Size" back to 1MB, and tried 8KB on the "DVR Read Chunk Size", no go on that one. As with 4KB, you get about 1/4 to 1/2 second of audio, before the audio stops. Kodi isn't "frozen" at this point, and Milkdrop 2 is still active. What causes the eventual freeze is any action that will force Milkdrop 2 to attempt to load a diffferent preset (e.g: Exiting Vis screen back to main menu, or attempting to stop the stream). Set to 16KB seems to work as with my initial setting of 32KB. I'm running the Zuki HDHomeRun add-on from your own repo, so any updates to that branch will automatically be applied to my systems here on each Kodi startup. Another thing i've noticed that can cause issues, is the "commoncache.db" database not being clear prior to tuning or playback, and I think this might of be partially responsible for the weird timing issues we saw on previous versions too. I no longer get those time discrepencies btw, but if Kodi crashes for some reason, due to issues unrelated to this add-on, and subsequently, the commoncache.db file is not empty, I've seen hangs due to that too. It's simply a case of deleting the file, and allowing Kodi to recreate it on launch. Carry on and send me your 12KB mod when ready and i'll try that. Dan / Gib. P.S: Don't worry about the string labels, that was just my P.Poor memory lol. |
|
I stand corrected, it's just hung again after exiting and re-launching a new Kodi session @16KB. This was the behaviour I saw last time I messed with the buffer settings. I'll try going up one step (again). |
|
It's looking like 32KB is the minimum that will work consistently with Zuki and MilkDrop 2. I notice that (since Kodi is set to switch instantly to Music Vis Screen on playback start), the "Buffer Progress Bar" displayed in this skin is only about 1/3rd to 1/2 full at stream start. It goes to 3/4 full after about 500ms (gradually) and floats about 3/4 to 4/5 for one or two minutes before eventually going full. Please bear in mind that this might be down to network latency, Powerline QoS, etc. But as it stands, 32KB is the only setting that works consistently across Kodi sessions on my setup at this time. Acronis backups ocurring might well throw a spanner in the works, as even though I have it's resource / network usage for all backup jobs set to "minimum / low", this will only have an effect on the PC the backup task is running on at the time, not overall network saturation. However, I can work around this issue if / when it occurs. EDIT: For the reasons outlined above, i wouldn't nessecarily say that my experiences should be used as a benchmark to base this add-ons defaults on. My setup here (borne out of necessity rather than desire), is far from "mainstream" by any means. At least now, I know what works and what doesn't, and potentially why. I can work around this issue now (potentially). Dan / Gib. |
|
The powerline adapter is my main concern with such a large chunk size. The small chunk size is used to combat intermittent latency problems. The bigger you make it, the longer the PVR will wait to get that much data before sending anything back to Kodi, which could cause drop outs and buffering. Even on a fast network if I set that value super high that ultimately happens. I looked into how Kodi is using this chunk size value, it turns out that it doesn't appear to be that important to it. It's primary function is to tell ffmpeg how big of a buffer to allocate. Here is what I am going to try tonight here, I'm going to tell Kodi to always expect a 32K chunk, but send less (the configured chunk size) if that much isn't available yet. My expectation is that there will be zero impact to anything at all. If that turns out to be the case, I will then have the ability to wait for the first chunk to be a full 32K before it goes across, which I'm 90% certain will still solve your problem. If that pans out, then there would be 2 settings: the existing chunk size one and a new "initial chunk size" one that I can default higher without impacting startup performance at all. The default for that still will be too low for you (probably 8K), so you'd need to change it to 32K in your situation. So, if that all works out and my assumptions are correct, we can solve your problem without the potential for momentary latency issues causing you new problems. Also, if you can split that file you have into something reasonable, I would like to look at what TS packets might not be coming across until we hit that magic 32K boundary. It could still be something that SiliconDust and/or Kodi may want to address. I think 50MB would be a good size? Long enough to play back for a while, and certainly more than enough to examine the first 100K or so. Heck, if you can even get me the first 128K of the file I can work with it. I'm going to find the old sample you sent me from "Live 5", I know I have it here somewhere, but I recall that particular one ended up working fine in the end. It would be a good comparison point, though. BTW - This is fun, thank you. I greatly appreciate all of the detective work you're putting in to help me try to help you sir! |
|
Hi Dan, I may need to extend an apology, after taking a look at how Krypton and Leia differ, this could all be the result of a mistake on my part. Krypton always used exactly a (you guessed it) 32K buffer size for ffmpeg for PVR addons. Leia made this configurable so ... I configured it. I give it a 50/50 chance of success, but can you try this build? (2.4.1~alpha1) : The difference here is that I am telling Kodi to always use 32K for the buffer size, but still enforcing the same minimum read chunk size, which should realign things with how Krypton worked. To test you will need to change the PVR setting back to 4K. If this works, all I need to do is rename the setting back to what it was (and what you see, apparently). If it doesn't work, I would like to follow up with a build that adds the proposed "first chunk of data" being 32K then falling back to the setting. I hope it works, even if it means I completely screwed the proverbial pooch :) |
|
Hi Mike, Downloaded and gave 2.4.1 - Alpha1 a quick try this morning. Unfortunately no joy. The audio plays for longer, or at least it did with this test, at about 5 seconds or thereabouts. But the instant Kodi switched from EPG to Music Vis (factoring in the Milkdrop startup delay), I lose audio and we're back to the same scenario as before. No more time this morning. EDIT: I've tried all the settings up to 32KB Mike, and the only difference between this version, and 2.4.0 is that the stream plays for a bit longer before dying. 32KB is still the only setting that seems to work consistently. Here's another sample for you: https://onedrive.live.com/?id=7CAC01C8D93B566E%21254&cid=7CAC01C8D93B566E Dan / Gib. |
|
Thanks for the sample, will queue it up tonight. Based on this feedback and your test I think we may have reached the limit of what can be done in the PVR and it's time to go looking into Kodi again. Ugh. The stream lasting a little longer at 4K makes sense, the change allows the PVR to return more than 4K if it has it available, with a cap at 32K. The stream is being buffered in the background while Kodi starts things up, so there is usually going to be a handful of extra bits available at that time, but those extra bits run out quickly. I think the change itself is a good one to keep in place, it allows Kodi to receive more data more quickly when the PVR buffer happens to have a surplus. For now it seems the only way to get this to work for you is to ensure we always send Kodi lots of data, even if we have to wait for it. I don't think adding something to buffer the first 32K and then falling back to smaller chunks would help, but am willing to cook that up if you think it might. Let me see if I can figure out what Kodi is asking for and why it's unhappy if it doesn't get these very large chunks of data every time. I think you're stuck with the minimum 32K chunk/read size setting for the time being. Hopefully it won't cause drop outs or buffering on you and keep you fully functional until we can figure out the next step to try or finger to point :) |
|
Just to make sure I'm looking at the correct issue, playback technically "continues", as in the progress bar moves along, but the audio itself ceases to exist? If that is what happens, I was able to duplicate it by making things take longer than they should artificially; this gives me something to work with - thank you! |
|
There is definitely some wacky stuff happening here with that sample, but so far they seem to be seek related. I don't see Milkdrop (or 2) causing it specifically, I can replicate it easily enough by trying to "resume from xx:xx" on Windows or by seeking at all from a Wifi connected Android device. In these cases, Kodi is asking to seek to very near the end of the file, then reading whatever data is left, and aborting completely because the end was reached. More as I have it. As a side note, I've remapped a Recorded TV category of 'audio' to 'series' like I did with 'news', so next version you shouldn't see an 'audio' folder anymore. I wish SD would stop changing these things; even in their own application you won't see "news", "special", or "audio" recordings anywhere but "All Recordings". |
|
Hi Mike, the buffer progress bar is still going up and down at about 3/4 after audio is lost, and Milkdrop 2 is still "active", (not frozen). I can't be certain if what's being displayed is as a response to the sttream, because Milkdrop will show itself even with a silent track. All activity freezes once a preset change is forced, which usually happens at a window change, or when playback is stopped. It doesn't freeze at normal transitions between presets in the same window (i.e: Milkdrop auto preset changes). Hope that makes sense. Dan / Gib. |
|
Well, I realize it's not much of an answer but everything I'm seeing so far is inside of Kodi. I'm having extreme problems with it trying to seek beyond the end of the stream repeatedly, which ultimately causes it to die. It appears to be related to either an issue with calculating timestamps and subsequently trying very hard to seek into a place that just doesn't exist, or has to do with the lack of a video component on the stream. As stated, it isn't much, but digging into ffmpeg is extremely difficult. |
|
Another thing I've noticed, is that even at 32KB read buffer, it's not possible to do a direct channel change between one radio station and another, as the stream also dies in the same way as before. This and the fact there is one station (TalkSPORT) that broadcasts as 64KB/s, which I can't tune at all, even at 32KB buffer, as the stream dies exactly the same as at the 4KB setting. Since I run my setups in portable mode (to keep Kodi's clutter off my SSDs), I can start a vanilla install fairly easily (which I also tried - no change). But I can do some testing in this way with debug logging on, where the only errors shown will be either Kodi or the addon. I may give this a go over the weekend, and send you the findings. I don't run any dodgy addons, but I do have a couple of addons, including the skin I currently use, that throw non-critical errors that will add to the confusion in the log (log spam). Dan / Gib. |
|
I don't think it's any addons, so far it has something to do with how Kodi/ffmpeg is generating the timestamps for the stream. Unfortunately that math is buried in non-debuggable code, so I'm building Kodi 18.4 from scratch with ffmpeg debugging enabled to dig deeper. Info as it becomes available. ffmpeg spelunking is not an enjoyable process, but it has borne fruit for us before. This may even be a side-effect of the changes we successfully got into Kodi last time; it's looking suspect. |
|
Hi Mike, Disregard my previous comment about 64K streams. I was unaware at the time that an Acronis backup was in progress on the HTPC. It seems any buffer underrun is enough to cause the issue to appear. It's my belief that the PVR code in Kodi needs an overhaul. It's optimised for the old style of PVR backend, where the hardware separates analogue "TV" and "Radio" channels into different categories / streams, where TV channels are audio / video streams and Radio are pure audio-only streams. This isn't the case with modern digital TV tuner hardware, where everything is an MPEG2 transport stream. I don't think Kodi's PVR module is optimized to deal with that, any FFMPEG issues aside. Sorry m8, I've not had time to do the additional testing I had planned to do over the weekend, hopefully I'll be able to do some testing this week. I think there may be a specific "FFMPEG" setting in component-specific debug logging, in Kodi. If so, when I do get around to doing some additional testing, i'll enable that, and send you whatever I find. |
|
No worries. Getting a fully debuggable build in place helped a lot. I'm 75% sure I know what the problem is. I now believe the problem is the stream data itself. 80% of the data is junk (program 7201). Since the primary stream is low bitrate, ffmpeg is actually unable to find the PTS (timestamp) it wants within a reasonable number of tries, so it's giving up. This is why the large chunk size helps, it's getting more of program 1302 each time, so it can latch onto it. I demuxed out program 7201 and will try that .MPG as-is later tonight. FWIW, the file went from 272MB to 55MB without the junk data in it. It plays fine and seeks fine in non-Kodi players like this, so we'll see. |
|
Hi Mike, OK m8. There is a lot of ADS (Alternate data streams) broadcast alongside the actual program data. OTA EPG data (ironic that the HDHomeRun can't make use of this and instead charges you for a SchedulesDirect Subscription, which is far less complete) and TV Text / Red Button / menu data as per the screenshot I previously thought was "video" data. at 32K, i did experience a micro-stutter at the beginning of the stream, but it only lasted for a split-second, and never more than once or twice while the buffer was low. Once the buffer catches up / fills, no issues with stuttering or buffer dropouts. There's no issue with throughput on my Powerline adapters once a successful playback of PVR or any other data over the network from my NASes actually starts. It's just that getting a stream actually started can occasionally take a while if there's network saturation. For example, late last night, when the HDHomeRun was recording two seperate channels to one (or both) of the NASes, I'd tried to queue up a music album to play. It took about 5 minutes for Kodi to become responsive and begin playback, because of network saturation. But once started, no issues thereafter. I hate the QoS on the things, and have found the least problematic way of getting around it is to set it to one of the 4 modes it offers that i'll never use, in my case VoIP (I have no VoIP hardware), because then it won't block everything else to prioritise one type of traffic. Also, TP-Link hsve a nasty habit of releasing hardware then abandoning it firmware-wise, in preference to just pushing out a new hardware version. I'll never buy anything by them again. |
|
If you have something you want me to test prior to me doing my own testing, let me know and i'll give it a go. |
|
I believe I found the cause and a solution for the main problem I'm seeing here. It is related to the fix we made in Kodi the first time around. My first attempt to quick-fix it failed yesterday but will take another crack at it. It seems like it needs to be more invasive than I was hoping. What I found was that pretty much all seek operations on an audio-only stream fail to an extent. They can recover most of the time, but I can get certain operations to fail 100% of the time. What's happening is that my PR to Kodi added stuff to say "if there is no video, use audio to set things up". Works OK when you are starting a new stream, but seeking requires Kodi to find the timestamp it wants. My PR didn't take that into account properly, so it's trying to find an infinitely large timestamp, which it never finds. Depending on the operation, it either takes way too long to figure that out, or if you happen to be looking for the end of the stream, will fail outright and stop since the PVR keeps saying "I have no more data for you". The problem I'm having fixing it is that there is all kinds of metadata stored for Video streams that don't apply for audio-only streams so I can't use that to fix things. Since Video always has an Audio stream with it, I can't just hack it for "IF AUDIO", that's always true. I may be able to get you a new kodi.exe to try (18.4 Leia) if I can hobble something together, but forming it into a PR that Kodi may accept will take time and I assume will get kicked back more than once. I agree with your thought patterns, audio-only MPEG-TS is just plain problematic for Kodi. It's pretty clear after digging in that the code was written to always expect both a video and an audio stream to use. How this is affecting the visualization is unknown to me right now, since I can't seem to make that fail, but I'll keep going and see where we end up! FWIW, I'm still not seeing anything wrong with the PVR module at least. It seems to be doing exactly what it should, it's just not being asked to do the right things .... (Can we rename this problem to "Dan and Mike's White Whale"?) |
|
Ha. Found it. What's happening is that when the stream stalls out or has a sync error, the visualization is being destroyed, but it's not being unregistered from the audio player (ActiveAE). The audio player still thinks it has a visualization and calls into a deleted object and poof. This is why it only happens when you have a visualization enabled. Locating point to fix it, will pass along a kodi.exe when found and if it has the desired effect. |
|
Good catch m8! I look forward to testing whatever you provide. Hopefully we are within touching distance of a solution to this annoying issue. Will probably have to spread any testing over a few sessions / days if it's an initial success to be sure. Well done in any case! :) Dan / Gib. |
|
Don't be too impressed, I can't fix it properly. I have no idea how it "should" behave so I'm going to open an Issue against Kodi instead for this. But, I have a quick-fix hack in place that should bypass the issue and prevent the lock-up/crash from happening. I'm cleaning out the build folders and will cut a Release build (this time from VS 2017 instead of 2019 - sorry about that!) and share it. I built a couple visualizations from source and was a bit wrong :) What is really and truly happening with 100% certainty now is that the visualization is entering an infinite loop because it's being asked to process zero bytes of data, and there is no protection against that. They (Kodi) may want to fix the visualization(s) to deal with this, or they may want to quick-fix hack it like I did, or they may decide that there is a more systemic problem they need to solve (like stopping the visualization and restarting it when there is data again). Anyhoooo ... build will take around an hour to cut and test. I am leaving my currently ill-fated PR for the other problem in there as well. It should be stable enough to use for at least a while. The only remaining problem I've seen is that seeking on audio-only programs is dreadfully slow. If that's not a problem for you, I'm willing to let it drop. If I ultimately decide to write my own demuxer that problem will go away. I'm sure it has to do with the constant assumption that MPEG-TS always has a video elementary stream. TTYL, in about an hour. |
|
See what this does for you ... https://1drv.ms/u/s!AgEGEEVzGNq-jIUAK4gCc4AWiV0Smg?e=A12vr2 It will take me until tomorrow to enter a proper Issue, but I do honestly think that Kodi needs to deal with/fix this on their own. Hopefully this .exe will solve the lionshare of your concerns. |
|
Well the initial test was successful. However this was also the case with the previous compile. I'll have another go tonight when I have more time on all 3 systems, and let you know how it goes. Many thanks for this. Dan / Gib. |
|
Hi Mike, Done a few test playbacks on all 3 systems, and have had success on all 3 at the default stream chunk size of 4KB. This is only across one session on the HTPC/Laptop, and 2 on the desktop. But the results (thus far) look promising. I noticed on the Kodi RSS feed a news item for Kodi "18.5", so i'm guessing they've put out an update. So we've probably missed the bus on getting a permanent fix into Leia, as I doubt there will be another release on the 18.x branch from this point. A check of the website will confirm it (Not had chance yet, at the time of writing). So if you could build me a copy of 18.5 with your fix / hack contained, i'd apprieciate it. I'll keep testing for the rest of the week and post up a definite conclusion sometime over the weekend period (The time of the week I do the most radio listening), and let you know one way or another. I'm assuming you'll be contacting the Kodi team with regards this issue and your fix? If so, and you wish to use any of the assets I've sent you so far, feel free to do so. Thanks for your persistence on this issue, I greatly apprieciate it. :) Dan / Gib. |
|
Yes on all counts. There is a new .zip for kodi-18.5 in the same OneDrive folder for you: https://1drv.ms/u/s!AgEGEEVzGNq-jIUAK4gCc4AWiV0Smg?e=A12vr2 I strongly suggest installing Kodi 18.5 first, then replacing the .EXE. You don't necessarily need the .PDB (debug symbols), but it may enable better crash dumps if kodi crashes. I'm going to take one more crack and finding a reasonable place to suggest a fix for the second issue since it only happens with streams like MPEG-TS, but if not I'll enter the Issue with what information I have for them. I still need to redo the PR for the first problem to their liking. |
|
Here are the relevant links to the generated PR and Issue: Problem 1 - PR (not accepted, needs work): xbmc/xbmc#16910 |
Hi Mike, Thanks for that. Will do as advised. I'll keep the kodi.pdb file in there if you can make use of the crash logs it may generate, should we encounter any further issues? If it's of no use to you personally, i'll bin it. I've monitored (followed) xbmc/xbmc#16949, and should recieve an email if the topic gains any traction. If you need me to validate your/our findings by posting in the thread, let me know. Otherwise i'll let it run it's course. Thanks again. Dan / Gib. |
|
I think we're OK to let it fester for a bit to see if anyone picks it up and runs with it. I redid the PR tonight to hopefully better align with the expectations, if that ends up being accepted I can cycle back to this Issue and see if I can find the root cause and submit a PR for that as well. I can pretty much find anything given enough time :) Keep me informed if you want/need a new kodi.exe (for Windows, of course), I'm happy to build you whatever you need, it only takes me about an hour from start to finish. In the event that the PR/Issue are only fixed/accepted for Matrix I am fine with back-porting them to Leia specifically for you to use. I intend to do the same here on my PCs. |
Thanks for that. There won't be any need for you to do me any new builds, unless there's an 18.6 release that doesn't fix the issue (unlikely in my view, in both cases). I expect this to remain outstanding until Matrix goes gold, at least. I gave up installing nightlies a long time ago, as Kodi has enough "issues" in regular builds / releases, without adding alpha status kodi builds into the mix. And to be fair to the Kodi team, a lot of the issues are often related to 3rd-party addons anyway, so i'll stick with 18.5 for now. I spend too much time as it is, diagnosing problems with my setups (not just with Kodi), so i'd like to spend at least some of my free time actually using them for their intended purpose, rather than "fixing" them. With that being the case, I won't be installing Matrix until it becomes an official release. If I do get any issues, i'll be in touch. However, since we've ascertained that the issue is not Zuki's fault, you can close this issue if you so wish. I can always comment on it after the fact, or message you on the Kodi / SiliconDust forums, or failing that, open up a new issue here. Thanks again for your help, you helped nail down a really frustrating / annoying issue. :) EDIT: There is a minor quirk i have noticed with these custom builds, and I don't think it's to do with anything you've done, but I have seen a "fullscreen" of random white (on black) random ASCII characters appear on screen / window transitions. It's only visible for about 1/10th of a second, but it's there. Not a big issue for me, Kodi works as expected in every other aspect, so don't devote much (if any) time to this on my behalf. Dan / Gib. |
|
I'll PM you contact details on the Kodi forums (email, social media, etc). That might be easier, since I know your not a Kodi forums regular, and i'm not a SiliconDust forums regular either. Dan / Gib. |
|
I decided to throw a PR for the second problem Kodi's way as well. I assume it will, as usual, require extensive changes and explanation for them before they accept it, but we'll see. Given that, I concur with closing this issue since the dead horse has clearly been beaten beyond recognition and the ball is now in Kodi's court :) Ping me anytime, sir. Always happy to do what I can for you. |
|
Something else I was thinking about the other night, was why this only rears it's ugly head (in the main) on PVR streams, and not when playing back other audio only streams. I know we looked at the fact that we're trying to deal with an audio-only MPEG-TS stream here, but the way I understand the problem, is that, in theory, it should suffer the same problem on any audio-only stream, regardless of the container format. For example, I have a "Digitally Imported" account, and that plays back fine. I have over 50,000 audio tracks, not all of which are music (Some are audio books, etc). Then It struck me. The one difference between, say Radio 5 Live, and my other audio sources is the bitrate. As Radio 5 Live is a Sports commentary / Talk channel, it almost never has music. So they have reduced the bitrate on that channel down to 96Kbps. Similar issue with TalkSport, an independant station that also broadcasts OTA on analogue, on the AM band. On Digital, they have the bitrate at 64Kbps. I'm therefore wondering if, the buffering engine in Kodi might be running out of data because of the small data size of the packets being sent. Remember, at 32K buffer in Zuki, we have no issues. Network connections do not handle small packet sizes very well, starting at a fairly low speed, gaining momentum as the connection proceeds. I've witnessed this myself with file transfers (via Windows Explorer) to and from my two NAS devices. To test out this theory, I need to try a station that has a higher bitrate than 96Kbps. BBC Radio 1 would be a prime candidate. I'll let you know how it goes, when I have time to revert the install(s) to stock Kodi and test. Dan / Gib. |
|
Well, yes and no. The problems we've found so far are pretty much exclusive to things where video is "expected" but not found. The lack of a video stream to provide the i-frames and the timing information for the audio stream seems to be where these issues lie. One of our problems is 100% because video is expected and wasn't there (what I called the "resume from" bug) -- back when we first fixed this, I didn't fully understand the ramifications of needing to wait for the stream data to be ready -- it was fine when it first started (from the beginning), but if you tried to start in the middle my original change said "you're good to go", but it wasn't. Everything changed took place in a function called "IsVideoReady()", if that helps to clarify how Kodi wasn't expecting this scenario at all. The second issue with the visualizations locking up could possibly happen from other sources, but I doubt it? This turns out to be (so far) a timing issue, in that Kodi detects that the audio has become out of sync with real time for whatever reason (paused, dropped out, etc) and corrects itself. Unfortunately during that correction it's still passing data to the Visualization, which couldn't handle it and locks up. My theory as to why 32K blocks worked is that Kodi has enough data to correct itself without needing to read more data, which bypassed the lock-up. I'm often wrong, this application is large and complicated, but if you run into more problems I'm going to bet it will ultimately be related to a stream/media type where video is typically present and it's not. Using a straight MP2/MP3 stream, for example, wouldn't run into this problem since it would never hit all this "DVD Demuxer" code where we found the issues. I'm happy to help as usual if you find something else, but nothing personal - I'm hoping you don't :) LOL. |
|
Hi Mike, Your probably right. I forgot about the "expected" video condition (I have bad short-term memory). I have a feeling I have had this crash when playing from the local library, but it's so rare, I can't remember the last time it occurred, and even then, it might have been for a differing reason anyway, with similar outward-looking results (visually). Sorry m8, thinking out-loud here. I will do the test though, just to rule it out as a contender. As for the last point... I hear that, LOL! ;) Dan / Gib. |
|
Mike, Can you check your e-mail spam folder please? I've sent you a message, via e-mail and a Kodi forum PM. Got a (potential) side-effect with your workaround: In the screenshot I'm getting both PVR info and audio info, where I should only be getting PVR on the home screen / main menu (LHS - Ignore the preview window on the right, that's supposed to be there). This only occurs on PVR streams, not local file playback, hence my hunch. It may be a skin issue, but it looks at this stage more likely to be Kodi detecting the stream (initially) as audio-only. Dan / Gib. |
|
Seems like a reasonable assumption, actually. The change tells Kodi the stream is ready when it's locked onto either audio or video now. Sigh. That's going to be a show stopper for the PR. |
|
Sorry for being a PITA m8, and sorry if i've invalidated any of the hard work you've put in so far, by reporting this. :( Dan / Gib |
|
Don't be! Way better to know it's flawed before mainline Kodi accepting it and causing all sorts of problems. I'm sorry it ended up not being correct. Might take another crack at it, but I doubt I'll be able to duplicate this one :( |
|
If you need any more assets / captures / logs from me, let me know. Always glad to help. Dan / Gib. |
|
Update: The skin author made some changes to the skin yesterday that solved the dual info display issue, that at the time I hadn't applied to my installs. So don't bust your guts trying to work around this at this stage. Visually, the issue is fixed (see before & after screenshots, below). There are still issues with Kodi's PVR module, of this I have no doubt. Before: https://imgur.com/kCoOhBs After: https://imgur.com/foltFGD While your changes have improved the issue, I think the whole PVR module is in need of a complete overhaul, as I mentioned before. It was designed at a time when analogue TV was still a thing, and has not evolved to be optimised for Digital TV, and more modern TV capture devices, like the HDHomeRun, as an example. At least, that's the way I see it. I cannot provide you a link to the skin's GitHub page as it's a private beta test, so you wouldn't have the access rights to view it. The skin author may post here in this thread, and I have suggested he grant you access, so you can test the interaction between your addon, Kodi, and his skin, yourself. He's had comparable issues to the ones I've experienced with the MediaPortal, and NextPVR PVR backends, so a common pooling of ideas, experience, and testing may be the order of the day here. As to whether re-opening this issue here is warranted, on the back of (potential) future changes to the code by yourself, I'll leave that up to you. :) Dan / Gib |
|
Hi djp952, gibxxi suggested i post something in regards to this issue. Looks like this is solved in the skin code. I did however find some odd behaviour in kodi using nextpvr and mediaportal backend (I assume this would apply to HDHomeRun too) Upon starting a livetv, the skin boolean Player.HasAudio is true, until livetv begins playback. Then Player.HasVideo becomes true. For me this is literally 1sec. Depending on your network setup and speed of the backend, this time would likely increase. A couple of issue in kodi;
Something seems a bit off on how the Player.HasAudio and Player.HasVideo booleans are filled when beginning playback. Not sure if this is a kodi or backend issue. Here is an invite link to the skin If you drop this file in any skin xml folder you can see the values being filled as you start live tv. It will look like this when media is playing |
|
@djp952 |
|
@mistervee, thanks for the invite! I'll install the skin and see if I can help in any way. I'm in the middle of a mandated updated to my PVR (the EPG data is going to disappear), but I'll do what I can. I did mention it in my related PR, and the Kodi guys think this is "normal" behavior and the the Player/Skin UI needs to be able to handle it. That said, I understand the problem and I actually have a couple ideas about it. I think there are a handful of issues with the Kodi videoplayer, this could be related to one of them. I personally have major issues when an audio stream format changes, for example. It's in the FFMPEG code somewhere, and it didn't used to happen under Krypton. It's difficult for me to narrow it down, I fear this concern would be equally difficult! |
|
Mike, I replied to your last e-mail (Verizon). If your not seeing it, you need to remove my email address from your spam blocker, or not send me e-mails via that account, as I can't respond otherwise. :) Dan / Gib. |
|
I got it! Haven't responded yet. Never fear on the dot release, I follow
kodi's weird version rules so "3.0.0" comes after "3.0.0~beta2". It will
update fine :)
Waiting on a reply from Nick K and then will push the button on it. Then
can go back and read emails
…On Sat, Dec 14, 2019, 8:09 PM Dan Morgan ***@***.***> wrote:
Mike, I replied to your last e-mail (Verizon). If your not seeing it, you
need to remove my email address from your spam blocker, or not send me
e-mails via that account, as I can't respond otherwise.
:)
Dan / Gib.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#65?email_source=notifications&email_token=AAFMMB6S62EK5E7WBMNNWCLQYV7V7A5CNFSM4JFM43WKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG4O2UY#issuecomment-565767507>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAFMMB42CL5LNBVB5BTLRPTQYV7V7ANCNFSM4JFM43WA>
.
|
|
OK m8, fair enough. Not trying to be "pushy", was just in fear of a repeat of the last attempt, is all. ;) Dan / Gib. |
Carrying on from the previous (now closed) issue: #48 and the conversation started here: #64 ...
We're treating radio stations on DVB-T/T2 as pure audio streams. They're not. They're MPEG2 streams, with video and audio, as I can see when tuning radio channels on my TV (Which has nothing to do with the HDHomeRun box).
https://i.imgur.com/YOme55I.jpg
https://i.imgur.com/x36CAoG.jpg
NOTE: The above images are not just showing static content. The synopsis text changes at regular intervals (at a guess, about once every 5-10 seconds).
The HDHomeRun VIEW Windows App, does not display the above in it's own app, you just get a black screen for radio channels.
It's my belief that the video portion may be being (purposefully) delayed to allow for DAB-based devices that have no video capabilities, so they latch onto the audio portion of the stream, ignoring / dropping the video portion, but that's just supposition on my part.
Here in the UK, digital radio stations are broadcast on multiple standard definition (SD) muxes. There are no radio channels on HD (H.264) muxes. NextPVR lists both TV channels and radio channels as "MPEG2" - i.e. a DVD-style MPEG2 stream with no differentation for either. There must be some kind of "flag" being broadcast signalling that it is a radio station, as the HDHomeRun is able to tell them apart.
Milkdrop 2 takes control of the graphics adapter whenever Kodi is told the source being played is "audio-only", it will not relinquish hold of that device until the stream is stopped. The sound portion of the stream plays for as long as there is buffered data to play, but since the video data has no output (because the graphics card is in use by Milkdrop), the audio cuts out as soon as it gets to a point where the audio will become too out of sync with the video which is not able to be rendered, therefore resulting in a freeze of Kodi as soon as any action is taken that will modify the play state of the current stream from the HDHomeRun. The skin i use shows buffer queue on screen for streamed audio, and it's not running out of buffered data (in as far as audio is concerned).
I have successfully managed to do "channel changes" when this ocurrs using the up / down arrow keys, and by keying in the channel number, e.g. "1" for BBC ONE (TV). The correct channel icon will appear in the music vis window, but no audio (or video) is played, again, for the reasons above stated above in my view.
This is the only explanation that makes logical sense to me (baseed on what I've witnessed). None of the other HDHomeRun PVR addons I've tried have any issue, with Milkdrop being active other than this one, and that may well be down to the fact they use the "VIEW" portion of the HDHomeRun box for playback, and not the "RECORD" portion as this add-on seems to do. Directly streaming from the HDHomeRun (Advanced Setting) also has no effect. The only thing that does is disabling Milkdrop which is a deal-breaker, long term.
The confusing thing would be that it's still the music vis screen that's being displayed, regardless. The skin I'm using has a setting I use to instantly "switch" to the music vis screen on music / audio playback, which might have a bearing, but I've not testing disabling that as yet. And I don't know who (Kodi itself, or this add-on) is making the determination that the stream is "audio-only".
The crux of this post, and henceforth creating another "issue" thread, after our previous conversation, is that I think the only workaround for this is that if the RECORD engine must be the engine of choice for playback, we need to treat all streams it provides as video streams, regardless. How we get to that point if the stream differs in any way from the regular TV channels, I do not know.
What I do know is, constantly having to switch on/off Milkdrop 2 to use PVR / listen to music is far from ideal.
One other thing I've noticed. Tuning channels (TV or radio) can take anywhere from 3 to 5 seconds for anything to appear in Kodi, even on the HTPC which is on the same powerline switch / segment as the HDHomeRun box itself. Another queer fact is that it doesn't ALWAYS happen, just 95% of the time.
Thoughts?
Dan / Gib.
The text was updated successfully, but these errors were encountered: