Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Playing a videostream and audiostream simultaneously #797

Closed
1 task done
gerion0 opened this issue Sep 12, 2021 · 9 comments
Closed
1 task done

Playing a videostream and audiostream simultaneously #797

gerion0 opened this issue Sep 12, 2021 · 9 comments
Labels
Issue Type: Question issue to ask for information

Comments

@gerion0
Copy link

gerion0 commented Sep 12, 2021

Bug report

Describe the bug

This isn't directly a bug, just a question to understand inputstream.adaptive's behavior without reading the source code.

I'm currently working on the Youtube support for plugin.video.sendtokodi. Currently, this plugin asks Youtube with youtube-dl for a (video and audio combined) stream URL, gets one stream back and plays it. For these cases (video and audio together), Youtube provides only a low resolution stream (720p or lower).

However, youtube-dl can request higher resolutions. In this case, it delivers two URLs (with some metadata), one for video and one for audio. I searched a lot to get Kodi to play these stream together and one common suggestion was inputstream.adaptive (here, partly here).

According to the wiki I need to build a custom mpd file for that. With a lot of try and error, I found out that the minimal mpd file for the example mpd file in the README looks so:

<?xml version="1.0"?>
<MPD type="static" xmlns="urn:mpeg:dash:schema:mpd:2011" profiles="urn:mpeg:dash:profile:isoff-live:2011">
 <ProgramInformation>
  <Title>mp4-live-mpd-AV-BS.mpd generated by GPAC</Title>
 </ProgramInformation>

 <Period start="PT0S">
  <AdaptationSet>
   <ContentComponent id="1" contentType="video"/>
   <SegmentTemplate startNumber="1" initialization="mp4-live-mpd-AV-BS_set1_init.mp4"/>
   <Representation id="foobar" mimeType="video/mp4" codecs="avc1.42c028" width="1920" height="1080">
   </Representation>
  </AdaptationSet>
  <AdaptationSet>
   <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="1"/>
   <ContentComponent id="1" contentType="audio" />
   <SegmentTemplate startNumber="1" initialization="mp4-live-mpd-AV-BS_set2_init.mp4"/>
   <Representation id="blabla" mimeType="audio/mp4" codecs="mp4a.40.02">
   </Representation>
  </AdaptationSet>
 </Period>
</MPD>

This mpd file is accepted by inputstream.adaptive while missing a lot of fields compared to the original.

My next step was to replace the URLs in the initialization field with the one that youtube provides. inputstream.adaptive resolves this correctly, but downloads the full media files first, and then complains that the stream selection was not successful (see log below).

My questions:

  • Is inputstream.adaptive capable of playing video and audio together in this way?
  • Is there a simpler way? I don't need any adaptive behavior, just play two stream together simultaneously.
  • Is my MPD file correct for that?

Additional information

  • To get the Youtube URLs you can run youtube-dl -j <youtube-url> (it provides a bunch of information but the URLs are contained in the block "requested_formats").
  • Just downloading (with wget) and remuxing the streams works fine.

Debuglog

2021-09-12 14:57:06.751 T:15202    INFO <general>: Creating InputStream
2021-09-12 14:57:06.751 T:15202   DEBUG <general>: ADDON: Dll Initializing - InputStream Adaptive
2021-09-12 14:57:06.751 T:15202   DEBUG <general>: SECTION:LoadDLL(/usr/lib64/kodi/addons/inputstream.adaptive/inputstream.adaptive.so.2.6.23)
2021-09-12 14:57:06.751 T:15202   DEBUG <general>: Loading: /usr/lib64/kodi/addons/inputstream.adaptive/inputstream.adaptive.so.2.6.23
2021-09-12 14:57:06.752 T:15203   DEBUG <general>: Loading settings for plugin://plugin.video.sendtokodi/?foo
2021-09-12 14:57:06.756 T:15202    INFO <general>: AddOnLog: inputstream.adaptive: SetVideoResolution (1234 x 629)
2021-09-12 14:57:06.756 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: Open()
2021-09-12 14:57:06.756 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: found inputstream.adaptive.manifest_type: mpd
2021-09-12 14:57:06.756 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: Initial bandwidth: 4000000 
2021-09-12 14:57:06.757 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: MAXRESOLUTION selected: 0 
2021-09-12 14:57:06.757 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: MAXRESOLUTIONSECURE selected: 0 
2021-09-12 14:57:06.757 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: STREAMSELECTION selected: 0 
2021-09-12 14:57:06.757 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: FORCENONSECUREDECODER selected: 0 
2021-09-12 14:57:06.757 T:15202   DEBUG <general>: CurlFile::Open(0x7f6ca00af8e0) http://localhost:10001/test.mpd
2021-09-12 14:57:06.757 T:15202   DEBUG <general>: easy_acquire - Created session to http://localhost
2021-09-12 14:57:06.758 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: Download finished: http://localhost:10001/test.mpd
2021-09-12 14:57:06.758 T:15202    INFO <general>: AddOnLog: inputstream.adaptive: Successfully parsed manifest file. #Periods: 1, #Streams in first period: 2, Type: VOD, Download speed: 0.0000 Bytes/s
2021-09-12 14:57:06.758 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: New period, dispose sample decrypter and reinitialize
2021-09-12 14:57:06.758 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: GetCapabilities()
2021-09-12 14:57:06.758 T:15202    INFO <general>: Creating Demuxer
2021-09-12 14:57:06.758 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: GetStreamIds()
2021-09-12 14:57:06.758 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: GetStream(1001)
2021-09-12 14:57:06.758 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: GetStream(1002)
2021-09-12 14:57:06.758 T:15202   DEBUG <general>: CDVDDemuxClient::RequestStream(): added/updated stream 1001 with codec_id 167
2021-09-12 14:57:06.758 T:15202   DEBUG <general>: CDVDDemuxClient::RequestStream(): added/updated stream 1002 with codec_id 86076
2021-09-12 14:57:06.758 T:15202    INFO <general>: Opening stream: 1001 source: 256
2021-09-12 14:57:06.758 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: OpenStream(1001)
2021-09-12 14:57:06.758 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: Selecting stream with conditions: w: 1234, h: 629, bw: 3600000
2021-09-12 14:57:06.760 T:15201   DEBUG <general>: Thread GUIDialogCache 140104651765312 terminating (autodelete)
2021-09-12 14:57:06.778 T:15202   DEBUG <general>: CurlFile::ParseAndCorrectUrl() adding custom header option 'connection: keep-alive'
2021-09-12 14:57:06.778 T:15202   DEBUG <general>: CurlFile::Open(0x7f6ca00af8e0) https://r5---sn-i5heen7s.googlevideo.com/videoplayback?expire=1631472954&ei=2vg9YZLXG4Lf1gKHlKbIAg&ip=<censored>&id=o-AKJk_Z79Srv8Ce8vtuK1eXS-goSleeyNnfg4Gjj_gb0G&itag=244&source=youtube&requiressl=yes&mh=la&mm=31%2C29&mn=sn-i5heen7s%2Csn-i5h7lned&ms=au%2Crdu&mv=m&mvi=5&pl=24&initcwndbps=2023750&vprv=1&mime=video%2Fwebm&gir=yes&clen=51160592&dur=2224.933&lmt=1540482793553708&mt=1631451180&fvip=5&keepalive=yes&fexp=24001373%2C24007246&c=ANDROID&txp=5532432&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRgIhAMf7LAUXd_G90HQdaIqilI-5jxN6ClialuY0BKdbek55AiEAtoykgDdNhtL6glBWuYbWw8ORvoqqE_p7UpP_d1phWsc%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIgcyuoo5w_zuDimk92EX8-icc9_kSdh2Y8KHlsy6M_dSQCIQCJwsZglWtLnM1A2TDq4Pi82UyRdO6sWikvFKpq6CPjYA%3D%3D
2021-09-12 14:57:06.778 T:15202   DEBUG <general>: easy_acquire - Created session to https://r5---sn-i5heen7s.googlevideo.com
2021-09-12 14:57:06.787 T:14995   DEBUG <general>: CPlayerGUIInfo::InitCurrentItem(plugin://plugin.video.sendtokodi/?foo)
2021-09-12 14:57:06.887 T:14995   DEBUG <general>: ------ Window Init (DialogBusy.xml) ------
2021-09-12 14:57:19.061 T:14995   DEBUG <general>: ------ Window Init (Pointer.xml) ------
2021-09-12 14:57:24.133 T:14995   DEBUG <general>: ------ Window Deinit (Pointer.xml) ------
2021-09-12 14:57:36.752 T:15203   DEBUG <general>: Thread JobWorker 140104070768192 terminating (autodelete)
2021-09-12 14:57:37.188 T:14995   DEBUG <general>: CheckIdle - Closing session to http://localhost (easy=0x7f6ca0069630, multi=0x7f6ca00bbf50)
2021-09-12 15:01:13.615 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: Download finished: https://r5---sn-i5heen7s.googlevideo.com/videoplayback?expire=1631472954&ei=2vg9YZLXG4Lf1gKHlKbIAg&ip=<censored>&id=o-AKJk_Z79Srv8Ce8vtuK1eXS-goSleeyNnfg4Gjj_gb0G&itag=244&source=youtube&requiressl=yes&mh=la&mm=31%2C29&mn=sn-i5heen7s%2Csn-i5h7lned&ms=au%2Crdu&mv=m&mvi=5&pl=24&initcwndbps=2023750&vprv=1&mime=video%2Fwebm&gir=yes&clen=51160592&dur=2224.933&lmt=1540482793553708&mt=1631451180&fvip=5&keepalive=yes&fexp=24001373%2C24007246&c=ANDROID&txp=5532432&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRgIhAMf7LAUXd_G90HQdaIqilI-5jxN6ClialuY0BKdbek55AiEAtoykgDdNhtL6glBWuYbWw8ORvoqqE_p7UpP_d1phWsc%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIgcyuoo5w_zuDimk92EX8-icc9_kSdh2Y8KHlsy6M_dSQCIQCJwsZglWtLnM1A2TDq4Pi82UyRdO6sWikvFKpq6CPjYA%3D%3D , avg speed: 207264.00byte/s, current speed: 207264.00byte/s
2021-09-12 15:01:13.615 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: Build segments from SIDX atom...
2021-09-12 15:01:13.615 T:15202   ERROR <general>: AddOnLog: inputstream.adaptive: Unable to select stream!
2021-09-12 15:01:13.615 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: GetStream(1001)
2021-09-12 15:01:13.615 T:15202    INFO <general>: Creating video codec with codec id: 167
2021-09-12 15:01:13.615 T:15202    INFO <general>: CDVDVideoCodecFFmpeg::Open() Using codec: Google VP9
2021-09-12 15:01:13.615 T:15202   DEBUG <general>: CDVDVideoCodecFFmpeg - Updated codec: ff-vp9
2021-09-12 15:01:13.615 T:15202   DEBUG <general>: CVideoPlayerVideo::OpenStream - open stream with codec id: 167
2021-09-12 15:01:13.615 T:15202    INFO <general>: Creating video thread
2021-09-12 15:01:13.615 T:15349   DEBUG <general>: Thread VideoPlayerVideo start, auto delete: false
2021-09-12 15:01:13.615 T:15349    INFO <general>: running thread: video_thread
2021-09-12 15:01:13.615 T:15349   DEBUG <general>: CVideoPlayerVideo - CDVDMsg::GENERAL_PAUSE: 0
2021-09-12 15:01:13.615 T:15202    INFO <general>: Opening stream: 1002 source: 256
2021-09-12 15:01:13.615 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: OpenStream(1002)
2021-09-12 15:01:13.615 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: Selecting stream with conditions: w: 0, h: 0, bw: 400000
2021-09-12 15:01:13.615 T:15202   DEBUG <general>: CurlFile::ParseAndCorrectUrl() adding custom header option 'connection: keep-alive'
2021-09-12 15:01:13.615 T:15202   DEBUG <general>: CurlFile::Open(0x7f6ca00af8e0) https://r5---sn-i5heen7s.googlevideo.com/videoplayback?expire=1631472954&ei=2vg9YZLXG4Lf1gKHlKbIAg&ip=<censored>&id=o-AKJk_Z79Srv8Ce8vtuK1eXS-goSleeyNnfg4Gjj_gb0G&itag=251&source=youtube&requiressl=yes&mh=la&mm=31%2C29&mn=sn-i5heen7s%2Csn-i5h7lned&ms=au%2Crdu&mv=m&mvi=5&pl=24&initcwndbps=2023750&vprv=1&mime=audio%2Fwebm&gir=yes&clen=36249101&dur=2225.021&lmt=1540486891949880&mt=1631451180&fvip=5&keepalive=yes&fexp=24001373%2C24007246&c=ANDROID&txp=5511222&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRgIhAN-ESW3-tUyWj8OHGi3UR72zrKaMNfkqpi-toqWCP-bwAiEAnHRvNS3W92Rs4Av7lxrw2iaQkJHbW2JONZP5ZuskxxQ%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIgcyuoo5w_zuDimk92EX8-icc9_kSdh2Y8KHlsy6M_dSQCIQCJwsZglWtLnM1A2TDq4Pi82UyRdO6sWikvFKpq6CPjYA%3D%3D
2021-09-12 15:01:13.903 T:14995   DEBUG <general>: Inhibiting OS screen saver
2021-09-12 15:01:13.903 T:15351   DEBUG <general>: Thread Timer start, auto delete: false
2021-09-12 15:03:15.707 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: Download finished: https://r5---sn-i5heen7s.googlevideo.com/videoplayback?expire=1631472954&ei=2vg9YZLXG4Lf1gKHlKbIAg&ip=<censored>&id=o-AKJk_Z79Srv8Ce8vtuK1eXS-goSleeyNnfg4Gjj_gb0G&itag=251&source=youtube&requiressl=yes&mh=la&mm=31%2C29&mn=sn-i5heen7s%2Csn-i5h7lned&ms=au%2Crdu&mv=m&mvi=5&pl=24&initcwndbps=2023750&vprv=1&mime=audio%2Fwebm&gir=yes&clen=36249101&dur=2225.021&lmt=1540486891949880&mt=1631451180&fvip=5&keepalive=yes&fexp=24001373%2C24007246&c=ANDROID&txp=5511222&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRgIhAN-ESW3-tUyWj8OHGi3UR72zrKaMNfkqpi-toqWCP-bwAiEAnHRvNS3W92Rs4Av7lxrw2iaQkJHbW2JONZP5ZuskxxQ%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIgcyuoo5w_zuDimk92EX8-icc9_kSdh2Y8KHlsy6M_dSQCIQCJwsZglWtLnM1A2TDq4Pi82UyRdO6sWikvFKpq6CPjYA%3D%3D , avg speed: 296901.00byte/s, current speed: 296901.00byte/s
2021-09-12 15:03:15.707 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: Build segments from SIDX atom...
2021-09-12 15:03:15.707 T:15202   ERROR <general>: AddOnLog: inputstream.adaptive: Unable to select stream!
2021-09-12 15:03:15.707 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: GetStream(1002)
2021-09-12 15:03:15.707 T:15202    INFO <general>: Finding audio codec for: 86076
2021-09-12 15:03:15.707 T:15202    INFO <general>: CDVDAudioCodecFFmpeg::Open() Successful opened audio decoder opus
2021-09-12 15:03:15.707 T:15202    INFO <general>: Creating audio thread
2021-09-12 15:03:15.707 T:15405   DEBUG <general>: Thread VideoPlayerAudio start, auto delete: false
2021-09-12 15:03:15.707 T:15405    INFO <general>: running thread: CVideoPlayerAudio::Process()
2021-09-12 15:03:15.707 T:15405   DEBUG <general>: CVideoPlayerAudio - CDVDMsg::GENERAL_PAUSE: 0
2021-09-12 15:03:15.707 T:15202   DEBUG <general>: CVideoPlayer::SetCaching - caching state 2
2021-09-12 15:03:15.707 T:15202   DEBUG <general>: CDVDClock::SetSpeedAdjust - adjusted:0.000000
2021-09-12 15:03:15.707 T:15202   DEBUG <general>: CVideoPlayer::HandleMessages - player 2 reported state: 0
2021-09-12 15:03:15.707 T:15406   DEBUG <general>: Thread JobWorker start, auto delete: true
2021-09-12 15:03:15.707 T:15202   DEBUG <general>: CVideoPlayer::HandleMessages - player 1 reported state: 0
2021-09-12 15:03:15.707 T:15406   DEBUG <general>: OnAVChange: CApplication::OnAVChange
2021-09-12 15:03:15.707 T:15202    INFO <general>: Process - eof reading from demuxer
2021-09-12 15:03:15.707 T:15202    INFO <general>: CVideoPlayer::OnExit()
2021-09-12 15:03:15.707 T:15202    INFO <general>: VideoPlayer: eof, waiting for queues to empty
2021-09-12 15:03:15.707 T:15202    INFO <general>: Closing stream player 1
2021-09-12 15:03:15.707 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: EnableStream(1002: false)
2021-09-12 15:03:15.707 T:15202    INFO <general>: Waiting for audio thread to exit
2021-09-12 15:03:15.717 T:15405    INFO <general>: thread end: CVideoPlayerAudio::OnExit()
2021-09-12 15:03:15.717 T:15405   DEBUG <general>: Thread VideoPlayerAudio 140105880524352 terminating
2021-09-12 15:03:15.717 T:15202    INFO <general>: Closing audio device
2021-09-12 15:03:15.717 T:15202    INFO <general>: Deleting audio codec
2021-09-12 15:03:15.717 T:15202    INFO <general>: Closing stream player 2
2021-09-12 15:03:15.717 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: EnableStream(1001: false)
2021-09-12 15:03:15.717 T:15202    INFO <general>: waiting for video thread to exit
2021-09-12 15:03:15.717 T:15349   ERROR <general>: Got MSGQ_ABORT or MSGO_IS_ERROR return true
2021-09-12 15:03:15.717 T:15349    INFO <general>: thread end: video_thread
2021-09-12 15:03:15.717 T:15349   DEBUG <general>: Thread VideoPlayerVideo 140104060401216 terminating
2021-09-12 15:03:15.717 T:15202    INFO <general>: deleting video codec
2021-09-12 15:03:15.718 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: Close()
2021-09-12 15:03:15.718 T:15202   DEBUG <general>: AddOnLog: inputstream.adaptive: Session::~Session()
2021-09-12 15:03:15.722 T:15202   DEBUG <general>: SECTION:UnloadDll(/usr/lib64/kodi/addons/inputstream.adaptive/inputstream.adaptive.so.2.6.23)
2021-09-12 15:03:15.722 T:15202    INFO <general>: ADDON: Dll Destroyed - InputStream Adaptive
2021-09-12 15:03:15.722 T:15202   DEBUG <general>: Thread VideoPlayer 140104643372608 terminating
2021-09-12 15:03:15.723 T:15407   DEBUG <general>: Thread JobWorker start, auto delete: true
2021-09-12 15:03:15.723 T:15407   DEBUG <general>: OnPlayBackEnded: CApplication::OnPlayBackEnded
2021-09-12 15:03:15.726 T:14995    INFO <general>: CVideoPlayer::CloseFile()
2021-09-12 15:03:15.726 T:14995   DEBUG <general>: DeleteRenderer - deleting renderer
2021-09-12 15:03:15.726 T:14995   DEBUG <general>: LinuxRendererGL: Cleaning up GL resources
2021-09-12 15:03:15.799 T:14995    INFO <general>: VideoPlayer: waiting for threads to exit
2021-09-12 15:03:15.799 T:14995    INFO <general>: VideoPlayer: finished waiting
2021-09-12 15:03:15.799 T:14995    INFO <general>: CVideoPlayer::CloseFile()
2021-09-12 15:03:15.799 T:14995    INFO <general>: VideoPlayer: waiting for threads to exit
2021-09-12 15:03:15.799 T:14995    INFO <general>: VideoPlayer: finished waiting
2021-09-12 15:03:15.799 T:14995   DEBUG <general>: Radio UECP (RDS) Processor - delete ~CDVDRadioRDSData
2021-09-12 15:03:15.801 T:14995   DEBUG <general>: ------ Window Deinit (DialogBusy.xml) ------

MPD/M3U8s/ISM

An example or copy of a manifest (or manifests for HLS - master and variants) can be found here (this file, however, will not work. The URLs therein are temporary only):

<?xml version="1.0"?>
<MPD type="static" xmlns="urn:mpeg:dash:schema:mpd:2011" profiles="urn:mpeg:dash:profile:isoff-live:2011">
 <ProgramInformation>
  <Title>F͟l͟a͟s͟h͟d͟a͟nce͟ ͟S͟o͟u͟n͟d͟t͟r͟ack͟ (FULL ALBUM) Original Cd Press HQ</Title>
 </ProgramInformation>

 <Period start="PT0S">
  <AdaptationSet>
   <ContentComponent id="1" contentType="video"/>
   <SegmentTemplate startNumber="1" initialization="https://r5---sn-i5h7lned.googlevideo.com/videoplayback?expire=1631468877&amp;ei=7eg9YfrvMcCmmLAPoLGyUA&amp;ip=<censored>&amp;id=o-AONrmuJpV7iitIznfr_c3htFdIs42W57PaFJ0vOkFgaa&amp;itag=244&amp;source=youtube&amp;requiressl=yes&amp;mh=la&amp;mm=31%2C29&amp;mn=sn-i5h7lned%2Csn-i5heen7s&amp;ms=au%2Crdu&amp;mv=m&amp;mvi=5&amp;pl=24&amp;initcwndbps=2057500&amp;vprv=1&amp;mime=video%2Fwebm&amp;gir=yes&amp;clen=51160592&amp;dur=2224.933&amp;lmt=1540482793553708&amp;mt=1631446847&amp;fvip=5&amp;keepalive=yes&amp;fexp=24001373%2C24007246&amp;beids=24027536&amp;c=ANDROID&amp;txp=5532432&amp;sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&amp;sig=AOq0QJ8wRgIhAMS0NkaALYPZ6qEz0Dvt7UvyxZ-GXxuxYWUkwjZjXi2HAiEAgyL51ouNMTgf3KOwtCpZJvr2qX_wobq3NzSmdvRLSII%3D&amp;lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&amp;lsig=AG3C_xAwRgIhAIHLwg8axBK3N1jPVlo0X70LlEZSTYzg4NO52dZqhEUCAiEAlw1hZX6_RGMDVvgEJT8loMW7U3cbHJrkhUbuWHGkzpo%3D"/>
   <Representation id="vid" mimeType="video/webm" codecs="vp9" width="640" height="480">
   </Representation>
  </AdaptationSet>

  <AdaptationSet>
   <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="1"/>
   <ContentComponent id="1" contentType="audio" />
   <SegmentTemplate startNumber="1" initialization="https://r5---sn-i5h7lned.googlevideo.com/videoplayback?expire=1631468877&amp;ei=7eg9YfrvMcCmmLAPoLGyUA&amp;ip=<censored>&amp;id=o-AONrmuJpV7iitIznfr_c3htFdIs42W57PaFJ0vOkFgaa&amp;itag=251&amp;source=youtube&amp;requiressl=yes&amp;mh=la&amp;mm=31%2C29&amp;mn=sn-i5h7lned%2Csn-i5heen7s&amp;ms=au%2Crdu&amp;mv=m&amp;mvi=5&amp;pl=24&amp;initcwndbps=2057500&amp;vprv=1&amp;mime=audio%2Fwebm&amp;gir=yes&amp;clen=36249101&amp;dur=2225.021&amp;lmt=1540486891949880&amp;mt=1631446847&amp;fvip=5&amp;keepalive=yes&amp;fexp=24001373%2C24007246&amp;beids=24027536&amp;c=ANDROID&amp;txp=5511222&amp;sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&amp;sig=AOq0QJ8wRgIhAJzNATT1PrfkShSeY5lbfBAw0kfCklv2_JIBA4BInO0nAiEA2Z0tf8-xPNFG-7IkKBw65eXwrOKmliRcMDyhSQWQpdY%3D&amp;lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&amp;lsig=AG3C_xAwRgIhAIHLwg8axBK3N1jPVlo0X70LlEZSTYzg4NO52dZqhEUCAiEAlw1hZX6_RGMDVvgEJT8loMW7U3cbHJrkhUbuWHGkzpo%3D"/>
   <Representation id="audio" mimeType="audio/webm" codecs="opus">
   </Representation>
  </AdaptationSet>

 </Period>
</MPD>

Your Environment

Used Operating system:

  • Linux

  • Operating system version/name: Gentoo Linux

  • Kodi version: 19.1

  • inputstream.adaptive version: 2.6.23

@anxdpanic
Copy link
Member

The YouTube and Tubed add-ons both do this, generate a custom mpd to use for playback based on quality selection and availability.
https://github.com/anxdpanic/script.module.tubed.api/blob/master/resources/lib/src/tubed_api/usher/lib/mpeg_dash.py

@gerion0
Copy link
Author

gerion0 commented Sep 12, 2021

Thanks, I will take a look into it. From a quick look in the code the generated mpd contains a lot of more information. Do you know, if this information (bitrate, framerate, ...) is necessary for a proper working?

@anxdpanic
Copy link
Member

I'm not sure if all of it is necessary, but some of it definitely helps avoid unwanted side-effects (ie. framerate avoids a refresh rate switch in Kodi). All the information available was used just to avoid any other potential situations. The majority if not all the required information should be parsable from the url.

@glennguy
Copy link
Contributor

@gerion0 how did you go?
Can the issue be closed?

@gerion0
Copy link
Author

gerion0 commented Sep 29, 2021

No progress on the main issue.

I played around a little bit und tried to reproduce the MPD files from the Tubed addon. The produced files looks like:

<?xml version="1.0" encoding="UTF-8"?>
<MPD xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:xlink="http://www.w3.org/1999/xlink" xsi:schemaLocation="urn:mpeg:dash:schema:mpd:2011 http://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-DASH_schema_files/DASH-MPD.xsd" minBufferTime="PT1.5S" mediaPresentationDuration="PT556S" type="static" profiles="urn:mpeg:dash:profile:isoff-main:2011">
        <Period>
                <AdaptationSet id="0" mimeType="video/webm" lang="" subsegmentAlignment="true" subsegmentStartsWithSAP="1" bitstreamSwitching="true" default="true">
                        <Role schemeIdUri="urn:mpeg:DASH:role:2011" value="main"/>
                        <Representation id="video" codecs="vp9" startWithSAP="1" bandwidth="2628.373" width="1920" height="1080" frameRate="25">
                                <BaseURL>https://r5---sn-i5heen7z.googlevideo.com/videoplayback?expire=163292548...</BaseURL>
                                <SegmentBase indexRange="0-0">
                                                <Initialization range="0-0" />
                                </SegmentBase>
                        </Representation>
                </AdaptationSet>
                <AdaptationSet id="1" mimeType="audio/webm" lang="" subsegmentAlignment="true" subsegmentStartsWithSAP="1" bitstreamSwitching="true" default="true">
                        <Role schemeIdUri="urn:mpeg:DASH:role:2011" value="main"/>
                        <Representation id="audio" codecs="opus" bandwidth="133.128">
                                <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
                                <BaseURL>https://r5---sn-i5heen7z.googlevideo.com/videoplayback?expire=1632925488...</BaseURL>
                                <SegmentBase indexRange="0-0">
                                                <Initialization range="0-0" />
                                </SegmentBase>
                        </Representation>
                </AdaptationSet>
        </Period>
</MPD>

These files do not play at all, the correct encoders are loaded but the stream itself is not downloaded.

My guess it that the <Initialization range="0-0" /> is the problematic part since it says the inputstream.adaptive addon that the relevant media header has no length. However, <Initialization range="0-0" /> is the default/fallback in the Tubed addon and , unfortunately, the correct valeus are not returned by youtube-dl/yt-dlp.

Do you know a minimal MPD that has long media streams and is playable with inputstream.adaptive? Ideally, this would only require the two URLs so inputstream.adaptive downloads the first bytes on it's own and figures out the formats, the length, the seeking capabilities etc.

Is inputstream.adaptive capable of doing so in general? Otherwise, the addon must utilize FFMpeg, download the first bytes, read the stream and feed that into a fake mpd or remux the streams into one.

@gerion0
Copy link
Author

gerion0 commented Sep 29, 2021

In it's core my question is:

Given the two URLs:

How must a MPD file which contains these URLS look like so Kodi is playing them simultaineously (without completely downloading them first)?

@glennguy
Copy link
Contributor

glennguy commented Oct 4, 2021

Ok

Mp4 can be packaged in two ways:

  • One large file
  • Separate segment files (fragmented mp4)

All mp4s have initialization data, usually a small size (~1000 bytes) that tells the video player information like resolution, codecs, profiles etc.

Some segmented mp4 will include the init data with each segment but this is wasteful, so a 'initialization segment' is created separately so the same 1000 bytes aren't having to be downloaded with each segment.

In the one large file scenario it's just at the start of the file.

The range parameter is to be used with the single file case, so <Initialization range="0-0" /> is telling us that this is the range of the file the init data is in - it doesn't make sense.

Then when you link to the separate init files in your post above, this is telling me you're not using one large file, but trying to play a stream with separate segments (in this case there is no range attribute).

Assuming that you're getting the single file - you'll need to figure out where the init segment ends. Of course it should start at 0.
Maybe there's a python library out there that can help with that, or you can simply search for whatever sequence of characters would signify the end of this sequence. Obviously, only get the first 2 or 3000 bytes of the file to check.

MPD files are normally generated by a packager which can read the source files and so it's assumed the init range information is contained within the MPD.

@gerion0
Copy link
Author

gerion0 commented Oct 4, 2021

Thanks for the explanation.

Assuming that you're getting the single file - you'll need to figure out where the init segment ends. Of course it should start at 0. Maybe there's a python library out there that can help with that, or you can simply search for whatever sequence of characters would signify the end of this sequence. Obviously, only get the first 2 or 3000 bytes of the file to check.

What happens, if no such ending exists in the MPD? In the example given in my initial post no init segment is given. In this case, inputstream.adaptive seems to download the whole stream. Is this to search for the init segment?

What happens, if the ending mentioned in the MPD is incorrect/point to data behind the actual ending (e.g. always the first MiB)? Does inputstream.adaptive only download 1 MiB in this case but parses the data for the correct ending anyway?

@glennguy
Copy link
Contributor

glennguy commented Oct 4, 2021

The answer to the second question could be found by just trying.
The example in the OP isn't making sense to me compared to the log, are you sure that exact mpd generated that log?

I've done a little more research for you.

An example of a working mpd is here

The initialization range refers to the location of the init data, and the segmentbase indexrange refers to the location of the SIDX (segment index) data. I'm not sure what happens if you don't provide the indexrange, try and see.

@CastagnaIT CastagnaIT added the Issue Type: Question issue to ask for information label Nov 28, 2021
@CastagnaIT CastagnaIT closed this as not planned Won't fix, can't repro, duplicate, stale Feb 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Issue Type: Question issue to ask for information
Projects
None yet
Development

No branches or pull requests

4 participants