Order of variants in HLS master playlist - can this be customized? #560

fuzing · 2019-02-23T00:53:49Z

The Apple HLS authoring spec says:

1.32. The default video variant(s) SHOULD be the 2000 kb/s (average bit rate) variant. (Defaults are the first variant listed in the master playlist within a group of variants having compatible audio.)

I'm trying to set the order for the EXT-X-STREAM-INF's in the master.m3u8 so that my 2000 kb/s appears first but no matter what I try I cannot manipulate it so it appears in the top spot. I've tried reordering my inputs, different naming (noting the std::sort()) in packager.cc, and various other ploys to try to trick shaka packager into ordering things the way I want - but to no avail. Shaka packager always puts my lowest resolution and bandwidth stream first in the hls master playlist.

I'm probably missing something obvious but could someone point me in the right direction as to how I can manipulate the ordering.

Thank you for your help (and amazing product!).

kqyang · 2019-02-25T17:02:03Z

@fuzing I am glad that you like Shaka Packager!

Unfortunately the order cannot be customized right now. And yes, it is something we need to support in Shaka Packager. Thanks for raising the issue! I'll add it to our next milestone.

fuzing · 2019-02-25T17:16:31Z

Awesome - thank you!

…

On Mon, Feb 25, 2019 at 10:02 AM Kongqun Yang ***@***.***> wrote: @fuzing <https://github.com/fuzing> I am glad that you like Shaka Packager! Unfortunately the order cannot be customized right now. And yes, it is something we need to support in Shaka Packager. I'll add it to our next milestone. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#560 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALHBFUff0o_NisTiyPgCwhHEJYNqLAmDks5vRBcVgaJpZM4bNOGQ> .

Nixon197 · 2019-07-12T07:05:24Z

Hi

I am also having trouble with EXT-X-STREAM-INF variant order.
It seems that order is random each time I start packager.

My command:

  /opt/packager-linux \
  'in=udp://224.10.197.5:1234?reuse=1&buffer_size=212992&interface=10.200.5.82,stream=audio,segment_template=audio/$Time$.aac,playlist_name=audio/main.m3u8,hls_group_id=audio,hls_name=ENGLISH,drm_label=SD' \
  'in=udp://224.10.197.5:1234?reuse=1&buffer_size=212992&interface=10.200.5.82,stream=video,segment_template=stream_p1/$Time$.ts,playlist_name=stream_p1/main.m3u8,drm_label=SD,iframe_playlist_name=stream_p1/iframe.m3u8' \
  'in=udp://224.10.196.5:1234?reuse=1&buffer_size=212992&interface=10.200.5.82,stream=video,segment_template=stream_p2/$Time$.ts,playlist_name=stream_p2/main.m3u8,drm_label=SD,iframe_playlist_name=stream_p2/iframe.m3u8' \
  'in=udp://224.10.195.5:1234?reuse=1&buffer_size=212992&interface=10.200.5.82,stream=video,segment_template=stream_p3/$Time$.ts,playlist_name=stream_p3/main.m3u8,drm_label=SD,iframe_playlist_name=stream_p3/iframe.m3u8' \
  'in=udp://224.10.194.5:1234?reuse=1&buffer_size=212992&interface=10.200.5.82,stream=video,segment_template=stream_p4/$Time$.ts,playlist_name=stream_p4/main.m3u8,drm_label=SD,iframe_playlist_name=stream_p4/iframe.m3u8' \
  'in=udp://224.10.193.5:1234?reuse=1&buffer_size=212992&interface=10.200.5.82,stream=video,segment_template=stream_p5/$Time$.ts,playlist_name=stream_p5/main.m3u8,drm_label=SD,iframe_playlist_name=stream_p5/iframe.m3u8' \
  'in=udp://224.10.192.5:1234?reuse=1&buffer_size=212992&interface=10.200.5.82,stream=video,segment_template=stream_p6/$Time$.ts,playlist_name=stream_p6/main.m3u8,drm_label=SD,iframe_playlist_name=stream_p6/iframe.m3u8' \
  --hls_master_playlist_output playlist.m3u8 \
  --clear_lead=0 \
  --hls_playlist_type LIVE \
  --time_shift_buffer_depth 10800 \
  --preserved_segments_outside_live_window 100`

Playlist output:

#EXTM3U
## Generated with https://github.com/google/shaka-packager version v2.3.0-5bf8ad5-release

#EXT-X-MEDIA:TYPE=AUDIO,URI="audio/main.m3u8",GROUP-ID="audio",NAME="ENGLISH",AUTOSELECT=YES,CHANNELS="2"

#EXT-X-STREAM-INF:BANDWIDTH=3174648,AVERAGE-BANDWIDTH=2980616,CODECS="avc1.4d401f,mp4a.40.2",RESOLUTION=1280x720,AUDIO="audio"
stream_p5/main.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1331994,AVERAGE-BANDWIDTH=1245153,CODECS="avc1.4d401f,mp4a.40.2",RESOLUTION=640x360,AUDIO="audio"
stream_p2/main.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2503863,AVERAGE-BANDWIDTH=2366505,CODECS="avc1.4d401f,mp4a.40.2",RESOLUTION=960x540,AUDIO="audio"
stream_p4/main.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=801081,AVERAGE-BANDWIDTH=734293,CODECS="avc1.4d401f,mp4a.40.2",RESOLUTION=480x270,AUDIO="audio"
stream_p1/main.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1996763,AVERAGE-BANDWIDTH=1856162,CODECS="avc1.4d401f,mp4a.40.2",RESOLUTION=960x540,AUDIO="audio"
stream_p3/main.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=4210154,AVERAGE-BANDWIDTH=4003925,CODECS="avc1.640029,mp4a.40.2",RESOLUTION=1280x720,AUDIO="audio"
stream_p6/main.m3u8

#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=991888,AVERAGE-BANDWIDTH=416696,CODECS="avc1.4d401f",RESOLUTION=1280x720,URI="stream_p5/iframe.m3u8"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=362464,AVERAGE-BANDWIDTH=177904,CODECS="avc1.4d401f",RESOLUTION=640x360,URI="stream_p2/iframe.m3u8"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=689584,AVERAGE-BANDWIDTH=324967,CODECS="avc1.4d401f",RESOLUTION=960x540,URI="stream_p4/iframe.m3u8"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=194016,AVERAGE-BANDWIDTH=103949,CODECS="avc1.4d401f",RESOLUTION=480x270,URI="stream_p1/iframe.m3u8"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=630176,AVERAGE-BANDWIDTH=276964,CODECS="avc1.4d401f",RESOLUTION=960x540,URI="stream_p3/iframe.m3u8"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=1080624,AVERAGE-BANDWIDTH=478163,CODECS="avc1.640029",RESOLUTION=1280x720,URI="stream_p6/iframe.m3u8"

Manual order or order by bitrate would be great because native iOS player seems to have problem with random variant order.

kqyang · 2019-07-12T23:11:59Z

Manual order or order by bitrate would be great because native iOS player seems to have problem with random variant order.

@Nixon197 This is my first time heard about it, but we can certainly order the output by bitrate.

@fuzing Will that work for you as well?

fuzing · 2019-08-06T19:55:04Z

Sorry for the delay - ordering by bitrate won't work for us, as we want to be able to select a middle of the road bitrate as the first stream in the playlist - this is the stream that the IOS player will start playing before adapting (in the case of playback over wifi at least, but read on for cellular).

Apple's HLS requirements

Spec

demand (in 9.18) a 192 Kb/s stream for video played over cellular. They also suggest (in 1.32a) that the default bitrate for wifi should be a 2mb/s stream, and that this should appear first in the playlist - they don't demand this, but they have good reason for suggesting it, i.e. it's a compromise "intermediate" bitrate to start playing before determining actual data-rate prior to adapting. They do also suggest a default bitrate of 730kb/s for cellular connections (1.32b), but our tests of the IOS player seem to suggest that it will initially target the adaptation with bitrate around the 730kb/s mark, no matter where it appears in the playlist when it detects that it's playing over cellular (vs. wifi where it will select the first).

For the case of WIFI, starting with the lowest bitrate stream (i.e. 192kb/s) for the first segment when a user has a 1gb/s connection translates to a very poor user experience (we use Apple's recommended 6 second segment duration, per 7.5 of the spec, meaning the user will get 6 seconds of 192kb/s before transitioning to our 3mb/s 1080 stream, which is nasty).

Hence, our preference would be to see some way of ordering the variants as opposed to sorting by bandwidth (order of input would be fine for us but we'd defer to any other suggestions).

We've tested nixon197's assertion that random ordering of the playlists causes problems with the IOS player and we cannot reproduce in recent versions of IOS across a number of devices (nixon197, can you expound more on the problems you're having when the playlist ordering is randomized).

Hope this makes sense!

kqyang · 2019-08-06T20:41:58Z

@fuzing Thanks for the detail explanations. That is very helpful. We'll see if we can simply order the variants according to the input order (stream descriptor order) then.

fuzing · 2019-08-06T20:46:09Z

No worries @kqyang - thanks for all your hard work on this. Ordering per the order of our input will work great!

patrickkunka · 2019-08-20T18:14:19Z

An option to order by bitrate would be very helpful too.

kickermeister · 2020-04-08T12:59:07Z

The random order of audio tracks is for me especially problematic when I have multiple tracks with the same language (but different versions such as different mixes or channel configurations). If I understood correctly, it is not reliably possible to generate a manifest where the default track is the one I want it to be (i.e. the first which I pass as input with the defined default language):

packager-osx \
’in=Eng_Default.mp4,stream=audio,init_segment=audio/init_Eng_Default.mp4,segment_template=audio/Eng_Default.mp4$Number$.m4s,playlist_name=audio/main_Eng_Default.mp4.m3u8,hls_name=English_Default,language=eng’\
’in=Eng_Mix1.mp4,stream=audio,init_segment=audio/init_Eng_Mix1.mp4,segment_template=audio/Eng_Mix1.mp4$Number$.m4s,playlist_name=audio/main_Eng_Mix1.mp4.m3u8,hls_name=English_Mix1,language=eng’\
’in=Eng_Mix2.mp4,stream=audio,init_segment=audio/init_Eng_Mix2.mp4,segment_template=audio/Eng_Mix2.mp4$Number$.m4s,playlist_name=audio/main_Eng_Mix2.mp4.m3u8,hls_name=English_Mix2,language=eng’\
’in=Eng_Multichannel.mp4,stream=audio,init_segment=audio/init_Eng_Multichannel.mp4,segment_template=audio/Eng_Multichannel.mp4$Number$.m4s,playlist_name=audio/main_Eng_Multichannel.mp4.m3u8,hls_name=English_Multichannel,language=eng’\
--default_language eng 
--hls_master_playlist_output master.m3u8

kqyang · 2020-04-09T05:29:45Z

@kickermeister Correct, there will be some work involved to make it work.

Here is a possible implementation:

Add a new field order_number or index to StreamDescriptor struct: https://github.com/google/shaka-packager/blob/master/packager/packager.h#L74
The field can be filled in after parsing StreamDescriptor: https://github.com/google/shaka-packager/blob/55349aa4c88b43b6bd6fd996efd15f8d9bfdfdb1/packager/app/packager_main.cc#L524
The value can then be passed to HlsNotifyMuxerListener in the constructor: https://github.com/google/shaka-packager/blob/a1dd82d478c4622dff1a0d0489f170ed6e1814d5/packager/media/event/muxer_listener_factory.cc#L71
Then passed to SimpleHlsNotifier in NotifyNewStream: https://github.com/google/shaka-packager/blob/ab8fa87d1807e2f8a5756e365f9bf7b41cc4a992/packager/media/event/hls_notify_muxer_listener.cc#L281, which can then be used to create media playlist: https://github.com/google/shaka-packager/blob/ab8fa87d1807e2f8a5756e365f9bf7b41cc4a992/packager/hls/base/simple_hls_notifier.cc#L313
Then it can be used to order playlists: https://github.com/google/shaka-packager/blob/master/packager/hls/base/master_playlist.cc#L358.

@kickermeister Let us know if you are interested in helping us implement it.

kickermeister · 2020-05-27T06:46:57Z

@kqyang I wish I had the skills to implement these changes, but I'm afraid I simply can't. Hope this issue is important enough for others to give it a try!

vasudevavarun · 2021-03-17T11:06:22Z

Hey @kqyang ,
I've no intentions to push but do you have any plan to implement it soon? I'm facing the same issue.

kqyang · 2021-03-17T16:42:26Z

No, we do not have resources to work on it ourselves right now. If anyone wants to help on this issue, it will be really appreciated!

xavierlaffargue · 2022-04-27T03:35:08Z

This is also an important feature that I am waiting for, thanks again for your work :)

xavierlaffargue · 2023-09-04T13:29:53Z

For information: I started a branch for this (https://github.com/xavierlaffargue/shaka-packager/tree/hls-order-track)

Note: I started from the cosmin:cmake-all-the-things branch which should be merged into shaka-packager:cmake.

bmurphy1976 · 2023-09-27T19:45:23Z

@xavierlaffargue is your branch usable? I'm happy to spend some time testing it. I'm running into problems right now because the order is effectively random. Not only does usability suffer, but so does debuggability because my inputs do not equal my outputs.

How does your branch work? Does it preserve the input order (my personal preference for default behavior)?

joeyparrish · 2023-09-27T23:42:00Z

Would it be feasible to preserve the input order? I would find that most intuitive, and it would be a benefit that existing users get for free without changing commands.

You could keep the order_number field internally, but default it to the index from the input arguments.

bmurphy1976 · 2023-09-28T02:47:58Z

@xavierlaffargue how do I build your branch? I tried the following commands but they are failing:

$ git clone --recurse-submodules https://github.com/xavierlaffargue/shaka-packager.git .
$ git checkout hls-order-track
$ cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at packager/third_party/abseil-cpp/CMakeLists.txt:22 (add_subdirectory):
  The source directory

    /src/packager/third_party/abseil-cpp/source

  does not contain a CMakeLists.txt file.


CMake Error at packager/third_party/curl/CMakeLists.txt:42 (add_subdirectory):
  The source directory

    /src/packager/third_party/curl/source

  does not contain a CMakeLists.txt file.

SNIP

If it makes a difference I'm using Ubuntu 22.04 and trying to follow the build directions from this pull request: #1270

bmurphy1976 · 2023-09-28T03:08:41Z

I figured out the error I posted in the previous comment. I'm running these build commands instead:

git clone https://github.com/xavierlaffargue/shaka-packager.git .
git checkout hls-order-track
git submodule update --init --recursive
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build --parallel

I'm getting much father, but the build is failing with the following:

SNIP
90.56 [970/1160] Building CXX object packager/media/event/CMakeFiles/media_event_unittest.dir/hls_notify_muxer_listener_unittest.cc.o
90.56 FAILED: packager/media/event/CMakeFiles/media_event_unittest.dir/hls_notify_muxer_listener_unittest.cc.o
90.56 /usr/bin/c++ -DCURL_STATICLIB -DGLOG_CUSTOM_PREFIX_SUPPORT -DLIBXML_STATIC -I/src/. -I/src/packager/third_party/abseil-cpp/source -I/src/packager/third_party/curl/source/include -I/src/packager/third_party/mbedtls/source/include -I/src/build/packager/third_party/glog/source -I/src/packager/third_party/glog/source/src -I/src/build/packager/third_party/libxml2/source -I/src/packager/third_party/libxml2/source/include -I/src/build -I/src/packager/third_party/protobuf/source/src -isystem /src/packager/third_party/googletest/source/googlemock/include -isystem /src/packager/third_party/googletest/source/googlemock -isystem /src/packager/third_party/googletest/source/googletest/include -isystem /src/packager/third_party/googletest/source/googletest -std=c++11  -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Wall -Wextra -Wnarrowing -Wno-deprecated -Werror -O3 -DNDEBUG -Wall -Wextra -Werror -Wno-unknown-warning-option -Wno-shorten-64-to-32 -Wno-unused-parameter -Wno-stringop-overflow -std=gnu++17 -MD -MT packager/media/event/CMakeFiles/media_event_unittest.dir/hls_notify_muxer_listener_unittest.cc.o -MF packager/media/event/CMakeFiles/media_event_unittest.dir/hls_notify_muxer_listener_unittest.cc.o.d -o packager/media/event/CMakeFiles/media_event_unittest.dir/hls_notify_muxer_listener_unittest.cc.o -c /src/packager/media/event/hls_notify_muxer_listener_unittest.cc
90.56 /src/packager/media/event/hls_notify_muxer_listener_unittest.cc:142:19: error: cannot declare field 'shaka::media::HlsNotifyMuxerListenerTest::mock_notifier_' to be of abstract type 'shaka::media::{anonymous}::MockHlsNotifier'
90.56   142 |   MockHlsNotifier mock_notifier_;
90.56       |                   ^~~~~~~~~~~~~~
90.56 /src/packager/media/event/hls_notify_muxer_listener_unittest.cc:30:7: note:   because the following virtual functions are pure within 'shaka::media::{anonymous}::MockHlsNotifier':
90.56    30 | class MockHlsNotifier : public hls::HlsNotifier {
90.56       |       ^~~~~~~~~~~~~~~
90.56 In file included from /src/packager/media/event/hls_notify_muxer_listener_unittest.cc:10:
90.56 /src/./packager/hls/base/hls_notifier.h:38:16: note:     'virtual bool shaka::hls::HlsNotifier::NotifyNewStream(const shaka::MediaInfo&, const string&, const string&, const string&, const int32_t&, uint32_t*)'
90.56    38 |   virtual bool NotifyNewStream(const MediaInfo& media_info,
90.56       |                ^~~~~~~~~~~~~~~
90.56 /src/packager/media/event/hls_notify_muxer_listener_unittest.cc: In constructor 'shaka::media::HlsNotifyMuxerListenerTest::HlsNotifyMuxerListenerTest()':
90.56 /src/packager/media/event/hls_notify_muxer_listener_unittest.cc:118:9: error: no matching function for call to 'shaka::media::HlsNotifyMuxerListener::HlsNotifyMuxerListener(const char [22], bool, const char [12], const char [15], std::vector<std::__cxx11::basic_string<char> >, shaka::media::{anonymous}::MockHlsNotifier*)'
90.56   118 |       : listener_(kDefaultPlaylistName,
90.56       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
90.56   119 |                   !kIFramesOnlyPlaylist,
90.56       |                   ~~~~~~~~~~~~~~~~~~~~~~
90.56   120 |                   kDefaultName,
90.56       |                   ~~~~~~~~~~~~~
90.56   121 |                   kDefaultGroupId,
90.56       |                   ~~~~~~~~~~~~~~~~
90.56   122 |                   std::vector<std::string>{kCharactersticA, kCharactersticB},
90.56       |                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
90.56   123 |                   &mock_notifier_) {}
90.56       |                   ~~~~~~~~~~~~~~~~
90.56 In file included from /src/packager/media/event/hls_notify_muxer_listener_unittest.cc:13:
90.56 /src/./packager/media/event/hls_notify_muxer_listener.h:43:3: note: candidate: 'shaka::media::HlsNotifyMuxerListener::HlsNotifyMuxerListener(const string&, bool, const string&, const string&, const std::vector<std::__cxx11::basic_string<char> >&, const int32_t&, shaka::hls::HlsNotifier*)'
90.56    43 |   HlsNotifyMuxerListener(const std::string& playlist_name,
90.56       |   ^~~~~~~~~~~~~~~~~~~~~~
90.56 /src/./packager/media/event/hls_notify_muxer_listener.h:43:3: note:   candidate expects 7 arguments, 6 provided
90.56 /src/packager/media/event/hls_notify_muxer_listener_unittest.cc: At global scope:
90.56 /src/packager/media/event/hls_notify_muxer_listener_unittest.cc:462:19: error: cannot declare field 'shaka::media::HlsNotifyMuxerListenerKeyFrameTest::mock_notifier_' to be of abstract type 'shaka::media::{anonymous}::MockHlsNotifier'
90.56   462 |   MockHlsNotifier mock_notifier_;
90.56       |                   ^~~~~~~~~~~~~~
90.56 /src/packager/media/event/hls_notify_muxer_listener_unittest.cc: In constructor 'shaka::media::HlsNotifyMuxerListenerKeyFrameTest::HlsNotifyMuxerListenerKeyFrameTest()':
90.56 /src/packager/media/event/hls_notify_muxer_listener_unittest.cc:455:9: error: no matching function for call to 'shaka::media::HlsNotifyMuxerListener::HlsNotifyMuxerListener(const char [22], const ParamType&, const char [12], const char [15], std::vector<std::__cxx11::basic_string<char> >, shaka::media::{anonymous}::MockHlsNotifier*)'
90.56   455 |       : listener_(kDefaultPlaylistName,
90.56       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
90.56   456 |                   GetParam(),
90.56       |                   ~~~~~~~~~~~
90.56   457 |                   kDefaultName,
90.56       |                   ~~~~~~~~~~~~~
90.56   458 |                   kDefaultGroupId,
90.56       |                   ~~~~~~~~~~~~~~~~
90.56   459 |                   std::vector<std::string>(),  // no characteristics.
90.56       |                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
90.56   460 |                   &mock_notifier_) {}
90.56       |                   ~~~~~~~~~~~~~~~~
90.56 In file included from /src/packager/media/event/hls_notify_muxer_listener_unittest.cc:13:
90.56 /src/./packager/media/event/hls_notify_muxer_listener.h:43:3: note: candidate: 'shaka::media::HlsNotifyMuxerListener::HlsNotifyMuxerListener(const string&, bool, const string&, const string&, const std::vector<std::__cxx11::basic_string<char> >&, const int32_t&, shaka::hls::HlsNotifier*)'
90.56    43 |   HlsNotifyMuxerListener(const std::string& playlist_name,
90.56       |   ^~~~~~~~~~~~~~~~~~~~~~
90.56 /src/./packager/media/event/hls_notify_muxer_listener.h:43:3: note:   candidate expects 7 arguments, 6 provided
90.56 At global scope:
90.56 cc1plus: note: unrecognized command-line option '-Wno-shorten-64-to-32' may have been intended to silence earlier diagnostics
90.57 cc1plus: note: unrecognized command-line option '-Wno-unknown-warning-option' may have been intended to silence earlier diagnostics
90.59 [971/1160] Building CXX object packager/media/formats/mp4/CMakeFiles/mp4.dir/box.cc.o
90.88 [972/1160] Building CXX object packager/media/chunking/CMakeFiles/media_chunking_unittest.dir/cue_alignment_handler_unittest.cc.o
90.93 [973/1160] Building CXX object packager/media/formats/dvb/CMakeFiles/dvb_unittest.dir/dvb_image_unittest.cc.o
91.19 [974/1160] Building CXX object packager/media/formats/packed_audio/CMakeFiles/packed_audio.dir/packed_audio_writer.cc.o
91.26 [975/1160] Building CXX object packager/media/event/CMakeFiles/media_event_unittest.dir/vod_media_info_dump_muxer_listener_unittest.cc.o
91.32 [976/1160] Building CXX object packager/media/formats/ttml/CMakeFiles/ttml.dir/ttml_muxer.cc.o
91.43 [977/1160] Building CXX object packager/media/formats/dvb/CMakeFiles/dvb_unittest.dir/subtitle_composer_unittest.cc.o
91.87 [978/1160] Building CXX object packager/media/formats/dvb/CMakeFiles/dvb_unittest.dir/dvb_sub_parser_unittest.cc.o
91.92 [979/1160] Building CXX object packager/media/event/CMakeFiles/mock_muxer_listener.dir/mock_muxer_listener.cc.o
92.78 [980/1160] Building CXX object packager/media/formats/ttml/CMakeFiles/ttml.dir/ttml_to_mp4_handler.cc.o
93.58 [981/1160] Building CXX object packager/media/crypto/CMakeFiles/media_crypto_unittest.dir/subsample_generator_unittest.cc.o
94.02 [982/1160] Building CXX object packager/media/event/CMakeFiles/media_event_unittest.dir/multi_codec_muxer_listener_unittest.cc.o
94.91 [983/1160] Building CXX object packager/media/event/CMakeFiles/media_event_unittest.dir/mpd_notify_muxer_listener_unittest.cc.o
95.17 [984/1160] Building CXX object packager/media/formats/mp4/CMakeFiles/mp4.dir/box_definitions.cc.o
95.53 [985/1160] Building CXX object packager/media/formats/packed_audio/CMakeFiles/packed_audio_unittest.dir/packed_audio_segmenter_unittest.cc.o
96.76 [986/1160] Building CXX object packager/media/formats/packed_audio/CMakeFiles/packed_audio_unittest.dir/packed_audio_writer_unittest.cc.o
98.52 [987/1160] Building CXX object packager/media/crypto/CMakeFiles/media_crypto_unittest.dir/encryption_handler_unittest.cc.o
98.52 ninja: build stopped: subcommand failed.

bmurphy1976 · 2023-09-28T17:24:58Z

For anybody else following up after me, I'm able to get just the packager binary built (skipping other build artifacts such as the broken tests) on @xavierlaffargue's branch with the following commands:

git clone https://github.com/xavierlaffargue/shaka-packager.git .
git checkout hls-order-track
git submodule update --init --recursive
cmake -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cd build
ninja packager/packager

cosmin · 2023-09-28T18:58:53Z

On a side note git clone --recurse-submodules https://github.com/xavierlaffargue/shaka-packager.git . this doesn't work because it would only init submodules in the default branch, which is main, and does not contain any submodules. It would only work in one shot if you directly specify the name of the branch as part of the git clone with -b hls-order-track

bmurphy1976 · 2023-09-28T19:14:58Z

@xavierlaffargue I have a video that is not respecting the order_number flag when I try it:

Here is the command I'm running:

packager \
	'in=output/tmp/mtaudio-a0-v0.mp4,order_number=1,drm_label=AUDIO,playlist_name=output/final/mtaudio-a0-v0.m3u8,hls_name=English - Stereo - Main,stream=audio,init_segment=output/final/mtaudio-a0-v0-init.mp4,segment_template=output/final/mtaudio-a0-v0-$Number$.m4s' \
	'in=output/tmp/mtaudio-a0-v1.mp4,order_number=2,drm_label=AUDIO,playlist_name=output/final/mtaudio-a0-v1.m3u8,hls_name=English - 5.1 AC-3 - Main,stream=audio,init_segment=output/final/mtaudio-a0-v1-init.mp4,segment_template=output/final/mtaudio-a0-v1-$Number$.m4s' \
	'in=output/tmp/mtaudio-a1-v0.mp4,order_number=3,drm_label=AUDIO,playlist_name=output/final/mtaudio-a1-v0.m3u8,hls_name=English - Stereo - Visually Impaired,stream=audio,init_segment=output/final/mtaudio-a1-v0-init.mp4,segment_template=output/final/mtaudio-a1-v0-$Number$.m4s' \
	'in=output/tmp/mtaudio-a1-v1.mp4,order_number=4,drm_label=AUDIO,playlist_name=output/final/mtaudio-a1-v1.m3u8,hls_name=English - 5.1 AC-3 - Visually Impaired,stream=audio,init_segment=output/final/mtaudio-a1-v1-init.mp4,segment_template=output/final/mtaudio-a1-v1-$Number$.m4s' \
	'in=output/tmp/mtaudio-v0-v0.mp4,order_number=5,drm_label=HD,playlist_name=output/final/mtaudio-v0-v0.m3u8,hls_name=1596k video,stream=video,init_segment=output/final/mtaudio-v0-v0-init.mp4,segment_template=output/final/mtaudio-v0-v0-$Number$.m4s' \
	'in=output/tmp/mtaudio-v0-v1.mp4,order_number=6,drm_label=HD,playlist_name=output/final/mtaudio-v0-v1.m3u8,hls_name=512k video,stream=video,init_segment=output/final/mtaudio-v0-v1-init.mp4,segment_template=output/final/mtaudio-v0-v1-$Number$.m4s' \
	--segment_duration 60 \
	--fragment_duration 60 \
	--mpd_output output/final/mtaudio.mpd \
	--hls_master_playlist_output output/final/mtaudio.m3u8 \
	--generate_static_live_mpd

And here's an example output m3u8:

#EXTM3U
## Generated with https://github.com/shaka-project/shaka-packager version 45c556c289-release

#EXT-X-INDEPENDENT-SEGMENTS

#EXT-X-MEDIA:TYPE=AUDIO,URI="mtaudio-a1-v0.m3u8",GROUP-ID="default-audio-group",LANGUAGE="en",NAME="English - Stereo - Visually Impaired",DEFAULT=NO,AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="mtaudio-a1-v1.m3u8",GROUP-ID="default-audio-group",LANGUAGE="en",NAME="English - 5.1 AC-3 - Visually Impaired",DEFAULT=NO,CHANNELS="6"
#EXT-X-MEDIA:TYPE=AUDIO,URI="mtaudio-a0-v0.m3u8",GROUP-ID="default-audio-group",LANGUAGE="en",NAME="English - Stereo - Main",DEFAULT=NO,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="mtaudio-a0-v1.m3u8",GROUP-ID="default-audio-group",LANGUAGE="en",NAME="English - 5.1 AC-3 - Main",DEFAULT=NO,CHANNELS="6"

#EXT-X-STREAM-INF:BANDWIDTH=1515575,AVERAGE-BANDWIDTH=1515575,CODECS="avc1.64001f,ac-3,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE=23.976,AUDIO="default-audio-group",CLOSED-CAPTIONS=NONE
mtaudio-v0-v0.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1055793,AVERAGE-BANDWIDTH=1055793,CODECS="avc1.64001f,ac-3,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE=23.976,AUDIO="default-audio-group",CLOSED-CAPTIONS=NONE
mtaudio-v0-v1.m3u8

Here is the console output:

WARNING: Logging before InitGoogleLogging() is written to STDERR
I20230928 18:13:57.768424    16 demuxer.cc:91] Demuxer::Run() on file 'output/tmp/mtaudio-v0-v0.mp4'.
I20230928 18:13:57.768501    17 demuxer.cc:91] Demuxer::Run() on file 'output/tmp/mtaudio-v0-v1.mp4'.
I20230928 18:13:57.768319    12 demuxer.cc:91] Demuxer::Run() on file 'output/tmp/mtaudio-a0-v0.mp4'.
I20230928 18:13:57.768397    15 demuxer.cc:91] Demuxer::Run() on file 'output/tmp/mtaudio-a1-v1.mp4'.
I20230928 18:13:57.768585    17 demuxer.cc:157] Initialize Demuxer for file 'output/tmp/mtaudio-v0-v1.mp4'.
I20230928 18:13:57.768319    13 demuxer.cc:91] Demuxer::Run() on file 'output/tmp/mtaudio-a0-v1.mp4'.
I20230928 18:13:57.768721    13 demuxer.cc:157] Initialize Demuxer for file 'output/tmp/mtaudio-a0-v1.mp4'.
I20230928 18:13:57.768576    16 demuxer.cc:157] Initialize Demuxer for file 'output/tmp/mtaudio-v0-v0.mp4'.
I20230928 18:13:57.768595    12 demuxer.cc:157] Initialize Demuxer for file 'output/tmp/mtaudio-a0-v0.mp4'.
I20230928 18:13:57.768646    15 demuxer.cc:157] Initialize Demuxer for file 'output/tmp/mtaudio-a1-v1.mp4'.
I20230928 18:13:57.768362    14 demuxer.cc:91] Demuxer::Run() on file 'output/tmp/mtaudio-a1-v0.mp4'.
I20230928 18:13:57.768826    14 demuxer.cc:157] Initialize Demuxer for file 'output/tmp/mtaudio-a1-v0.mp4'.
I20230928 18:13:57.893694    12 mp4_muxer.cc:186] MP4 file 'output/final/mtaudio-a0-v0-init.mp4' finalized.
I20230928 18:13:57.913518    14 mp4_muxer.cc:186] MP4 file 'output/final/mtaudio-a1-v0-init.mp4' finalized.
I20230928 18:13:57.924500    16 mp4_muxer.cc:186] MP4 file 'output/final/mtaudio-v0-v0-init.mp4' finalized.
I20230928 18:13:57.935506    15 mp4_muxer.cc:186] MP4 file 'output/final/mtaudio-a1-v1-init.mp4' finalized.
I20230928 18:13:57.946743    13 mp4_muxer.cc:186] MP4 file 'output/final/mtaudio-a0-v1-init.mp4' finalized.
I20230928 18:13:57.957764    17 mp4_muxer.cc:186] MP4 file 'output/final/mtaudio-v0-v1-init.mp4' finalized.
Packaging completed successfully.

I've run this multiple times and the order is still completely random.

vish91 · 2023-09-28T20:26:10Z

Would it be feasible to preserve the input order? I would find that most intuitive, and it would be a benefit that existing users get for free without changing commands.

You could keep the order_number field internally, but default it to the index from the input arguments.

@joeyparrish yeah we have been working on something like this internally on shaka packager fork and got it working. If you would like to take a look , I or my teammate can open a PR for that.
Shaka order is random today but instead we can preserve the order if the user passes a singe feature flag on command , so doesn't have to do it on each input or specify order number. This way the users not using this feature flag will basically just get backwards compatible behavior support.

However our stuff is not on the cmake branch since we started working on this quite a while back. But we can open a PR and take it from there or rebase against a certain branch you prefer.

xavierlaffargue · 2023-09-29T21:05:02Z

@xavierlaffargue I have a video that is not respecting the order_number flag when I try it:

Here is the command I'm running:

packager \
	'in=output/tmp/mtaudio-a0-v0.mp4,order_number=1,drm_label=AUDIO,playlist_name=output/final/mtaudio-a0-v0.m3u8,hls_name=English - Stereo - Main,stream=audio,init_segment=output/final/mtaudio-a0-v0-init.mp4,segment_template=output/final/mtaudio-a0-v0-$Number$.m4s' \
	'in=output/tmp/mtaudio-a0-v1.mp4,order_number=2,drm_label=AUDIO,playlist_name=output/final/mtaudio-a0-v1.m3u8,hls_name=English - 5.1 AC-3 - Main,stream=audio,init_segment=output/final/mtaudio-a0-v1-init.mp4,segment_template=output/final/mtaudio-a0-v1-$Number$.m4s' \
	'in=output/tmp/mtaudio-a1-v0.mp4,order_number=3,drm_label=AUDIO,playlist_name=output/final/mtaudio-a1-v0.m3u8,hls_name=English - Stereo - Visually Impaired,stream=audio,init_segment=output/final/mtaudio-a1-v0-init.mp4,segment_template=output/final/mtaudio-a1-v0-$Number$.m4s' \
	'in=output/tmp/mtaudio-a1-v1.mp4,order_number=4,drm_label=AUDIO,playlist_name=output/final/mtaudio-a1-v1.m3u8,hls_name=English - 5.1 AC-3 - Visually Impaired,stream=audio,init_segment=output/final/mtaudio-a1-v1-init.mp4,segment_template=output/final/mtaudio-a1-v1-$Number$.m4s' \
	'in=output/tmp/mtaudio-v0-v0.mp4,order_number=5,drm_label=HD,playlist_name=output/final/mtaudio-v0-v0.m3u8,hls_name=1596k video,stream=video,init_segment=output/final/mtaudio-v0-v0-init.mp4,segment_template=output/final/mtaudio-v0-v0-$Number$.m4s' \
	'in=output/tmp/mtaudio-v0-v1.mp4,order_number=6,drm_label=HD,playlist_name=output/final/mtaudio-v0-v1.m3u8,hls_name=512k video,stream=video,init_segment=output/final/mtaudio-v0-v1-init.mp4,segment_template=output/final/mtaudio-v0-v1-$Number$.m4s' \
	--segment_duration 60 \
	--fragment_duration 60 \
	--mpd_output output/final/mtaudio.mpd \
	--hls_master_playlist_output output/final/mtaudio.m3u8 \
	--generate_static_live_mpd

And here's an example output m3u8:

#EXTM3U
## Generated with https://github.com/shaka-project/shaka-packager version 45c556c289-release

#EXT-X-INDEPENDENT-SEGMENTS

#EXT-X-MEDIA:TYPE=AUDIO,URI="mtaudio-a1-v0.m3u8",GROUP-ID="default-audio-group",LANGUAGE="en",NAME="English - Stereo - Visually Impaired",DEFAULT=NO,AUTOSELECT=YES,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="mtaudio-a1-v1.m3u8",GROUP-ID="default-audio-group",LANGUAGE="en",NAME="English - 5.1 AC-3 - Visually Impaired",DEFAULT=NO,CHANNELS="6"
#EXT-X-MEDIA:TYPE=AUDIO,URI="mtaudio-a0-v0.m3u8",GROUP-ID="default-audio-group",LANGUAGE="en",NAME="English - Stereo - Main",DEFAULT=NO,CHANNELS="2"
#EXT-X-MEDIA:TYPE=AUDIO,URI="mtaudio-a0-v1.m3u8",GROUP-ID="default-audio-group",LANGUAGE="en",NAME="English - 5.1 AC-3 - Main",DEFAULT=NO,CHANNELS="6"

#EXT-X-STREAM-INF:BANDWIDTH=1515575,AVERAGE-BANDWIDTH=1515575,CODECS="avc1.64001f,ac-3,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE=23.976,AUDIO="default-audio-group",CLOSED-CAPTIONS=NONE
mtaudio-v0-v0.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1055793,AVERAGE-BANDWIDTH=1055793,CODECS="avc1.64001f,ac-3,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE=23.976,AUDIO="default-audio-group",CLOSED-CAPTIONS=NONE
mtaudio-v0-v1.m3u8

Here is the console output:

WARNING: Logging before InitGoogleLogging() is written to STDERR
I20230928 18:13:57.768424    16 demuxer.cc:91] Demuxer::Run() on file 'output/tmp/mtaudio-v0-v0.mp4'.
I20230928 18:13:57.768501    17 demuxer.cc:91] Demuxer::Run() on file 'output/tmp/mtaudio-v0-v1.mp4'.
I20230928 18:13:57.768319    12 demuxer.cc:91] Demuxer::Run() on file 'output/tmp/mtaudio-a0-v0.mp4'.
I20230928 18:13:57.768397    15 demuxer.cc:91] Demuxer::Run() on file 'output/tmp/mtaudio-a1-v1.mp4'.
I20230928 18:13:57.768585    17 demuxer.cc:157] Initialize Demuxer for file 'output/tmp/mtaudio-v0-v1.mp4'.
I20230928 18:13:57.768319    13 demuxer.cc:91] Demuxer::Run() on file 'output/tmp/mtaudio-a0-v1.mp4'.
I20230928 18:13:57.768721    13 demuxer.cc:157] Initialize Demuxer for file 'output/tmp/mtaudio-a0-v1.mp4'.
I20230928 18:13:57.768576    16 demuxer.cc:157] Initialize Demuxer for file 'output/tmp/mtaudio-v0-v0.mp4'.
I20230928 18:13:57.768595    12 demuxer.cc:157] Initialize Demuxer for file 'output/tmp/mtaudio-a0-v0.mp4'.
I20230928 18:13:57.768646    15 demuxer.cc:157] Initialize Demuxer for file 'output/tmp/mtaudio-a1-v1.mp4'.
I20230928 18:13:57.768362    14 demuxer.cc:91] Demuxer::Run() on file 'output/tmp/mtaudio-a1-v0.mp4'.
I20230928 18:13:57.768826    14 demuxer.cc:157] Initialize Demuxer for file 'output/tmp/mtaudio-a1-v0.mp4'.
I20230928 18:13:57.893694    12 mp4_muxer.cc:186] MP4 file 'output/final/mtaudio-a0-v0-init.mp4' finalized.
I20230928 18:13:57.913518    14 mp4_muxer.cc:186] MP4 file 'output/final/mtaudio-a1-v0-init.mp4' finalized.
I20230928 18:13:57.924500    16 mp4_muxer.cc:186] MP4 file 'output/final/mtaudio-v0-v0-init.mp4' finalized.
I20230928 18:13:57.935506    15 mp4_muxer.cc:186] MP4 file 'output/final/mtaudio-a1-v1-init.mp4' finalized.
I20230928 18:13:57.946743    13 mp4_muxer.cc:186] MP4 file 'output/final/mtaudio-a0-v1-init.mp4' finalized.
I20230928 18:13:57.957764    17 mp4_muxer.cc:186] MP4 file 'output/final/mtaudio-v0-v1-init.mp4' finalized.
Packaging completed successfully.

I've run this multiple times and the order is still completely random.

Indeed I ordering only video!, I'm adding the audio :)

xavierlaffargue · 2023-09-29T22:14:50Z

@bmurphy1976 can you pull the last commit and retry?

xavierlaffargue · 2023-09-30T15:55:09Z

Would it be feasible to preserve the input order? I would find that most intuitive, and it would be a benefit that existing users get for free without changing commands.

You could keep the order_number field internally, but default it to the index from the input arguments.

Yes, good idea, I will preserve the input order :)

bmurphy1976 · 2023-10-02T15:46:56Z

@xavierlaffargue I pulled, but I'm getting a new set of build errors now:

[3/14] Building CXX object packager/mpd/CMakeFiles/mpd_builder.dir/base/period.cc.o
FAILED: packager/mpd/CMakeFiles/mpd_builder.dir/base/period.cc.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DCURL_STATICLIB -DGLOG_CUSTOM_PREFIX_SUPPORT -DLIBXML_STATIC -I/Users/bmurphy/w/shaka-packager/. -I/Users/bmurphy/w/shaka-packager/packager/third_party/abseil-cpp/source -I/Users/bmurphy/w/shaka-packager/build/packager/third_party/glog/source -I/Users/bmurphy/w/shaka-packager/packager/third_party/glog/source/src -I/Users/bmurphy/w/shaka-packager/build/packager/third_party/libxml2/source -I/Users/bmurphy/w/shaka-packager/packager/third_party/libxml2/source/include -I/Users/bmurphy/w/shaka-packager/packager/third_party/curl/source/include -I/Users/bmurphy/w/shaka-packager/packager/third_party/mbedtls/source/include -I/Users/bmurphy/w/shaka-packager/build -I/Users/bmurphy/w/shaka-packager/packager/third_party/protobuf/source/src -isystem /opt/local/include -std=c++11  -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Wall -Wextra -Wnarrowing -Wno-deprecated -Wshorten-64-to-32 -Werror -O3 -DNDEBUG -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk -Wall -Wextra -Werror -Wno-unknown-warning-option -Wno-shorten-64-to-32 -Wno-unused-parameter -Wno-stringop-overflow -std=gnu++17 -MD -MT packager/mpd/CMakeFiles/mpd_builder.dir/base/period.cc.o -MF packager/mpd/CMakeFiles/mpd_builder.dir/base/period.cc.o.d -o packager/mpd/CMakeFiles/mpd_builder.dir/base/period.cc.o -c /Users/bmurphy/w/shaka-packager/packager/mpd/base/period.cc
In file included from /Users/bmurphy/w/shaka-packager/packager/mpd/base/period.cc:7:
In file included from /Users/bmurphy/w/shaka-packager/./packager/mpd/base/period.h:16:
/Users/bmurphy/w/shaka-packager/./packager/mpd/base/adaptation_set.h:184:3: error: 'const' type qualifier on return type has no effect [-Werror,-Wignored-qualifiers]
  const uint32_t transfer_characteristics() const {
  ^~~~~~
1 error generated.
[4/14] Building CXX object packager/mpd/CMakeFiles/mpd_builder.dir/base/simple_mpd_notifier.cc.o
FAILED: packager/mpd/CMakeFiles/mpd_builder.dir/base/simple_mpd_notifier.cc.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DCURL_STATICLIB -DGLOG_CUSTOM_PREFIX_SUPPORT -DLIBXML_STATIC -I/Users/bmurphy/w/shaka-packager/. -I/Users/bmurphy/w/shaka-packager/packager/third_party/abseil-cpp/source -I/Users/bmurphy/w/shaka-packager/build/packager/third_party/glog/source -I/Users/bmurphy/w/shaka-packager/packager/third_party/glog/source/src -I/Users/bmurphy/w/shaka-packager/build/packager/third_party/libxml2/source -I/Users/bmurphy/w/shaka-packager/packager/third_party/libxml2/source/include -I/Users/bmurphy/w/shaka-packager/packager/third_party/curl/source/include -I/Users/bmurphy/w/shaka-packager/packager/third_party/mbedtls/source/include -I/Users/bmurphy/w/shaka-packager/build -I/Users/bmurphy/w/shaka-packager/packager/third_party/protobuf/source/src -isystem /opt/local/include -std=c++11  -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Wall -Wextra -Wnarrowing -Wno-deprecated -Wshorten-64-to-32 -Werror -O3 -DNDEBUG -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk -Wall -Wextra -Werror -Wno-unknown-warning-option -Wno-shorten-64-to-32 -Wno-unused-parameter -Wno-stringop-overflow -std=gnu++17 -MD -MT packager/mpd/CMakeFiles/mpd_builder.dir/base/simple_mpd_notifier.cc.o -MF packager/mpd/CMakeFiles/mpd_builder.dir/base/simple_mpd_notifier.cc.o.d -o packager/mpd/CMakeFiles/mpd_builder.dir/base/simple_mpd_notifier.cc.o -c /Users/bmurphy/w/shaka-packager/packager/mpd/base/simple_mpd_notifier.cc
In file included from /Users/bmurphy/w/shaka-packager/packager/mpd/base/simple_mpd_notifier.cc:10:
/Users/bmurphy/w/shaka-packager/./packager/mpd/base/adaptation_set.h:184:3: error: 'const' type qualifier on return type has no effect [-Werror,-Wignored-qualifiers]
  const uint32_t transfer_characteristics() const {
  ^~~~~~
1 error generated.
[5/14] Building CXX object packager/mpd/CMakeFiles/mpd_builder.dir/base/mpd_utils.cc.o
FAILED: packager/mpd/CMakeFiles/mpd_builder.dir/base/mpd_utils.cc.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DCURL_STATICLIB -DGLOG_CUSTOM_PREFIX_SUPPORT -DLIBXML_STATIC -I/Users/bmurphy/w/shaka-packager/. -I/Users/bmurphy/w/shaka-packager/packager/third_party/abseil-cpp/source -I/Users/bmurphy/w/shaka-packager/build/packager/third_party/glog/source -I/Users/bmurphy/w/shaka-packager/packager/third_party/glog/source/src -I/Users/bmurphy/w/shaka-packager/build/packager/third_party/libxml2/source -I/Users/bmurphy/w/shaka-packager/packager/third_party/libxml2/source/include -I/Users/bmurphy/w/shaka-packager/packager/third_party/curl/source/include -I/Users/bmurphy/w/shaka-packager/packager/third_party/mbedtls/source/include -I/Users/bmurphy/w/shaka-packager/build -I/Users/bmurphy/w/shaka-packager/packager/third_party/protobuf/source/src -isystem /opt/local/include -std=c++11  -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Wall -Wextra -Wnarrowing -Wno-deprecated -Wshorten-64-to-32 -Werror -O3 -DNDEBUG -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk -Wall -Wextra -Werror -Wno-unknown-warning-option -Wno-shorten-64-to-32 -Wno-unused-parameter -Wno-stringop-overflow -std=gnu++17 -MD -MT packager/mpd/CMakeFiles/mpd_builder.dir/base/mpd_utils.cc.o -MF packager/mpd/CMakeFiles/mpd_builder.dir/base/mpd_utils.cc.o.d -o packager/mpd/CMakeFiles/mpd_builder.dir/base/mpd_utils.cc.o -c /Users/bmurphy/w/shaka-packager/packager/mpd/base/mpd_utils.cc
In file included from /Users/bmurphy/w/shaka-packager/packager/mpd/base/mpd_utils.cc:18:
/Users/bmurphy/w/shaka-packager/./packager/mpd/base/adaptation_set.h:184:3: error: 'const' type qualifier on return type has no effect [-Werror,-Wignored-qualifiers]
  const uint32_t transfer_characteristics() const {
  ^~~~~~
1 error generated.
[6/14] Building CXX object packager/mpd/CMakeFiles/mpd_builder.dir/base/mpd_builder.cc.o
FAILED: packager/mpd/CMakeFiles/mpd_builder.dir/base/mpd_builder.cc.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DCURL_STATICLIB -DGLOG_CUSTOM_PREFIX_SUPPORT -DLIBXML_STATIC -I/Users/bmurphy/w/shaka-packager/. -I/Users/bmurphy/w/shaka-packager/packager/third_party/abseil-cpp/source -I/Users/bmurphy/w/shaka-packager/build/packager/third_party/glog/source -I/Users/bmurphy/w/shaka-packager/packager/third_party/glog/source/src -I/Users/bmurphy/w/shaka-packager/build/packager/third_party/libxml2/source -I/Users/bmurphy/w/shaka-packager/packager/third_party/libxml2/source/include -I/Users/bmurphy/w/shaka-packager/packager/third_party/curl/source/include -I/Users/bmurphy/w/shaka-packager/packager/third_party/mbedtls/source/include -I/Users/bmurphy/w/shaka-packager/build -I/Users/bmurphy/w/shaka-packager/packager/third_party/protobuf/source/src -isystem /opt/local/include -std=c++11  -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Wall -Wextra -Wnarrowing -Wno-deprecated -Wshorten-64-to-32 -Werror -O3 -DNDEBUG -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk -Wall -Wextra -Werror -Wno-unknown-warning-option -Wno-shorten-64-to-32 -Wno-unused-parameter -Wno-stringop-overflow -std=gnu++17 -MD -MT packager/mpd/CMakeFiles/mpd_builder.dir/base/mpd_builder.cc.o -MF packager/mpd/CMakeFiles/mpd_builder.dir/base/mpd_builder.cc.o.d -o packager/mpd/CMakeFiles/mpd_builder.dir/base/mpd_builder.cc.o -c /Users/bmurphy/w/shaka-packager/packager/mpd/base/mpd_builder.cc
In file included from /Users/bmurphy/w/shaka-packager/packager/mpd/base/mpd_builder.cc:20:
/Users/bmurphy/w/shaka-packager/./packager/mpd/base/adaptation_set.h:184:3: error: 'const' type qualifier on return type has no effect [-Werror,-Wignored-qualifiers]
  const uint32_t transfer_characteristics() const {
  ^~~~~~
1 error generated.
[7/14] Building CXX object packager/mpd/CMakeFiles/mpd_builder.dir/base/adaptation_set.cc.o
FAILED: packager/mpd/CMakeFiles/mpd_builder.dir/base/adaptation_set.cc.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DCURL_STATICLIB -DGLOG_CUSTOM_PREFIX_SUPPORT -DLIBXML_STATIC -I/Users/bmurphy/w/shaka-packager/. -I/Users/bmurphy/w/shaka-packager/packager/third_party/abseil-cpp/source -I/Users/bmurphy/w/shaka-packager/build/packager/third_party/glog/source -I/Users/bmurphy/w/shaka-packager/packager/third_party/glog/source/src -I/Users/bmurphy/w/shaka-packager/build/packager/third_party/libxml2/source -I/Users/bmurphy/w/shaka-packager/packager/third_party/libxml2/source/include -I/Users/bmurphy/w/shaka-packager/packager/third_party/curl/source/include -I/Users/bmurphy/w/shaka-packager/packager/third_party/mbedtls/source/include -I/Users/bmurphy/w/shaka-packager/build -I/Users/bmurphy/w/shaka-packager/packager/third_party/protobuf/source/src -isystem /opt/local/include -std=c++11  -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Wall -Wextra -Wnarrowing -Wno-deprecated -Wshorten-64-to-32 -Werror -O3 -DNDEBUG -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.3.sdk -Wall -Wextra -Werror -Wno-unknown-warning-option -Wno-shorten-64-to-32 -Wno-unused-parameter -Wno-stringop-overflow -std=gnu++17 -MD -MT packager/mpd/CMakeFiles/mpd_builder.dir/base/adaptation_set.cc.o -MF packager/mpd/CMakeFiles/mpd_builder.dir/base/adaptation_set.cc.o.d -o packager/mpd/CMakeFiles/mpd_builder.dir/base/adaptation_set.cc.o -c /Users/bmurphy/w/shaka-packager/packager/mpd/base/adaptation_set.cc
In file included from /Users/bmurphy/w/shaka-packager/packager/mpd/base/adaptation_set.cc:7:
/Users/bmurphy/w/shaka-packager/./packager/mpd/base/adaptation_set.h:184:3: error: 'const' type qualifier on return type has no effect [-Werror,-Wignored-qualifiers]
  const uint32_t transfer_characteristics() const {
  ^~~~~~
/Users/bmurphy/w/shaka-packager/packager/mpd/base/adaptation_set.cc:299:12: error: use of undeclared identifier 'base'; did you mean 'google::base'?
    return base::nullopt;
           ^~~~
           google::base
/Users/bmurphy/w/shaka-packager/build/packager/third_party/glog/source/glog/logging.h:1867:11: note: 'google::base' declared here
namespace base {
          ^
/Users/bmurphy/w/shaka-packager/packager/mpd/base/adaptation_set.cc:299:12: error: no member named 'nullopt' in namespace 'google::base'; did you mean 'absl::nullopt'?
    return base::nullopt;
           ^~~~~~~~~~~~~
           absl::nullopt
/Users/bmurphy/w/shaka-packager/packager/third_party/abseil-cpp/source/absl/types/optional.h:51:12: note: 'absl::nullopt' declared here
using std::nullopt;
           ^
3 errors generated.
[11/14] Building CXX object packager/mpd/CMakeFiles/mpd_builder.dir/base/xml/xml_node.cc.o
ninja: build stopped: subcommand failed.

xavierlaffargue · 2023-10-02T18:39:50Z

@bmurphy1976 Sorry, fixed, now it will be ok :)

bmurphy1976 · 2023-10-02T20:55:23Z

@xavierlaffargue I was able to build the tool but it's failing with this error now:

WARNING: Logging before InitGoogleLogging() is written to STDERR
E20231002 15:50:42.442212 4910076 stream_descriptor.cc:253] Unknown field in stream descriptor ("order_number").

I did a full clean checkout just to be sure I didn't have something lingering from a previous build, no luck.

$ git rev-parse --verify HEAD
1cfd10ec087d37ae7c764a228764a69aa3585e08
$ git status
On branch hls-order-track
Your branch is up to date with 'origin/hls-order-track'.

nothing to commit, working tree clean
$ git remote show origin
* remote origin
  Fetch URL: https://github.com/xavierlaffargue/shaka-packager
  Push  URL: https://github.com/xavierlaffargue/shaka-packager
  HEAD branch: main
  Remote branches:
    cmake           tracked
    gh-pages        tracked
    hls-order-track tracked
    main            tracked
    v1.4.x          tracked
    v1.5.x          tracked
    v1.6.x          tracked
    v2.0.x          tracked
    v2.1.x          tracked
    v2.2.x          tracked
    v2.3.x          tracked
    v2.4.x          tracked
  Local branches configured for 'git pull':
    hls-order-track merges with remote hls-order-track
    main            merges with remote main
  Local refs configured for 'git push':
    hls-order-track pushes to hls-order-track (up to date)
    main            pushes to main            (up to date)
$ grep -rl order_number *
packager/app/packager_main.cc
packager/packager.cc
packager/packager.h
packager/hls/base/media_playlist.h
packager/hls/base/simple_hls_notifier.cc
packager/hls/base/hls_notifier.h
packager/hls/base/media_playlist.cc
packager/hls/base/master_playlist.cc
packager/hls/base/simple_hls_notifier.h
packager/media/event/hls_notify_muxer_listener.h
packager/media/event/muxer_listener_factory.cc
packager/media/event/hls_notify_muxer_listener.cc
packager/media/event/muxer_listener_factory.h

xavierlaffargue · 2023-10-02T22:13:30Z

Ho yes sorry, I deleted this field :D Now I preserve the input order! like propose @joeyparrish

bmurphy1976 · 2023-10-02T22:22:30Z

Ah, I understand! I just ran a quick test and it looks like it is working. I'm going to be bulk testing a lot of videos this week, I'll test it out and if I run into anything I'll let you know.

bmurphy1976 · 2023-10-05T02:20:03Z

I ran a bunch of tests, but I had to switch back to the release build. The ordering code appears to be working as expected, but I'm running into a bunch of other issues which I think are related to the underlying branch not being fully baked yet. I have other priorities right now so I can't spend any more time on it, but I'll revisit it in the future. I have a lot of messy post-shaka-processing code to keep the track orders in check that I would love to remove some day.

Fixed shaka-project#560 with suggested implementation: shaka-project#560 (comment)

wjywbs · 2023-10-17T22:43:13Z

Hi, #1280 is my implementation of this issue for reference, based on the @kqyang's suggestion at #560 (comment)

Fixed shaka-project#560 with suggested implementation: shaka-project#560 (comment)

This will force the muxer to order streams in the order given on the command-line. Closes #560 Closes #1280 Closes #1313 --------- Co-authored-by: Joey Parrish <joeyparrish@users.noreply.github.com> Co-authored-by: Cosmin Stejerean <cstejerean@meta.com>

shaka-bot added the needs triage label Feb 23, 2019

kqyang added type: enhancement New feature or request and removed needs triage labels Feb 25, 2019

kqyang added this to the v2.5 milestone Feb 25, 2019

kqyang self-assigned this Jul 12, 2019

kqyang modified the milestones: v2.5, v2.4 Jul 12, 2019

kqyang modified the milestones: v2.4, v2.5 Nov 17, 2019

kqyang added the flag: seeking PR We are actively seeking PRs for this; we do not currently expect the core team will resolve this label Apr 8, 2020

kqyang modified the milestones: v2.5, v2.6 Aug 20, 2020

joeyparrish modified the milestones: v2.6, Backlog Jun 24, 2022

joeyparrish assigned xavierlaffargue and unassigned kqyang Sep 27, 2023

joeyparrish added the priority: P3 Useful but not urgent label Sep 27, 2023

wjywbs mentioned this issue Oct 17, 2023

feat: Implement stream ordering in HLS playlist based on input order. #1280

Closed

wjywbs added a commit to wjywbs/shaka-packager that referenced this issue Oct 17, 2023

Implement stream ordering in HLS playlist based on input order.

2eff0a5

Fixed shaka-project#560 with suggested implementation: shaka-project#560 (comment)

wjywbs added a commit to wjywbs/shaka-packager that referenced this issue Oct 17, 2023

Implement stream ordering in HLS playlist based on input order.

4e520c5

Fixed shaka-project#560 with suggested implementation: shaka-project#560 (comment)

xavierlaffargue mentioned this issue Dec 3, 2023

feat: Preserve order of variants in HLS master playlist #1313

Closed

joeyparrish mentioned this issue Feb 8, 2024

feat: order streams in manifest based on command-line order #1329

Merged

joeyparrish closed this as completed in #1329 Feb 14, 2024

github-actions bot added the status: archived Archived and locked; will not be updated label Apr 14, 2024

github-actions bot locked as resolved and limited conversation to collaborators Apr 14, 2024

cosmin removed this from the Backlog milestone May 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Order of variants in HLS master playlist - can this be customized? #560

Order of variants in HLS master playlist - can this be customized? #560

fuzing commented Feb 23, 2019 •

edited

kqyang commented Feb 25, 2019 •

edited

fuzing commented Feb 25, 2019 via email

Nixon197 commented Jul 12, 2019 •

edited

kqyang commented Jul 12, 2019

fuzing commented Aug 6, 2019 •

edited

kqyang commented Aug 6, 2019

fuzing commented Aug 6, 2019

patrickkunka commented Aug 20, 2019

kickermeister commented Apr 8, 2020

kqyang commented Apr 9, 2020

kickermeister commented May 27, 2020

vasudevavarun commented Mar 17, 2021

kqyang commented Mar 17, 2021

xavierlaffargue commented Apr 27, 2022

xavierlaffargue commented Sep 4, 2023

bmurphy1976 commented Sep 27, 2023

joeyparrish commented Sep 27, 2023

bmurphy1976 commented Sep 28, 2023

bmurphy1976 commented Sep 28, 2023

bmurphy1976 commented Sep 28, 2023

cosmin commented Sep 28, 2023

bmurphy1976 commented Sep 28, 2023

vish91 commented Sep 28, 2023 •

edited

xavierlaffargue commented Sep 29, 2023

xavierlaffargue commented Sep 29, 2023

xavierlaffargue commented Sep 30, 2023

bmurphy1976 commented Oct 2, 2023

xavierlaffargue commented Oct 2, 2023

bmurphy1976 commented Oct 2, 2023

xavierlaffargue commented Oct 2, 2023

bmurphy1976 commented Oct 2, 2023

bmurphy1976 commented Oct 5, 2023

wjywbs commented Oct 17, 2023

Order of variants in HLS master playlist - can this be customized? #560

Order of variants in HLS master playlist - can this be customized? #560

Comments

fuzing commented Feb 23, 2019 • edited

kqyang commented Feb 25, 2019 • edited

fuzing commented Feb 25, 2019 via email

Nixon197 commented Jul 12, 2019 • edited

kqyang commented Jul 12, 2019

fuzing commented Aug 6, 2019 • edited

kqyang commented Aug 6, 2019

fuzing commented Aug 6, 2019

patrickkunka commented Aug 20, 2019

kickermeister commented Apr 8, 2020

kqyang commented Apr 9, 2020

kickermeister commented May 27, 2020

vasudevavarun commented Mar 17, 2021

kqyang commented Mar 17, 2021

xavierlaffargue commented Apr 27, 2022

xavierlaffargue commented Sep 4, 2023

bmurphy1976 commented Sep 27, 2023

joeyparrish commented Sep 27, 2023

bmurphy1976 commented Sep 28, 2023

bmurphy1976 commented Sep 28, 2023

bmurphy1976 commented Sep 28, 2023

cosmin commented Sep 28, 2023

bmurphy1976 commented Sep 28, 2023

vish91 commented Sep 28, 2023 • edited

xavierlaffargue commented Sep 29, 2023

xavierlaffargue commented Sep 29, 2023

xavierlaffargue commented Sep 30, 2023

bmurphy1976 commented Oct 2, 2023

xavierlaffargue commented Oct 2, 2023

bmurphy1976 commented Oct 2, 2023

xavierlaffargue commented Oct 2, 2023

bmurphy1976 commented Oct 2, 2023

bmurphy1976 commented Oct 5, 2023

wjywbs commented Oct 17, 2023

fuzing commented Feb 23, 2019 •

edited

kqyang commented Feb 25, 2019 •

edited

Nixon197 commented Jul 12, 2019 •

edited

fuzing commented Aug 6, 2019 •

edited

vish91 commented Sep 28, 2023 •

edited