Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MediaElementAudioSourceNode #321

Merged
merged 12 commits into from Nov 15, 2019
Merged

Conversation

@ferjm
Copy link
Member

ferjm commented Oct 31, 2019

No description provided.

@ferjm
Copy link
Member Author

ferjm commented Oct 31, 2019

I am not sure why this isn't currently working as expected. I am creating an appsink element and setting it as the playbin audio-sink property before playback starts, but the playbin is still creating an autoaudiosink (osxaudiosink in my case) element and ignoring the provided appsink.

0:00:00.026250000 98775 0x7fb0694f1a30 INFO                 playbin gstplaybin2.c:2331:void gst_play_bin_set_sink(GstPlayBin *, GstPlaySinkType, const gchar *, GstElement **, GstElement *):<playbin> Setting video sink to <appsink0>
0:00:00.026502000 98775 0x7fb0699162d0 DEBUG                playbin gstplaybin2.c:1704:void gst_play_bin_set_uri(GstPlayBin *, const gchar *): set new uri to servosrc://
0:00:00.026533000 98775 0x7fb0699162d0 DEBUG                playbin gstplaybin2.c:1721:void gst_play_bin_set_suburi(GstPlayBin *, const gchar *): setting new .sub uri to (null)
0:00:00.026785000 98775 0x7fb0699162d0 DEBUG                playbin gstplaybin2.c:5626:GstStateChangeReturn setup_next_source(GstPlayBin *, GstState):<playbin> setup sources
0:00:00.026797000 98775 0x7fb0699162d0 DEBUG                playbin gstplaybin2.c:5220:GstStateChangeReturn activate_group(GstPlayBin *, GstSourceGroup *, GstState):<playbin> activating group 0x7fb06a00a4c8
0:00:00.026994000 98775 0x7fb0699162d0 DEBUG                playbin gstplaybin2.c:5290:GstStateChangeReturn activate_group(GstPlayBin *, GstSourceGroup *, GstState):<playbin> making new uridecodebin
0:00:00.027152000 98775 0x7fb0699162d0 DEBUG                playbin gstplaybin2.c:2915:void gst_play_bin_handle_message(GstBin *, GstMessage *):<playbin> Ignoring async state change of uridecodebin: uridecodebin0
0:00:00.029948000 98775 0x7fb0699162d0 DEBUG                playbin gstplaybin2.c:5195:gboolean group_set_locked_state_unlocked(GstPlayBin *, GstSourceGroup *, gboolean):<playbin> locked_state 0 on group 0x7fb06a00a4c8
0:00:00.033459000 98775 0x7fb069604440 INFO                 playbin gstplaybin2.c:2331:void gst_play_bin_set_sink(GstPlayBin *, GstPlaySinkType, const gchar *, GstElement **, GstElement *):<playbin> Setting audio sink to <appsink1>
0:00:00.033766000 98775 0x7fb0699162d0 DEBUG                playbin gstplaybin2.c:2782:void gst_play_bin_update_cached_duration_from_query(GstPlayBin *, gboolean, GstQuery *):<playbin> Updating cached duration from query
Finished pushing data
0:00:00.036938000 98775 0x7fb06b0150f0 DEBUG                playbin gstplaybin2.c:2915:void gst_play_bin_handle_message(GstBin *, GstMessage *):<playbin> Ignoring async state change of uridecodebin: uridecodebin0
0:00:00.038599000 98775 0x7fb06c817b20 DEBUG                playbin gstplaybin2.c:4553:gboolean autoplug_continue_cb(GstElement *, GstPad *, GstCaps *, GstSourceGroup *):<playbin> continue autoplugging group 0x7fb06a00a4c8 for '':decodepad0, audio/ogg: 1
0:00:00.038621000 98775 0x7fb06c817b20 DEBUG                playbin gstplaybin2.c:4181:GValueArray *autoplug_factories_cb(GstElement *, GstPad *, GstCaps *, GstSourceGroup *):<playbin> factories group 0x7fb06a00a4c8 for '':decodepad0, audio/ogg
0:00:00.047422000 98775 0x7fb06c817b20 DEBUG                playbin gstplaybin2.c:4191:GValueArray *autoplug_factories_cb(GstElement *, GstPad *, GstCaps *, GstSourceGroup *):<playbin> found factories 0x7fb06b80a360
0:00:00.047512000 98775 0x7fb06c817b20 DEBUG                playbin gstplaybin2.c:4600:GstAutoplugSelectResult autoplug_select_cb(GstElement *, GstPad *, GstCaps *, GstElementFactory *, GstSourceGroup *):<playbin> select group 0x7fb06a00a4c8 for '':decodepad0, audio/ogg
0:00:00.047520000 98775 0x7fb06c817b20 DEBUG                playbin gstplaybin2.c:4602:GstAutoplugSelectResult autoplug_select_cb(GstElement *, GstPad *, GstCaps *, GstElementFactory *, GstSourceGroup *):<playbin> checking factory oggdemux
0:00:00.050325000 98775 0x7fb069995de0 DEBUG                playbin gstplaybin2.c:4553:gboolean autoplug_continue_cb(GstElement *, GstPad *, GstCaps *, GstSourceGroup *):<playbin> continue autoplugging group 0x7fb06a00a4c8 for '':decodepad1, audio/x-vorbis, rate=(int)44100, channels=(int)2, streamheader=(buffer)< 01766f72626973000000000244ac00000000000080b5010000000000b801, 03766f726269730d0000004c61766635372e38332e3130300800000017000000414c42554d3d4f66204669726520416e6420537465656c120000005449544c453d486f726e65642056697065721100000050415254204f462041205345543d312f3110000000545241434b4e554d4245523d362f3130180000004152544953543d436f6e7175657374204f6620537465656c09000000444154453d323031330900000047454e52453d506f7020000000656e636f6465723d4c61766335372e3130372e313030206c6962766f7262697301, 05766f726269732542435601004000002473182a46a5731684101a425019e31c42ce6bec19424c11821c324c5bcb25739021a4a042885b2881d09055000040000087417814848a41082184253d589283273d082184883978148469410821841042082184104208218445396892832741081d84e330380c83e538f81c8445395810832741e820840f42b89a83ac3908218424354850830639e81c84c22c288a82c430b816840435288c82e430c8d4830b42889a834935f81a846741781684694108218424414890830641c8188446415892830639b81484cb41a81a842a39081f842034641500900000a0a2288aa2280a101ab20a00c8000010405114c7711cc9911cc9b11c0b080d590500000100080000a0488aa4488ee44892245992255992255992e689aa2ccbb22ccbb22ccb32101ab20a0048000050510c45711407080d59050064000008a0388aa5588aa5688ae7888e088486ac0200800000040000103443533c479444cf5455d7b66ddbb66ddbb66ddbb66ddbb66d5b966519080d59050040000010d26966a9068830031906424356010008000080118a30c480d09055000040000080184a0ea209ad39df9ce3a0590e9a4ab1391d9c48b579929b8ab939e79c73cec9e69c31ce39e79ca29c590c9a09ad39e79cc4a0590a9a09ad39e79c27b179d09a2aad39e79c71cee9609c11c639e79c26ad79909a8db539e79c05ad698e9a4bb139e79c48b979529b4bb539e79c73ce39e79c73ce39e79ceac5e91c9c13ce39e79ca8bdb9969bd0c539e79c4fc6e9de9c10ce39e79c73ce39e79c73ce39e79c20346415000004004010868d61dc2908d2e768204611621a32e941f7e830091a839c42ead1e868a4943a08259571524a27080d590500000200400821851452482185145248218514628821861872ca29a7a0824a2aa9a8a28c32cb2cb3cc32cb2cb3cc3aecacb30e3b0c31c410432badc452536d35d6586bee39e79a83b4565a6badb5524a29a594520a42435601002000000442061964905148218514628829a79c720a2aa880d090550000200080000000004ff21cd1111dd1111dd1111dd1111dd1f11ccf112551122551122dd33235d35345557565d7967559b77d5bd8855df77dddf77dddf8756158966559966559966559966559966559962034641500000200002084104248218514524829c61873cc39e8249410080d59050000020008000000701447711cc9911c49b2244bd224cdd22c4ff3344f133d511445d33455d1155d51376d513665d3355d53365d55566d57966d5bb675db9765dbf77ddff77ddff77ddff77ddff77d5d0742435601001200003a92232992222992e3388e24494068c82a004006004000008ae2288ee33892244992256992677996a8999ae9999e2aaa4068c82a00001000400000000000008aa6788aa9788aa8788ee88892689996a8a99a2bcaa6ecbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaae0b8486ac02002400007424477224475224455224477280d0905500800c008000001cc3312445722ccbd2344ff3344f133dd1133dd35345577481d0905500002000800000000000000cc9b014cbd11c4d1225d5522d55532dd55245d5535555555555555555555555555555555555555555555555555555555555555555554dd3344d13080d5909009001009010532d2dc69a098b2462d26aaba0630c52eca5b1482a67b5b7ca318518b55e1a879451107ba924638a41cc2da4d02926add654428514a498632a15520e522034648500109a01e0701c40b22c40b22c0000000000000090340dd03c0fb0340f00000000000000244d032c4f0334cf030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000040d23440f33c40f33c00000000000000d03c0ff03c11f04411000000000000002ccf0334d1033c51040000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000040d23440f33c40f33c00000000000000b03c0ff04411d03c11000000000000002ccf033c51043cd103000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000010e00000106021141ab2220088130070481224099204cd034896054d83a6c13401926541d3a069304d00000000000000000000244d83a641d3208a0049d3a069d03488220000000000000000000092a641d3a069104580a469d034681a441100000000000000000000cf34218a10459826c0334d88224411a60900000000000000000000000000000000000000000000080000187000000830a10c141ab2220088130070388a65010080e338960500008ee3581600005896258a000060599a280200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000080000187000000830a10c141ab2120088020070288a6501c7b12ce038960524c9b2009605d03c80a6014411000800002870000008b04153627180424356020051000006c5b12c4d134592a4699a278a2449d33c4f14699ae7799e69c2f33ccf34218aa2689a104551344d98a669aa2a304d5515000050e00000106083a6c4e2008586ac04004202001c8a62599ae6799e278aa6a99a2449d33c4f1445d1344d53554992a6799e288aa2699aa6aab22c4df33c511445d354555585a6799e288aa269aaaaeac2f33c4f1445d13455d575e1799e278aa2689aaaeaba104551344dd3544d55755d208aa6699aaaaaaaae0b444f144d53555dd77581e789a269aaaaabba2e104dd3545555755d590698a669aaaaebca32405555d5755d579601aaaaaaaeebbab20c5055d7755d59966500aeebbab22ccb0200000e1c0000028ca0938c2a8bb0d1840b0f40a1212b0280280000c018a61453ca302621a4101ac6248414422625a5d252aa20a45252291584544a2a25a394526a2955105229a9940a422a2595520000d8810300d881855068c84a00200f0080304629c618734e22a41463ce392711528a31e79c934a31e69c73ce49291973cc39e7a494ce39e79c73524ae69c73ce3929a573ce39e79c94524ae79c734e4a292584ce4127a594d239e79c13000054e000001060a3c8e60423418586ac04005201000c8e63599ae679a2689a9624699ae7799e289aa626499ae6799e278aaac9f33c4f1445d1345595e7799e288aa269aa2ad71545d3344d55555db22c8aa6699aaaeaba304dd35455d7755d98a669aaaaebba2e6c5b5555d5756519b6adaaaaeabab20c5cd77565d996812cbbaeecdab20000f0040700a0021b564738291a0b2c34642500900100401883904208216510420a2184945208090000187000000830a10c141ab21200480500008cb1d65a6badb5d64067adb5d65a6bad80cc5a6badb5d65a6badb5d65a6badb5d6526badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6b2da594524a29a594524a29a594524a29a594524a0500fa553800f83fd8b03ac249d15860a1212b0180700000c018a518730c4229a5540831e69c74545a8bb1428831e724a4d45a6cc573ce41282195d6622c9e730e422929c5566351298452524a2db6588b4aa1a392524aadd5588c31a9a4d65a8badc6628c4929b4d45a8b3116236c4da9b5d86aabb118636b2a2db418638cc5085f646c2da6da6a0dc608235b2c2dd55a6b30c618dd5b8ba5b69a8b313ef8da522c31d65c00007783030044828d33ac249d158e06171ab212000809002010528a31c61873ce39e7a4528c39e69c730e4208a1548a31c69c730e420821948c31e69c731042082184524ac69c7310420821849052ea9c731042082184104a299d730e42082184104229a5831042082184104a28a5a41442082184104208a9a494420821845242282195945208218410422925a494520a2184524208a18494524a2985104208a59492524a29a5124a09258412522929a5144a082194524a4a29a5544a09a184124a2925a594524a2184104a290500001c3800000418412719551661a309171e80424356020064000090a29452292d458222a518a4184b461573505a8aa8720c52cda952ce20e6249688318494935432e614420c42ea1c754c29062d951842c618a4d8724ba1730e000000410080809000000304053300c0e000e17310740204471b00802044668844c3427078500910115301406282422e0054585ca45d5c4097012ee8e2ae0321042108412c0ea080041c9c70c3136f78c20d4ed0292a752000000000000d00f00000905c001111d1cc6164686c7074787c8084888c9008000000000019007c00002425404444347318191a1b1c1d1e1f20212223240100800002000000002080000404040000000000020000000404 >: 1
0:00:00.051687000 98775 0x7fb069995de0 DEBUG                playbin gstplaybin2.c:4181:GValueArray *autoplug_factories_cb(GstElement *, GstPad *, GstCaps *, GstSourceGroup *):<playbin> factories group 0x7fb06a00a4c8 for '':decodepad1, audio/x-vorbis, rate=(int)44100, channels=(int)2, streamheader=(buffer)< 01766f72626973000000000244ac00000000000080b5010000000000b801, 03766f726269730d0000004c61766635372e38332e3130300800000017000000414c42554d3d4f66204669726520416e6420537465656c120000005449544c453d486f726e65642056697065721100000050415254204f462041205345543d312f3110000000545241434b4e554d4245523d362f3130180000004152544953543d436f6e7175657374204f6620537465656c09000000444154453d323031330900000047454e52453d506f7020000000656e636f6465723d4c61766335372e3130372e313030206c6962766f7262697301, 05766f726269732542435601004000002473182a46a5731684101a425019e31c42ce6bec19424c11821c324c5bcb25739021a4a042885b2881d09055000040000087417814848a41082184253d589283273d082184883978148469410821841042082184104208218445396892832741081d84e330380c83e538f81c8445395810832741e820840f42b89a83ac3908218424354850830639e81c84c22c288a82c430b816840435288c82e430c8d4830b42889a834935f81a846741781684694108218424414890830641c8188446415892830639b81484cb41a81a842a39081f842034641500900000a0a2288aa2280a101ab20a00c8000010405114c7711cc9911cc9b11c0b080d590500000100080000a0488aa4488ee44892245992255992255992e689aa2ccbb22ccbb22ccb32101ab20a0048000050510c45711407080d59050064000008a0388aa5588aa5688ae7888e088486ac0200800000040000103443533c479444cf5455d7b66ddbb66ddbb66ddbb66ddbb66d5b966519080d59050040000010d26966a9068830031906424356010008000080118a30c480d09055000040000080184a0ea209ad39df9ce3a0590e9a4ab1391d9c48b579929b8ab939e79c73cec9e69c31ce39e79ca29c590c9a09ad39e79cc4a0590a9a09ad39e79c27b179d09a2aad39e79c71cee9609c11c639e79c26ad79909a8db539e79c05ad698e9a4bb139e79c48b979529b4bb539e79c73ce39e79c73ce39e79ceac5e91c9c13ce39e79ca8bdb9969bd0c539e79c4fc6e9de9c10ce39e79c73ce39e79c73ce39e79c20346415000004004010868d61dc2908d2e768204611621a32e941f7e830091a839c42ead1e868a4943a08259571524a27080d590500000200400821851452482185145248218514628821861872ca29a7a0824a2aa9a8a28c32cb2cb3cc32cb2cb3cc3aecacb30e3b0c31c410432badc452536d35d6586bee39e79a83b4565a6badb5524a29a594520a42435601002000000442061964905148218514628829a79c720a2aa880d090550000200080000000004ff21cd1111dd1111dd1111dd1111dd1f11ccf112551122551122dd33235d35345557565d7967559b77d5bd8855df77dddf77dddf8756158966559966559966559966559966559962034641500000200002084104248218514524829c61873cc39e8249410080d59050000020008000000701447711cc9911c49b2244bd224cdd22c4ff3344f133d511445d33455d1155d51376d513665d3355d53365d55566d57966d5bb675db9765dbf77ddff77ddff77ddff77ddff77d5d0742435601001200003a92232992222992e3388e24494068c82a004006004000008ae2288ee33892244992256992677996a8999ae9999e2aaa4068c82a00001000400000000000008aa6788aa9788aa8788ee88892689996a8a99a2bcaa6ecbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaae0b8486ac02002400007424477224475224455224477280d0905500800c008000001cc3312445722ccbd2344ff3344f133dd1133dd35345577481d0905500002000800000000000000cc9b014cbd11c4d1225d5522d55532dd55245d5535555555555555555555555555555555555555555555555555555555555555555554dd3344d13080d5909009001009010532d2dc69a098b2462d26aaba0630c52eca5b1482a67b5b7ca318518b55e1a879451107ba924638a41cc2da4d02926add654428514a498632a15520e522034648500109a01e0701c40b22c40b22c0000000000000090340dd03c0fb0340f00000000000000244d032c4f0334cf030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000040d23440f33c40f33c00000000000000d03c0ff03c11f04411000000000000002ccf0334d1033c51040000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000040d23440f33c40f33c00000000000000b03c0ff04411d03c11000000000000002ccf033c51043cd103000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000010e00000106021141ab2220088130070481224099204cd034896054d83a6c13401926541d3a069304d00000000000000000000244d83a641d3208a0049d3a069d03488220000000000000000000092a641d3a069104580a469d034681a441100000000000000000000cf34218a10459826c0334d88224411a60900000000000000000000000000000000000000000000080000187000000830a10c141ab2220088130070388a65010080e338960500008ee3581600005896258a000060599a280200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000080000187000000830a10c141ab2120088020070288a6501c7b12ce038960524c9b2009605d03c80a6014411000800002870000008b04153627180424356020051000006c5b12c4d134592a4699a278a2449d33c4f14699ae7799e69c2f33ccf34218aa2689a104551344d98a669aa2a304d5515000050e00000106083a6c4e2008586ac04004202001c8a62599ae6799e278aa6a99a2449d33c4f1445d1344d53554992a6799e288aa2699aa6aab22c4df33c511445d354555585a6799e288aa269aaaaeac2f33c4f1445d13455d575e1799e278aa2689aaaeaba104551344dd3544d55755d208aa6699aaaaaaaae0b444f144d53555dd77581e789a269aaaaabba2e104dd3545555755d590698a669aaaaebca32405555d5755d579601aaaaaaaeebbab20c5055d7755d59966500aeebbab22ccb0200000e1c0000028ca0938c2a8bb0d1840b0f40a1212b0280280000c018a61453ca302621a4101ac6248414422625a5d252aa20a45252291584544a2a25a394526a2955105229a9940a422a2595520000d8810300d881855068c84a00200f0080304629c618734e22a41463ce392711528a31e79c934a31e69c73ce49291973cc39e7a494ce39e79c73524ae69c73ce3929a573ce39e79c94524ae79c734e4a292584ce4127a594d239e79c13000054e000001060a3c8e60423418586ac04005201000c8e63599ae679a2689a9624699ae7799e289aa626499ae6799e278aaac9f33c4f1445d1345595e7799e288aa269aa2ad71545d3344d55555db22c8aa6699aaaeaba304dd35455d7755d98a669aaaaebba2e6c5b5555d5756519b6adaaaaeabab20c5cd77565d996812cbbaeecdab20000f0040700a0021b564738291a0b2c34642500900100401883904208216510420a2184945208090000187000000830a10c141ab21200480500008cb1d65a6badb5d64067adb5d65a6bad80cc5a6badb5d65a6badb5d65a6badb5d6526badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6b2da594524a29a594524a29a594524a29a594524a0500fa553800f83fd8b03ac249d15860a1212b0180700000c018a518730c4229a5540831e69c74545a8bb1428831e724a4d45a6cc573ce41282195d6622c9e730e422929c5566351298452524a2db6588b4aa1a392524aadd5588c31a9a4d65a8badc6628c4929b4d45a8b3116236c4da9b5d86aabb118636b2a2db418638cc5085f646c2da6da6a0dc608235b2c2dd55a6b30c618dd5b8ba5b69a8b313ef8da522c31d65c00007783030044828d33ac249d158e06171ab212000809002010528a31c61873ce39e7a4528c39e69c730e4208a1548a31c69c730e420821948c31e69c731042082184524ac69c7310420821849052ea9c731042082184104a299d730e42082184104229a5831042082184104a28a5a41442082184104208a9a494420821845242282195945208218410422925a494520a2184524208a18494524a2985104208a59492524a29a5124a09258412522929a5144a082194524a4a29a5544a09a184124a2925a594524a2184104a290500001c3800000418412719551661a309171e80424356020064000090a29452292d458222a518a4184b461573505a8aa8720c52cda952ce20e6249688318494935432e614420c42ea1c754c29062d951842c618a4d8724ba1730e000000410080809000000304053300c0e000e17310740204471b00802044668844c3427078500910115301406282422e0054585ca45d5c4097012ee8e2ae0321042108412c0ea080041c9c70c3136f78c20d4ed0292a752000000000000d00f00000905c001111d1cc6164686c7074787c8084888c9008000000000019007c00002425404444347318191a1b1c1d1e1f20212223240100800002000000002080000404040000000000020000000404 >
0:00:00.052674000 98775 0x7fb069995de0 DEBUG                playbin gstplaybin2.c:4191:GValueArray *autoplug_factories_cb(GstElement *, GstPad *, GstCaps *, GstSourceGroup *):<playbin> found factories 0x7fb06b83f880
0:00:00.053269000 98775 0x7fb069995de0 DEBUG                playbin gstplaybin2.c:4600:GstAutoplugSelectResult autoplug_select_cb(GstElement *, GstPad *, GstCaps *, GstElementFactory *, GstSourceGroup *):<playbin> select group 0x7fb06a00a4c8 for '':decodepad1, audio/x-vorbis, rate=(int)44100, channels=(int)2, streamheader=(buffer)< 01766f72626973000000000244ac00000000000080b5010000000000b801, 03766f726269730d0000004c61766635372e38332e3130300800000017000000414c42554d3d4f66204669726520416e6420537465656c120000005449544c453d486f726e65642056697065721100000050415254204f462041205345543d312f3110000000545241434b4e554d4245523d362f3130180000004152544953543d436f6e7175657374204f6620537465656c09000000444154453d323031330900000047454e52453d506f7020000000656e636f6465723d4c61766335372e3130372e313030206c6962766f7262697301, 05766f726269732542435601004000002473182a46a5731684101a425019e31c42ce6bec19424c11821c324c5bcb25739021a4a042885b2881d09055000040000087417814848a41082184253d589283273d082184883978148469410821841042082184104208218445396892832741081d84e330380c83e538f81c8445395810832741e820840f42b89a83ac3908218424354850830639e81c84c22c288a82c430b816840435288c82e430c8d4830b42889a834935f81a846741781684694108218424414890830641c8188446415892830639b81484cb41a81a842a39081f842034641500900000a0a2288aa2280a101ab20a00c8000010405114c7711cc9911cc9b11c0b080d590500000100080000a0488aa4488ee44892245992255992255992e689aa2ccbb22ccbb22ccb32101ab20a0048000050510c45711407080d59050064000008a0388aa5588aa5688ae7888e088486ac0200800000040000103443533c479444cf5455d7b66ddbb66ddbb66ddbb66ddbb66d5b966519080d59050040000010d26966a9068830031906424356010008000080118a30c480d09055000040000080184a0ea209ad39df9ce3a0590e9a4ab1391d9c48b579929b8ab939e79c73cec9e69c31ce39e79ca29c590c9a09ad39e79cc4a0590a9a09ad39e79c27b179d09a2aad39e79c71cee9609c11c639e79c26ad79909a8db539e79c05ad698e9a4bb139e79c48b979529b4bb539e79c73ce39e79c73ce39e79ceac5e91c9c13ce39e79ca8bdb9969bd0c539e79c4fc6e9de9c10ce39e79c73ce39e79c73ce39e79c20346415000004004010868d61dc2908d2e768204611621a32e941f7e830091a839c42ead1e868a4943a08259571524a27080d590500000200400821851452482185145248218514628821861872ca29a7a0824a2aa9a8a28c32cb2cb3cc32cb2cb3cc3aecacb30e3b0c31c410432badc452536d35d6586bee39e79a83b4565a6badb5524a29a594520a42435601002000000442061964905148218514628829a79c720a2aa880d090550000200080000000004ff21cd1111dd1111dd1111dd1111dd1f11ccf112551122551122dd33235d35345557565d7967559b77d5bd8855df77dddf77dddf8756158966559966559966559966559966559962034641500000200002084104248218514524829c61873cc39e8249410080d59050000020008000000701447711cc9911c49b2244bd224cdd22c4ff3344f133d511445d33455d1155d51376d513665d3355d53365d55566d57966d5bb675db9765dbf77ddff77ddff77ddff77ddff77d5d0742435601001200003a92232992222992e3388e24494068c82a004006004000008ae2288ee33892244992256992677996a8999ae9999e2aaa4068c82a00001000400000000000008aa6788aa9788aa8788ee88892689996a8a99a2bcaa6ecbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaaeebbaae0b8486ac02002400007424477224475224455224477280d0905500800c008000001cc3312445722ccbd2344ff3344f133dd1133dd35345577481d0905500002000800000000000000cc9b014cbd11c4d1225d5522d55532dd55245d5535555555555555555555555555555555555555555555555555555555555555555554dd3344d13080d5909009001009010532d2dc69a098b2462d26aaba0630c52eca5b1482a67b5b7ca318518b55e1a879451107ba924638a41cc2da4d02926add654428514a498632a15520e522034648500109a01e0701c40b22c40b22c0000000000000090340dd03c0fb0340f00000000000000244d032c4f0334cf030000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000040d23440f33c40f33c00000000000000d03c0ff03c11f04411000000000000002ccf0334d1033c51040000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000040d23440f33c40f33c00000000000000b03c0ff04411d03c11000000000000002ccf033c51043cd103000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000010e00000106021141ab2220088130070481224099204cd034896054d83a6c13401926541d3a069304d00000000000000000000244d83a641d3208a0049d3a069d03488220000000000000000000092a641d3a069104580a469d034681a441100000000000000000000cf34218a10459826c0334d88224411a60900000000000000000000000000000000000000000000080000187000000830a10c141ab2220088130070388a65010080e338960500008ee3581600005896258a000060599a280200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000080000187000000830a10c141ab2120088020070288a6501c7b12ce038960524c9b2009605d03c80a6014411000800002870000008b04153627180424356020051000006c5b12c4d134592a4699a278a2449d33c4f14699ae7799e69c2f33ccf34218aa2689a104551344d98a669aa2a304d5515000050e00000106083a6c4e2008586ac04004202001c8a62599ae6799e278aa6a99a2449d33c4f1445d1344d53554992a6799e288aa2699aa6aab22c4df33c511445d354555585a6799e288aa269aaaaeac2f33c4f1445d13455d575e1799e278aa2689aaaeaba104551344dd3544d55755d208aa6699aaaaaaaae0b444f144d53555dd77581e789a269aaaaabba2e104dd3545555755d590698a669aaaaebca32405555d5755d579601aaaaaaaeebbab20c5055d7755d59966500aeebbab22ccb0200000e1c0000028ca0938c2a8bb0d1840b0f40a1212b0280280000c018a61453ca302621a4101ac6248414422625a5d252aa20a45252291584544a2a25a394526a2955105229a9940a422a2595520000d8810300d881855068c84a00200f0080304629c618734e22a41463ce392711528a31e79c934a31e69c73ce49291973cc39e7a494ce39e79c73524ae69c73ce3929a573ce39e79c94524ae79c734e4a292584ce4127a594d239e79c13000054e000001060a3c8e60423418586ac04005201000c8e63599ae679a2689a9624699ae7799e289aa626499ae6799e278aaac9f33c4f1445d1345595e7799e288aa269aa2ad71545d3344d55555db22c8aa6699aaaeaba304dd35455d7755d98a669aaaaebba2e6c5b5555d5756519b6adaaaaeabab20c5cd77565d996812cbbaeecdab20000f0040700a0021b564738291a0b2c34642500900100401883904208216510420a2184945208090000187000000830a10c141ab21200480500008cb1d65a6badb5d64067adb5d65a6bad80cc5a6badb5d65a6badb5d65a6badb5d6526badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6badb5d65a6b2da594524a29a594524a29a594524a29a594524a0500fa553800f83fd8b03ac249d15860a1212b0180700000c018a518730c4229a5540831e69c74545a8bb1428831e724a4d45a6cc573ce41282195d6622c9e730e422929c5566351298452524a2db6588b4aa1a392524aadd5588c31a9a4d65a8badc6628c4929b4d45a8b3116236c4da9b5d86aabb118636b2a2db418638cc5085f646c2da6da6a0dc608235b2c2dd55a6b30c618dd5b8ba5b69a8b313ef8da522c31d65c00007783030044828d33ac249d158e06171ab212000809002010528a31c61873ce39e7a4528c39e69c730e4208a1548a31c69c730e420821948c31e69c731042082184524ac69c7310420821849052ea9c731042082184104a299d730e42082184104229a5831042082184104a28a5a41442082184104208a9a494420821845242282195945208218410422925a494520a2184524208a18494524a2985104208a59492524a29a5124a09258412522929a5144a082194524a4a29a5544a09a184124a2925a594524a2184104a290500001c3800000418412719551661a309171e80424356020064000090a29452292d458222a518a4184b461573505a8aa8720c52cda952ce20e6249688318494935432e614420c42ea1c754c29062d951842c618a4d8724ba1730e000000410080809000000304053300c0e000e17310740204471b00802044668844c3427078500910115301406282422e0054585ca45d5c4097012ee8e2ae0321042108412c0ea080041c9c70c3136f78c20d4ed0292a752000000000000d00f00000905c001111d1cc6164686c7074787c8084888c9008000000000019007c00002425404444347318191a1b1c1d1e1f20212223240100800002000000002080000404040000000000020000000404 >
0:00:00.054006000 98775 0x7fb069995de0 DEBUG                playbin gstplaybin2.c:4602:GstAutoplugSelectResult autoplug_select_cb(GstElement *, GstPad *, GstCaps *, GstElementFactory *, GstSourceGroup *):<playbin> checking factory vorbisdec
0:00:00.054022000 98775 0x7fb069995de0 DEBUG                playbin gstplaybin2.c:4669:GstAutoplugSelectResult autoplug_select_cb(GstElement *, GstPad *, GstCaps *, GstElementFactory *, GstSourceGroup *):<playbin> Trying to create sink 'osxaudiosink' for decoder 'vorbisdec'

Player state changed to Buffering

NeedData

Player state changed to Paused
0:00:00.085070000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3345:void pad_added_cb(GstElement *, GstPad *, GstSourceGroup *):<playbin> pad uridecodebin0:src_0 with caps audio/x-raw, format=(string)F32LE, layout=(string)interleaved, rate=(int)44100, channels=(int)2, channel-mask=(bitmask)0x0000000000000003 added in group 0x7fb06a00a4c8
0:00:00.085089000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3386:void pad_added_cb(GstElement *, GstPad *, GstSourceGroup *):<playbin> creating new input selector
0:00:00.085164000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3421:void pad_added_cb(GstElement *, GstPad *, GstSourceGroup *):<playbin> adding new stream combiner 0x7fb06c0550a0
0:00:00.085187000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3442:void pad_added_cb(GstElement *, GstPad *, GstSourceGroup *):<playbin> blocking <inputselector0:src>
0:00:00.085216000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3453:void pad_added_cb(GstElement *, GstPad *, GstSourceGroup *):<playbin> got pad inputselector0:sink_0 from combiner
0:00:00.085225000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3487:void pad_added_cb(GstElement *, GstPad *, GstSourceGroup *):<playbin> pad 0x7fb06c05c200 added to array
0:00:00.085271000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3499:void pad_added_cb(GstElement *, GstPad *, GstSourceGroup *):<playbin> linked pad uridecodebin0:src_0 to combiner 0x7fb06c0550a0
0:00:00.085293000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3714:void no_more_pads_cb(GstElement *, GstSourceGroup *):<playbin> no more pads in group 0x7fb06a00a4c8
0:00:00.085305000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3727:void no_more_pads_cb(GstElement *, GstSourceGroup *):<playbin> requesting new sink pad 0
0:00:00.085471000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3745:void no_more_pads_cb(GstElement *, GstSourceGroup *):<playbin> linked type audio/, result: 0
0:00:00.085480000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3754:void no_more_pads_cb(GstElement *, GstSourceGroup *):<playbin> pending 1 > 0
0:00:00.085485000 98775 0x7fb06c025050 INFO                 playbin gstplaybin2.c:3779:void no_more_pads_cb(GstElement *, GstSourceGroup *):<playbin> setting custom audio sink <osxaudiosink0>
0:00:00.085492000 98775 0x7fb06c025050 INFO                 playbin gstplaybin2.c:3786:void no_more_pads_cb(GstElement *, GstSourceGroup *):<playbin> setting custom video sink <appsink0>
0:00:00.085498000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3808:void no_more_pads_cb(GstElement *, GstSourceGroup *):<playbin> unblocking <inputselector0:src>
0:00:00.085607000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:1360:void do_async_done(GstPlayBin *):<playbin> posting ASYNC_DONE
0:00:00.085637000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:2915:void gst_play_bin_handle_message(GstBin *, GstMessage *):<playbin> Ignoring async state change of uridecodebin: uridecodebin0
0:00:00.088798000 98775 0x7fb06a8295e0 DEBUG                playbin gstplaybin2.c:2931:void gst_play_bin_handle_message(GstBin *, GstMessage *):<playbin> Stream start from new group 0x7fb06a00a4c8
0:00:00.126103000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3291:void notify_tags_cb(GObject *, GParamSpec *, gpointer):<playbin> Tags on pad <inputselector0:sink_0> with stream id 0 and type 0 have changed
0:00:00.126154000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3291:void notify_tags_cb(GObject *, GParamSpec *, gpointer):<playbin> Tags on pad <inputselector0:sink_0> with stream id 0 and type 0 have changed
0:00:00.126205000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3291:void notify_tags_cb(GObject *, GParamSpec *, gpointer):<playbin> Tags on pad <inputselector0:sink_0> with stream id 0 and type 0 have changed

Metadata updated! Metadata { duration: Some(0ns), width: 0, height: 0, format: "Ogg", is_seekable: true, video_tracks: [], audio_tracks: ["Vorbis"], is_live: false, title: Some("Horned Viper") }
0:00:00.126624000 98775 0x7fb0699162d0 DEBUG                playbin gstplaybin2.c:2782:void gst_play_bin_update_cached_duration_from_query(GstPlayBin *, gboolean, GstQuery *):<playbin> Updating cached duration from query

Metadata updated! Metadata { duration: Some(5s), width: 0, height: 0, format: "Ogg", is_seekable: true, video_tracks: [], audio_tracks: ["Vorbis"], is_live: false, title: Some("Horned Viper") }

Metadata updated! Metadata { duration: Some(5s), width: 0, height: 0, format: "Ogg", is_seekable: true, video_tracks: [], audio_tracks: ["Vorbis"], is_live: false, title: Some("Horned Viper") }

Player state changed to Buffering

Seeked to 0

Player state changed to Playing
0:00:03.908017000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3868:void drained_cb(GstElement *, GstSourceGroup *):<playbin> about to finish in group 0x7fb06a00a4c8
0:00:03.908036000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:5626:GstStateChangeReturn setup_next_source(GstPlayBin *, GstState):<playbin> setup sources
0:00:03.908041000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:5662:GstStateChangeReturn setup_next_source(GstPlayBin *, GstState):<playbin> no next group
0:00:03.908048000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:3868:void drained_cb(GstElement *, GstSourceGroup *):<playbin> about to finish in group 0x7fb06a00a4c8
0:00:03.908052000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:5626:GstStateChangeReturn setup_next_source(GstPlayBin *, GstState):<playbin> setup sources
0:00:03.908186000 98775 0x7fb06c025050 DEBUG                playbin gstplaybin2.c:5662:GstStateChangeReturn setup_next_source(GstPlayBin *, GstState):<playbin> no next group

EOF

@sdroege do you know what's wrong here? Thanks

@ceyusa
Copy link
Contributor

ceyusa commented Oct 31, 2019

It looks like you're setting the audio sink when the player is already in ready or playing state

Copy link
Member

Manishearth left a comment

looking good so far

Ideally should use the MediaStream stuff for the audio sink. It internally uses a proxy sink which should clean up most of this code.

One thing missing from the mediastream code is the ability to use a AudioChunk based interface when the proxy sink isn't available (the stream comes from a non-gstreamer source).

But the original design of mediastream was so that it could be used for these things.

@ferjm
Copy link
Member Author

ferjm commented Oct 31, 2019

Ideally should use the MediaStream stuff for the audio sink. It internally uses a proxy sink which should clean up most of this code.

I thought about it. But in this case we don't really need to connect two gstreamer pipelines (well, not directly). What we need is to extract the data from the player pipeline and take it into the webaudio render loop (which will end up in the webaudio gstreamer pipeline). And for that, appsink seemed to be the most appropriate element.

@ferjm
Copy link
Member Author

ferjm commented Oct 31, 2019

It looks like you're setting the audio sink when the player is already in ready or playing state

Yes. That was the case. Thanks.

I fixed it by reseting and recreating the player.

@Manishearth
Copy link
Member

Manishearth commented Oct 31, 2019

What we need is to extract the data from the player pipeline and take it into the webaudio render loop (which will end up in the webaudio gstreamer pipeline). And for that, appsink seemed to be the most appropriate element

Right, but eventually we'd also want to be able to hook up webrtc to webaudio.

You'd still need an appsink, but you instead make it possible to attach an appsink to a mediastream, so that webaudio can connect to any mediastream and attach an appsink on its proxysrc

ferjm added 4 commits Nov 4, 2019
@ferjm ferjm changed the title [WIP] MediaElementAudioSourceNode MediaElementAudioSourceNode Nov 5, 2019
@ferjm
Copy link
Member Author

ferjm commented Nov 5, 2019

This is currently playing sound coming from the player and through the webaudio pipeline, but the playback rate seems to be wrong, cause the audio is reproduced slower than it should.

@ferjm
Copy link
Member Author

ferjm commented Nov 15, 2019

I am unable to figure out why the playback rate is slower than it should. Let's merge this as is for now. I'll file a follow-up to investigate the issue further.

r? @Manishearth

@Manishearth
Copy link
Member

Manishearth commented Nov 15, 2019

r=me then. I think we should still try and use MediaStream here so that we have a standard way of plugging mediastreams into webaudio. That can be a followup.

@bors-servo r+

@bors-servo
Copy link
Contributor

bors-servo commented Nov 15, 2019

📌 Commit e70664c has been approved by Manishearth

@bors-servo
Copy link
Contributor

bors-servo commented Nov 15, 2019

Testing commit e70664c with merge 220ed13...

bors-servo added a commit that referenced this pull request Nov 15, 2019
MediaElementAudioSourceNode
@bors-servo
Copy link
Contributor

bors-servo commented Nov 15, 2019

☀️ Test successful - checks-travis
Approved by: Manishearth
Pushing 220ed13 to master...

@bors-servo bors-servo merged commit e70664c into servo:master Nov 15, 2019
3 checks passed
3 checks passed
Travis CI - Pull Request Build Passed
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
homu Test successful
Details
bors-servo added a commit to servo/servo that referenced this pull request Nov 18, 2019
WebAudio MediaElementAudioSourceNode implementation

- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix #22370
- [X] There are tests for these changes

Depends on servo/media#321

Tested with https://ferjm.github.io/webaudio-examples/media-source-buffer/

I am still investigating why it is playing back with a low rate.
bors-servo added a commit to servo/servo that referenced this pull request Nov 19, 2019
WebAudio MediaElementAudioSourceNode implementation

- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix #22370
- [X] There are tests for these changes

Depends on servo/media#321

Tested with https://ferjm.github.io/webaudio-examples/media-source-buffer/

I am still investigating why it is playing back with a low rate.
bors-servo added a commit to servo/servo that referenced this pull request Nov 19, 2019
WebAudio MediaElementAudioSourceNode implementation

- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix #22370
- [X] There are tests for these changes

Depends on servo/media#321

Tested with https://ferjm.github.io/webaudio-examples/media-source-buffer/

I am still investigating why it is playing back with a low rate.
bors-servo added a commit to servo/servo that referenced this pull request Nov 19, 2019
WebAudio MediaElementAudioSourceNode implementation

- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix #22370
- [X] There are tests for these changes

Depends on servo/media#321

Tested with https://ferjm.github.io/webaudio-examples/media-source-buffer/

I am still investigating why it is playing back with a low rate.
bors-servo added a commit to servo/servo that referenced this pull request Nov 19, 2019
WebAudio MediaElementAudioSourceNode implementation

- [X] `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- [X] These changes fix #22370
- [X] There are tests for these changes

Depends on servo/media#321

Tested with https://ferjm.github.io/webaudio-examples/media-source-buffer/

I am still investigating why it is playing back with a low rate.
@ferjm ferjm deleted the ferjm:mediaelementsourcenode branch May 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants
You can’t perform that action at this time.