wav audio files can't be played #26273

jdm · 2020-04-22T15:43:11Z

joshmatthews@joshmatthews-fcmd6r servo % ./mach run -r https://playcanv.as/p/Ikq6Uk6A/
Powered by PlayCanvas 0.182.1 b9c2b30
pre-main prep time: 0 ms
Audio format for files/assets/3667012/1/shop.wav?t=4fcb27bba9ab45b5dddebcb562fc3fb6 not supported
Audio format for files/assets/3667013/1/Explosion.wav?t=c1239dba84d7aa439de1357d120df1c3 not supported
Audio format for files/assets/3667073/1/plasma.wav?t=2399ce38c05b909324825dae2df74f5b not supported
Audio format for files/assets/3667639/1/button.wav?t=754cd4f8c6086f7fe22d632f1c74b274 not supported
Audio format for files/assets/3668734/1/laser.wav?t=a4ab93ea49e2043fed2ab4bce1f4a8a8 not supported
Audio format for files/assets/3671306/1/rocket launch.wav?t=19f862f64403cf06e7d840a4a8bd1373 not supported
Audio format for files/assets/3675472/1/engine.wav?t=61d6cb6a023cd8fff7072d9190a9a9f1 not supported
Audio format for files/assets/3676322/1/hit2.wav?t=ccd3f31ef7b4570dd8e0956fb9e72001 not supported
Audio format for files/assets/3942422/1/intro.ogg?t=5fc2c6dc799b7c57a149086a3c90232e not supported
Audio format for files/assets/3944808/1/cash.wav?t=49f4cd2b0de1cf180546cdc65b0421c8 not supported

khodzha · 2020-05-04T07:20:08Z

i'd like to take a look but i tried grepping both servo and servo-media and failed to find anything like that 🤷
could you point out where these log messages come from?

ferjm · 2020-05-04T13:02:19Z

could you point out where these log messages come from?

These log messages come from the app itself. This is probably canPlayType mistakenly reporting that Servo is unable to play this format.

I've just checked that we are actually able to play wav audio files, but canPlayType is reporting that we are NOT able to do it with this test:

<audio autoplay src="http://www.music.helsinki.fi/tmt/opetus/uusmedia/esim/a2002011001-e02-16kHz.wav"></audio>
<script>
  const canPlay = document.querySelector("audio").canPlayType('audio/wav');
  if (canPlay == 'maybe' || canPlay == 'probably') {
    console.log('Can play wav');
  } else {
    console.log('Can NOT play wav');
  }
</script>

The code for the canPlayType logic mostly lives in servo-media:

khodzha · 2020-05-04T19:23:58Z

also this link at best crashes servo and gives following stacktrace

Stacktrace


[2020-05-04T15:46:57Z ERROR canvas::canvas_data] Unimplemented canvas2d.fillText. Values received: undefined, 256, 64, None.
[2020-05-04T15:46:57Z ERROR canvas::canvas_data] Unimplemented canvas2d.fillText. Values received: undefined, 256, 64, None.
[2020-05-04T15:46:57Z ERROR canvas::canvas_data] Unimplemented canvas2d.fillText. Values received: undefined, 256, 64, None.
[2020-05-04T15:46:57Z ERROR canvas::canvas_data] Unimplemented canvas2d.fillText. Values received: 0, 256, 64, None.
[2020-05-04T15:46:57Z ERROR canvas::canvas_data] Unimplemented canvas2d.fillText. Values received: 0, 256, 64, None.
[2020-05-04T15:46:57Z ERROR canvas::canvas_data] Unimplemented canvas2d.fillText. Values received: 0          0, 256, 64, None.
[2020-05-04T15:47:01Z WARN  html5ever::tree_builder] foster parenting not implemented
[2020-05-04T15:47:01Z WARN  html5ever::tree_builder] foster parenting not implemented
[2020-05-04T15:47:01Z WARN  html5ever::tree_builder] foster parenting not implemented
[2020-05-04T15:47:01Z WARN  html5ever::tree_builder] foster parenting not implemented
servo: ../nouveau/pushbuf.c:723: nouveau_pushbuf_data: Assertion `kref' failed.
Stack trace for thread "main"
   0: servo::backtrace::print
             at ports/glutin/backtrace.rs:17
   1: servo::install_crash_handler::handler
             at ports/glutin/main2.rs:66
   2: <unknown>
             at /build/glibc-YYA7BZ/glibc-2.31/nptl/../sysdeps/unix/sysv/linux/x86_64/sigaction.c:0
   3: raise
   4: abort
   5: <unknown>
   6: __assert_fail
   7: nouveau_pushbuf_data
   8: nouveau_pushbuf_data
   9: nouveau_pushbuf_space
  10: <unknown>
  11: <unknown>
  12: <unknown>
  13: <unknown>
  14: <unknown>
  15: gleam::ffi_gl::Gl::Clear
             at /home/khodzha/programming/servo/target/debug/build/gleam-25aa1f82e03902d0/out/gl_bindings.rs:3515
  16: <gleam::gl::GlFns as gleam::gl::Gl>::clear
             at /home/khodzha/.cargo/registry/src/github.com-1ecc6299db9ec823/gleam-0.9.2/src/gl_fns.rs:1880
  17: compositing::compositor::IOCompositor<Window>::clear_background
             at /home/khodzha/programming/servo/components/compositing/compositor.rs:1466
  18: compositing::compositor::IOCompositor<Window>::composite_specific_target::{{closure}}
             at /home/khodzha/programming/servo/components/compositing/compositor.rs:1319
  19: profile_traits::time::profile
             at /home/khodzha/programming/servo/components/profile_traits/time.rs:147
  20: compositing::compositor::IOCompositor<Window>::composite_specific_target
             at /home/khodzha/programming/servo/components/compositing/compositor.rs:1307
  21: compositing::compositor::IOCompositor<Window>::composite
             at /home/khodzha/programming/servo/components/compositing/compositor.rs:1225
  22: compositing::compositor::IOCompositor<Window>::perform_updates
             at /home/khodzha/programming/servo/components/compositing/compositor.rs:1539
  23: servo::Servo<Window>::handle_events
             at /home/khodzha/programming/servo/components/servo/lib.rs:795
  24: servo::app::App::handle_events
             at ports/glutin/app.rs:212
  25: servo::app::App::run_loop
             at ports/glutin/app.rs:173
  26: servo::app::App::run
             at ports/glutin/app.rs:96
  27: servo::main
             at ports/glutin/main2.rs:178
  28: std::rt::lang_start::{{closure}}
             at /rustc/42abbd8878d3b67238f3611b0587c704ba94f39c/src/libstd/rt.rs:67
  29: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:52
      std::panicking::try::do_call
             at src/libstd/panicking.rs:331
      std::panicking::try
             at src/libstd/panicking.rs:274
      std::panic::catch_unwind
             at src/libstd/panic.rs:394
      std::rt::lang_start_internal
             at src/libstd/rt.rs:51
  30: std::rt::lang_start
             at /rustc/42abbd8878d3b67238f3611b0587c704ba94f39c/src/libstd/rt.rs:67
  31: main
  32: __libc_start_main
  33: _start
Servo exited with return value 6

or just kills my gnome-shell so i had to reset my pc 😱 (i think it does something to my nouveau driver? im not proficient with this stuff)


        Auto merge of #336 - khodzha:can_play_type_wav, r=Manishearth

added demuxer presence check for audio/x-wav media type also fixed unused-extern-crates warning GStreamerRegistryScanner was checking if wav is supported by searching gstreamer factories for application/x-wav caps, but some hosts report wav mime as audio/x-wav. issue: servo/servo#26273

jdm · 2020-05-05T15:02:53Z

With the media update, we have some new test results:

  ▶ Unexpected subtest result in /html/semantics/embedded-content/media-elements/mime-types/canPlayType.html:
  └ PASS [expected FAIL] audio/wav; codecs="1" (optional)
  ▶ Unexpected subtest result in /html/semantics/embedded-content/media-elements/mime-types/canPlayType.html:
  │ FAIL [expected PASS] audio/wav with and without codecs
  │   → assert_equals: expected true but got false
  │ 
  │ type_codecs_test/<@http://web-platform.test:8000/html/semantics/embedded-content/media-elements/mime-types/canPlayType.html:111:18
  │ Test.prototype.step@http://web-platform.test:8000/resources/testharness.js:1988:25
  │ test@http://web-platform.test:8000/resources/testharness.js:535:30
  │ type_codecs_test@http://web-platform.test:8000/html/semantics/embedded-content/media-elements/mime-types/canPlayType.html:109:7
  └ @http://web-platform.test:8000/html/semantics/embedded-content/media-elements/mime-types/canPlayType.html:117:17

The new failure corresponds with audio/wav; codecs=1 based on https://searchfox.org/mozilla-central/rev/54f965e51e4f77866bec42737978d40d4c1fdfc5/testing/web-platform/tests/html/semantics/embedded-content/media-elements/mime-types/canPlayType.html#109-117. Perhaps we need to have a more complicated mimetype check that supports codecs as well?

ferjm · 2020-05-07T05:51:25Z

Perhaps we need to have a more complicated mimetype check that supports codecs as well?

AFAICT we are already checking codecs as well.

https://github.com/servo/media/blob/master/backends/gstreamer/lib.rs#L267
https://github.com/servo/media/blob/master/backends/gstreamer/registry_scanner.rs#L185

ferjm · 2020-05-07T09:07:09Z

The problem with this test seems to be that we are failing to parse mime types like audio/wav;codecs and we are reporting that we are unable to play in that case.

ferjm · 2020-05-07T09:19:13Z

We saw this in the past #23229 (comment)

ferjm · 2020-05-07T09:22:37Z

And hyperium/mime#91 likely solves the problem

jdm added the A-content/webaudio label Apr 22, 2020

khodzha mentioned this issue May 4, 2020

added demuxer presence check for audio/x-wav media type servo/media#336

Merged

servo / servo

wav audio files can't be played #26273

wav audio files can't be played #26273

jdm commented Apr 22, 2020 •

edited

khodzha commented May 4, 2020

ferjm commented May 4, 2020 •

edited

khodzha commented May 4, 2020

jdm commented May 5, 2020

ferjm commented May 7, 2020

ferjm commented May 7, 2020

ferjm commented May 7, 2020

ferjm commented May 7, 2020

servo / servo

Join GitHub today

wav audio files can't be played #26273

wav audio files can't be played #26273

Comments

jdm commented Apr 22, 2020 • edited

khodzha commented May 4, 2020

ferjm commented May 4, 2020 • edited

khodzha commented May 4, 2020

jdm commented May 5, 2020

ferjm commented May 7, 2020

ferjm commented May 7, 2020

ferjm commented May 7, 2020

ferjm commented May 7, 2020

jdm commented Apr 22, 2020 •

edited

ferjm commented May 4, 2020 •

edited