Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash when using the GvrAudioProcessor with mono audio channel video #2710

Closed
Oleur opened this issue Apr 20, 2017 · 3 comments
Closed

Crash when using the GvrAudioProcessor with mono audio channel video #2710

Oleur opened this issue Apr 20, 2017 · 3 comments
Labels
enhancement

Comments

@Oleur
Copy link

Oleur commented Apr 20, 2017 •
edited

Issue description

Videos with one audio channel cannot be played when the player uses the GvrAudioProcessor. An exception (UnhandledFormatException) is thrown from the GvrAudioProcessor in the configure method.

Reproduction steps

Override the buildAudioProcessors method in SimpleExoPlayer.

player = new SimpleExoPlayer(...) {
  @Override
  public AudioProcessor[] buildAudioProcessors() {
    gvrAudioProcessor = new GvrAudioProcessor();
    return new AudioProcessor[] {gvrAudioProcessor};
  }
};

Setup the player with the mono audio channel video and start playback.

Link to test content

Every video with only one audio channel

Version of ExoPlayer being used

ExoPlayer 2.3.1

Device(s) and version(s) of Android being used

Nexus 5X - Android 7.1.2

A full bug report captured from the device

04-20 11:06:46.458 19432-19642/cinemur.apptest E/ExoPlayerImplInternal: Renderer error.
                                                                        com.google.android.exoplayer2.ExoPlaybackException
                                                                            at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.onOutputFormatChanged(SourceFile:250)
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.b(SourceFile:967)
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.a(SourceFile:909)
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(SourceFile:506)
                                                                            at com.google.android.exoplayer2.ExoPlayerImplInternal.f(SourceFile:466)
                                                                            at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(SourceFile:300)
                                                                            at android.os.Handler.dispatchMessage(Handler.java:98)
                                                                            at android.os.Looper.loop(Looper.java:154)
                                                                            at android.os.HandlerThread.run(HandlerThread.java:61)
                                                                            at com.google.android.exoplayer2.util.PriorityHandlerThread.run(SourceFile:40)
                                                                         Caused by: com.google.android.exoplayer2.audio.AudioTrack$ConfigurationException: com.google.android.exoplayer2.audio.AudioProcessor$UnhandledFormatException: Unhandled format: 48000 Hz, 1 channels in encoding 2
                                                                            at com.google.android.exoplayer2.audio.AudioTrack.configure(SourceFile:492)
                                                                            at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.onOutputFormatChanged(SourceFile:248)
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.b(SourceFile:967) 
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.a(SourceFile:909) 
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(SourceFile:506) 
                                                                            at com.google.android.exoplayer2.ExoPlayerImplInternal.f(SourceFile:466) 
                                                                            at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(SourceFile:300) 
                                                                            at android.os.Handler.dispatchMessage(Handler.java:98) 
                                                                            at android.os.Looper.loop(Looper.java:154) 
                                                                            at android.os.HandlerThread.run(HandlerThread.java:61) 
                                                                            at com.google.android.exoplayer2.util.PriorityHandlerThread.run(SourceFile:40) 
                                                                         Caused by: com.google.android.exoplayer2.audio.AudioProcessor$UnhandledFormatException: Unhandled format: 48000 Hz, 1 channels in encoding 2
                                                                            at com.google.android.exoplayer2.ext.gvr.GvrAudioProcessor.configure(SourceFile:106)
                                                                            at com.google.android.exoplayer2.audio.AudioTrack.configure(SourceFile:490)
                                                                            at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.onOutputFormatChanged(SourceFile:248) 
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.b(SourceFile:967) 
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.a(SourceFile:909) 
                                                                            at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(SourceFile:506) 
                                                                            at com.google.android.exoplayer2.ExoPlayerImplInternal.f(SourceFile:466) 
                                                                            at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(SourceFile:300) 
                                                                            at android.os.Handler.dispatchMessage(Handler.java:98) 
                                                                            at android.os.Looper.loop(Looper.java:154) 
                                                                            at android.os.HandlerThread.run(HandlerThread.java:61) 
                                                                            at com.google.android.exoplayer2.util.PriorityHandlerThread.run(SourceFile:40) 
04-20 11:06:46.460 19432-19432/cinemur.apptest E/EventLogger: playerFailed [0.53]
                                                              com.google.android.exoplayer2.ExoPlaybackException
                                                                  at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.onOutputFormatChanged(SourceFile:250)
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.b(SourceFile:967)
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.a(SourceFile:909)
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(SourceFile:506)
                                                                  at com.google.android.exoplayer2.ExoPlayerImplInternal.f(SourceFile:466)
                                                                  at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(SourceFile:300)
                                                                  at android.os.Handler.dispatchMessage(Handler.java:98)
                                                                  at android.os.Looper.loop(Looper.java:154)
                                                                  at android.os.HandlerThread.run(HandlerThread.java:61)
                                                                  at com.google.android.exoplayer2.util.PriorityHandlerThread.run(SourceFile:40)
                                                               Caused by: com.google.android.exoplayer2.audio.AudioTrack$ConfigurationException: com.google.android.exoplayer2.audio.AudioProcessor$UnhandledFormatException: Unhandled format: 48000 Hz, 1 channels in encoding 2
                                                                  at com.google.android.exoplayer2.audio.AudioTrack.configure(SourceFile:492)
                                                                  at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.onOutputFormatChanged(SourceFile:248)
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.b(SourceFile:967) 
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.a(SourceFile:909) 
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(SourceFile:506) 
                                                                  at com.google.android.exoplayer2.ExoPlayerImplInternal.f(SourceFile:466) 
                                                                  at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(SourceFile:300) 
                                                                  at android.os.Handler.dispatchMessage(Handler.java:98) 
                                                                  at android.os.Looper.loop(Looper.java:154) 
                                                                  at android.os.HandlerThread.run(HandlerThread.java:61) 
                                                                  at com.google.android.exoplayer2.util.PriorityHandlerThread.run(SourceFile:40) 
                                                               Caused by: com.google.android.exoplayer2.audio.AudioProcessor$UnhandledFormatException: Unhandled format: 48000 Hz, 1 channels in encoding 2
                                                                  at com.google.android.exoplayer2.ext.gvr.GvrAudioProcessor.configure(SourceFile:106)
                                                                  at com.google.android.exoplayer2.audio.AudioTrack.configure(SourceFile:490)
                                                                  at com.google.android.exoplayer2.audio.MediaCodecAudioRenderer.onOutputFormatChanged(SourceFile:248) 
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.b(SourceFile:967) 
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.a(SourceFile:909) 
                                                                  at com.google.android.exoplayer2.mediacodec.MediaCodecRenderer.render(SourceFile:506) 
                                                                  at com.google.android.exoplayer2.ExoPlayerImplInternal.f(SourceFile:466) 
                                                                  at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(SourceFile:300) 
                                                                  at android.os.Handler.dispatchMessage(Handler.java:98) 
                                                                  at android.os.Looper.loop(Looper.java:154) 
                                                                  at android.os.HandlerThread.run(HandlerThread.java:61) 
                                                                  at com.google.android.exoplayer2.util.PriorityHandlerThread.run(SourceFile:40) 

Partial fix

I came up with a solution but I don't know if this is the right way to do it. (https://gist.github.com/Oleur/b2770cf7c225bf5c0d3861925d6fb576). It would be nice to pass through the spatialization if the audio format is not supported by the GvrAudioSurround and process the audio normaly.
@andrewlewis
Copy link
Collaborator

I'm following up to find out whether GVR could handle mono audio. I'm not sure that just passing through the the input is the best option, as the audio wouldn't respond to head rotation.

If you only need to handle mono input, as a temporary workaround you could try duplicating the channel to make it stereo. One way to do that is to make ChannelMappingAudioProcessor publicly visible and add one before the GvrAudioProcessor in your buildAudioProcessors method. Then call setChannelMap(new int[] {0, 0}); to make it output a stereo stream with the first channel of input duplicated. I'm not sure about the correctness of this option from a spatial audio standpoint, though.

@andrewlewis
Copy link
Collaborator

There's work underway to add support for handling mono input in the GVR audio SDK. I don't have an ETA yet but I'll update this issue when the functionality is available via GvrAudioProcessor.

@Oleur
Copy link
Author

Oleur commented Apr 27, 2017

Thanks for the update!

ojw28 pushed a commit that referenced this issue Jun 15, 2017
@andrewlewis @ojw28
Add support for mono input to the GVR extension
a913fd9 
Issue: #2710

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159082518
@ojw28 ojw28 closed this as completed Jun 30, 2017
ojw28 pushed a commit that referenced this issue Jun 30, 2017
@andrewlewis @ojw28
Add support for mono input to the GVR extension
e8ee868 
Issue: #2710

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159082518
@google google locked and limited conversation to collaborators Oct 29, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@andrewlewis @Oleur @ojw28