[big refactoring] Audio latency fix for Android. Support to preload effects on Android now. #15875

dumganhar · 2016-06-17T09:52:04Z

PLEASE DONT MERGE BEFORE DEVELOPERS TEST IT. AND MORE COMMENTS NEED TO BE ADDED.

This Pull Request has done the following things:

Supports to preload small audio files and cache them with PCM data buffer
MP3/Ogg/... files are decoded to PCM data by OpenSLES API
Decoded PCM buffer data matches the best sample rate & buffer size in frames of device by using resampling source code in Android Source (Android 6.0, platform/frameworks/av)
Big refactoring for the implementation of AudioEngineImpl for Android, better class responsibility
Mixing audio clips by ourself, in this way, we only need a single OpenSLES AudioPlayer for playing many clips and this will fix latency of making a AudioPlayer active and play a track. For now, we only support 32 audio clips being played simultaneously. But AudioEngine module has limited it to 24 clips by default.
Adds audio performance test case
Supports to preload audio files asynchronously.

Please help to test it on the Android devices which have low performance. Thanks :)

xpol · 2016-06-19T13:25:54Z

I see the linter errors:

Checking headers in: /android_slave/workspace/ccs-pr/node/android/cocos
audio/android/AudioResamplerSinc.cpp:
    #include "AudioResamplerSinc.h" should be #include "audio/android/AudioResamplerSinc.h"
    #include "AudioResamplerSincDown.h" should be #include "audio/android/AudioResamplerSincDown.h"
    #include "AudioResamplerSincUp.h" should be #include "audio/android/AudioResamplerSincUp.h"
audio/android/UrlAudioPlayer.cpp:
    #include "UrlAudioPlayer.h" should be #include "audio/android/UrlAudioPlayer.h"
audio/android/PcmAudioPlayer.h:
    #include "OpenSLHelper.h" should be #include "audio/android/OpenSLHelper.h"
    #include "IAudioPlayer.h" should be #include "audio/android/IAudioPlayer.h"
    #include "PcmData.h" should be #include "audio/android/PcmData.h"
audio/android/AudioResampler.cpp:
    #include "AudioResamplerSinc.h" should be #include "audio/android/AudioResamplerSinc.h"
    #include "AudioResamplerCubic.h" should be #include "audio/android/AudioResamplerCubic.h"
    #include "AudioResampler.h" should be #include "audio/android/AudioResampler.h"
audio/android/AudioResamplerCubic.h:
    #include "AudioBufferProvider.h" should be #include "audio/android/AudioBufferProvider.h"
    #include "AudioResampler.h" should be #include "audio/android/AudioResampler.h"
audio/android/AudioDecoder.h:
    #include "OpenSLHelper.h" should be #include "audio/android/OpenSLHelper.h"
    #include "PcmData.h" should be #include "audio/android/PcmData.h"
audio/android/AudioResamplerCubic.cpp:
    #include "AudioResamplerCubic.h" should be #include "audio/android/AudioResamplerCubic.h"
    #include "AudioResampler.h" should be #include "audio/android/AudioResampler.h"
audio/android/UrlAudioPlayer.h:
    #include "OpenSLHelper.h" should be #include "audio/android/OpenSLHelper.h"
    #include "IAudioPlayer.h" should be #include "audio/android/IAudioPlayer.h"
audio/android/AudioResamplerSinc.h:
    #include "AudioBufferProvider.h" should be #include "audio/android/AudioBufferProvider.h"
    #include "AudioResampler.h" should be #include "audio/android/AudioResampler.h"
audio/android/AudioPlayerProvider.h:
    #include "OpenSLHelper.h" should be #include "audio/android/OpenSLHelper.h"
    #include "IAudioPlayer.h" should be #include "audio/android/IAudioPlayer.h"
    #include "PcmData.h" should be #include "audio/android/PcmData.h"
audio/android/PcmAudioPlayerPool.cpp:
    #include "PcmAudioPlayerPool.h" should be #include "audio/android/PcmAudioPlayerPool.h"
    #include "PcmAudioPlayer.h" should be #include "audio/android/PcmAudioPlayer.h"
audio/android/PcmAudioPlayerPool.h:
    #include "OpenSLHelper.h" should be #include "audio/android/OpenSLHelper.h"
audio/android/PcmBufferProvider.cpp:
    #include "PcmBufferProvider.h" should be #include "audio/android/PcmBufferProvider.h"
audio/android/AudioResampler.h:
    #include "AudioBufferProvider.h" should be #include "audio/android/AudioBufferProvider.h"
audio/android/AudioDecoder.cpp:
    #include "AudioDecoder.h" should be #include "audio/android/AudioDecoder.h"
    #include "PcmBufferProvider.h" should be #include "audio/android/PcmBufferProvider.h"
    #include "AudioResampler.h" should be #include "audio/android/AudioResampler.h"
audio/android/AudioEngine-inl.cpp:
    #include "AudioPlayerProvider.h" should be #include "audio/android/AudioPlayerProvider.h"
audio/android/PcmAudioPlayer.cpp:
    #include "PcmAudioPlayer.h" should be #include "audio/android/PcmAudioPlayer.h"
audio/android/AudioPlayerProvider.cpp:
    #include "AudioPlayerProvider.h" should be #include "audio/android/AudioPlayerProvider.h"
    #include "AudioDecoder.h" should be #include "audio/android/AudioDecoder.h"
    #include "PcmAudioPlayerPool.h" should be #include "audio/android/PcmAudioPlayerPool.h"
    #include "PcmAudioPlayer.h" should be #include "audio/android/PcmAudioPlayer.h"
    #include "UrlAudioPlayer.h" should be #include "audio/android/UrlAudioPlayer.h"
audio/android/PcmBufferProvider.h:
    #include "AudioBufferProvider.h" should be #include "audio/android/AudioBufferProvider.h"
Total: 39 errors in 19 files
Rerun this script with -f to fixes these errors

Please do the follow to fixes:

run python ./tools/coding-style/include-linter.py -f to fixes these errors
Check the changed code and commit the changes.
the ci should pass after push the new commit.

dumganhar · 2016-06-19T15:17:51Z

@xpol
This code style is terrible. All my files are in the same folder. Why do I have to
#include "audio/android/..." ??

I developed this functionality on Android Studio by a simple demo.
I don't want the include path to be coupled with cocos2d-x include path.

I think we need to change the rule of linter. If included file is in the same directory of source file, need to support to include header file with just file name.

xpol · 2016-06-20T02:28:47Z

@dumganhar

Currently:

cocos2d-x has header (.h) files in same folder as source (.cpp) files (the $EngineRoot/cocos folder).
And we also do not distinguish public and private headers.

As result, using style as #include "basename.h" is OK.

But libraries like:

boost it have public headers in a separated folder, and thus, #include "basename.h" should not compile.
google-api-cpp-client it has cpp and h in same folder but it use #include "googleapis/client/auth/credential_store.h" rather than #include "credential_store.h"

Introducing tools/coding-style/include-linter.py is to help get ready for moving public headers into separated folder (eg. $EngineRoot/include).
So I suggest not to change the linter rule.

But anyway, it is only a choice of flavor, you and your team are the keeper of keys at cocos2d-x.
If you choose not to change the coding style, I suggest we just remove the linter form the CI and code base.

dumganhar · 2016-06-20T05:04:41Z

@xpol , thanks for adding this useful static analysis tool.
Yep, some libraries like boost or google-api-cpp-client do in that way.
But could we make some changes to fix our needs better?
I don't want to draw the line of use linter or not, could we improve it? could it be better for us?

For developing an independent module, including a header file( with cocos2d-x path coupled ) in the same directory as .cpp file is painful.

I suggest to keep current tool linter, but support to #include "xxx.h" for xxx.cpp file which are in the same directory.

That's my opinion. :)

@xpol @ricardoquesada @minggo @zilongshanren

What's your opinions?

minggo · 2016-06-20T05:48:48Z

@dumganhar I am not sure if it is able to modify linter to just satisfy your requirements. But i think we should keep it like this if:

need to add more search path to fit the requirements

dumganhar · 2016-06-20T05:59:19Z

@minggo , I looked at the linter source code, found that currently it only supports one root path (cocos2d-x/cocos/). I think modifying linter source code to fix our requirement isn't difficult.
So just think about that, do we have to keep the only one mode of include header strategy?

minggo · 2016-06-20T06:33:20Z

@dumganhar Only have one search path is good:

less search path needed
every module starts from its root will reduce conflict, and it is easy to know which module is included, for example,

#include "ui/xxx.h"

Then i know i uses xxx UI module. I think it is good.

dumganhar · 2016-06-20T06:38:02Z

@minggo , if it's in another module. yep, it should include the module prefix.
But if it's in the same module, this module know that header file is myself. Adding prefix will not take enough advantage.

BTW, current directory is the default include path for all compliers. Not extra search paths is added.

minggo · 2016-06-20T06:45:29Z

@dumganhar Yep, if the .cpp and .h file in the same directory, it is reasonable not to include full path. But as i said, it should not add additional search path.

xpol · 2016-06-20T14:47:56Z

for example we have the follow audio/a.cpp

#include "a.h" // (1) include the audio/a.h
#include "b.h" // (2) include the audio/b.h
#include "./b.h" // (3) include the audio/b.h
#include "android/c.h" // (4) include the audio/android/c.h
#include "../ui/d.h" // (5) include the ui/d.h

and the follow is audio/a.h

#include "b.h" // (6) include the audio/b.h
#include "./b.h" // (7) include the audio/b.h
#include "android/c.h" // (8) include the audio/android/c.h
#include "../ui/d.h" // (9) include the ui/d.h

The line 1~8. which is allowed which is not allowed?

dumganhar · 2016-06-21T01:57:57Z

@xpol
(1)(2)(4)(6)(8) are allowed, others are not.

dumganhar · 2016-06-21T02:05:26Z

I'm thinking about if current directory is supported for header search path.

e.g. audio/a.cpp

#include "audio/a.h"  // allowed
#include "a.h" // allowed
// a sub folder like `audio/android/b.(h | cpp)`, could a.cpp be included like:
#include "android/b.h"  ??

j-jorge · 2016-06-21T15:33:17Z

cocos/audio/android/AudioDecoder.cpp

+    if (_pcmMetaData != nullptr)
+    {
+        free(_pcmMetaData);
+        _pcmMetaData = nullptr;


_pcmMetaData won't be read after the destructor, you can skip this assignment.

Should not skip this assignment since AudioDecoder::start may be interrupted and return false with _pcmMetaData != null, and the destructor should handle it.

alfogrillo · 2016-06-22T10:19:11Z

@dumganhar
I'm testing your PR.

I configured cocos2d-x as follows:

Cloned last cocos2d-x version
Merged your PR in a new branch
Executed "setup.py"
Generated the "cocos2d" folder with "cocos new" and copied in my existing project
Substituted folder "cocos2d/cocos/audio/android" with the one in your PR

My configuration

Android 5.1.1 on Galaxy S4
Compiling in debug mode with API 18
Using AudioEngine
Using AudioEngine::preload for each sound (using relative path and search paths)
Sound type: .wav (only one soundtrack is .mp3)

My game experiment FPS variation in the range 30-60 fps, without audio the game is more enjoyable with FPS in range 45-60.
The FPS drops especially when multiple sound effects are executed in short intervals.
With audio it seems that FPS are more stable with time (as if preload does not work).
On iOS (iPhone 6) the game FPS in stable in the range 55-60 fps.

Tell me if you want I make some other tests.

dumganhar · 2016-06-22T10:23:52Z

@Drakon-Cocos , thanks for testing.
Could you please upload the log from logcat? I could analyze what the problem is.

alfogrillo · 2016-06-22T13:25:14Z

@dumganhar
Yes of course! :-)
Here the log.

I saw in the log various lines of kind:
D/AudioPlayerProvider(29538): File ("filename") is too large, ignore preload!

UPDATE:
log.txt

dumganhar · 2016-06-22T14:49:23Z

Hi @Drakon-Cocos , preload only works for audio files with small file size.
But you could configure it by modify AudioPlayerProvider.cpp:

bool AudioPlayerProvider::isSmallFile(long fileSize)
{
    //TODO: If file size is smaller than 30k, we think it's a small file. This value should be set by evelopers.
    return fileSize < 30000;
}

I looked at the log, you have effect files that bigger than 60k, so try to change its value bigger and test.
BTW, what's the performance while not using this changes in this PR?

alfogrillo · 2016-06-22T15:30:38Z

@dumganhar
I increased "isSmallFile" threshold from 30k to 300k without visible performance improvements.
I did not made quantified performance test before and after your PR merge, but the perceived game experience seems the same.
With SimpleAudioEngine things seem better, but I experience sound effect truncation due to max number of sound effect playing in parallel (5 if I remember well)!
Using your PR sometimes happens that some sound effects eventually do not play, while others play fine!

dumganhar · 2016-06-22T16:10:59Z

Do you mind to send me an apk of your game?
My email: jianhua.chen@cocos2d-x.org

alfogrillo · 2016-06-22T21:31:33Z

@dumganhar
Sorry currently I cannot distribute the apk to third parties.
Anyway I would like to continue to test your PR with you.
In the next days I will try to understand exactly when fps drops in my game!
If I discover something useful I will post immediately! ;-)

alfogrillo · 2016-06-23T09:22:21Z

@dumganhar
I began to do some performance test.
It seems that sound preloading has problems.

I tried two kind of preload (in my Layer init):

AudioEngine::preload
AudioEngine::play2d (with volume 0)

However the first play of each sound effect continue to take a time between 30-60 ms for the first play!
With small .mp3 latency reach even 100ms for first play!
Does preloading work for you?

UPDATE1:
Similar behavior with SimpleAudioEngine and its "preloadEffect" on Android

UPDATE2 :
On iOS AudioEngine preload seems to work well.
The sound effect play time is less than 0.5 ms on iPhone 6.
On Galaxy S4 the play time (without counting the first play!) is, in worst case, about 10x slower!! :-(

dumganhar · 2016-06-23T10:41:23Z

@Drakon-Cocos , thank you for testing.
Could you share the testing code? I'd have a try.

Preload does work on my device. I don't know why it works badly on your android device. :(

dumganhar · 2016-06-23T13:18:43Z

@xpol , updated, made linter happy now. :)

alfogrillo · 2016-07-21T10:20:43Z

@dumganhar
Setting APP_STL := c++_static in Application.mk does not prevent the crash.
In CCThreadPool.cpp I added a log line at the beginning of each function.
If you do:
cat s4_crash442_ThreadPool.txt | grep ThreadPool
you get

D/ThreadPool(25491): ThreadPool::newCachedThreadPool
D/ThreadPool(25491): ThreadPool::ThreadPool
D/ThreadPool(25491): ThreadPool::init()
D/ThreadPool(25491): ThreadPool::setThread
D/ThreadPool(25491): ThreadPool::setFixedSize
D/ThreadPool(25491): ThreadPool::setShrinkInterval
D/ThreadPool(25491): ThreadPool::setShrinkStep
D/ThreadPool(25491): ThreadPool::setStretchStep
D/ThreadPool(25491): ThreadPool::pushTask
D/ThreadPool(25491): ThreadPool::pushTask
D/ThreadPool(25491): ThreadPool::stretchPool
D/ThreadPool(25491): ThreadPool::setThread
E/ActivityThread(25694):    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
E/ActivityThread(25694):    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)

Here the full log:
s4_crash442_ThreadPool.txt

dumganhar · 2016-07-21T10:46:24Z

@Drakon-Cocos , could you add more logs in

AudioDecoder::~AudioDecoder()
{
    ALOGV("~AudioDecoder() %p", this);
    SL_DESTROY_OBJ(_playObj);
    ALOGV("After destroying SL play object");
    if (_assetFd > 0)
    {
        ALOGV("Closing assetFd: %d", _assetFd);
        ::close(_assetFd);
        _assetFd = 0;
    }
    ALOGV("before free _pcmData");  // add this
    free(_pcmData);
    ALOGV("after free _pcmData"); // add this
}

I wanna know the exact line which cause the crash.

alfogrillo · 2016-07-21T11:06:47Z

@dumganhar
I added the two lines.
Full log:
s4_crash442_ThreadPool_v2.txt

When I did:
adb logcat | grep _pcmData
I got

V/AudioDecoder( 3301): before free _pcmData
V/AudioDecoder( 3301): after free _pcmData

ryanxia2016 · 2016-07-21T14:53:04Z

@dumganhar

Another Issue, but I don't know if it's related to this PR.
When call playEffect with same SMALL mp3 files, game crashed.
Crash Device : ASUS ZENPHONE2 (Android 5.0 x86)

COCOS Version 3.3 with this PR merged.

The game worked right at most time, but if it hits the specified mp3 files, then crash occur.

attachment is the mp3 file, which will cause crash.
crash_voice.mp3.zip

dumganhar · 2016-07-22T01:57:52Z

@Drakon-Cocos, could you help to find the exact line which causes the crash since I don't have a device to reproduce it. Thanks.

@ryanxia2016 , thanks for your feedback. For a glance, it seems to be a PCM decoding issue of OpenSLES. Anyway, I will check it.

dumganhar · 2016-07-22T03:42:00Z

@ryanxia2016 , it seems like a bug of decoding small files by OpenSLES API.
AudioDecoder::decodeToPcmCallback wasn't invoked if the file is too small.

ryanxia2016 · 2016-07-22T04:04:34Z

@dumganhar
so It's a OS level bug, saddly story...
then the quick solution is to check file size and to choose Play it with JNI Call or AudioEngine call?

or, we should ensure all the sound file's size > xxxx bytes?

dumganhar · 2016-07-22T04:51:18Z

@ryanxia2016 , basically, it shouldn't crash while dealing with small file.

dumganhar · 2016-07-22T05:04:49Z

@ryanxia2016 , I created an issue (#16192).
Let's go to there to communicate. :)

alfogrillo · 2016-07-22T08:15:54Z

@dumganhar
I'm doing some other test with 4.4.2.
The game seems to crash in main thread.
The last lines I see (in main thread) are (notice the QuadCommand line!):

07-22 09:59:27.398 16660 16678 D cocos2d-x debug info: Time for loading effect s07.wav: 25.311000 ms
07-22 09:59:27.408 16660 16678 V AudioPlayerProvider: (assets/sounds/s08.wav) file size: 39320
07-22 09:59:27.408 16660 16678 D ThreadPool: ThreadPool::pushTask start
07-22 09:59:27.408 16660 16678 D ThreadPool: ThreadPool::stretchPool
07-22 09:59:27.408 16660 16678 D ThreadPool: ThreadPool::pushTask end
07-22 09:59:27.408 16660 16678 V AssetFd : ~AssetFd: 74
07-22 09:59:27.408 16660 16678 D cocos2d-x debug info: Time for loading effect s08.wav: 5.046000 ms
07-22 09:59:27.608 16660 16678 D cocos2d-x debug info: cocos2d: QuadCommand: resizing index size from [-1] to [2560]
07-22 09:59:27.658 16660 16678 F libc    : Fatal signal 11 (SIGSEGV) at 0x0086b3c0 (code=1), thread 16678 (Thread-1932)

Here the log with grep on main thread id:
crashMain.txt

any clue?

EDIT: I start to think that could be a rendering bug in v3 branch
EDIT2: without using audio the game does not crash

Obg1 · 2016-07-24T00:16:50Z

I'm also getting a crash but with different error:

I/OMXCodec: Successfully allocated OMX node 'OMX.google.vorbis.decoder'
I/OMXCodec: >>>UHQA initOutputFormat 16
I/OMXCodec: [OMX.google.vorbis.decoder] OMXCodec::start mState=1
I/OMXCodec: [OMX.google.vorbis.decoder] allocating 4 buffers of size 8192 on input port
I/OMXCodec: [OMX.google.vorbis.decoder] allocating 4 buffers of size 32768 on output port
I/OMXCodec: [OMX.google.vorbis.decoder] Now Idle. Component sends idle done Event 
I/OMXCodec: [OMX.google.vorbis.decoder] 0onCmdComplete  mState =  7
I/OMXCodec: [OMX.google.vorbis.decoder] PORT_DISABLED(1)
I/OMXCodec: >>>UHQA initOutputFormat 16
I/OMXCodec: [OMX.google.vorbis.decoder] allocating 4 buffers of size 32768 on output port
I/OMXCodec: [OMX.google.vorbis.decoder] End Of Stream
I/OMXCodec: [OMX.google.vorbis.decoder] stop mState=4
I/OMXCodec: [OMX.google.vorbis.decoder] stop() sendCommand(1, OMX_CommandStateSet, OMX_StateIdle)
I/OMXCodec: [OMX.google.vorbis.decoder] Now Idle. Component sends idle done Event 
I/OMXCodec: [OMX.google.vorbis.decoder] stopOmxComponent_l() mstate = 1
I/OggExtractor: OggSource::stop() mExtractor ref count = 1
I/OggExtractor: OggSource::~OggSource() mExtractor !mStarted ref count = 1
I/OggExtractor: ~OggSource --
I/OggExtractor: ~OggExtractor ++
I/OggExtractor: ~MyOggExtractor ++ 
I/OggExtractor: ~MyOggExtractor --
I/OggExtractor: ~OggExtractor --
A/libc: Fatal signal 8 (SIGFPE), code -6, fault addr 0x46f9 in tid 18979 (GLThread 30507)

superman-t · 2016-07-25T02:27:07Z

@dumganhar I transplant the new audioengine to my cocos2dx 2.1.2 version, and i tested in my project found that in Huawei Nexus 6 (android 6.0.1) performance is low, play one effect（MP3 ） that not preload consumed about 500ms, but i tested the same effect in Samsang GALAXY S6 （Android 5.1.1）that consumed only 5ms, it is very different。

dumganhar · 2016-07-25T02:30:56Z

@superman-t , please attach the log.
How about playing an effect file after preloading?

superman-t · 2016-07-25T02:36:38Z

@dumganhar Playing the effect file after preloading it only consumed 5ms. I found the preload in GLThread？

dumganhar · 2016-07-25T02:46:26Z

@superman-t , it's in a sub thread. You could refer to AudioPlayerProvider.cpp:

_threadPool->pushTask([this, audioFilePath](int tid) {
            ALOGV("AudioPlayerProvider::preloadEffect: (%s)", audioFilePath.c_str());
            PcmData d;
            AudioDecoder decoder(_engineItf, audioFilePath, _bufferSizeInFrames, _deviceSampleRate);
            bool ret = decoder.start(_fdGetterCallback);
            if (ret)

Putting decoding to thread pool for executing.

Preload is needed before entering game scene. Otherwise, AudioEngine::play2d will wait AudioEngine::preload to finish, the GLThread will be blocked.

Currently, we use OpenSLES API for decoding, I also found that its performance isn't really great.
Probably, we need to switch to MediaCodec for this stuff.

Another probable improvement is to play & preload at the same time, I mean to support play an effect not until it's preloaded, maybe some frames of pcm data are decoded, play2d is possible.

superman-t · 2016-07-25T02:59:07Z

@dumganhar Tks，sometimes there are many effects, the scene can't preload all the effects, so it is only play the effect not preloading.The sub thread locks the GLThread, I know the reason why play2d consume many times. it must decode the effect into memory Thank you for doing this work. I will solve my problem using preload.

alfogrillo · 2016-07-28T10:42:57Z

@dumganhar
Could you please show me the steps to integrate this PR in cocos2d-x 3.12?
I don't want branch v3 version for now.
I tried to replace cocos/audio/android and cocos/platform/androidfolders but I get a crash at game boot time:

E/AndroidRuntime( 1630): Process: org.cocos2dx.SpaceRush, PID: 1630
E/AndroidRuntime( 1630): java.lang.UnsatisfiedLinkError: Native method not found: org.cocos2dx.lib.Cocos2dxHelper.nativeSetAudioDeviceInfo:(ZII)V
E/AndroidRuntime( 1630):    at org.cocos2dx.lib.Cocos2dxHelper.nativeSetAudioDeviceInfo(Native Method)
E/AndroidRuntime( 1630):    at org.cocos2dx.lib.Cocos2dxHelper.init(Cocos2dxHelper.java:139)
E/AndroidRuntime( 1630):    at org.cocos2dx.lib.Cocos2dxActivity.onCreate(Cocos2dxActivity.java:269)
W/ActivityManager(  547):   Force finishing activity org.cocos2dx.SpaceRush/org.cocos2dx.cpp.AppActivity

piotrros · 2016-08-24T09:13:46Z

@Drakon-Cocos there are few more additional files changed outside audio directory. Check out "Files changed" to ensure you have everything.
I tried the same thing with cocos2d-x 3.11. After making sure I've copied all necessary files I had to remove obb stuff from code (obb was introtuced after 3.11) then it finally compiled. And sadly game crashes on android 4.4.4 on one of my devices, probably after calling uncache (but before that audio plays well). On the other hand on my Nexus 6P with android 7.0 nougat each "play" freezes app for a 2 seconds and it's not playing. So I guess this is more compilacted stuff.

alfogrillo · 2016-08-24T09:19:37Z

@piotrros in few days Cocos2d-x 3.13 will be released. This release will include also this PR.
I think it is better to wait! :-)

dumganhar · 2016-08-24T09:22:12Z

@piotrros , about freezes for 2 seconds, could you upload the logcat output? Thanks.

piotrros · 2016-08-24T09:35:40Z

@Drakon-Cocos I'd like to use 3.13 or even 3.12, but I'm stuck on 3.11, because 3.12 and above breaks cocos studio compatibility and there's no alternative for now.

@dumganhar Well yeah I forgot to post them:

08-24 11:31:44.646 23913-24246/com.rosapp.Numbers E/ACodec: [OMX.google.vorbis.decoder] ERROR(0x80001005)
08-24 11:31:44.646 23913-24246/com.rosapp.Numbers E/ACodec: signalError(omxError 0x80001005, internalError -2147483648)
08-24 11:31:44.646 23913-24246/com.rosapp.Numbers E/MediaCodec: Codec reported err 0x80001005, actionCode 0, while in state 6

                                                                [ 08-24 11:31:44.646 23913:24229 W/         ]
                                                                [OMX.google.vorbis.decoder] could not get input buffer #1
08-24 11:31:44.646 23913-24229/com.rosapp.Numbers E/libOpenSLES: MediaSource::read returned error -2147483648
08-24 11:31:44.653 23913-24253/com.rosapp.Numbers E/ACodec: [OMX.google.vorbis.decoder] ERROR(0x80001005)
08-24 11:31:44.653 23913-24253/com.rosapp.Numbers E/ACodec: signalError(omxError 0x80001005, internalError -2147483648)
08-24 11:31:44.654 23913-24248/com.rosapp.Numbers E/ACodec: [OMX.google.vorbis.decoder] ERROR(0x80001005)
08-24 11:31:44.654 23913-24248/com.rosapp.Numbers E/ACodec: signalError(omxError 0x80001005, internalError -2147483648)
08-24 11:31:44.654 23913-24248/com.rosapp.Numbers E/MediaCodec: Codec reported err 0x80001005, actionCode 0, while in state 6
08-24 11:31:44.654 23913-24253/com.rosapp.Numbers E/MediaCodec: Codec reported err 0x80001005, actionCode 0, while in state 6

                                                                [ 08-24 11:31:44.654 23913:24231 W/         ]
                                                                [OMX.google.vorbis.decoder] could not get input buffer #1

                                                                [ 08-24 11:31:44.654 23913:24226 I/         ]
                                                                [OMX.google.vorbis.decoder] failed to queue input buffer #1
08-24 11:31:44.654 23913-24231/com.rosapp.Numbers E/libOpenSLES: MediaSource::read returned error -2147483648

                                                                 [ 08-24 11:31:44.654 23913:24226 W/         ]
                                                                 [OMX.google.vorbis.decoder] could not get input buffer #1
08-24 11:31:44.654 23913-24226/com.rosapp.Numbers E/libOpenSLES: MediaSource::read returned error -2147483648

and later on:

08-24 11:32:00.593 23913-23943/com.rosapp.Numbers E/cocos2d-x debug info: Oops, player is null ...
08-24 11:32:03.369 23913-23943/com.rosapp.Numbers W/AudioPlayerProvider: Couldn't find the pcm cache: (assets/sounds_ogg/lector/avatar_photo.ogg)

dumganhar · 2016-08-24T09:41:01Z

@piotrros , it seems that there was an OpenSLES error while decoding audio to PCM buffer.

 E/libOpenSLES: MediaSource::read returned error -2147483648

Did it happen only on Nexus 6p Android 7.0? How about other devices?

piotrros · 2016-08-24T10:21:19Z

Today for testing I only have Nexus 6P and old lenovo tablet with android 4.4.4, which I mentioned before.

That's strange that it can't decode .ogg files, because as for as I know it should be the most compatible audio format. That's why I use it.

piotrros · 2016-08-25T06:13:01Z

I've uploaded modified cocos2d-x 3.11.1 here: https://drive.google.com/file/d/0ByXjTjdhFfLAZXhWdW5qRTdWYnM/view?usp=sharing

It includes this new audio engine as well as performance bugfix from 3.12 and few minor changes.

dumganhar · 2016-08-25T06:44:43Z

Ogg should be supported by Android. I don't know why decoder doesn't love it....
Anyway, I could not reproduce the issue you got.

ghost · 2016-10-06T12:33:13Z

Hello,
I have similar issue to that of piotrros.
I am using cocos v3.13, and have no issues on my test devices which are not very recent (I have an HTC phone running android 4.0.4 and a samsung one running android 4.1.2). With these devices, my game is running at about 60fps with many audio playing (they are all preloaded).
However, my friend as a Nexus 5X with android 7.0 and each played sounds makes the game freeze. If I remove all calls to play2d(), there is no freeze anymore, so I am sure the issue si coming from the newaudio engine.
I have no access to the log file since it is not my phone, but all my sounds are ogg files so it should be the same issue as for @piotrros.

Do you have any idea of how to fix this ?

Thanks

ghost · 2016-11-02T08:11:10Z

No news on this issue ? @dumganhar @piotrros

piotrros · 2016-11-02T08:28:28Z

No news.. I just went back to the old cocos2d::experimental::AudioEngine and it's working fine.

hgokturk · 2018-01-22T12:58:35Z

3.16 audio effect with preload plays with delay both for AudioEngine & SimpleAudioEngine. Tested on emulator & LG G5 Android 7. Ndk 16b, 13b, 11c. Target & compile sdk 19, min. sdk 15. Tried wav, mp3, ogg.

dumganhar changed the title ~~Audio latency fix for Android. Support to preload effects on Android now.~~ [big refactoring] Audio latency fix for Android. Support to preload effects on Android now. Jun 17, 2016

dumganhar force-pushed the audio-latency-fix branch from 1113e6a to a03be61 Compare June 17, 2016 09:56

dumganhar mentioned this pull request Jun 17, 2016

Android rendering performance issue #15652

Closed

j-jorge reviewed Jun 21, 2016
View reviewed changes

dumganhar force-pushed the audio-latency-fix branch from a49585f to 54675b6 Compare June 23, 2016 13:01

alfogrillo mentioned this pull request Jul 27, 2016

Crashed with Android 4.4.2 #16229

Closed

[big refactoring] Audio latency fix for Android. Support to preload effects on Android now. #15875

[big refactoring] Audio latency fix for Android. Support to preload effects on Android now. #15875

Conversation

dumganhar commented Jun 17, 2016 • edited Loading

xpol commented Jun 19, 2016

dumganhar commented Jun 19, 2016 • edited Loading

xpol commented Jun 20, 2016

dumganhar commented Jun 20, 2016 • edited Loading

minggo commented Jun 20, 2016

dumganhar commented Jun 20, 2016

minggo commented Jun 20, 2016

dumganhar commented Jun 20, 2016

minggo commented Jun 20, 2016

xpol commented Jun 20, 2016

dumganhar commented Jun 21, 2016

dumganhar commented Jun 21, 2016 • edited Loading

j-jorge Jun 21, 2016

Choose a reason for hiding this comment

dumganhar Jun 22, 2016

Choose a reason for hiding this comment

alfogrillo commented Jun 22, 2016

dumganhar commented Jun 22, 2016

alfogrillo commented Jun 22, 2016 • edited Loading

dumganhar commented Jun 22, 2016 • edited Loading

alfogrillo commented Jun 22, 2016

dumganhar commented Jun 22, 2016

alfogrillo commented Jun 22, 2016

alfogrillo commented Jun 23, 2016 • edited Loading

dumganhar commented Jun 23, 2016 • edited Loading

dumganhar commented Jun 23, 2016

alfogrillo commented Jul 21, 2016

dumganhar commented Jul 21, 2016 • edited Loading

alfogrillo commented Jul 21, 2016

ryanxia2016 commented Jul 21, 2016 • edited Loading

dumganhar commented Jul 22, 2016

dumganhar commented Jul 22, 2016

ryanxia2016 commented Jul 22, 2016

dumganhar commented Jul 22, 2016

dumganhar commented Jul 22, 2016

alfogrillo commented Jul 22, 2016 • edited Loading

Obg1 commented Jul 24, 2016

superman-t commented Jul 25, 2016

dumganhar commented Jul 25, 2016

superman-t commented Jul 25, 2016

dumganhar commented Jul 25, 2016 • edited Loading

superman-t commented Jul 25, 2016

alfogrillo commented Jul 28, 2016

piotrros commented Aug 24, 2016

alfogrillo commented Aug 24, 2016

dumganhar commented Aug 24, 2016

piotrros commented Aug 24, 2016

dumganhar commented Aug 24, 2016

piotrros commented Aug 24, 2016

piotrros commented Aug 25, 2016

dumganhar commented Aug 25, 2016

ghost commented Oct 6, 2016

ghost commented Nov 2, 2016

piotrros commented Nov 2, 2016

hgokturk commented Jan 22, 2018

dumganhar commented Jun 17, 2016 •

edited

Loading

dumganhar commented Jun 19, 2016 •

edited

Loading

dumganhar commented Jun 20, 2016 •

edited

Loading

dumganhar commented Jun 21, 2016 •

edited

Loading

alfogrillo commented Jun 22, 2016 •

edited

Loading

dumganhar commented Jun 22, 2016 •

edited

Loading

alfogrillo commented Jun 23, 2016 •

edited

Loading

dumganhar commented Jun 23, 2016 •

edited

Loading

dumganhar commented Jul 21, 2016 •

edited

Loading

ryanxia2016 commented Jul 21, 2016 •

edited

Loading

alfogrillo commented Jul 22, 2016 •

edited

Loading

dumganhar commented Jul 25, 2016 •

edited

Loading