New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix "out of buffers" crash #3783

Merged
merged 9 commits into from Sep 26, 2017

Conversation

Projects
None yet
4 participants
@lukas-w
Member

lukas-w commented Aug 30, 2017

Fixes #3223

Replaces BufferManager's implementation with simple calls to `MemoryManager::alloc. Re-use buffers where they are allocated (AudioPort.cpp & PlayHandle.cpp). I don't experience any performance regression from this change.

My initial approach to fix #3223 was to re-implement BufferManager using a lock-free stack, see b6421d4. However, I couldn't get Travis to build with it, and I realized BufferManager::acquire is only used in two places that can be optimized easily.

lukas-w added some commits Aug 29, 2017

Reimplement BufferManager
Re-implement BufferManager using a single lock-free stack instead of two
fixed-size arrays. This has the following advantages:
 * Less code
 * Immediate re-use of released buffers, no blocking "refresh" required
 * Dynamic size, no crash when out of buffers (fixes #3670)
Remove BufferManager implementation
Use MemoryManager allocation instead and re-use buffers where they are
allocated (AudioPort.cpp & PlayHandle.cpp)
Revert "Find Boost in CMake, fix Travis"
Boost not used anymore.

This reverts commit 6b557d2.

@lukas-w lukas-w changed the title from Remove BufferManager implementation to Fix "out of buffers" crash Sep 1, 2017

@Umcaruje

In general, there is a lot of inconsistency with the spaces in the brackets, I vote they all get removed because spaces are not a coding convention anymore.

Show outdated Hide outdated include/MemoryHelper.h
Show outdated Hide outdated include/MemoryManager.h

@tresf tresf added this to the 1.2.0 milestone Sep 11, 2017

@lukas-w

This comment has been minimized.

Show comment
Hide comment
@lukas-w

lukas-w Sep 17, 2017

Member

@Umcaruje I fixed some inconsistencies. However, I'd like to avoid touching any code just for the sake of fixing code format, because it makes reviewing commit history (especially using git blame) a lot harder.

Member

lukas-w commented Sep 17, 2017

@Umcaruje I fixed some inconsistencies. However, I'd like to avoid touching any code just for the sake of fixing code format, because it makes reviewing commit history (especially using git blame) a lot harder.

Fix C++03 compatibility again
Same as 73728d7, accidentally removed
space in aebbe22
@Umcaruje

This comment has been minimized.

Show comment
Hide comment
@Umcaruje

Umcaruje Sep 17, 2017

Member

The original project from the bug report still crashes for me with this PR. backtrace:

Thread 11 "QThread" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff859a0700 (LWP 7287)]
0x0000000000579116 in QGenericAtomicOps<QBasicAtomicOps<8> >::storeRelease<ThreadableJob*, ThreadableJob*> (newValue=0x7fffe7ff8f48, _q_value=@0xffffffff40998c60: <error reading variable>)
    at /usr/include/x86_64-linux-gnu/qt5/QtCore/qgenericatomic.h:111
111	        *static_cast<volatile T *>(&_q_value) = newValue;
(gdb) bt full
#0  0x0000000000579116 in QGenericAtomicOps<QBasicAtomicOps<8> >::storeRelease<ThreadableJob*, ThreadableJob*> (newValue=0x7fffe7ff8f48, _q_value=@0xffffffff40998c60: <error reading variable>)
    at /usr/include/x86_64-linux-gnu/qt5/QtCore/qgenericatomic.h:111
No locals.
#1  QBasicAtomicPointer<ThreadableJob>::storeRelease (this=0xffffffff40998c60, newValue=0x7fffe7ff8f48) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qbasicatomic.h:259
No locals.
#2  0x0000000000578f14 in QAtomicPointer<ThreadableJob>::operator= (this=0xffffffff40998c60, other=...) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qatomic.h:178
No locals.
#3  0x0000000000578968 in MixerWorkerThread::JobQueue::addJob (this=0x9d1a60 <MixerWorkerThread::globalJobQueue>, _job=0x7fffe7ff8f48) at /home/umcaruje/github/lmmstest/src/core/MixerWorkerThread.cpp:57
No locals.
#4  0x0000000000555538 in MixerWorkerThread::addJob (_job=0x7fffe7ff8f48) at /home/umcaruje/github/lmmstest/include/MixerWorkerThread.h:87
No locals.
#5  0x00000000005753fa in MixerWorkerThread::fillJobQueue<QList<PlayHandle*> > (_vec=..., _opMode=MixerWorkerThread::JobQueue::Static) at /home/umcaruje/github/lmmstest/include/MixerWorkerThread.h:99
        it = {i = 0x7fff7806cf78}
#6  0x0000000000571fe5 in Mixer::renderNextBuffer (this=0xba56c0) at /home/umcaruje/github/lmmstest/src/core/Mixer.cpp:438
        last_metro_pos = {<MidiTime> = {m_ticks = -1, static s_ticksPerTact = 192}, m_timeLine = 0x0, m_currentFrame = 0}
        song = 0xe2d990
        currentPlayMode = Song::Mode_PlaySong
        p = {<MidiTime> = {m_ticks = 8212, static s_ticksPerTact = 192}, m_timeLine = 0x2a9df60, m_currentFrame = 312.87793}
        playModeSupportsMetronome = true
        it_rem = {i = 0x7fff780013b0}
        fxMixer = 0xea2800
#7  0x0000000000573c3f in Mixer::fifoWriter::run (this=0xb1f3e0) at /home/umcaruje/github/lmmstest/src/core/Mixer.cpp:1102
        buffer = 0x7fff780257e0
        b = 0xe0ce10
        frames = 256
#8  0x00007ffff4b0c7be in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#9  0x00007ffff7bc16ba in start_thread (arg=0x7fff859a0700) at pthread_create.c:333
        __res = <optimized out>
        pd = 0x7fff859a0700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140735434852096, 8462573384777015085, 1, 140737488344511, 140735434852800, 11662208, -8462806189127819475, -8462555276406087891}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#10 0x00007ffff44223dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
No locals.
Member

Umcaruje commented Sep 17, 2017

The original project from the bug report still crashes for me with this PR. backtrace:

Thread 11 "QThread" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff859a0700 (LWP 7287)]
0x0000000000579116 in QGenericAtomicOps<QBasicAtomicOps<8> >::storeRelease<ThreadableJob*, ThreadableJob*> (newValue=0x7fffe7ff8f48, _q_value=@0xffffffff40998c60: <error reading variable>)
    at /usr/include/x86_64-linux-gnu/qt5/QtCore/qgenericatomic.h:111
111	        *static_cast<volatile T *>(&_q_value) = newValue;
(gdb) bt full
#0  0x0000000000579116 in QGenericAtomicOps<QBasicAtomicOps<8> >::storeRelease<ThreadableJob*, ThreadableJob*> (newValue=0x7fffe7ff8f48, _q_value=@0xffffffff40998c60: <error reading variable>)
    at /usr/include/x86_64-linux-gnu/qt5/QtCore/qgenericatomic.h:111
No locals.
#1  QBasicAtomicPointer<ThreadableJob>::storeRelease (this=0xffffffff40998c60, newValue=0x7fffe7ff8f48) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qbasicatomic.h:259
No locals.
#2  0x0000000000578f14 in QAtomicPointer<ThreadableJob>::operator= (this=0xffffffff40998c60, other=...) at /usr/include/x86_64-linux-gnu/qt5/QtCore/qatomic.h:178
No locals.
#3  0x0000000000578968 in MixerWorkerThread::JobQueue::addJob (this=0x9d1a60 <MixerWorkerThread::globalJobQueue>, _job=0x7fffe7ff8f48) at /home/umcaruje/github/lmmstest/src/core/MixerWorkerThread.cpp:57
No locals.
#4  0x0000000000555538 in MixerWorkerThread::addJob (_job=0x7fffe7ff8f48) at /home/umcaruje/github/lmmstest/include/MixerWorkerThread.h:87
No locals.
#5  0x00000000005753fa in MixerWorkerThread::fillJobQueue<QList<PlayHandle*> > (_vec=..., _opMode=MixerWorkerThread::JobQueue::Static) at /home/umcaruje/github/lmmstest/include/MixerWorkerThread.h:99
        it = {i = 0x7fff7806cf78}
#6  0x0000000000571fe5 in Mixer::renderNextBuffer (this=0xba56c0) at /home/umcaruje/github/lmmstest/src/core/Mixer.cpp:438
        last_metro_pos = {<MidiTime> = {m_ticks = -1, static s_ticksPerTact = 192}, m_timeLine = 0x0, m_currentFrame = 0}
        song = 0xe2d990
        currentPlayMode = Song::Mode_PlaySong
        p = {<MidiTime> = {m_ticks = 8212, static s_ticksPerTact = 192}, m_timeLine = 0x2a9df60, m_currentFrame = 312.87793}
        playModeSupportsMetronome = true
        it_rem = {i = 0x7fff780013b0}
        fxMixer = 0xea2800
#7  0x0000000000573c3f in Mixer::fifoWriter::run (this=0xb1f3e0) at /home/umcaruje/github/lmmstest/src/core/Mixer.cpp:1102
        buffer = 0x7fff780257e0
        b = 0xe0ce10
        frames = 256
#8  0x00007ffff4b0c7be in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
No symbol table info available.
#9  0x00007ffff7bc16ba in start_thread (arg=0x7fff859a0700) at pthread_create.c:333
        __res = <optimized out>
        pd = 0x7fff859a0700
        now = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140735434852096, 8462573384777015085, 1, 140737488344511, 140735434852800, 11662208, -8462806189127819475, -8462555276406087891}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 
              0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
        pagesize_m1 = <optimized out>
        sp = <optimized out>
        freesize = <optimized out>
        __PRETTY_FUNCTION__ = "start_thread"
#10 0x00007ffff44223dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
No locals.
@lukas-w

This comment has been minimized.

Show comment
Hide comment
@lukas-w

lukas-w Sep 17, 2017

Member

@Umcaruje That's an unrelated crash (job queue is full), see #777 (comment).

Member

lukas-w commented Sep 17, 2017

@Umcaruje That's an unrelated crash (job queue is full), see #777 (comment).

@Umcaruje

This comment has been minimized.

Show comment
Hide comment
@Umcaruje

Umcaruje Sep 17, 2017

Member

Ah, ok, good, this fixes the out of buffers issue and I can't seem to pick up any issues with it. This is ready to merge.

Member

Umcaruje commented Sep 17, 2017

Ah, ok, good, this fixes the out of buffers issue and I can't seem to pick up any issues with it. This is ready to merge.

@PhysSong

This comment has been minimized.

Show comment
Hide comment
@PhysSong

PhysSong Sep 19, 2017

Member

It seems to fix the issue clearly, anyway.

test.mmp.zip

This project file shows the bug clearly and it seems to be fixed by this PR.

Member

PhysSong commented Sep 19, 2017

It seems to fix the issue clearly, anyway.

test.mmp.zip

This project file shows the bug clearly and it seems to be fixed by this PR.

@PhysSong

This comment has been minimized.

Show comment
Hide comment
@PhysSong

PhysSong Sep 22, 2017

Member

@lukas-w I think it is ready to merge. However, I suggest
fixing inconsistent and unintentional things before you merge. Would this pach be fine?

Member

PhysSong commented Sep 22, 2017

@lukas-w I think it is ready to merge. However, I suggest
fixing inconsistent and unintentional things before you merge. Would this pach be fine?

@lukas-w

This comment has been minimized.

Show comment
Hide comment
@lukas-w

lukas-w Sep 22, 2017

Member

Would this pach be fine?

@PhysSong Sure. Feel free to apply it if you find the time before I do.

Member

lukas-w commented Sep 22, 2017

Would this pach be fine?

@PhysSong Sure. Feel free to apply it if you find the time before I do.

@lukas-w

This comment has been minimized.

Show comment
Hide comment
@lukas-w

lukas-w Sep 26, 2017

Member

Thanks @PhysSong. Merging.

Member

lukas-w commented Sep 26, 2017

Thanks @PhysSong. Merging.

@lukas-w lukas-w merged commit f23cf4e into stable-1.2 Sep 26, 2017

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@lukas-w lukas-w deleted the fix/buffer-manager branch Sep 26, 2017

@lukas-w

This comment has been minimized.

Show comment
Hide comment
@lukas-w

lukas-w Sep 29, 2017

Member

Commit message says "fixes #3670", but should say "#3223" instead.

Member

lukas-w commented on b6421d4 Sep 29, 2017

Commit message says "fixes #3670", but should say "#3223" instead.

@@ -60,6 +65,10 @@ void PlayHandle::doProcessing()
void PlayHandle::releaseBuffer()
{
BufferManager::release( m_playHandleBuffer );

This comment has been minimized.

@PhysSong

PhysSong Oct 15, 2017

Member

@lukas-w Why did you move this to destructor?

@PhysSong

PhysSong Oct 15, 2017

Member

@lukas-w Why did you move this to destructor?

This comment has been minimized.

@lukas-w

lukas-w Oct 15, 2017

Member

@PhysSong Because I moved buffer allocation to the constructor. The buffer's lifetime matches the PlayHandle's lifetime now, instead of being allocated everytime doProcessing is called.

@lukas-w

lukas-w Oct 15, 2017

Member

@PhysSong Because I moved buffer allocation to the constructor. The buffer's lifetime matches the PlayHandle's lifetime now, instead of being allocated everytime doProcessing is called.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment