Skip to content
This repository

RSP race during connection #103

Closed
dulley opened this Issue March 16, 2012 · 3 comments

2 participants

Lucas Peetz Dulley Stefan Eilemann
Lucas Peetz Dulley
Collaborator

eqPly clients segfault when 30+ clients (1 client per node) are pre-launched sequencially in multicast (RSP) mode.

Launch Script:

for node in `cat nodesList.txt`; do
  echo $node;
  ssh $node -f ~/bin/eqPly --eq-client --eq-listen $node:22345 --eq-listen RSP\#102400\#239.255.42.42\#$node\#11147\#\# --eq-logfile ~/logs/eqply.$node.log;
  usleep 100000; 
done

It seems that the RSP and Receiver threads are competing when _findConnection( id ) is called.
When _findConnection is called from RSPConnection::_addNewConnection[receiver thread]), there is no lock preventing _children of being accessed when a new connection is being pushed in RSPConnection::acceptSync()

Possible Fix:

diff --git a/libs/co/rspConnection.cpp b/libs/co/rspConnection.cpp
index c0b5989..c7c4074 100644
--- a/libs/co/rspConnection.cpp
+++ b/libs/co/rspConnection.cpp
@@ -1397,9 +1397,6 @@ RSPConnection::RSPConnectionPtr RSPConnection::_findConnection(

 bool RSPConnection::_addNewConnection( const uint16_t id )
 {
-    if( _findConnection( id ).isValid() )
-        return false;
-
     base::ScopedWrite mutex( _mutexConnection );
     if( _findConnection( id ).isValid( ))
         return false;
Lucas Peetz Dulley
Collaborator

Log for crashed client

23431  /srv/hudson-slave/workspace/Equalizer_Linux/libs/eq/client/init.cpp:60 2 Equalizer v1.2 (9ec4fca022f73871ab6e97704e17b8ffcedd1b59) initializing
23431 Main /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/base/init.cpp:50 3 Log level INFO topics 64
23431 Main /srv/hudson-slave/workspace/Equalizer_Linux/libs/eq/client/client.cpp:71 10 New client at 0xc755450
23431 Main /srv/hudson-slave/workspace/Equalizer_Linux/libs/eq/client/client.cpp:232 10 Launching 999a6302b03faf82:d212f69958039d26
23431 Main /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/socketConnection.cpp:693 12 Listening on n122002[10.129.3.201]:22345 (TCPIP#102400#n122002##22345#default#)
23431 Main /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 13 New RSP connection, 64 buffers of 65000 bytes
23431 Main /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:259 13 Joining 239.255.42.42 on 10.129.3.201
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:488 26 Started RSP protocol thread
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:492 26 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 36 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 46 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 56 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 66 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 76 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 86 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 96 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 106 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 116 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 126 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 136 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 146 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 156 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 166 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 176 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 186 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 196 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 206 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:427 216 Announce 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:432 226 Confirm 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:433 226 opened RSP connection 30689
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 226 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 534 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 849 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 1164 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 1474 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 1785 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 2096 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 2406 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 2713 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 3021 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 3330 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 3641 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 3948 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 4257 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 4564 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 4877 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 5184 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 5495 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 5806 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 6113 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6304 Got 29 nodes from 36042
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 6304 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6305 Got 26 nodes from 24794
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 6305 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6305 Got 5 nodes from 4549
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6305 Got 7 nodes from 22490
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6305 Got 8 nodes from 22490
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6305 Got 29 nodes from 48112
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 6306 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 14 nodes from 39971
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 6 nodes from 4549
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 7 nodes from 4549
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 15 nodes from 39971
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 27 nodes from 24794
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 9 nodes from 22490
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 10 nodes from 22490
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 30 nodes from 35974
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 6306 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 8 nodes from 4549
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 9 nodes from 4549
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 11 nodes from 22490
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 28 nodes from 24794
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 16 nodes from 39971
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 30 nodes from 48112
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 30 nodes from 36042
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 20 nodes from 50849
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6306 Got 29 nodes from 4468
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 6307 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 21 nodes from 1760
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 10 nodes from 4549
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 12 nodes from 22490
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 21 nodes from 50849
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 22 nodes from 1760
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 17 nodes from 39971
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 8 nodes from 33102
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 29 nodes from 24794
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 9 nodes from 36210
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 22 nodes from 50849
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 13 nodes from 22490
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 24 nodes from 42258
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 11 nodes from 4549
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 12 nodes from 4549
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 10 nodes from 36210
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 13 nodes from 4549
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 9 nodes from 33102
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 14 nodes from 22490
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 23 nodes from 1760
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6307 Got 23 nodes from 50849
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 18 nodes from 39971
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 19 nodes from 39971
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 14 nodes from 4549
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 11 nodes from 36210
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 27 nodes from 28857
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:107 6308 New RSP connection, 64 buffers of 65000 bytes
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 10 nodes from 33102
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/base/thread.cpp:92 6308 Thread co::RSPConnection::Thread successfully initialized
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 15 nodes from 22490
23431 Main /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:299 6308 Listening on 239.255.42.42:11147 (RSP#102400#239.255.42.42#n122002#11147## @0xc759890)
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 20 nodes from 39971
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 12 nodes from 36210
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 24 nodes from 1760
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 15 nodes from 4549
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 24 nodes from 50849
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 16 nodes from 33723
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 25 nodes from 42258
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 15 nodes from 37458
23431 Cmd eqPly:: /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/base/thread.cpp:92 6308 Thread co::LocalNode::CommandThread successfully initialized
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 16 nodes from 22490
23431 Rcv eqPly:: /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/base/thread.cpp:92 6308 Thread co::LocalNode::ReceiverThread successfully initialized
23431 Main /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/localNode.cpp:215 6308 node 999a6302b03faf82:d212f69958039d26 listening, TCPIP#102400#n122002##22345#default#, RSP#102400#239.255.42.42#n122002#11147##
23431 Main /srv/hudson-slave/workspace/Equalizer_Linux/libs/eq/client/client.cpp:301 6308 Entered client loop
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 17 nodes from 33723
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 13 nodes from 36210
23431 Rcv eqPly:: /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:321 6308 30689 accepted RSP connection 28857
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 21 nodes from 39971
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 22 nodes from 30196
23431 Rcv eqPly:: /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:321 6308 30689 accepted RSP connection 4468
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 16 nodes from 4549
23431 Rcv eqPly:: /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:321 6308 30689 accepted RSP connection 35974
23431 Rcv eqPly:: /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:321 6308 30689 accepted RSP connection 48112
23431 co::RSPConn /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:1368 6308 Got 23 nodes from 17951
23431 Rcv eqPly:: /srv/hudson-slave/workspace/Equalizer_Linux/libs/co/rspConnection.cpp:321 6308 30689 accepted RSP connection 24794
Stefan Eilemann
Owner
eile commented March 17, 2012

Good analysis! This is indeed a race. The real fix is to not have the app thread modify the _child connections vector. I'll look into it.

Stefan Eilemann eile referenced this issue from a commit March 17, 2012
Stefan Eilemann Untested fix for #103: RSP race during connection [UNTESTED]
Only modify _children from the RSP thread. New connections are still
stored in _newChildren for accepting them from the app, but are directly
added and activated in the _children vector. Clean up naming somewhat.

Bump version to 1.2.1

[ ] May break build
[ ] May break existing applications (see CHANGES.txt)
[x] Bugfix
[ ] New Feature
[x] Cleanup
[ ] Optimization
[ ] Documentation
50dc80f
Stefan Eilemann eile referenced this issue from a commit March 19, 2012
Stefan Eilemann Fix for #103 fix
[ ] May break build
[ ] May break existing applications (see CHANGES.txt)
[x] Bugfix
[ ] New Feature
[ ] Cleanup
[ ] Optimization
[ ] Documentation
74daf87
Stefan Eilemann eile referenced this issue from a commit March 19, 2012
Stefan Eilemann Fix for #103 fix
[ ] May break build
[ ] May break existing applications (see CHANGES.txt)
[x] Bugfix
[ ] New Feature
[ ] Cleanup
[ ] Optimization
[ ] Documentation
4496daa
Stefan Eilemann
Owner
eile commented March 20, 2012

Last commit nailed this one.

Stefan Eilemann eile closed this March 20, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.