Seedlink and multi threads

[sbonaime]

I have a problem when I use seedlink clients in threads.
Here is my code:

from obspy.seedlink.slclient import SLClient
import threading
from obspy.core.utcdatetime import UTCDateTime

if __name__ == '__main__':

    #the first SLClient()
    slClient_1 = SLClient(loglevel='DEBUG')
    slClient_1.slconn.setSLAddress("rtserver.ipgp.fr:18000")
    slClient_1.multiselect = ("G_SSB:BHN")
    dt = UTCDateTime()
    slClient_1.begin_time = (dt - 35000).formatSeedLink()
    print slClient_1.end_time
    slClient_1.verbose = 2
    slClient_1.initialize()

    #the second SLClient()
    slClient_2 = SLClient(loglevel='DEBUG')
    slClient_2.slconn.setSLAddress("rtserver.ipgp.fr:18000")
    slClient_2.multiselect =  ("G_TAM:BHZ")
    dt = UTCDateTime()
    slClient_2.begin_time = (dt - 35000).formatSeedLink()
    print slClient_2.end_time
    slClient_2.verbose = 2
    slClient_2.initialize()

    # start slClient_1 in a thread
    thread_1 = threading.Thread(target=slClient_1.run)
    thread_1.start()

    # start slClient_2 in a thread
    thread_2 = threading.Thread(target=slClient_2.run)
    thread_2.start()

With just one thread, I don't have any problem, but with at least two, I have the following errors and warnings

Warning: msr_unpack_steim2(G_SSB_10_BHN_D): number of samples indicated in header (408) does not equal data (406)
G_SSB_10_BHN_D: Warning: Data integrity check for Steim-2 failed, last_data=4077, xn=3199
ERROR: obspy.seedlink [rtserver.ipgp.fr:18000]:unexpected data received: G SSB
Warning: msr_unpack_steim2(G_SSB_10_BHN_D): number of samples indicated in header (408) does not equal data (406)
G_SSB_10_BHN_D: Warning: Data integrity check for Steim-2 failed, last_data=4077, xn=3199

with Mac OS 10.6.8, latest obspy dev version, python 2.7.4

on linux Opensuse , python 2.7.4:

ERROR: obspy.seedlink [rtserver.ipgp.fr:18000]:unexpected data received: G TAM
ERROR: obspy.seedlink [rtserver.ipgp.fr:18000]:unexpected data received: G TAM
G_TAM_00_BHZ_D: Warning: Data integrity check for Steim-2 failed, last_data=8217, xn=1149
ERROR: obspy.seedlink [rtserver.ipgp.fr:18000]:unexpected data received: G TAM
G_TAM_00_BHZ_D: Warning: Data integrity check for Steim-2 failed, last_data=8217, xn=1149
ERROR: obspy.seedlink [rtserver.ipgp.fr:18000]:unexpected data received: G TAM
G_TAM_00_BHZ_D: Warning: Data integrity check for Steim-2 failed, last_data=13628, xn=1149
G_TAM_00_BHZ_D: Warning: Data integrity check for Steim-2 failed, last_data=13628, xn=1149
ERROR: obspy.seedlink [rtserver.ipgp.fr:18000]:unexpected data received: G SSB
G_TAM_00_BHZ_D: Warning: Data integrity check for Steim-2 failed, last_data=13628, xn=1149
ERROR: obspy.seedlink [rtserver.ipgp.fr:18000]:unexpected data received: G TAM
ERROR: obspy.seedlink [rtserver.ipgp.fr:18000]:unexpected data received: G SSB
Warning: msr_unpack_steim2(G_SSB_00_BHN_D): number of samples indicated in header (343) does not equal data (333)
G_SSB_00_BHN_D: Warning: Data integrity check for Steim-2 failed, last_data=272, xn=2212
ERROR: obspy.seedlink [rtserver.ipgp.fr:18000]:unexpected data received: G SSB
Warning: msr_unpack_steim2(G_SSB_00_BHN_D): number of samples indicated in header (343) does not equal data (333)
G_SSB_00_BHN_D: Warning: Data integrity check for Steim-2 failed, last_data=272, xn=2212
Warning: msr_unpack_steim2(G_SSB_00_BHN_D): number of samples indicated in header (343) does not equal data (333)

After the backtrace has been recovered, the errors seems to disappear. Most of the first paquets are corrupted and not usable

When I use two different seedlink server (not in the same place ), I have less errors and warnings.

Maybe the threads share the same receiving buffer which is maybe too small and/or not updated quickly enough

WIth just one thread I don't have any problem

[megies]

I think it is usually not necessary to connect to one seedlink server with more than one thread. Fetching multiple stations/channels can be handled in one client (see also sbonaime/seedlink_plotter#6). If multiple clients in threads connecting to different servers is not a problem (as I understood), then fixing this / looking into it is probably pretty low priority.

[bmorg]

Hi!

The anomalous thread behaviour is related to how the SeedLink connection object stores the retrieved data internally.

The data is stored in two bytearrays inside an SLState object (databuf and packed_buf). These bytearrays are however initialized as class attributes and hence shared between all instances of SLState (and thus SeedLinkConnection).

This is fixed in the commit referenced above.

@bonaime: Your code runs fine for me now. Can you confirm this with the fix_seedlink_multithread branch in my ObsPy fork?

[sbonaime]

Hi
@bmorg: Your fix runs fine with my code.
Thank you !

[bmorg]

@bonaime: Thank you for testing. I opened a new pull request.

[megies]

Fixed in #769.