Analyse CWB Query data issues

[niketchhajed]

The following cwb query data issues have been analysed so far:

1. For some of the network/station/channel combinations and for a particular time window, the data being returned just blows up. For example, for the below query string:

java -jar -Xmx1600m ~/CWBQuery/CWBQuery.jar -h 54.153.144.205 -t dcc -s "IUPET..BH200" -b "2015/03/01 00:00:00" -d 31d

The output generated is:

centos@proc:~/miniseed$ java -jar -Xmx1600m ~/CWBQuery/CWBQuery.jar -h 54.153.144.205 -t dcc -s "IUPET..BH200" -b "2015/03/01 00:00:00" -d 31d
04:13:25 Query on IUPET BH200 184868 mini-seed blks 2015 060:00:00:00.0499 2015 090:00:00:00.000 ns=51312360 #dups=8
04:15:25.869 Thread-2 ReadTimeout went off. close the socket 120602 waitlen=120000 sock.isCLosed()=false loop=162

And the size of miniseed file being generated just blows up. It reaches sizes of 100G plus. If I abort this query, the returned file's size turns to zero after sometime so I cannot analyse it either. Is this a known issue for some network/station/channel combinations?

2. For some net/sta/cha combinations, the number of blocks for a given time period is higher than that of others. Its possible that the sampling frequency for those combinations is higher than that of others. Can we query the sampling frequency of such combinations? so we can better manage the query by diving into appropriate time periods. Otherwise, the response for such combinations takes too long (sometimes half an hour) and is returned in batches which makes it difficult to manage. Or if you know some pertinent information, please do share.

3. On querying CWB for a given time period, the response miniseed files have data that overflows the requested time window. For e.g. when I query as below:

centos@proc:~/miniseed$ java -jar -Xmx1600m /CWBQuery/CWBQuery.jar -h 54.153.144.205 -t dcc -s "GEFAKI.BHZ" -b "2015/03/15 10:00:00" -d 7200
04:34:01 Query on GEFAKI BHZ 000299 mini-seed blks 2015 074:09:59:42.8695 2015 074:12:00:15.370 ns=144651 #dups=0
299 Total blocks transferred in 463 ms 645 b/s 0 #dups=0
centos@proc:/miniseed$ ls -la
total 53424
drwxrwxr-x 2 centos centos 126 Sep 19 04:34 .
drwx------. 7 centos centos 270 Sep 19 04:14 ..
-rw-rw-r-- 1 centos centos 139264 Sep 19 04:34 GEFAKI_BHZ__.msd

And when I look at the starttime and endtime for this miniseed:

In [1]: from obspy.core import read
In [2]: st = read('GEFAKI_BHZ__.msd')
In [3]: tr = st[0]
In [4]: tr.stats
Out[4]:
network: GE
station: FAKI
location:
channel: BHZ
starttime: 2015-03-15T09:59:42.869538Z
endtime: 2015-03-15T12:00:15.369538Z
sampling_rate: 20.0
delta: 0.05
npts: 144651
calib: 1.0
_format: MSEED
mseed: AttribDict({'record_length': 4096, 'encoding': u'STEIM2', 'filesize': 139264, u'dataquality': u'D', 'number_of_records': 34, 'byteorder': u'>'})