/usr/local/lib/python2.7/dist-packages/dask/context.py:23: UserWarning: The dask.set_options function has been deprecated. Please use dask.config.set instead warnings.warn("The dask.set_options function has been deprecated. " Using 'pyrap' casacore binding to produce MS Extract MS for spw 0: central frequency 1284.00 MHz Will create MS output in 1536252666.ms #### Producing a full polarisation MS (HH,HV,VH,VV) #### Cross-correlations only. Using m000 as the reference antenna. All targets and activity detection will be based on this antenna. Iterating through scans in file(s)... Writing static meta data... scan 38 ( 2 samples) loaded. Target: 'J2129+0528'. Writing to disk... Wrote scan data (499.179222 MiB) in 10.151202 s (49.174396 MiBps) scan 64 ( 4 samples) loaded. Target: 'J0841-7540'. Writing to disk... Wrote scan data (998.358444 MiB) in 14.546914 s (68.630256 MiBps) scan 66 ( 7 samples) loaded. Target: 'J0833-7540'. Writing to disk... Wrote scan data (1747.127277 MiB) in 24.877881 s (70.228139 MiBps) scan 68 ( 7 samples) loaded. Target: 'J0845-7606'. Writing to disk... Wrote scan data (1747.127277 MiB) in 21.799393 s (80.145685 MiBps) scan 130 ( 3 samples) loaded. Target: 'J2157-6941'. Writing to disk... Wrote scan data (748.768833 MiB) in 9.935242 s (75.364930 MiBps) scan 132 ( 5 samples) loaded. Target: 'J2151-6941'. Writing to disk... Wrote scan data (1247.948055 MiB) in 17.229203 s (72.432141 MiBps) scan 139 ( 2 samples) loaded. Target: 'J0405-6519'. Writing to disk... Wrote scan data (499.179222 MiB) in 7.343559 s (67.975111 MiBps) scan 141 ( 7 samples) loaded. Target: 'J0413-6545'. Writing to disk... Wrote scan data (1747.127277 MiB) in 23.692395 s (73.742113 MiBps) scan 143 ( 6 samples) loaded. Target: 'J0408-6545'. Writing to disk... Wrote scan data (1497.537666 MiB) in 20.154647 s (74.302352 MiBps) scan 145 ( 7 samples) loaded. Target: 'J0403-6545'. Writing to disk... Wrote scan data (1747.127277 MiB) in 22.229525 s (78.594900 MiBps) scan 147 ( 7 samples) loaded. Target: 'J0410-6611'. Writing to disk... Wrote scan data (1747.127277 MiB) in 21.769590 s (80.255405 MiBps) scan 149 ( 7 samples) loaded. Target: 'J0405-6611'. Writing to disk... Wrote scan data (1747.127277 MiB) in 23.347110 s (74.832700 MiBps) scan 151 ( 4 samples) loaded. Target: 'J0012-4127'. Writing to disk... Wrote scan data (998.358444 MiB) in 13.283489 s (75.157848 MiBps) scan 153 ( 7 samples) loaded. Target: 'J0009-4127'. Writing to disk... Wrote scan data (1747.127277 MiB) in 23.601578 s (74.025867 MiBps) scan 155 ( 7 samples) loaded. Target: 'J0013-4153'. Writing to disk... Wrote scan data (1747.127277 MiB) in 23.601286 s (74.026782 MiBps) scan 157 ( 6 samples) loaded. Target: 'J0010-4153'. Writing to disk... Wrote scan data (1497.537666 MiB) in 18.796682 s (79.670320 MiBps) scan 159 ( 7 samples) loaded. Target: 'J0008-4153'. Writing to disk... Wrote scan data (1747.127277 MiB) in 23.874211 s (73.180524 MiBps) scan 161 ( 6 samples) loaded. Target: 'J0012-4219'. Writing to disk... Wrote scan data (1497.537666 MiB) in 19.244931 s (77.814655 MiBps) scan 163 ( 7 samples) loaded. Target: 'J0009-4219'. Writing to disk... Wrote scan data (1747.127277 MiB) in 22.308270 s (78.317471 MiBps) scan 165 ( 4 samples) loaded. Target: 'J0117-2026'. Writing to disk... Wrote scan data (998.358444 MiB) in 14.872306 s (67.128692 MiBps) scan 167 ( 7 samples) loaded. Target: 'J0115-2026'. Writing to disk... Wrote scan data (1747.127277 MiB) in 23.200225 s (75.306480 MiBps) scan 169 ( 7 samples) loaded. Target: 'J0118-2052'. Writing to disk... Wrote scan data (1747.127277 MiB) in 20.697213 s (84.413649 MiBps) scan 179 ( 6 samples) loaded. Target: 'J1821-0912'. Writing to disk... Wrote scan data (1497.537666 MiB) in 19.559459 s (76.563348 MiBps) scan 191 ( 3 samples) loaded. Target: 'J1924+3329'. Writing to disk... Wrote scan data (748.768833 MiB) in 9.683845 s (77.321439 MiBps) scan 193 ( 7 samples) loaded. Target: 'J1921+3329'. Writing to disk... Wrote scan data (1747.127277 MiB) in 23.671299 s (73.807833 MiBps) scan 195 ( 3 samples) loaded. Target: 'J1925+3303'. Writing to disk... Wrote scan data (748.768833 MiB) in 9.475906 s (79.018178 MiBps) scan 200 ( 7 samples) loaded. Target: 'J2131+0528'. Writing to disk... Wrote scan data (1747.127277 MiB) in 22.650845 s (77.132984 MiBps) scan 202 ( 6 samples) loaded. Target: 'J2129+0528'. Writing to disk... Wrote scan data (1497.537666 MiB) in 19.833935 s (75.503811 MiBps) scan 204 ( 7 samples) loaded. Target: 'J2132+0502'. Writing to disk... Wrote scan data (1747.127277 MiB) in 24.426505 s (71.525880 MiBps) scan 206 ( 6 samples) loaded. Target: 'J2130+0502'. Writing to disk... Wrote scan data (1497.537666 MiB) in 19.755460 s (75.803735 MiBps) scan 208 ( 7 samples) loaded. Target: 'J2128+0502'. Writing to disk... Wrote scan data (1747.127277 MiB) in 24.439050 s (71.489165 MiBps) scan 210 ( 7 samples) loaded. Target: 'J2131+0436'. Writing to disk... Wrote scan data (1747.127277 MiB) in 26.138954 s (66.839984 MiBps) scan 212 ( 7 samples) loaded. Target: 'J2129+0436'. Writing to disk... Wrote scan data (1747.127277 MiB) in 24.689381 s (70.764320 MiBps) scan 214 ( 6 samples) loaded. Target: 'J2137+0107'. Writing to disk... Wrote scan data (1497.537666 MiB) in 21.432409 s (69.872578 MiBps) scan 230 ( 4 samples) loaded. Target: 'J2138-2108'. Writing to disk... Wrote scan data (998.358444 MiB) in 13.582469 s (73.503458 MiBps) scan 232 ( 7 samples) loaded. Target: 'J2136-2108'. Writing to disk... Wrote scan data (1747.127277 MiB) in 23.647790 s (73.881208 MiBps) scan 235 ( 7 samples) loaded. Target: 'J0837-7514'. Writing to disk... Wrote scan data (1747.127277 MiB) in 23.528235 s (74.256623 MiBps) scan 241 ( 6 samples) loaded. Target: 'J0845-7606'. Writing to disk... Wrote scan data (1497.537666 MiB) in 20.349738 s (73.590022 MiBps) scan 248 ( 3 samples) loaded. Target: 'J1734-5632'. Writing to disk... Wrote scan data (748.768833 MiB) in 10.344520 s (72.383140 MiBps) scan 250 ( 7 samples) loaded. Target: 'J1739-5658'. Writing to disk... Wrote scan data (1747.127277 MiB) in 24.664167 s (70.836663 MiBps) scan 263 ( 4 samples) loaded. Target: 'J0258-7104'. Writing to disk... Wrote scan data (998.358444 MiB) in 13.787020 s (72.412924 MiBps) scan 292 ( 4 samples) loaded. Target: 'J1937-6408'. Writing to disk... Wrote scan data (998.358444 MiB) in 15.836254 s (63.042588 MiBps) scan 322 ( 4 samples) loaded. Target: 'J0408-6545'. Writing to disk... Wrote scan data (998.358444 MiB) in 14.728446 s (67.784371 MiBps) scan 339 ( 4 samples) loaded. Target: 'J0118-2052'. Writing to disk... Wrote scan data (998.358444 MiB) in 12.952528 s (77.078270 MiBps) scan 341 ( 7 samples) loaded. Target: 'J0116-2052'. Writing to disk... Wrote scan data (1747.127277 MiB) in 23.920723 s (73.038230 MiBps) scan 343 ( 7 samples) loaded. Target: 'J0114-2052'. Writing to disk... Wrote scan data (1747.127277 MiB) in 26.030900 s (67.117436 MiBps) scan 345 ( 6 samples) loaded. Target: 'J0117-2118'. Writing to disk... Wrote scan data (1497.537666 MiB) in 22.284780 s (67.200020 MiBps) scan 347 ( 7 samples) loaded. Target: 'J0115-2118'. Writing to disk... Wrote scan data (1747.127277 MiB) in 26.600027 s (65.681410 MiBps) scan 351 ( 9 samples) loaded. Target: 'J0135+3309'. Writing to disk... Wrote scan data (2246.306499 MiB) in 30.901088 s (72.693444 MiBps) scan 353 ( 6 samples) loaded. Target: 'J0138+3243'. Writing to disk... Wrote scan data (1497.537666 MiB) in 23.094830 s (64.842983 MiBps) scan 355 ( 7 samples) loaded. Target: 'J0136+3243'. Writing to disk... Wrote scan data (1747.127277 MiB) in 23.374981 s (74.743474 MiBps) scan 357 ( 6 samples) loaded. Target: 'J0205+1540'. Writing to disk... Wrote scan data (1497.537666 MiB) in 22.922397 s (65.330762 MiBps) scan 359 ( 6 samples) loaded. Target: 'J0203+1540'. Writing to disk... Wrote scan data (1497.537666 MiB) in 21.347636 s (70.150047 MiBps) scan 361 ( 7 samples) loaded. Target: 'J0206+1514'. Writing to disk... Wrote scan data (1747.127277 MiB) in 24.947355 s (70.032566 MiBps) scan 363 ( 7 samples) loaded. Target: 'J0204+1514'. Writing to disk... Wrote scan data (1747.127277 MiB) in 25.182043 s (69.379886 MiBps) scan 365 ( 7 samples) loaded. Target: 'J0202+1514'. Writing to disk... Wrote scan data (1747.127277 MiB) in 26.286824 s (66.463993 MiBps) scan 367 ( 6 samples) loaded. Target: 'J0205+1448'. Writing to disk... Wrote scan data (1497.537666 MiB) in 21.466948 s (69.760157 MiBps) scan 369 ( 7 samples) loaded. Target: 'J0203+1448'. Writing to disk... Wrote scan data (1747.127277 MiB) in 26.173250 s (66.752401 MiBps) scan 371 ( 7 samples) loaded. Target: 'J0241-2243'. Writing to disk... Wrote scan data (1747.127277 MiB) in 24.517854 s (71.259388 MiBps) scan 373 ( 6 samples) loaded. Target: 'J0239-2243'. Writing to disk... Wrote scan data (1497.537666 MiB) in 20.302791 s (73.760187 MiBps) scan 375 ( 8 samples) loaded. Target: 'J0242-2309'. Writing to disk... Wrote scan data (1996.716888 MiB) in 26.103013 s (76.493732 MiBps) scan 377 ( 6 samples) loaded. Target: 'J0240-2309'. Writing to disk... Wrote scan data (1497.537666 MiB) in 19.503907 s (76.781420 MiBps) scan 379 ( 2 samples) loaded. Target: 'J0237-2309'. Writing to disk... Wrote scan data (499.179222 MiB) in 7.396046 s (67.492715 MiBps) scan 398 ( 4 samples) loaded. Target: 'J1824-0938'. Writing to disk... Wrote scan data (998.358444 MiB) in 13.968691 s (71.471153 MiBps) scan 410 ( 3 samples) loaded. Target: 'J2131+0436'. Writing to disk... Wrote scan data (748.768833 MiB) in 10.356314 s (72.300708 MiBps) scan 412 ( 6 samples) loaded. Target: 'J2129+0436'. Writing to disk... Traceback (most recent call last): File "/usr/local/bin/mvftoms.py", line 791, in main() File "/usr/local/bin/mvftoms.py", line 541, in main scan_vis_data, scan_weight_data, scan_flag_data) File "/usr/local/bin/mvftoms.py", line 75, in load [vis, weights, flags], lock=False) File "/usr/local/lib/python2.7/dist-packages/dask/array/core.py", line 1019, in store result.compute(**kwargs) File "/usr/local/lib/python2.7/dist-packages/dask/base.py", line 156, in compute (result,) = compute(self, traverse=False, **kwargs) File "/usr/local/lib/python2.7/dist-packages/dask/base.py", line 395, in compute results = schedule(dsk, keys, **kwargs) File "/usr/local/lib/python2.7/dist-packages/dask/threaded.py", line 75, in get pack_exception=pack_exception, **kwargs) File "/usr/local/lib/python2.7/dist-packages/dask/local.py", line 501, in get_async raise_exception(exc, tb) File "/usr/local/lib/python2.7/dist-packages/dask/local.py", line 272, in execute_task result = _execute_task(task, data) File "/usr/local/lib/python2.7/dist-packages/dask/local.py", line 252, in _execute_task args2 = [_execute_task(a, cache) for a in args] File "/usr/local/lib/python2.7/dist-packages/dask/local.py", line 253, in _execute_task return func(*args2) File "/usr/local/lib/python2.7/dist-packages/katdal/chunkstore.py", line 209, in get_chunk_or_zeros return self.get_chunk(array_name, slices, dtype) File "/usr/local/lib/python2.7/dist-packages/katdal/chunkstore_s3.py", line 333, in get_chunk chunk = np.lib.format.read_array(data, allow_pickle=False) File "/usr/lib/python2.7/dist-packages/numpy/lib/format.py", line 664, in read_array data = _read_bytes(fp, read_size, "array data") File "/usr/lib/python2.7/dist-packages/numpy/lib/format.py", line 803, in _read_bytes r = fp.read(size - len(data)) File "/usr/local/lib/python2.7/dist-packages/urllib3/response.py", line 430, in read raise IncompleteRead(self._fp_bytes_read, self.length_remaining) File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__ self.gen.throw(type, value, traceback) File "/usr/local/lib/python2.7/dist-packages/urllib3/response.py", line 336, in _error_catcher raise ReadTimeoutError(self._pool, None, 'Read timed out.') urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='archive-gw-1.kat.ac.za', port=7480): Read timed out.