unable to read mf4 with variable length of channels

[gdu2003]

Pyhton version

Python 3.5.3

Platform information

Windows 7

Numpy version

numpy = 1.14.0

mdfreader version

mdfdata.MDFVersionNumber
: 400

Description

I have mf4 mdf file. It have more than 3000 signals or channels whatever it is called. All channels / signals are recorded at different frequencies as they are from CAN BUS recording of a car using ETAS INCA data logger. When i am trying to load it , it gives error saying structure must have same size. Looks like it is looking for same length of all signals. Where as in MDFREADER 0.2.7 it was all working fine. If i install MDFREADER 0.2.3 it works well. There seems to be some problem in 2.7.4

MDF Version of file is MF4.

mdf_file = filedialog.askopenfilename()
mdfdata = mdfreader.mdf(fileName=mdf_file)
Traceback (most recent call last):

File "", line 11, in
mdfdata = mdfreader.mdf(fileName=mdf_file)

File "c:\python35\lib\site-packages\mdfreader\mdf.py", line 153, in init
compression=compression)

File "c:\python35\lib\site-packages\mdfreader\mdfreader.py", line 399, in read
convertAfterRead, filterChannelNames, compression)

File "c:\python35\lib\site-packages\mdfreader\mdf4reader.py", line 1248, in read4
buf.read(channelSet, info, self.fileName)

File "c:\python35\lib\site-packages\mdfreader\mdf4reader.py", line 340, in read
self[recordID]['data'] = self.load(record, info, nameList=channelSet, sortedFlag=True)

File "c:\python35\lib\site-packages\mdfreader\mdf4reader.py", line 453, in load
sortedFlag=sortedFlag, vlsd=vlsd)

ValueError: structures must have the same size

[ratal]

Hi
There might be some regression between versions.
The end of the error stack is missing maybe ? Could you inform what is line 453 to be sure ?
If it is temps.update(temp), coudl reproduce this error and introduce a print of both temp and temps just before to better undersand the error ?

[gdu2003]

Following is the If block which has line 453 in mdfreader.py

401 temps = defaultdict()
402 # block header
403 temps.update(_loadHeader(self.fid, self.pointerTodata))
404 if temps['id'] in ('##DL', b'##DL'): # data list block
405 temps.update(DLBlock(self.fid, temps['link_count']))
406 if temps['dl_dl_next']:
407 index = 1
408 while temps['dl_dl_next']: # reads pointers to all data blocks (DT, RD, SD, DZ)
409 temp = defaultdict()
410 temp.update(_loadHeader(self.fid, temps['dl_dl_next']))
411 temps['dl_dl_next'] = structunpack('<Q', self.fid.read(8))[0]
412 temps['dl_data'][index] =
413 structunpack('<{}Q'.
414 format(temp['link_count'] - 1),
415 self.fid.read(8 * (temp['link_count'] - 1)))
416 index += 1
417 if temps['dl_count']:
418 # read and concatenate raw blocks
419 buf = bytearray()
420 for DL in temps['dl_data']:
421 for pointer in temps['dl_data'][DL]:
422 # read fist data blocks linked by DLBlock to identify data block type
423 data_block = defaultdict()
424 data_block.update(_loadHeader(self.fid, pointer))
425 if data_block['id'] in ('##DT', '##RD', b'##DT',
426 b'##RD', '##SD', b'##SD'):
427 buf.extend(self.fid.read(
428 data_block['length'] - 24))
429 elif data_block['id'] in ('##DZ', b'##DZ'):
430 data_block.update(DZBlock(self.fid))
431 data_block['data'] =
432 decompress_datablock(
433 self.fid.read(data_block['dz_data_length']),
434 data_block['dz_zip_type'],
435 data_block['dz_zip_parameter'],
436 data_block['dz_org_data_length'])
437 buf.extend(data_block['data'])
438 data_block['id'] = '##DT' # do not uncompress in DATABlock function
439 data_block['data'] = buf
440 temps['data'] = DATABlock(record, info, parent_block=data_block, channelSet=nameList, sortedFlag=sortedFlag)
441 else: # empty datalist
442 temps['data'] = None
443 elif temps['id'] in ('##HL', b'##HL'): # header list block for DZBlock
444 # link section
445 temps.update(HLBlock(self.fid))
446 self.pointerTodata = temps['hl_dl_first']
447 temps['data'] = self.load(record, info, zip=temps['hl_zip_type'], nameList=nameList, sortedFlag=sortedFlag)
448 elif temps['id'] in ('##DT', '##RD', b'##DT', b'##RD'): # normal sorted data block, direct read
449 temps['data'] = record.readSortedRecord(self.fid, self.pointerTodata, info, channelSet=nameList)
450 elif temps['id'] in ('##SD', b'##SD'): # VLSD
451 temps['data'] = self.fid.read(temps['length'] - 24)
452 temps['data'] = DATABlock(record, info, parent_block=temps, channelSet=nameList, sortedFlag=sortedFlag)
453 elif temps['id'] in ('##DZ', b'##DZ'): # zipped data block
454 temps.update(DZBlock(self.fid))
455 temps['data'] = self.fid.read(temps['dz_data_length'])
456 temps['data'] = DATABlock(record, info, parent_block=temps, channelSet=nameList, sortedFlag=sortedFlag)

[ratal]

This is weird because temps['id'] in ('##DZ', b'##DZ') should not raise this kind of error, especially that is not so different from similar previous condition tests
Are you sure you execute the same code ?

[ratal]

Long time no feedback, closing