Unable to merge two files, possibly due to hardcoded 'master' master channel name

[darth3PO]

Python version

3.7.4 (default, Aug 9 2019, 18:34:13) [MSC v.1915 64 bit (AMD64)]

Platform information

Windows-10-10.0.16299-SP0

Numpy version

1.18.1

mdfreader version

4.0

Description

Unable to merge two mdf files. Files have no common channels, but were recorded simultaneously to different computers. One is version 300 and another 310.

yop = mdfreader.Mdf(file_name=r'File1.dat')
poy = mdfreader.Mdf(file_name=r'File2.dat')

yop.resample(sampling=0.5)
poy.resample(sampling=0.5)

After resampling, yop and poy each have one master channel, but they are named from one of the channels in the file, not 'time' or 'master'.

yop.masterChannelList.keys()
['time_1']

poy.masterChannelList.keys()
['TEST_TIME_1']

I've tried merging in each direction:
combined = yop.merge_mdf(poy)
and
combined = poy.merge_mdf(yop)
and both will yield the same error:

File "C:\ProgramData\Anaconda3\lib\site-packages\mdfreader\mdfreader.py", line 1293, in merge_mdf:
refill = empty(len(mdf_class.get_channel_data('master')))

TypeError: object of type 'NoneType' has no len()

Neither mdf has a master channel named master.

Is 'master' a reserved name? or should resample() force a name rather than inherit a name from the file?

[ratal]

Thanks for pointing out.
Yes, the code is rather old, I will revise it and remove reference to 'master' channel name.
However, I do not think merge_mdf() is what you are thinking. It is concatenating (sorry, name is misleading, I should change it), meaning it is putting 2 files/mdf one after the other in time. Basically biggest use case is to have one logger that records files once big enough for a long recording period of time, for endurance runs for instance.
Having 2 parallel loggers is kinda new use case -> new method should be written.
But if you want a quick fix, you could use pandas conversion and use the merge method.
Otherwise, just doing a quick yop.update(poy) could do it for your use case but you would have inconsistencies in attribute .masterChannelList to fix

[ratal]

In dev branch you will find merge_mdf() has changed name into concat_mdf() and a new merge_mdf() appears that answers your use case -> you can try it.

[ecalpy]

Hello Mr. Rateau,

I'm having a similar issue, but instead with data from the same vehicle and have all common channels.

Error text:
_File "C:\Users\ecalder.conda\envs\ecalpy\lib\site-packages\mdfreader\mdf4reader.py", line 1556, in _get_channel_data4
vector = self.get_channel(channel_name)[dataField]

KeyError: 'data'_

Will your updated concat_mdf()/merge_mdf() code help me as well?

Thanks,
Ecal

[ratal]

Hi Ecal,
Did you try with last version 4.1 of mdfreader ?
It includes concat_mdf() that should fit your need.
In my tests, I tried to concatenate twice the same file and no issue. If your issue still exists, can you please detail what you are doing ?

[ratal]

I guess issue was solved after 5 months, do hesitate to reopen if not.