Concatenate changes underlying mdf obj group count

[BGorenc]

Python version

('python=3.12.2 (tags/v3.12.2:6abddd9, Feb  6 2024, 21:26:36) [MSC v.1937 64 ' 'bit (AMD64)]')
'os=Windows-10-10.0.19045-SP0'
'numpy=1.26.4'
'asammdf=7.4.2'

Code

MDF version

4.10

Code snippet

I can't share much of the code unfortunately.

mdf_obj_list = [obj1, obj2]
current_mdf = MDF.concatenate(mdf_obj_list)
mdf_obj_list.clear()
mdf_obj_list.append(current_mdf)
mdf_obj_list.append(obj3)
new_mdf = MDF.concatenate(mdf_obj_list) # <---- Failure Point

Traceback

When opening the full file with all channels: mdf_obj = MDF(file_path)

Traceback (most recent call last):
  File "c:\Users\gorencb\Documents\Projects\ETS_nikolai\Brute_Force_.py", line 134, in <module>
    main()
  File "c:\Users\gorencb\Documents\Projects\ETS_nikolai\Brute_Force_.py", line 36, in main
    current_mdf = MDF.concatenate(mdf_list)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\gorencb\Documents\Projects\ETS_nikolai\ets_env\Lib\site-packages\asammdf\mdf.py", line 2389, in concatenate
    _file.determine_max_vlsd_sample_size(_second_gp_idx, _second_ch_idx),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\gorencb\Documents\Projects\ETS_nikolai\ets_env\Lib\site-packages\asammdf\blocks\mdf_v4.py", line 7905, in _determine_max_vlsd_sample_size
    ch = group.channels[index]
         ~~~~~~~~~~~~~~^^^^^^^
IndexError: list index out of range

When opening the same files with only a couple channels: mdf_obj = MDF(file_path, channels=['channel_1', 'channel_2'])

Traceback (most recent call last):
  File "c:\Users\gorencb\Documents\Projects\ETS_nikolai\Brute_Force_.py", line 133, in <module>
    main()
  File "c:\Users\gorencb\Documents\Projects\ETS_nikolai\Brute_Force_.py", line 36, in main
    current_mdf = MDF.concatenate(mdf_list)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\gorencb\Documents\Projects\ETS_nikolai\ets_env\Lib\site-packages\asammdf\mdf.py", line 2402, in concatenate
    raise MdfException(
asammdf.blocks.utils.MdfException: internal structure of file <data_2\254-1553_W1NKM5GB6PF001154_6427_8427_20240301_022249.mf4> is different; different channel groups count

Description

I have many files (~2.5GB a piece) that need to be stitched together at specific points. I am scanning for these points then cutting the mdf object twice. I save the first cut as a new file and concatenate onto the remainder with the new opened object and continue scanning for new points.

The issue is when I concatenate, the number of groups reduces. I believe this is causing the issue. In this case when I read in the MDF object I have 51 groups, then when I concatenate I only end up with 47 in the resulting object. This prevents me from continuing to concatenate new objects. I also tried only opening the file with 2 channels and the file still opens with 51 groups but then after concatenate I end up with 3 leading to the same problem and failure point but slightly different Traceback.

I'm trying to avoid concatenating everything all at once to save some space in RAM, but it does work when I just concatenate every file together in one go.