Reading Modem data

[CedricPatzer]

reading of modem data file causes index error

md.read_data_file(fname)
File ~\Anaconda3\envs\mtpy-v2\lib\site-packages\mtpy\modeling\modem\data.py:1094 in read_data_file
n_periods, n_stations = self._read_header(
File ~\Anaconda3\envs\mtpy-v2\lib\site-packages\mtpy\modeling\modem\data.py:891 in _read_header
self._parse_header_line(head_line, inv_mode)
File ~\Anaconda3\envs\mtpy-v2\lib\site-packages\mtpy\modeling\modem\data.py:930 in _parse_header_line
value = item_list[1].replace("%", "").split()[0]
IndexError: list index out of range

Current Behavior

The first line of all of our ModeEM data files looks like this:
# ModEM impedance responses for Description:
item_list will contain ['description', ''] for each of our files and than fail

Your Environment

• MtPy version: 2.0.5

[kujaku11]

@CedricPatzer Could you provide an example of the header where it fails?

Also, are you using MTData.from_modem()? If not try that.

from mtpy import MTData

md = MTData()
md.from_modem(modem_data_filename)

[VicenteYanez]

Hello

I am having a similar issue as @CedricPatzer. I have tested with two ModEM files, one file of my own and one that I downloaded from the old repo of mtpy (https://github.com/MTgeophysics/mtpy/blob/develop/examples/data/ModEM_files/ModEM_Data.dat).

My version of MTpy is 2.0.6

import mtpy
from mtpy import MTData

datafile = 'datos3.dat'
#datafile = 'ModEM_Data.dat'

md = MTData()
md.from_modem(datafile)

With my modem file (datos3.dat) I get the following error

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[9], line 8
      5 datafile = 'datos3.dat'
      7 md = MTData()
----> 8 md.from_modem(datafile)

File [~/projects/test_pymt/envs/lib/python3.12/site-packages/mtpy/core/mt_data.py:917](http://localhost:8888/lab/tree/envs/lib/python3.12/site-packages/mtpy/core/mt_data.py#line=916), in MTData.from_modem(self, data_filename, file_type, **kwargs)
    905 """
    906 read in a modem data file
    907 
   (...)
    914 
    915 """
    916 modem_data = Data(**kwargs)
--> 917 mdf = modem_data.read_data_file(data_filename)
    918 if file_type in ["data"]:
    919     mdf.dataframe["survey"] = "data"

File [~/projects/test_pymt/envs/lib/python3.12/site-packages/mtpy/modeling/modem/data.py:1141](http://localhost:8888/lab/tree/envs/lib/python3.12/site-packages/mtpy/modeling/modem/data.py#line=1140), in Data.read_data_file(self, data_fn)
   1138     dlines = dfid.readlines()
   1140 # read header information
-> 1141 n_periods, n_stations = self._read_header(
   1142 [line for line in dlines if ">" in line or "#" in line]
   1143 )
   1145 # create a list of dictionaries to make into a pandas dataframe
   1146 entries = []

File [~/projects/test_pymt/envs/lib/python3.12/site-packages/mtpy/modeling/modem/data.py:939](http://localhost:8888/lab/tree/envs/lib/python3.12/site-packages/mtpy/modeling/modem/data.py#line=938), in Data._read_header(self, header_lines)
    936             n_stations = int(hline[1:].strip().split()[1])
    938 for head_line, inv_mode in zip(header_list, inv_list):
--> 939     self._parse_header_line(head_line, inv_mode)
    941 self._get_inv_mode(inv_list)
    943 return n_periods, n_stations

File [~/projects/test_pymt/envs/lib/python3.12/site-packages/mtpy/modeling/modem/data.py:978](http://localhost:8888/lab/tree/envs/lib/python3.12/site-packages/mtpy/modeling/modem/data.py#line=977), in Data._parse_header_line(self, header_line, mode)
    976 if len(item_list) == 2:
    977     key = item_list[0]
--> 978     value = item_list[1].replace("%", "").split()[0]
    979     if key in ["error_value", "data_rotation"]:
    980         try:

IndexError: list index out of range

and with ModEM_Data.dat I get

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[8], line 8
      5 #datafile = 'datos3.dat'
      7 md = MTData()
----> 8 md.from_modem(datafile)

File [~/projects/test_pymt/envs/lib/python3.12/site-packages/mtpy/core/mt_data.py:917](http://localhost:8888/lab/tree/envs/lib/python3.12/site-packages/mtpy/core/mt_data.py#line=916), in MTData.from_modem(self, data_filename, file_type, **kwargs)
    905 """
    906 read in a modem data file
    907 
   (...)
    914 
    915 """
    916 modem_data = Data(**kwargs)
--> 917 mdf = modem_data.read_data_file(data_filename)
    918 if file_type in ["data"]:
    919     mdf.dataframe["survey"] = "data"

File [~/projects/test_pymt/envs/lib/python3.12/site-packages/mtpy/modeling/modem/data.py:1141](http://localhost:8888/lab/tree/envs/lib/python3.12/site-packages/mtpy/modeling/modem/data.py#line=1140), in Data.read_data_file(self, data_fn)
   1138     dlines = dfid.readlines()
   1140 # read header information
-> 1141 n_periods, n_stations = self._read_header(
   1142 [line for line in dlines if ">" in line or "#" in line]
   1143 )
   1145 # create a list of dictionaries to make into a pandas dataframe
   1146 entries = []

File [~/projects/test_pymt/envs/lib/python3.12/site-packages/mtpy/modeling/modem/data.py:939](http://localhost:8888/lab/tree/envs/lib/python3.12/site-packages/mtpy/modeling/modem/data.py#line=938), in Data._read_header(self, header_lines)
    936             n_stations = int(hline[1:].strip().split()[1])
    938 for head_line, inv_mode in zip(header_list, inv_list):
--> 939     self._parse_header_line(head_line, inv_mode)
    941 self._get_inv_mode(inv_list)
    943 return n_periods, n_stations

File [~/projects/test_pymt/envs/lib/python3.12/site-packages/mtpy/modeling/modem/data.py:1016](http://localhost:8888/lab/tree/envs/lib/python3.12/site-packages/mtpy/modeling/modem/data.py#line=1015), in Data._parse_header_line(self, header_line, mode)
   1010     setattr(obj, item_dict["error_value"], value)
   1012 if "deg" in item:
   1013     setattr(
   1014         self,
   1015         item_dict["data_rotation"],
-> 1016         float(item.split("_")[0]),
   1017     )

ValueError: could not convert string to float: ' data rotated 0.0'