docs/examples/notebooks/transfer_function_example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b0edfb4a-e73d-42e0-8bf1-62c7042c7406",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Transfer Functions\n",
    "\n",
    "There exists the ability to store transfer functions in an MTH5 for a comprehensive representation of data.  The transfer functions are stored at the `Station` level in a group called `TransferFunctionsGroup`.  Within this group station transfer functions are stored as a single group for each transfer function `TransferFunctionGroup`.  Each `TransferFunctionGroup` has a data set for each statistical estimate provided within the transfer function.  Currently supported estimates are\n",
    "\n",
    "| Estimate           | Description                  |  Shape | Data Type |\n",
    "|--------------------|------------------------------|--------|---------- |\n",
    "| transfer_function  | Full transfer function       | n_periods x n_inputs x n_outputs | complex |\n",
    "| transfer_function_error  | Full transfer function error estimation      | n_periods x n_inputs x n_outputs | real |\n",
    "| impedance  | Only horizontal components, traditional impedance tensor       | n_periods x 2 x 2 |complex |\n",
    "| impedance_error  | Only horizontal components, traditional impedance tensor error       | n_periods x 2 x 2 |real |\n",
    "| tipper  | horizontal and vertical magnetic transfer function, Tipper  | n_periods x 1 x 2 | complex |\n",
    "| tipper_error  | horizontal and vertical magnetic transfer function, Tipper error  | n_periods x 1 x 2 |real |\n",
    "| inverse_signal_power  | covariance of input channels (sources)  | n_periods x n_inputs x n_inputs |complex |\n",
    "| residual_covariance  | covariance of output channels (responses) | n_periods x n_outputs x n_outputs |complex |\n",
    "\n",
    "**Note:** There are plans to add phase tensor and resitivity/phase estimations in the future.\n",
    "\n",
    "This examples demonstrates how transfer functions can be added to an MTH5 file.  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6ad495f-36cc-437b-9214-e4a503985720",
   "metadata": {},
   "source": [
    "## TF object\n",
    "\n",
    "The TF object comes from `mt_metadata.transfer_functions.core.TF` and is meant to be the common container for transfer functions.  It has readers for:\n",
    "\n",
    "* EDI\n",
    "* EMFT XML\n",
    "* Z-files (EMTF output)\n",
    "* J-files (BIRRP output)\n",
    "* AVG files (Zonge output)\n",
    "\n",
    "The `TF` object has two important metadata objects `survey_metadata` and `station_metadata`.  Metadata from the other files are translated into these containers and translated back when writing a file.  \n",
    "\n",
    "The statistical estimates are stored as `xarray.Datasets` that have coordinates of `period`, `input_channels`, `output_channels`.  This way the transfer function can be generalized.  `impedance` and `tipper` are stored in `transfer_function` and `TF` provides convenience functions to access `impedance` and `tipper` and associated errors.  Variances are stored as covariances for input channels (`inverse_signal_power`) and output channels (`residual_covariance`) when possible, and the `transfer_function_error` is stored as well.  \n",
    "\n",
    "**Note:** There are future plans to include phase tensor and resistivity/phase representation as well.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "389f02a1-a84f-4be3-bdf1-1109c29dbf10",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-21 10:31:36,442 [line 141] mth5.setup_logger - INFO: Logging file can be found C:\\Users\\jpeacock\\OneDrive - DOI\\Documents\\GitHub\\mth5\\logs\\mth5_debug.log\n"
     ]
    }
   ],
   "source": [
    "from mth5.mth5 import MTH5\n",
    "\n",
    "from mt_metadata import TF_XML, TF_EDI_SPECTRA, TF_ZMM, TF_EDI_CGG\n",
    "from mt_metadata.transfer_functions.core import TF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "5aaba4d3-6d50-4475-9f85-610f525413f1",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-21 10:31:42,346 [line 672] mth5.mth5.MTH5._initialize_file - INFO: Initialized MTH5 0.2.0 file transfer_function_example.h5 in mode w\n"
     ]
    }
   ],
   "source": [
    "m = MTH5(file_version=\"0.2.0\")\n",
    "m.open_mth5(r\"transfer_function_example.h5\", \"w\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5ecc33c-6822-4405-8f0f-87d4b99f50dc",
   "metadata": {},
   "source": [
    "## Read in Transfer functions\n",
    "\n",
    "* **TF_XML**: An example of EMTF XML format, the preferred format for archiving\n",
    "* **TF_EDI_SPECTRA**: An example of an EDI file stored as spectra\n",
    "* **TF_EDI_CGG**: An example of an output file from a contractor\n",
    "* **TF_ZMM**: An example of an output file from EMFT\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "eff7a3d1-b0df-49d3-b439-ce8ad3da8488",
   "metadata": {},
   "outputs": [],
   "source": [
    "tf1 = TF(TF_XML)\n",
    "tf1.read_tf_file()\n",
    "\n",
    "tf2 = TF(TF_EDI_SPECTRA)\n",
    "tf2.read_tf_file()\n",
    "\n",
    "tf3 = TF(TF_EDI_CGG)\n",
    "tf3.read_tf_file()\n",
    "\n",
    "tf4 = TF(TF_ZMM)\n",
    "tf4.read_tf_file()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92e1ede5-3481-4ef5-9373-dc5766f015c7",
   "metadata": {},
   "source": [
    "## Add TF_XML to the MTH5\n",
    "\n",
    "When we add a transfer function to an MTH5, it looks for the `survey.id` and `station.id`, if it doesn't find any then they are created.  If information is provided on which runs were processed and channels used those are filled in as well.\n",
    "\n",
    "**Note:** If you have multiple transfer functions for a given station be sure to rename the file, for an EDI this is in `HEADER` under the attribute `DATAID`. Name it something like `station_sample_rate` or `station_runs` "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "87d6bd11-644c-4a43-87c0-7a6d1bcf2394",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-21 10:31:49,566 [line 1054] mth5.mth5.MTH5.get_survey - WARNING: /Experiment/Surveys/CONUS_South does not exist, check survey_list for existing names.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "/Experiment/Surveys/CONUS_South/Stations/NMX20/Transfer_Functions/NMX20:\n",
       "====================\n",
       "    --> Dataset: inverse_signal_power\n",
       "    ...................................\n",
       "    --> Dataset: period\n",
       "    .....................\n",
       "    --> Dataset: residual_covariance\n",
       "    ..................................\n",
       "    --> Dataset: transfer_function\n",
       "    ................................\n",
       "    --> Dataset: transfer_function_error\n",
       "    ......................................"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf_group_01 = m.add_transfer_function(tf1)\n",
    "tf_group_01"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "524c8f00-c90a-4f3b-9ee1-259e670b3d7a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "est = tf_group_01.get_estimate(\"transfer_function\")\n",
    "tf_group_01.has_estimate(\"covariance\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0fb6f89-d4da-4b78-94e0-4db46dcb892e",
   "metadata": {},
   "source": [
    "### Have a look at what was added to the MTH5\n",
    "\n",
    "Note that an EMTF XML has comprehensive metadata which can be used to populate the MTH5 as necessary, including `Runs` and `Channels`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca68ec41-7013-41b2-80aa-8fd85c270380",
   "metadata": {},
   "source": [
    "## Add an example EDI\n",
    "\n",
    "Here the survey is not specified therefore we need to fill that information in ourselves otherwise an error is raised, see below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ced41eda-b1f7-40f2-8783-6d7f4aba9858",
   "metadata": {},
   "outputs": [],
   "source": [
    "#tf_group_02 = m.add_transfer_function(tf2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5244a069-1133-4434-803f-1a671931f383",
   "metadata": {},
   "source": [
    "Here we give the survey the id `unknown_survey`.  Also note that because the data are stored as spectra in the EDI we can calculate the `inverse_signal_power` and `residual_covariance`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b12a3baa-bebf-4020-9c24-46dc34845b89",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-21 10:32:07,195 [line 1054] mth5.mth5.MTH5.get_survey - WARNING: /Experiment/Surveys/unknown_survey does not exist, check survey_list for existing names.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n"
     ]
    }
   ],
   "source": [
    "tf2.survey_metadata.id = \"unknown_survey\"\n",
    "tf_group_02 = m.add_transfer_function(tf2)\n",
    "print(tf_group_02.has_estimate(\"covariance\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c8ee84e-7bcf-4ec2-9c39-73a925d38535",
   "metadata": {},
   "source": [
    "## Add typical EDI file\n",
    "\n",
    "This file only has impedance and tipper and minimal metadata, which are converted into a full transfer function for storage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "9e87dc23-6f76-461d-abd8-ffce1256d549",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-21 10:32:11,616 [line 222] mth5.groups.base.TransferFunction.get_estimate - ERROR: residual_covariance does not exist, check groups_list for existing names\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf3.survey_metadata.id = \"unknown_survey\"\n",
    "tf_group_03 = m.add_transfer_function(tf3)\n",
    "tf_group_03.has_estimate(\"covariance\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2a505e5-0132-437b-a409-a0473d91174a",
   "metadata": {},
   "source": [
    "## Add an output from EMTF\n",
    "\n",
    "A ZMM file contains the full covariance and transfer functions but has minimal metadata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "b18a925f-be0d-4499-8833-820ae4c2a09f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Station: 300\n",
      "--------------------------------------------------\n",
      "\tSurvey:            None\n",
      "\tProject:           None\n",
      "\tAcquired by:       None\n",
      "\tAcquired date:     1980-01-01\n",
      "\tLatitude:          34.727\n",
      "\tLongitude:         -115.735\n",
      "\tElevation:         0.000\n",
      "\tDeclination:   \n",
      "\t\tValue:     13.1\n",
      "\t\tModel:     WMM\n",
      "\tCoordinate System: geographic\n",
      "\tImpedance:         True\n",
      "\tTipper:            True\n",
      "\tN Periods:     38\n",
      "\tPeriod Range:\n",
      "\t\tMin:   1.16364E+00 s\n",
      "\t\tMax:   1.09227E+04 s\n",
      "\tFrequency Range:\n",
      "\t\tMin:   9.15527E-05 Hz\n",
      "\t\tMax:   8.59372E-01 Hz\n"
     ]
    }
   ],
   "source": [
    "print(tf4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "464966c3-1d72-4ff6-9ab5-cecfb1d08c3d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf4.survey_metadata.id = \"unknown_survey\"\n",
    "tf_group_04 = m.add_transfer_function(tf4)\n",
    "tf_group_04.has_estimate(\"impedance\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9afbeb5a-ff13-4d48-99b6-36a70f643a31",
   "metadata": {},
   "source": [
    "## Have a look at the MTH5 file\n",
    "\n",
    "Everything has been filled in now in the MTH5 including metadata about `runs` and `channels`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "ca6de2e4-7548-4dc6-8ab5-c6305c5b4460",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "/:\n",
       "====================\n",
       "    |- Group: Experiment\n",
       "    --------------------\n",
       "        |- Group: Reports\n",
       "        -----------------\n",
       "        |- Group: Standards\n",
       "        -------------------\n",
       "            --> Dataset: summary\n",
       "            ......................\n",
       "        |- Group: Surveys\n",
       "        -----------------\n",
       "            |- Group: CONUS_South\n",
       "            ---------------------\n",
       "                |- Group: Filters\n",
       "                -----------------\n",
       "                    |- Group: coefficient\n",
       "                    ---------------------\n",
       "                    |- Group: fap\n",
       "                    -------------\n",
       "                    |- Group: fir\n",
       "                    -------------\n",
       "                    |- Group: time_delay\n",
       "                    --------------------\n",
       "                    |- Group: zpk\n",
       "                    -------------\n",
       "                |- Group: Reports\n",
       "                -----------------\n",
       "                |- Group: Standards\n",
       "                -------------------\n",
       "                    --> Dataset: summary\n",
       "                    ......................\n",
       "                |- Group: Stations\n",
       "                ------------------\n",
       "                    |- Group: NMX20\n",
       "                    ---------------\n",
       "                        |- Group: NMX20a\n",
       "                        ----------------\n",
       "                            --> Dataset: ex\n",
       "                            .................\n",
       "                            --> Dataset: ey\n",
       "                            .................\n",
       "                            --> Dataset: hx\n",
       "                            .................\n",
       "                            --> Dataset: hy\n",
       "                            .................\n",
       "                            --> Dataset: hz\n",
       "                            .................\n",
       "                        |- Group: NMX20b\n",
       "                        ----------------\n",
       "                            --> Dataset: ex\n",
       "                            .................\n",
       "                            --> Dataset: ey\n",
       "                            .................\n",
       "                            --> Dataset: hx\n",
       "                            .................\n",
       "                            --> Dataset: hy\n",
       "                            .................\n",
       "                            --> Dataset: hz\n",
       "                            .................\n",
       "                        |- Group: Transfer_Functions\n",
       "                        ----------------------------\n",
       "                            |- Group: NMX20\n",
       "                            ---------------\n",
       "                                --> Dataset: inverse_signal_power\n",
       "                                ...................................\n",
       "                                --> Dataset: period\n",
       "                                .....................\n",
       "                                --> Dataset: residual_covariance\n",
       "                                ..................................\n",
       "                                --> Dataset: transfer_function\n",
       "                                ................................\n",
       "                                --> Dataset: transfer_function_error\n",
       "                                ......................................\n",
       "            |- Group: unknown_survey\n",
       "            ------------------------\n",
       "                |- Group: Filters\n",
       "                -----------------\n",
       "                    |- Group: coefficient\n",
       "                    ---------------------\n",
       "                    |- Group: fap\n",
       "                    -------------\n",
       "                    |- Group: fir\n",
       "                    -------------\n",
       "                    |- Group: time_delay\n",
       "                    --------------------\n",
       "                    |- Group: zpk\n",
       "                    -------------\n",
       "                |- Group: Reports\n",
       "                -----------------\n",
       "                |- Group: Standards\n",
       "                -------------------\n",
       "                    --> Dataset: summary\n",
       "                    ......................\n",
       "                |- Group: Stations\n",
       "                ------------------\n",
       "                    |- Group: 300\n",
       "                    -------------\n",
       "                        |- Group: 300a\n",
       "                        --------------\n",
       "                            --> Dataset: ex\n",
       "                            .................\n",
       "                            --> Dataset: ey\n",
       "                            .................\n",
       "                            --> Dataset: hx\n",
       "                            .................\n",
       "                            --> Dataset: hy\n",
       "                            .................\n",
       "                            --> Dataset: hz\n",
       "                            .................\n",
       "                        |- Group: Transfer_Functions\n",
       "                        ----------------------------\n",
       "                            |- Group: 300\n",
       "                            -------------\n",
       "                                --> Dataset: inverse_signal_power\n",
       "                                ...................................\n",
       "                                --> Dataset: period\n",
       "                                .....................\n",
       "                                --> Dataset: residual_covariance\n",
       "                                ..................................\n",
       "                                --> Dataset: transfer_function\n",
       "                                ................................\n",
       "                                --> Dataset: transfer_function_error\n",
       "                                ......................................\n",
       "                    |- Group: SAGE_2005\n",
       "                    -------------------\n",
       "                        |- Group: SAGE_2005a\n",
       "                        --------------------\n",
       "                            --> Dataset: ex\n",
       "                            .................\n",
       "                            --> Dataset: ey\n",
       "                            .................\n",
       "                            --> Dataset: hx\n",
       "                            .................\n",
       "                            --> Dataset: hy\n",
       "                            .................\n",
       "                            --> Dataset: hz\n",
       "                            .................\n",
       "                        |- Group: Transfer_Functions\n",
       "                        ----------------------------\n",
       "                            |- Group: SAGE_2005\n",
       "                            -------------------\n",
       "                                --> Dataset: inverse_signal_power\n",
       "                                ...................................\n",
       "                                --> Dataset: period\n",
       "                                .....................\n",
       "                                --> Dataset: residual_covariance\n",
       "                                ..................................\n",
       "                                --> Dataset: transfer_function\n",
       "                                ................................\n",
       "                                --> Dataset: transfer_function_error\n",
       "                                ......................................\n",
       "            |- Group: unknown_survey_001\n",
       "            ----------------------------\n",
       "                |- Group: Filters\n",
       "                -----------------\n",
       "                    |- Group: coefficient\n",
       "                    ---------------------\n",
       "                    |- Group: fap\n",
       "                    -------------\n",
       "                    |- Group: fir\n",
       "                    -------------\n",
       "                    |- Group: time_delay\n",
       "                    --------------------\n",
       "                    |- Group: zpk\n",
       "                    -------------\n",
       "                |- Group: Reports\n",
       "                -----------------\n",
       "                |- Group: Standards\n",
       "                -------------------\n",
       "                    --> Dataset: summary\n",
       "                    ......................\n",
       "                |- Group: Stations\n",
       "                ------------------\n",
       "                    |- Group: TEST01\n",
       "                    ----------------\n",
       "                        |- Group: TEST01a\n",
       "                        -----------------\n",
       "                            --> Dataset: ex\n",
       "                            .................\n",
       "                            --> Dataset: ey\n",
       "                            .................\n",
       "                            --> Dataset: hx\n",
       "                            .................\n",
       "                            --> Dataset: hy\n",
       "                            .................\n",
       "                            --> Dataset: hz\n",
       "                            .................\n",
       "                        |- Group: Transfer_Functions\n",
       "                        ----------------------------\n",
       "                            |- Group: TEST01\n",
       "                            ----------------\n",
       "                                --> Dataset: period\n",
       "                                .....................\n",
       "                                --> Dataset: transfer_function\n",
       "                                ................................\n",
       "                                --> Dataset: transfer_function_error\n",
       "                                ......................................\n",
       "        --> Dataset: channel_summary\n",
       "        ..............................\n",
       "        --> Dataset: tf_summary\n",
       "        ........................."
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95f954ce-7b3f-41f8-bab3-a002d25d5676",
   "metadata": {},
   "source": [
    "## Get a transfer function object from MTH5\n",
    "\n",
    "To retrieve a transfer function from the MTH5 file a convenience function `m.get_transfer_function` is supplied.  You only need to know the `station.id`, `tf.id`, and the `survey.id`.  Here the `tf.id` is the same as the `station.id`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "c5f3ee5b-784e-4ba9-a480-a36c13b2a09c",
   "metadata": {},
   "outputs": [],
   "source": [
    "tf1_h5 = m.get_transfer_function(tf1.station_metadata.id, tf1.tf_id, tf1.survey_metadata.id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "535ebb46-db7a-436f-a7e6-e2e2e500d17c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Station: NMX20\n",
      "--------------------------------------------------\n",
      "\tSurvey:            CONUS South\n",
      "\tProject:           USMTArray\n",
      "\tAcquired by:       National Geoelectromagnetic Facility\n",
      "\tAcquired date:     2020-09-20\n",
      "\tLatitude:          34.471\n",
      "\tLongitude:         -108.712\n",
      "\tElevation:         1940.050\n",
      "\tDeclination:   \n",
      "\t\tValue:     9.09\n",
      "\t\tModel:     WMM\n",
      "\tCoordinate System: geographic\n",
      "\tImpedance:         True\n",
      "\tTipper:            True\n",
      "\tN Periods:     33\n",
      "\tPeriod Range:\n",
      "\t\tMin:   4.65455E+00 s\n",
      "\t\tMax:   2.91271E+04 s\n",
      "\tFrequency Range:\n",
      "\t\tMin:   3.43323E-05 Hz\n",
      "\t\tMax:   2.14844E-01 Hz\n",
      "Station: NMX20\n",
      "--------------------------------------------------\n",
      "\tSurvey:            CONUS South\n",
      "\tProject:           USMTArray\n",
      "\tAcquired by:       National Geoelectromagnetic Facility\n",
      "\tAcquired date:     2020-09-20\n",
      "\tLatitude:          34.471\n",
      "\tLongitude:         -108.712\n",
      "\tElevation:         1940.050\n",
      "\tDeclination:   \n",
      "\t\tValue:     9.09\n",
      "\t\tModel:     WMM\n",
      "\tCoordinate System: geographic\n",
      "\tImpedance:         True\n",
      "\tTipper:            True\n",
      "\tN Periods:     33\n",
      "\tPeriod Range:\n",
      "\t\tMin:   4.65455E+00 s\n",
      "\t\tMax:   2.91271E+04 s\n",
      "\tFrequency Range:\n",
      "\t\tMin:   3.43323E-05 Hz\n",
      "\t\tMax:   2.14844E-01 Hz\n"
     ]
    }
   ],
   "source": [
    "print(tf1)\n",
    "print(tf1_h5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fee88c0c-59c1-4928-b827-bee3e766c35d",
   "metadata": {},
   "source": [
    "## Summarize what transfer functions are in the file\n",
    "\n",
    "In the MTH5 file there is a property called `tf_summary`, this provides an array that can be converted to a `pandas.DataFrame` of the transfer functions within the file.  There is a column called `hdf5_reference` which can be used to get the transfer function directly.  This table is updated when the file is close, so when you open the file next it should be up to date.  This table should be read only, if you want to change metadata, you should do it directly in the transfer function object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "4a63b5d4-207b-4c4a-be5a-621242501289",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>station</th>\n",
       "      <th>survey</th>\n",
       "      <th>latitude</th>\n",
       "      <th>longitude</th>\n",
       "      <th>elevation</th>\n",
       "      <th>tf_id</th>\n",
       "      <th>units</th>\n",
       "      <th>has_impedance</th>\n",
       "      <th>has_tipper</th>\n",
       "      <th>has_covariance</th>\n",
       "      <th>period_min</th>\n",
       "      <th>period_max</th>\n",
       "      <th>hdf5_reference</th>\n",
       "      <th>station_hdf5_reference</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NMX20</td>\n",
       "      <td>CONUS_South</td>\n",
       "      <td>34.470528</td>\n",
       "      <td>-108.712288</td>\n",
       "      <td>1940.05</td>\n",
       "      <td>NMX20</td>\n",
       "      <td>none</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>4.654550</td>\n",
       "      <td>29127.110000</td>\n",
       "      <td>&lt;HDF5 object reference&gt;</td>\n",
       "      <td>&lt;HDF5 object reference&gt;</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>300</td>\n",
       "      <td>unknown_survey</td>\n",
       "      <td>34.727000</td>\n",
       "      <td>-115.735000</td>\n",
       "      <td>0.00</td>\n",
       "      <td>300</td>\n",
       "      <td>none</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>1.163640</td>\n",
       "      <td>10922.666990</td>\n",
       "      <td>&lt;HDF5 object reference&gt;</td>\n",
       "      <td>&lt;HDF5 object reference&gt;</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>SAGE_2005</td>\n",
       "      <td>unknown_survey</td>\n",
       "      <td>35.550000</td>\n",
       "      <td>-106.283333</td>\n",
       "      <td>0.00</td>\n",
       "      <td>SAGE_2005</td>\n",
       "      <td>none</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>0.004196</td>\n",
       "      <td>209.731544</td>\n",
       "      <td>&lt;HDF5 object reference&gt;</td>\n",
       "      <td>&lt;HDF5 object reference&gt;</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>TEST01</td>\n",
       "      <td>unknown_survey_001</td>\n",
       "      <td>-30.930285</td>\n",
       "      <td>127.229230</td>\n",
       "      <td>175.27</td>\n",
       "      <td>TEST01</td>\n",
       "      <td>none</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>0.001212</td>\n",
       "      <td>1211.527490</td>\n",
       "      <td>&lt;HDF5 object reference&gt;</td>\n",
       "      <td>&lt;HDF5 object reference&gt;</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     station              survey   latitude   longitude  elevation      tf_id  \\\n",
       "0      NMX20         CONUS_South  34.470528 -108.712288    1940.05      NMX20   \n",
       "1        300      unknown_survey  34.727000 -115.735000       0.00        300   \n",
       "2  SAGE_2005      unknown_survey  35.550000 -106.283333       0.00  SAGE_2005   \n",
       "3     TEST01  unknown_survey_001 -30.930285  127.229230     175.27     TEST01   \n",
       "\n",
       "  units  has_impedance  has_tipper  has_covariance  period_min    period_max  \\\n",
       "0  none           True        True            True    4.654550  29127.110000   \n",
       "1  none           True        True            True    1.163640  10922.666990   \n",
       "2  none           True        True            True    0.004196    209.731544   \n",
       "3  none           True        True           False    0.001212   1211.527490   \n",
       "\n",
       "            hdf5_reference   station_hdf5_reference  \n",
       "0  <HDF5 object reference>  <HDF5 object reference>  \n",
       "1  <HDF5 object reference>  <HDF5 object reference>  \n",
       "2  <HDF5 object reference>  <HDF5 object reference>  \n",
       "3  <HDF5 object reference>  <HDF5 object reference>  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "m.tf_summary.clear_table()\n",
    "m.tf_summary.summarize()\n",
    "tf_df = m.tf_summary.to_dataframe()\n",
    "tf_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f09baf20-8cc0-4361-a2fc-bd88f4cbb7a7",
   "metadata": {},
   "source": [
    "### Get TF from reference "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "c16940b1-a77e-437b-9a2c-116bc92f9fec",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Station: NMX20\n",
      "--------------------------------------------------\n",
      "\tSurvey:            CONUS South\n",
      "\tProject:           USMTArray\n",
      "\tAcquired by:       National Geoelectromagnetic Facility\n",
      "\tAcquired date:     2020-09-20\n",
      "\tLatitude:          34.471\n",
      "\tLongitude:         -108.712\n",
      "\tElevation:         1940.050\n",
      "\tDeclination:   \n",
      "\t\tValue:     9.09\n",
      "\t\tModel:     WMM\n",
      "\tCoordinate System: geographic\n",
      "\tImpedance:         True\n",
      "\tTipper:            True\n",
      "\tN Periods:     33\n",
      "\tPeriod Range:\n",
      "\t\tMin:   4.65455E+00 s\n",
      "\t\tMax:   2.91271E+04 s\n",
      "\tFrequency Range:\n",
      "\t\tMin:   3.43323E-05 Hz\n",
      "\t\tMax:   2.14844E-01 Hz\n"
     ]
    }
   ],
   "source": [
    "tf_object = m.from_reference(tf_df.iloc[0].hdf5_reference)\n",
    "\n",
    "print(tf_object)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e6d46d3-e01b-4b64-8e3a-710fcacd96ce",
   "metadata": {},
   "source": [
    "## MTpy\n",
    "\n",
    "To analyze, plot, prepare input files one should look to use `MTpy`.  **Note:** `MTpy version 2.0` will use `MTH5` as the storage mechanism and `TF` to read/write files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "55c38861-3605-4e33-bc39-99325549f10f",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-21 10:32:58,441 [line 753] mth5.mth5.MTH5.close_mth5 - INFO: Flushing and closing transfer_function_example.h5\n"
     ]
    }
   ],
   "source": [
    "m.close_mth5()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ff2fbb2b-b746-49aa-846a-4cc42ef9d1fc",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}