Skip to content

Commit

Permalink
Adding Tutorial Notebook
Browse files Browse the repository at this point in the history 
* Added a notebook showing how GA QA/QC metadata can be
  extracted from stationXML files.
  • Loading branch information
@geojunky
geojunky committed Feb 3, 2023
1 parent 05ca38e commit bc37783
Showing 1 changed file with 249 additions and 0 deletions.
249 changes: 249 additions & 0 deletions notebooks/inventory-metadata-tutorial.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,249 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "58a1a8aa",
"metadata": {},
"source": [
"## Passive Seismic Metadata Standards (Geoscience Australia) v(0.2)"
]
},
{
"cell_type": "markdown",
"id": "8442fc0a",
"metadata": {},
"source": [
"This document describes how corrections for GPS clock-errors and station-orientations are encapsulated in StationXML files. "
]
},
{
"cell_type": "markdown",
"id": "32900926",
"metadata": {},
"source": [
"### 1. QA/QC Procedure"
]
},
{
"cell_type": "markdown",
"id": "4551d26a",
"metadata": {},
"source": [
"[HiPerSeis](https://github.com/GeoscienceAustralia/hiperseis) implements elaborate workflows for computing corrections for:\n",
"\n",
"1. [GPS clock errors](https://github.com/GeoscienceAustralia/hiperseis/tree/develop/seismic/gps_corrections)\n",
" \n",
" This workflow outputs simple csv files for each station processed, as follows:\n",
" ```\n",
" net,sta,loc,comp,date,clock_correction\n",
" OA,BM26,,HHZ,2019-05-25T00:00:00,0.040261028\n",
" OA,BM26,,HHZ,2019-05-26T00:00:00,0.269642617\n",
" OA,BM26,,HHZ,2019-05-27T00:00:00,0.518772089\n",
" OA,BM26,,HHZ,2019-05-28T00:00:00,0.787649446\n",
" OA,BM26,,HHZ,2019-05-29T00:00:00,1.076274687\n",
" ...\n",
" ```\n",
" Each row above lists the daily clock-correction (in seconds) for a given station. \n",
" \n",
" \n",
"2. [Station-orientation errors](https://github.com/GeoscienceAustralia/hiperseis/tree/develop/seismic#orientation-analysis)\n",
"\n",
" This workflow generates orientation correction estimates based on two separate \n",
" methods: (i) Receiver function ([Wilde-Piórko et al. 2017](https://doi.org/10.1007/s10950-017-9640-x)), (ii) Surface wave polarization ([Doran and Lakse, 2017](https://doi.org/10.1785/0120160165))\n",
"\n",
" Correction estimates are output in Json format as follows:\n",
" ```json\n",
" {\n",
" \"rf\": {\n",
" \"OA.BK24.\": {\n",
" \"date_range\": [\n",
" \"2019-05-30T15:45:33.505000Z\",\n",
" \"2020-09-13T11:58:30.490000Z\"\n",
" ],\n",
" \"azimuth_correction\": -4.0\n",
" }\n",
" ...\n",
" ...\n",
" }\n",
" \"swp\": {\n",
" \"OA.BK24.\": {\n",
" \"date_range\": [\n",
" \"2019-05-26T07:40:04.360000Z\",\n",
" \"2020-09-11T11:35:53.700000Z\"\n",
" ],\n",
" \"azimuth_correction\": 1.3485540759336345,\n",
" \"uncertainty\": 3.6205002297639233\n",
" }\n",
" ...\n",
" ...\n",
" }\n",
"}\n",
"``` \n",
" Correction estimates from the surface wave polarization method have associated uncertainties, which are currently unavailable for those from the receiver function method."
]
},
{
"cell_type": "markdown",
"id": "202bc997",
"metadata": {},
"source": [
"### 2. Embedding QC/QC Metadata into StationXML Files"
]
},
{
"cell_type": "markdown",
"id": "6d4d249f",
"metadata": {},
"source": [
"Obspy [inventory utilities](https://docs.obspy.org/packages/autogen/obspy.core.inventory.inventory.Inventory.html) are used to embed the corrections described above within StationXML files as [extra station metadata](https://docs.obspy.org/tutorial/code_snippets/stationxml_custom_tags.html).\n",
"\n",
"Once these QA/QC metadata are embedded, a typical StationXML file looks like as follows:\n",
"\n",
"```xml\n",
"<Network code=\"OA\" \n",
" startDate=\"2017-09-11T00:00:36\" \n",
" endDate=\"2018-11-28T23:06:20\">\n",
" <Station code=\"BY22\" \n",
" startDate=\"2017-10-01T13:58:58\" \n",
" endDate=\"2018-11-24T21:13:47\">\n",
" <GeoscienceAustralia:clock_corrections>[{'0M': [{'2018-07-28T00:00:00.000000 - 2018-07-29T00:00:00.000000': 0.432706767}, {'2018-07-29T00:00:00.000000 - 2018-07-30T00:00:00.000000': 0.768295739}, {'2018-07-30T00:00:00.000000 - 2018-07-31T00:00:00.000000': 1.124155978}, {'2018-07-31T00:00:00.000000 - 2018-08-01T00:00:00.000000': 1.500287483}, {'2018-08-01T00:00:00.000000 - 2018-08-02T00:00:00.000000': 1.896690255}, {'2018-08-02T00:00:00.000000 - 2018-08-03T00:00:00.000000': 2.313364293}, {'2018-08-03T00:00:00.000000 - 2018-08-04T00:00:00.000000': 2.750309598}, {'2018-08-04T00:00:00.000000 - 2018-08-05T00:00:00.000000': 3.207526168}, {'2018-08-05T00:00:00.000000 - 2018-08-06T00:00:00.000000': 3.685014006}, {'2018-08-06T00:00:00.000000 - 2018-08-07T00:00:00.000000': 4.182773109}, {'2018-08-07T00:00:00.000000 - 2018-08-08T00:00:00.000000': 4.700803479}, {'2018-08-08T00:00:00.000000 - 2018-08-09T00:00:00.000000': 5.239105116}, {'2018-08-09T00:00:00.000000 - 2018-08-10T00:00:00.000000': 5.797678019}, {'2018-08-10T00:00:00.000000 - 2018-08-11T00:00:00.000000': 6.376522188}, {'2018-08-11T00:00:00.000000 - 2018-08-12T00:00:00.000000': 6.975637623}, {'2018-08-12T00:00:00.000000 - 2018-08-13T00:00:00.000000': 7.595024326}, {'2018-08-13T00:00:00.000000 - 2018-08-14T00:00:00.000000': 8.234682294}, {'2018-08-14T00:00:00.000000 - 2018-08-15T00:00:00.000000': 8.894611529}, {'2018-08-15T00:00:00.000000 - 2018-08-16T00:00:00.000000': 9.57481203}]}]\n",
" </GeoscienceAustralia:clock_corrections>\n",
" \n",
" <GeoscienceAustralia:rf_orientation_corrections>[{'0M': [{'2017-10-06T08:08:02.844999 - 2018-11-22T16:21:07.070000': 5.0}]}] \n",
" </GeoscienceAustralia:rf_orientation_corrections>\n",
"\n",
" <GeoscienceAustralia:swp_orientation_corrections>[{'0M': [{'2017-10-03T06:52:09.700000 - 2018-11-15T01:21:51.440000': [2.373463113875232, 3.271484514337502]}]}]\n",
" </GeoscienceAustralia:swp_orientation_corrections>\n",
" \n",
"...\n",
"...\n",
"```\n",
"\n",
"For each station, `clock_corrections` contain lists of dictionaries with timespan as the key and clock-correction as the value in seconds, for each location code. Similarly, `rf_orientation_corrections` and `swp_orientation_corrections` contain orientation correction estimates. "
]
},
{
"cell_type": "markdown",
"id": "a801817b",
"metadata": {},
"source": [
"### 3. Extracting QA/QC Metadata\n",
"\n",
"QA/QC metadata can be extracted through the Obspy [Inventory](https://docs.obspy.org/tutorial/code_snippets/stationxml_custom_tags.html) interface as follows: "
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "b07ed886",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Clock corrections for station BS25:\n",
"\n",
"AttribDict({'namespace': 'https://github.com/GeoscienceAustralia/hiperseis', 'value': \"[{'0M': [{'2018-04-16T00:00:00.000000 - 2018-04-17T00:00:00.000000': -0.916666667}, {'2018-04-17T00:00:00.000000 - 2018-04-18T00:00:00.000000': -0.886904762}, {'2018-04-18T00:00:00.000000 - 2018-04-19T00:00:00.000000': -0.857142857}, {'2018-04-19T00:00:00.000000 - 2018-04-20T00:00:00.000000': -0.827380952}, {'2018-04-20T00:00:00.000000 - 2018-04-21T00:00:00.000000': -0.797619048}, {'2018-04-21T00:00:00.000000 - 2018-04-22T00:00:00.000000': -0.767857143}, {'2018-04-22T00:00:00.000000 - 2018-04-23T00:00:00.000000': -0.738095238}, {'2018-04-23T00:00:00.000000 - 2018-04-24T00:00:00.000000': -0.708333333}, {'2018-04-24T00:00:00.000000 - 2018-04-25T00:00:00.000000': -0.833334188}, {'2018-04-25T00:00:00.000000 - 2018-04-26T00:00:00.000000': -1.133324819}, {'2018-04-26T00:00:00.000000 - 2018-04-27T00:00:00.000000': -0.433329917}, {'2018-04-27T00:00:00.000000 - 2018-04-28T00:00:00.000000': 0.266667879}, {'2018-04-28T00:00:00.000000 - 2018-04-29T00:00:00.000000': 0.966668568}, {'2018-04-29T00:00:00.000000 - 2018-04-30T00:00:00.000000': -0.200005913}, {'2018-04-30T00:00:00.000000 - 2018-05-01T00:00:00.000000': -0.877278375}, {'2018-05-01T00:00:00.000000 - 2018-05-02T00:00:00.000000': -1.054550836}, {'2018-05-02T00:00:00.000000 - 2018-05-03T00:00:00.000000': -1.481823297}, {'2018-05-03T00:00:00.000000 - 2018-05-04T00:00:00.000000': -2.159095759}, {'2018-05-04T00:00:00.000000 - 2018-05-05T00:00:00.000000': -2.58636822}, {'2018-05-05T00:00:00.000000 - 2018-05-06T00:00:00.000000': -2.763640681}, {'2018-05-06T00:00:00.000000 - 2018-05-07T00:00:00.000000': -3.190913143}, {'2018-05-07T00:00:00.000000 - 2018-05-08T00:00:00.000000': -3.618185604}, {'2018-05-08T00:00:00.000000 - 2018-05-09T00:00:00.000000': -4.045458065}, {'2018-05-09T00:00:00.000000 - 2018-05-10T00:00:00.000000': -4.472730527}, {'2018-04-13T00:00:00.000000 - 2018-04-14T00:00:00.000000': 0.833333333}, {'2018-04-14T00:00:00.000000 - 2018-04-15T00:00:00.000000': -0.166666667}, {'2018-04-15T00:00:00.000000 - 2018-04-16T00:00:00.000000': -1.166666667}, {'2017-12-02T00:00:00.000000 - 2017-12-03T00:00:00.000000': -0.436435534}, {'2017-12-03T00:00:00.000000 - 2017-12-04T00:00:00.000000': 0.218395387}, {'2017-12-04T00:00:00.000000 - 2017-12-05T00:00:00.000000': 0.277598394}, {'2017-12-05T00:00:00.000000 - 2017-12-06T00:00:00.000000': -0.258826513}, {'2017-12-09T00:00:00.000000 - 2017-12-10T00:00:00.000000': 0.291396688}, {'2017-12-10T00:00:00.000000 - 2017-12-11T00:00:00.000000': 0.172606192}, {'2017-12-11T00:00:00.000000 - 2017-12-12T00:00:00.000000': 0.053815695}, {'2017-12-15T00:00:00.000000 - 2017-12-16T00:00:00.000000': -0.5}, {'2017-12-16T00:00:00.000000 - 2017-12-17T00:00:00.000000': -0.5}, {'2018-03-09T00:00:00.000000 - 2018-03-10T00:00:00.000000': 0.163461538}, {'2018-03-10T00:00:00.000000 - 2018-03-11T00:00:00.000000': 0.5}, {'2018-03-11T00:00:00.000000 - 2018-03-12T00:00:00.000000': 0.528846154}, {'2018-03-12T00:00:00.000000 - 2018-03-13T00:00:00.000000': 0.25}, {'2018-03-14T00:00:00.000000 - 2018-03-15T00:00:00.000000': 0.285714286}, {'2018-03-15T00:00:00.000000 - 2018-03-16T00:00:00.000000': 0.107142857}, {'2018-03-16T00:00:00.000000 - 2018-03-17T00:00:00.000000': 0.214285714}, {'2018-03-17T00:00:00.000000 - 2018-03-18T00:00:00.000000': 0.357142857}, {'2018-03-18T00:00:00.000000 - 2018-03-19T00:00:00.000000': 0.285714286}]}]\"})\n",
"\n",
"RF Orientation correction for station BS25:\n",
"\n",
"AttribDict({'namespace': 'https://github.com/GeoscienceAustralia/hiperseis', 'value': \"[{'': [{'2019-12-04T20:15:16.408000 - 2021-03-19T19:02:55.508000': -1.0}]}, {'0M': [{'2018-07-06T01:50:27.592000 - 2018-08-17T22:17:35.916000': -7.0}]}]\"})\n",
"\n",
"SWP Orientation correction and uncertainty for station BS25:\n",
"\n",
"AttribDict({'namespace': 'https://github.com/GeoscienceAustralia/hiperseis', 'value': \"[{'': [{'2019-12-15T06:10:41.648000 - 2021-03-06T04:15:42.108000': [-5.141686537338579, 3.30488525300443]}]}, {'0M': [{'2018-07-06T01:38:56.168000 - 2018-08-16T01:56:55.020000': [-0.7883395667681177, 5.303916191852374]}]}]\"})\n"
]
}
],
"source": [
"from obspy import Inventory\n",
"from obspy.core.inventory import Network, read_inventory\n",
"\n",
"inv = read_inventory('OA.xml')\n",
"sta = inv.networks[0].stations[1]\n",
"\n",
"# Extract corrections from the inventory \n",
"if(hasattr(sta.extra, 'clock_corrections')):\n",
" print('Clock corrections for station {}:\\n'.format(sta.code))\n",
" print(sta.extra.clock_corrections)\n",
"# end if\n",
"\n",
"if(hasattr(sta.extra, 'rf_orientation_corrections')):\n",
" print('\\nRF Orientation correction for station {}:\\n'.format(sta.code))\n",
" print(sta.extra.rf_orientation_corrections)\n",
"# end if\n",
"\n",
"if(hasattr(sta.extra, 'swp_orientation_corrections')):\n",
" print('\\nSWP Orientation correction and uncertainty ' \\\n",
" 'for station {}:\\n'.format(sta.code))\n",
" print(sta.extra.swp_orientation_corrections)\n",
"# end if"
]
},
{
"cell_type": "markdown",
"id": "d23417fa",
"metadata": {},
"source": [
"### 4. Metadata Extractor Script\n",
"\n",
"In addition to accessing metadata programmatically above, a script [export_corrections.py](https://github.com/GeoscienceAustralia/hiperseis/blob/develop/seismic/inventory/export_corrections.py) is provided to extract the metadata into csv files.\n",
"\n",
"#### Requirements\n",
"\n",
"python3, obspy (version 1.1.0), pandas\n",
"\n",
"#### Launch Exporter\n",
"\n",
"The meta-data exporter is launched as follows:\n",
"\n",
"`python export_corrections.py OA.xml meta`\n",
"\n",
"The inputs are: (i) a StationXML file with embedded metadata (ii) output-file base-name\n",
"\n",
"Upon successful completion, two csv files should be produced:\n",
"\n",
" 1. `meta.clock_corrections.csv`, with typical content as follows:\n",
" ```\n",
"# GPS clock-corrections grouped by network, station and location\n",
"Network,Station,Location,Start-time,End-time,Correction_in_seconds\n",
"OA,BS25,0M,2018-04-16T00:00:00.000000,2018-04-17T00:00:00.000000,-0.916666667\n",
"OA,BS25,0M,2018-04-17T00:00:00.000000,2018-04-18T00:00:00.000000,-0.886904762\n",
" ...\n",
" ...\n",
" ```\n",
" 2. `meta.orientation_corrections.csv`, with typical content as follows:\n",
"```\n",
"# Orientation corrections are derived from two separate methods: (i) Receiver Function (RF) (ii) Surface-wave Polarization (SWP). Only the latter method provides uncertainty estimates.\n",
"Network,Station,Location,Method,Start-time,End-time,Azimuth_correction_in_degrees,Uncertainty±\n",
"OA,BS24,0M,RF,2017-10-06T08:08:12.755000,2018-08-17T22:17:31.755000,2.0,\n",
"OA,BS25,,RF,2019-12-04T20:15:16.408000,2021-03-19T19:02:55.508000,-1.0,\n",
"...\n",
"...\n",
"```\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.8"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

0 comments on commit bc37783

Please sign in to comment.