Skip to content

Commit

Permalink
Merge 5a57e4f into 0e99071
Browse files Browse the repository at this point in the history
  • Loading branch information
@claudiodsf
claudiodsf committed Jun 30, 2016
2 parents 0e99071 + 5a57e4f commit b27648e
Show file tree
Hide file tree
Showing 4 changed files with 142 additions and 46 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.txt
View file
Expand Up @@ -29,6 +29,8 @@ master:
vertical, see #1445).
- obspy.io.css
* Read support for NNSA KB Core format waveform data. (see #1332)
- obspy.io.quakeml
* Read and write support for nested custom tags (see #1463)
- obspy.io.segy:
* Iterative reading of large SEG-Y and SU files with
`obspy.io.segy.segy.iread_segy` and `obspy.io.segy.segy.iread_su`.
Expand Down
106 changes: 85 additions & 21 deletions misc/docs/source/tutorial/code_snippets/quakeml_custom_tags.rst
View file
Expand Up @@ -4,12 +4,12 @@
Handling custom defined tags in QuakeML and the ObsPy Catalog/Event framework
=============================================================================

QuakeML allows use of custom elements in addition to the "usual" information
QuakeML allows use of custom elements in addition to the 'usual' information
defined by the QuakeML standard. It allows *a)* custom namespace attributes to
QuakeML namespace tags and *b)* custom namespace subtags to QuakeML namespace
elements.
ObsPy can handle both basic (non-nested) custom tags in event type objects
(*a)*) and custom attributes (*b)*) during input/output to/from QuakeML.
ObsPy can handle both basic custom tags in event type objects (*a*) and custom
attributes (*b*) during input/output to/from QuakeML.
The following basic example illustrates how to output a valid QuakeML file
with custom xml tags/attributes:

Expand All @@ -18,23 +18,23 @@ with custom xml tags/attributes:
from obspy import Catalog, UTCDateTime
extra = {'my_tag': {'value': True,
'namespace': r"http://some-page.de/xmlns/1.0",
'attrib': {r"{http://some-page.de/xmlns/1.0}my_attrib1": "123.4",
r"{http://some-page.de/xmlns/1.0}my_attrib2": "567"}},
'my_tag_2': {'value': u"True",
'namespace': r"http://some-page.de/xmlns/1.0"},
'namespace': 'http://some-page.de/xmlns/1.0',
'attrib': {'{http://some-page.de/xmlns/1.0}my_attrib1': '123.4',
'{http://some-page.de/xmlns/1.0}my_attrib2': '567'}},
'my_tag_2': {'value': u'True',
'namespace': 'http://some-page.de/xmlns/1.0'},
'my_tag_3': {'value': 1,
'namespace': r"http://some-page.de/xmlns/1.0"},
'namespace': 'http://some-page.de/xmlns/1.0'},
'my_tag_4': {'value': UTCDateTime('2013-01-02T13:12:14.600000Z'),
'namespace': r"http://test.org/xmlns/0.1"},
'namespace': 'http://test.org/xmlns/0.1'},
'my_attribute': {'value': 'my_attribute_value',
'type': 'attribute',
'namespace': r"http://test.org/xmlns/0.1"}}
'namespace': 'http://test.org/xmlns/0.1'}}
cat = Catalog()
cat.extra = extra
cat.write("my_catalog.xml", "QUAKEML",
nsmap={"my_ns": r"http://test.org/xmlns/0.1"})
cat.write('my_catalog.xml', format='QUAKEML',
nsmap={'my_ns': 'http://test.org/xmlns/0.1'})
All custom information to be stored in the customized QuakeML has to
be stored in form of a :class:`dict` or
Expand All @@ -58,25 +58,30 @@ The xml output of the above example looks like:
.. code-block:: xml
<?xml version='1.0' encoding='utf-8'?>
<q:quakeml xmlns:q="http://quakeml.org/xmlns/quakeml/1.2" xmlns:ns0="http://some-page.de/xmlns/1.0"
xmlns:my_ns="http://test.org/xmlns/0.1" xmlns="http://quakeml.org/xmlns/bed/1.2">
<eventParameters publicID="smi:local/b425518c-9445-40c7-8284-d1f299ed2eac" my_ns:my_attribute="my_attribute_value">
<ns0:my_tag ns0:my_attrib1="123.4" ns0:my_attrib2="567">true</ns0:my_tag>
<q:quakeml xmlns:q='http://quakeml.org/xmlns/quakeml/1.2'
xmlns:ns0='http://some-page.de/xmlns/1.0'
xmlns:my_ns='http://test.org/xmlns/0.1'
xmlns='http://quakeml.org/xmlns/bed/1.2'>
<eventParameters publicID='smi:local/b425518c-9445-40c7-8284-d1f299ed2eac'
my_ns:my_attribute='my_attribute_value'>
<ns0:my_tag ns0:my_attrib1='123.4' ns0:my_attrib2='567'>true</ns0:my_tag>
<my_ns:my_tag_4>2013-01-02T13:12:14.600000Z</my_ns:my_tag_4>
<ns0:my_tag_2>True</ns0:my_tag_2>
<ns0:my_tag_3>1</ns0:my_tag_3>
</eventParameters>
</q:quakeml>
When reading the above xml again, the custom tags get parsed and attached to
the respective Event type objects (in this example to the Catalog object) as
``.extra``:
When reading the above xml again, using
:meth:`read_events() <obspy.core.event.read_events>`, the custom tags get
parsed and attached to the respective Event type objects (in this example to
the Catalog object) as ``.extra``.
Note that all values are read as text strings:

.. code-block:: python
from obspy import read_events
cat = read("my_catalog.xml")
cat = read_events('my_catalog.xml')
print(cat.extra)
.. code-block:: python
Expand All @@ -94,3 +99,62 @@ the respective Event type objects (in this example to the Catalog object) as
u'value': 'True'},
u'my_tag_3': {u'namespace': u'http://some-page.de/xmlns/1.0',
u'value': '1'}})
Custom tags can be nested:

.. code-block:: python
from obspy import Catalog
from obspy.core import AttribDict
ns = 'http://some-page.de/xmlns/1.0'
my_tag = AttribDict()
my_tag.namespace = ns
my_tag.value = AttribDict()
my_tag.value.my_nested_tag1 = AttribDict()
my_tag.value.my_nested_tag1.namespace = ns
my_tag.value.my_nested_tag1.value = 1.23E+10
my_tag.value.my_nested_tag2 = AttribDict()
my_tag.value.my_nested_tag2.namespace = ns
my_tag.value.my_nested_tag2.value = True
cat = Catalog()
cat.extra = AttribDict()
cat.extra.my_tag = my_tag
cat.write('my_catalog.xml', 'QUAKEML')
This will produce an xml output similar to the following:

.. code-block:: xml
<?xml version='1.0' encoding='utf-8'?>
<q:quakeml xmlns:q='http://quakeml.org/xmlns/quakeml/1.2'
xmlns:ns0='http://some-page.de/xmlns/1.0'
xmlns='http://quakeml.org/xmlns/bed/1.2'>
<eventParameters publicID='smi:local/97d2b338-0701-41a4-9b6b-5903048bc341'>
<ns0:my_tag>
<ns0:my_nested_tag1>12300000000.0</ns0:my_nested_tag1>
<ns0:my_nested_tag2>true</ns0:my_nested_tag2>
</ns0:my_tag>
</eventParameters>
</q:quakeml>
The output xml can be read again using
:meth:`read_events() <obspy.core.event.read_events>` and the nested tags can be
retrieved in the following way:

.. code-block:: python
from obspy import read_events
cat = read_events('my_catalog.xml')
print(cat.extra.my_tag.value.my_nested_tag1.value)
print(cat.extra.my_tag.value.my_nested_tag2.value)
.. code-block:: python
12300000000.0
true
20 changes: 17 additions & 3 deletions obspy/io/quakeml/core.py
View file
Expand Up @@ -29,6 +29,7 @@
import os
import warnings

from collections import Mapping
from lxml import etree

from obspy.core.event import (Amplitude, Arrival, Axis, Catalog, Comment,
Expand Down Expand Up @@ -987,7 +988,13 @@ def _extra(self, element, obj):
for el in element.iterfind("{%s}*" % ns):
# remove namespace from tag name
_, name = el.tag.split("}")
value = el.text
# check if element has children (nested tags)
if len(el):
sub_obj = AttribDict()
self._extra(el, sub_obj)
value = sub_obj.extra
else:
value = el.text
try:
extra = obj.setdefault("extra", AttribDict())
# Catalog object is not based on AttribDict..
Expand Down Expand Up @@ -1171,7 +1178,10 @@ def _extra(self, obj, element):
"""
if not hasattr(obj, "extra"):
return
for key, item in obj.extra.items():
self._custom(obj.extra, element)

def _custom(self, obj, element):
for key, item in obj.items():
value = item["value"]
ns = item["namespace"]
attrib = item.get("attrib", {})
Expand All @@ -1182,7 +1192,11 @@ def _extra(self, obj, element):
if type_.lower() in ("attribute", "attrib"):
element.attrib[tag] = str(value)
elif type_.lower() == "element":
if isinstance(value, bool):
# check if value is dictionary-like
if isinstance(value, Mapping):
subelement = etree.SubElement(element, tag, attrib=attrib)
self._custom(value, subelement)
elif isinstance(value, bool):
self._bool(value, element, tag, attrib=attrib)
else:
self._str(value, element, tag, attrib=attrib)
Expand Down
60 changes: 38 additions & 22 deletions obspy/io/quakeml/tests/test_quakeml.py
View file
Expand Up @@ -871,32 +871,43 @@ def test_write_with_extra_tags_and_read(self):
# - tag with explicit namespace and namespace abbreviation
my_extra = AttribDict(
{'public': {'value': False,
'namespace': r"http://some-page.de/xmlns/1.0",
'attrib': {u"some_attrib": u"some_value",
u"another_attrib": u"another_value"}},
'custom': {'value': u"True",
'namespace': r'http://test.org/xmlns/0.1'},
'namespace': 'http://some-page.de/xmlns/1.0',
'attrib': {'some_attrib': 'some_value',
'another_attrib': 'another_value'}},
'custom': {'value': 'True',
'namespace': 'http://test.org/xmlns/0.1'},
'new_tag': {'value': 1234,
'namespace': r"http://test.org/xmlns/0.1"},
'namespace': 'http://test.org/xmlns/0.1'},
'tX': {'value': UTCDateTime('2013-01-02T13:12:14.600000Z'),
'namespace': r'http://test.org/xmlns/0.1'},
'dataid': {'namespace': r'http://anss.org/xmlns/catalog/0.1',
'type': 'attribute', 'value': '00999999'}})
nsmap = {"ns0": r"http://test.org/xmlns/0.1",
"catalog": r'http://anss.org/xmlns/catalog/0.1'}
'namespace': 'http://test.org/xmlns/0.1'},
'dataid': {'namespace': 'http://anss.org/xmlns/catalog/0.1',
'type': 'attribute', 'value': '00999999'},
# some nested tags :
'quantity': {'namespace': 'http://some-page.de/xmlns/1.0',
'attrib': {'attrib1': 'attrib_value1',
'attrib2': 'attrib_value2'},
'value': {
'my_nested_tag1': {
'namespace': 'http://some-page.de/xmlns/1.0',
'value': 1.23E10},
'my_nested_tag2': {
'namespace': 'http://some-page.de/xmlns/1.0',
'value': False}}}})
nsmap = {'ns0': 'http://test.org/xmlns/0.1',
'catalog': 'http://anss.org/xmlns/catalog/0.1'}
cat[0].extra = my_extra.copy()
# insert a pick with an extra field
p = Pick()
p.extra = {'weight': {'value': 2,
'namespace': r"http://test.org/xmlns/0.1"}}
'namespace': 'http://test.org/xmlns/0.1'}}
cat[0].picks.append(p)

with NamedTemporaryFile() as tf:
tmpfile = tf.name
# write file
cat.write(tmpfile, format="QUAKEML", nsmap=nsmap)
cat.write(tmpfile, format='QUAKEML', nsmap=nsmap)
# check contents
with open(tmpfile, "rb") as fh:
with open(tmpfile, 'rb') as fh:
# enforce reproducible attribute orders through write_c14n
obj = etree.fromstring(fh.read()).getroottree()
buf = io.BytesIO()
Expand Down Expand Up @@ -930,26 +941,31 @@ def test_write_with_extra_tags_and_read(self):
# - we always end up with a namespace definition, even if it was
# omitted when originally setting the custom tag
# - custom namespace abbreviations should attached to Catalog
self.assertTrue(hasattr(cat[0], "extra"))
self.assertTrue(hasattr(cat[0], 'extra'))

def _tostr(x):
if isinstance(x, bool):
if x:
return str("true")
return str('true')
else:
return str("false")
return str(x)
return str('false')
elif isinstance(x, AttribDict):
for key, value in x.items():
x[key].value = _tostr(value['value'])
return x
else:
return str(x)

for key, value in my_extra.items():
my_extra[key]['value'] = _tostr(value['value'])
self.assertEqual(cat[0].extra, my_extra)
self.assertTrue(hasattr(cat[0].picks[0], "extra"))
self.assertTrue(hasattr(cat[0].picks[0], 'extra'))
self.assertEqual(
cat[0].picks[0].extra,
{'weight': {'value': '2',
'namespace': r'http://test.org/xmlns/0.1'}})
self.assertTrue(hasattr(cat, "nsmap"))
self.assertEqual(getattr(cat, "nsmap")['ns0'], nsmap['ns0'])
'namespace': 'http://test.org/xmlns/0.1'}})
self.assertTrue(hasattr(cat, 'nsmap'))
self.assertEqual(getattr(cat, 'nsmap')['ns0'], nsmap['ns0'])


def suite():
Expand Down

0 comments on commit b27648e

Please sign in to comment.