StationXML 1.x handling

[nikosT]

Hi,
Concerning the issue with the StationXML versioning:
When a FDSN-station service is queried for station metadata the first line of the responded XML document is showing the schema Version. For example:

• If we request this information http://eida.gein.noa.gr/fdsnws/station/1/query?network=HL&level=network, the first line of the responded XML file will show: <FDSNStationXML schemaVersion="1.0">.

• Now, if we request this information http://ws.resif.fr/fdsnws/station/1/query?&network=CL&level=network, the first line of the responded XML file will show: <FDSNStationXML schemaVersion="1.1">.

I was wondering if the eida-federator could parse this information and convert performantly the StationXML version 1.1 to 1.0, even as a temporary fix.

About the conversion:

• For the missing fields that are presented at the StationXML 1.0 but not at the StationXML 1.1, N/A indication will be enough.
• For the fields that are presented at the StationXML 1.1 but not at the StationXML 1.0, discarding (of them) will be applied.

[damb]

@nikosT, thanks for your hints.

Solving the StationXML1.x issue is not that straight forward:

• Currently, eida-federator does some parsing. However, it uses a SAX parser, exclusively. Furthermore, only BaseNodeType attributes are interpreted (i.e. a hash is computed from those attributes) in order to detect BaseNodeTypes already seen before. This needs to be done for merging StationXML from different sources. Additional elements are neither interpreted nor validated (EIDA DCs are required to take the responsibility for the (meta)data integrity.).
  The approach does perform quite well and does not require to read the metadata with something such as obspy. (FYI: The current implementation is based on lxml which again is based on libxml2).
  A performant solution is required since a service such as eida-federator needs to serve multiple clients concurrently. It is exposed to a much higher aggregated load. Thus, an implementation by means of obspy is no option anymore. Again, the approach proposed by yourself would require to fully decompose the metadata.
• eida-federator can only support one single version of StationXML. Which version to be chosen? StationXML1.0 or rather StationXML1.1? How to convert between those versions is not defined anywhere. E.g. how to interpret an element/attribute which is optional in StationXML1.0 and thus for instance not defined but mandatory in StationXML1.1? AFAIK the conversion between those two formats is not lossless.

• For the missing fields that are presented at the StationXML 1.0 but not at the StationXML 1.1, N/A indication will be enough.

Did you verify if such a document would validate?

To conclude: From the current point of view, I doubt if the changes would be worth the effort.

[kaestli]

n/a does not seem to be a valid DateTime: https://www.w3.org/TR/xmlschema-2/#dateTime

[nikosT]

Hi guys,
@damb, concerning the first point, would it be possible to request or decompose -let's say- the "headers" of the meta-data info? Perhaps, the /endpoints info of the Routing Service could include information of such. For example, an XML field where the DC refers to which version of the StationXML is using, among other info too.

eida-federator can only support one single version of StationXML.
Maybe, we should consider supporting multiple cases even if effort and energy is wasted. What if a DC move to 1.3 next year?

@kaestli

n/a does not seem to be a valid DateTime: https://www.w3.org/TR/xmlschema-2/#dateTime

Could we consider an EIDA convention about the N/A, like a negative year as it's supported by the specification (-0001 = 1 BC)?

[damb]

@nikosT,

Perhaps, the /endpoints info of the Routing Service could include information of such. For example, an XML field where the DC refers to which version of the StationXML is using, among other info too.

Hmm. This would be another example of EIDA internal configuration and such a reason for setting up a configuration service. See also the discussion in EIDA/routing#62.

Maybe, we should consider supporting multiple cases even if effort and energy is wasted. What if a DC move to 1.3 next year?

As long EIDA webservices implement the API of webservices specified by FDSN it needs to be discussed at FDSN level how to move to a new StationXML1.x format and how a StationXML *.xsd incompatible modification is implemented with respect to /fdsnws/station webservices.

IIf FDSN would allow returning any StationXML *.xsd incompatible version using one and only the path /fdsnws/station/1/, clients such as eida-federator need to implement the expensive merging solution. However, then it still needs to be defined how those documents are merged, practically.

[kaestli]

@nikosT: i am not sure whether i get your first point. But I believe that (sooner or later) a next version of the routing protocol should (among other changes) include a version tag for the services dataselect, station, wfcatalog etc.

On the second point:
a) changing the data type of a StationXML element or attribute from type="xs:dateTime" (ISO 8601 formatted) to something else would require a new version of StationXML (or a violation of both StationXML 1.0 and 1.1) - so I don't think this can solve anything here.
b) Negative years are covered because they are covered by ISO 8601 the dateTime xml basetype refers to.
c)Replacing xs:dateTime by a proprietary time representation has a considerable impact on software development: Developers cannot leverage existing software packages as easily any more & may have to re-implement functionality such as sorting, converting etc.

[nikosT]

@kaestli

@nikosT: i am not sure whether i get your first point. But I believe that (sooner or later) a next version of the routing protocol should (among other changes) include a version tag for the services dataselect, station, wfcatalog etc.

I mean exposing EIDA node configuration as primary data (i.e. among mseed, inventory and routing).

a) changing the data type of a StationXML element or attribute from type="xs:dateTime" (ISO 8601 formatted) to something else would require a new version of StationXML (or a violation of both StationXML 1.0 and 1.1) - so I don't think this can solve anything here.

I agree!

b) Negative years are covered because they are covered by ISO 8601 the dateTime xml basetype refers to.

So, we could use, for example, the negative year to indicate N/A.

c)Replacing xs:dateTime by a proprietary time representation has a considerable impact on software development: Developers cannot leverage existing software packages as easily any more & may have to re-implement functionality such as sorting, converting etc.

I agree!

[damb]

Closed. StationXML1.1 is in place.