ANSS PDL Indexer Background

jmfee-usgs edited this page Mar 12, 2013 · 2 revisions

ANSS PDL/Indexer Background

DRAFT

Last updated: 2013-03-12

Overview

This document provides background information about products and the indexer, to provide context for how quakeml is mapped into products for the catalog.

Products

A product is a container for a specific version of one "type" of information. Each "source" assigns a unique "code" to each piece of information they send, and an "updateTime" when that information is sent. Type, source, and code is a unique identifier for each product, and updateTime is a version for that product.

A product has "status", "properties", "links", and "contents". The only reserved status is case insensitive DELETE. Properties are "key"/"value" pairs with information or metadata; each key has only one value. Links have a "relation" and "uri"; each relation may have many URIs. Contents have "path", "type", "modified", and "bytes" attributes.

Versions

More recent updateTimes for the same product (type,source,code) supersede earlier updateTimes for that same product (type,source,code). Each version of a product is immutable and can only be updated by sending the same product (type,source,code) with a later updateTime.

To delete a product, send an update with status=DELETE. To undelete a product, send an update with status=UPDATE (anything except DELETE).

PDL/Indexer

Events

The indexer receives products via PDL, and groups related products together into events. It associates using Event ID or preferred Location. The preferred Location and Event ID for an Event come from the most preferred "origin" type product; when there is no origin, from the most preferred product that has Location and Event ID.

While the indexer will create events without "origin" type products, for the purpose of ComCat an Event only exists if it has an associated origin product.

When all associated origins for an event have been deleted, the event is considered deleted.

Indexer specific Product properties

The indexer relies on several product properties to associate related information:

Event ID

A product has an event id when it has eventsource and eventsourcecode properties.

"eventsource"     - a unique source of events
"eventsourcecode" - a unique code assigned by the eventsource

Location

A product has a location when it has latitude, longitude, and eventtime properties.

"latitude"        - latitude of the event
"longitude"       - longitude of the event
"eventtime"       - time of the event

Other

The indexer also uses these properties:

"magnitude"       - magnitude of the event
"depth"           - depth of the event in km
"version"         - sender assigned version code, for information only

Preferred weights

When multiple contributors contribute the same type of product to an event, only one product of each type is most preferred across all contributors. The indexer assigns a weight to each product and considers the highest weight most preferred. When weights are equal, the most recent updateTime is more preferred.

The default rules for preferred weights are based on ANSS authoritative polygons:

weight = 1
if product source is authoritative at location
	weight += 100
if event source is authoritative at location
	weight += 50
if product source = event source
	weight += 5

The indexer also supports custom modules to compute preferred weights using different rules when needed. One example of this is a ShakeMap module, which additionally prefers maps centered on the epicenter.

Indexer specific products

Associate

Force the association of two events that would otherwise not associate. The two events "merge".

Disassociate

Force the disassociation of two events that would otherwise associate. The two events "split".

Trump

Set the preferred weight of a specific version of a product.

Indexer Association

When the indexer receives a product, it takes these steps to associate it:

  1. store product
  2. summarize product, determine preferred weight
  3. find previous version of product
  4. find existing event
    • based on Event ID
    • based on Location (within 16 seconds, ~111km)
    • if not found, based on association of previous version of product
    • when more than one event found, choose closest event
  5. add summary to index
  6. try to associate summary to event
    • found event? associate
    • else, can create event? create event, associate
    • else, unassociated
  7. if associated to event
    • check event for splits
      • compare each sub event (per eventid) location
      • new disassociate products
    • check event for merges
      • nearby events in space and time
      • existing products with event id but no location
      • new associate products
    • if new trump product, modify product weight
    • update preferred event attributes in index
  8. notify listeners

More Information

More information about PDL and the Indexer can be found in the PDL User Guide.