Preparing a set of labelled events from PyMSEC format

Overview

To conduct supervised machine learning, we need to prepare a set of labelled events. The Montserrat seismic event catalog (originally stored as a Seisan database, but now converted to PyMSEC format) uses the following labels (classifications):

• teleseism (D),
• regional (R),
• volcano-tectonic earthquake (t),
• hybrid earthquake (h),
• long-period earthquake (l),
• rockfall (r),
• noise (n),
• tremor (m),
• gages (g),
• unknown (u).

But MVO event classifications are unreliable (I say this as someone who originally classified many of those events while at MVO, 1996-2004), which is the main motivation for attempting supervised learning. PyMSEC format includes metrics that have been computed for each trace for each event. What we do next is:

1. **Compute a statistical fingerprint is created for each label. These fingerprints are used in the next step.
2. Iteratively select equal numbers of events of each class & verify or relabel them. At each iteration, the new information is used to recalculate the fingerprints. This is a primitive form of "active learning", i.e. trying to find the best set of events to manually label to train a Random Forest.

Event selection

The following steps are undertaken to improve the original manual classifications to prepare the Seisan database for supervised machine learning. These steps are necessary because the original event classifications are unreliable, classification is difficult so it makes more sense to apply a probability for each event class, and each event file may also contain several events, or may be garbage. So what we do is iteratively select the top N events of each class, manually reclassify them giving percentage probabilities for each class (while marking bad events for deleted, or multi-event files for splitting). We can also generate/update fingerprints for each class, which can help guide the manual classification, or be used to automatically classify events.

20_select_events: select the top N events of each class or accepted subclass. A file called 01_seisandb_classes.csv is required for the latter to be available. By default, only unchecked events are loaded. To choose the top unchecked events, the quality is used. Optionally, checked events can be picked instead. To choose the top checked events, the probability is used, and a minimum threshold given (default: 50%). In either case, a set of class fingerprints is generated. These are stored in two CSV files: checked_fingerprints.csv and unchecked_fingerprints.csv. The function fingerprint_events is used in both cases. For unchecked events, the original subclass is used. For checked events, the probabilistic subclasses are used and weighted using the weight column (see 61).`

21_reclassify_events_manually: manually (re)classify the selected events. a probability can be given for each mainclass or accepted subclass. a weight can also be given, indicating how clear the signal is. or the event can be marked for splitting because it is judged to contain multiple events. All processed events are marked as checked unless skipped.

22_reclassify_events_with_fingerprints: automatically reclassify the selected events. the probability for each class will be the result of comparing to event metrics to the fingerprints.

23_write_AAA_input_file: looks through the entire sfile_index.csv, extracts only the top probability N events of each mainclass/subclass that have been checked (and not marked for splitting or deletion). Not sure how to use the weight. Or does it just use the selected events?

For selecting events:

1. Enable user to manually split event files into separate events.
2. Enable "noise" and "unknown" classifications. Implement a "weight" for each classification, from 0-9. Unchecked events are given a weight of 3. Very good examples can be given a weight of 9 and therefore weighted more strongly in the fingerprints.
3. Enable user to assign a probability to each classification. Keep asking for further classifications until probability reaches 1.0 (including unknown and noise). (a) Implement classification using single characters as 'T'='teleseism', 'R'='regional', 'r'='rockfall/PDC', 'e'='long-period + rockfall', 'l'='long-period', 'h'='hybrid', 'm'='tremor', 't'='volcano-tectonic', 'n'='noise', 'u'='unknown'. (b) A classification code should be followed by ', p' where p is a percentage (an integer from 1-100). If omitted, percentage defaults to 100. (c) A percentage should be followed by ', w' where w is a weight. If omitted, weight defaults to 3. (d) To accept an existing classification with probability 100% and weight 3 . (e) To accept the probabilities from the fingerprints, enter 'f'. (f) To split an event, 's' (do not enter a percentage or weight). This will open a separate window to clip out each event. (g) If an event is garbage, 'd' should delete it and all corresponding files, including from the indexes. (h) 'k' skips to the next event.

Valid entries: 'h' # 100% hybrid, weight 3 'h, 100' # 100% hybrid, weight 3 'h, 100, 9' # 100% hybrid, weight 9 'h, 50, l, 50' # 50% hybrid, 50% long-period, weight 3 's' # mark for splitting 'd' # mark for deletion

Goal for Montserrat catalog

Beyond this, a goal is to label at least 100 events of each class for each phase/pause of the eruption (e.g. breakpoints at Jan 1992, Jul 1995, Mar 1998, Nov 1999, Aug 2003, etc.) and for each year of the eruption. This would then allow evolution of events to be studied. But first, I will just construct a single model for the entire dataset.

Outstanding questions

An important question is whether only uncorrected traces, only corrected traces, or whether a mixture of uncorrected and corrected traces, should be used for supervised learning. An argument for uncorrected traces is poorly corrected traces could have a lower similarity. An argument for corrected traces is that instrument corrections at a particular site can vary as digitizers and seismometers are swapped or reconfigured, and that corrected traces should eliminate if any influence of these providing instrument response records are complete. But given that some instrument responses are not recorded, the set of corrected traces will be smaller than the set of uncorrected traces. There isn't perhaps a good argument for mixing them, as a mixed set will still only contain the same number of traces as the uncorrected set. We could experiment by applying supervised learning to both uncorrected and corrected traces. This implies we need to create two parallel directory structures, ideally with access to the same list of labelled (i.e. reviewed/reclassified) events.

Improvements

• SeisanDB to Pickle/CSV:

• Extract station and volcano coordinates too.

• Metrics:

• Should a stronger high pass filter be used?
• I could use peak amplitudes across the network. Perhaps by running ASL on that. Only because MBLG is so often much larger than other traces for rockfalls, but not necessary for earthquakes.
• Spectrograms are often taken to be the most useful metric, since they combine amplitude, phase, frequency and time. Is there a way to include them, given that window length changes and start/end of event within time window is unknown?
• some "magnitude" of each event might be useful too, e.g. from energy-magnitude scale.
• is signal to noise used?
• If phase pick times, locations and magnitudes are available, I should include that information, though weighted in a way that reflects that such information is sparse. If P and S picks exist, event is likely t (or R). If only P, it might be h. If neither, r, e, or l are possibilities too. However, if no picks, could still be t, R or h (even l).
• I could examine the frequency metrics before and after peak amplitude time. And an envelope. What events would this help for:

• r might be expected to have a single peak. Frequency no different before and after.
• e could peak twice - in l and r phase. Frequency around each should be constant, but different from other peak.
• l should peak once.
• h should have a different frequency before and after peak. So this is where measuring frequency on time windows is potentially most useful.

• t should peak once and frequency be constant before and after peak.

• Fingerprints:

• I may need to build a statistical model for each NSLC. And then I could combine guesses from each good channel, perhaps weighted by amplitude or signal-to-noise. The event summary CSV files are then useless. This requires a mechanism to deselect bad traces during manual review, below.

• Picking:

• Would AR_pick be good for detecting impulsive onsets, to separate VTs and hybrids from rockfalls? Not sure how it would do with LPs.
• I could visually pick the start and end of waveforms (might need MATLAB). Could I create an efficient way to then update metrics?

• Keeping previous work:

• Need to keep the work I have done. So need a mechanism that can easily update metrics, and transfer information from previously reviewed events.

• Event review/labelling:

• Implement an interface more like eev (use MATLAB?) to navigate between events more easily? I may have already implemented skipping back or forwards one event, and this may be enough.
• 3-D traces should be included in the visual reclassification.
• A mechanism needs adding to deselect/weight out visually poor traces, rather than do this only at the event level. Those would then be removed from the corresponding tracesdf CSV file.
• Do I need a stronger filter for visual classification?
• Add option to replace full time window with the zoom in window, to sort-of autosplit.
• It might be better to move the manual picking to MATLAB. That would make it easier to select zoom windows, signal onsets and offsets, pick phases and signal peaks, and de-select channels, as well as apply tools like noise suppression and polarization analysis. To apply polarization analysis, I would need to pick a source location for rotating seismograms. This could be based on travel-time location - if available - or on peak-amplitude time differences, or ASL.

• Choosing events:

• It might be best to focus on events before the dome collapsing phase, as those are predominantly just earthquakes. However, we have only the ASN for that time period, and those data clip.

• Splitting events:

• A separate process needs to be divised for splitting marked events. Run processWAV to get updated metrics.

Ensure I save the subclass models each time, so workflow could be:

• Choose number of events for model.
• Select next event. (PYTHON)
• Show waveform plots. Try noise suppression. (MATLAB)
• Show spectrogram. (Create PNG with Python to fixed path. Open in MATLAB).
• Show spectrum. (Create PNG with Python to fixed path. Open in MATLAB).
• Show polarization plot (spectrogram? MATLAB)
• Show any travel-time location, magnitude, phase picks. (MATLAB: get from eventdf/tracedf).
• Show metrics for each trace. (tracedf)
• Show location from ASL, and event magnitude. (Do we run these in real-time in Python after selecting traces?)
• Show guess classification from model. One per channel. (Python.)
• Show on/off times and phase times from network detection and phase picker. (on MATLAB waveform plots).
• Allow user to deselect channels (MATLAB waveform plots). This triggers those channels to be deleted from the trace metrics df.
• Allow user to define multiple event on/off times. (MATLAB waveform plots). This triggers old event to be ignored, and new events to be created, and added to the queue for computing metrics. Any phase picks, locations, magnitudes within timewindow of a new event need to be copied too.
• Allow user to pick phases (MATLAB waveform plots). This will positively identify as earthquake, not rockfall.