Skip to content

Applying Malfante's AAA code base

Jump to bottom
Glenn Thompson edited this page Nov 4, 2021 · 5 revisions

For ease, I copied the AAA repository into a folder in this repository here. This allows me to modify this in a way to works best for the Montserrat dataset.

I've added a MONTSERRAT folder which contains configuration files.

I added several pre-computed features:

  • band_ratio_1 (bandratio_[1.0_6.0_11.0]),
  • band_ratio_2 (bandratio_[0.8_4.0_16.0]),
  • medianF,
  • peakF,
  • bw_min
  • bw_max by adding extra functions to featuresFunctions.py. These are loaded from from precomputed_metrics.csv. The read_montserrat function I added to DataReadingFunctions.py ensures that precomputed_metrics.csv is a copy of the appropriate event CSV file.

A test data flow based on the one used by Falcin is contained in a Python Notebook called test_montserrat.ipynb.

The workflow I had developed to loop over many different Trace IDs and included labels (jackknifing) is at 00_catalogue_reading_montserrat.ipynb.

Algorithm

Preparing the catalog

  1. Copy external file MVO_labelled_events.csv from PyMSEC folder to MONTSERRAT/catalog/MVO_labelled_events_filtered.csv
  2. Read external file MVO_labelled_events.csv from PyMSEC folder into a DataFrame called 'cat'.
  3. Print out total number of events, and by label
  4. Create a blank list called 'frames'
  5. Loop over rows fo 'cat' DataFrame.
  • Read row['path'] (points to WAV file).
  • Replace '/WAV' (change to name of PyMSEC folder) with 'eventFiles' and add '.pickle' to WAV filename.
  • If found, update the row in 'cat' to point to the pickle file instead. (If not found, next row).
  • Load the corresponding event CSV file into DataFrame called 'tracedf'.
  • Set tracedf['filename'] = row['filename']
  • Append tracedf to frames
  1. Concatenate all tracedf DataFrames from frames list into a single DataFrame called 'alltraces'.
  2. Save alltraces to MONTSERRAT/alltracesDFs.csv
  3. Print number of events each trace ID occurs in.
  4. Save 'cat' to MONTSERRAT/catalog/MVO_labelled_events_filtered.csv and corresponding .pd file.

Machine learning

  1. Read general configuration from config/general/newsettings_10.json. Currently this sets analysis_type to "continuous", specific configuration to "config/specific/usecase1_continuous_classification/usecase1_EXAMPLE.json", ML algorithm to "RandomForestClassifier(n_estimators=100,criterion='entropy',bootstrap=True, class_weight=None)", splits the data 30 times into 50% training, 50% testing, sets the folder to "montserrat", the path_to_catalogue to "catalog/MVO_labelled_events_filtered.pd", the features to "path_to_config":"config/specific/features/features_montserrat.json", computation_domains to "time spectral cepstral", and thresholds to 0.8 for each class. The specific configuration defines the reading_function as 'read_montserrat', data_files as '*.pickle' and a path_to_data as 'Dropbox/MVO_labelled_events/'. It also seems to set a single time window of 15-s, sampling rate of 100 sps, a 2-pole bandpass filter from 0.5-25 Hz, and 256 samples per FFT.
  2. Read and check the configuration
  3. Set list of trace IDs to loop over ['MV.MBWH..SHZ', 'MV.MBLG..SHZ', 'MV.MBRY..SHZ']
  4. Set list of minimum weight to loop over (1-4)
  5. Set list of lists of classes to include [ ['l', 't'], ['e', 'r'], ['h', 'l', 't'], ['h', 'l', 't', 'r'], ['e', 'h', 'l', 't', 'r'] ]
  6. Define functions to filter alltracesDFs by:
  • traceID
  • classes
  • minimum weight
  1. Define function to check that there are sufficient traces in each class to continue (I use 30)
  2. Initialize a counter to 0 and a results_list to a blank list
  3. Loop over traceIDs:
  • Write this traceID to MONTSERRAT/current_traceID.txt (used by DataReadingFunctions.py).
  • Loop over minimum weights
  • Loop over classes to include
  • Load the catalog pd file into 'cat' DataFrame
  • print traceID, classes included, minimum weight
  • initialize dict called 'results' with traceID, classes, minWeight, NtraceID, Nclasses, Nweight, counts
  • if there are enough events in each class:
  • print number of events, grouped by class
  • create Analyzer object called 'analyzer' using the config and 'cat' DataFrame. analyzer.catalogue is now the 'cat' DataFrame.
  • run Analyzer.learn method. This is where it gets complicated, so I broke it down into a separate section below. It returns a list of all signatures (the AAA code base sometimes calls the waveform traces, whether or not converted to the spectral or cepstral domains, signals or signatures), allLabels, and an accuracy (a list with one element per class, or per event?)
  • compute mean and standard deviation of accuracy
  • add a column called predicted_class to 'cat', set to allLabels
  • save this updated 'cat' DataFrame to MONTSERRAT/catalog/MVO_labelled_events_filtered_predicted_(counter).csv
  • append accuracy mean and std to results_list
  • increment the counter
  1. Create a DataFrame from the results_list and write to MONTSERRAT/results.csv

Analyzer.learn

  1. Create a FeatureVector object called 'features', which has attributes including domains, featuresFunctions, featuresValues, n_domains and n_values. It calls features._readFeaturesFunctions() to load config/specific/features/features_montserrat.json
  2. Loop over each row in analyzer.catalogue:
  1. Get the start time, duration, path.
  2. Call the requestObservation function in DataReadingFunctions.py which calls the read_montserrat function set in the config, and returns fs and signature.
  3. Return unfiltered signature as an element of allData list, and the label is returned as an element of allLabels list.
  4. Bandpass filter (which uses fs) is commented out currently, but okay because I pre-filter traces (and remove trend, etc.).
  5. Call extract_features from automatic_processing/tools.py which:
  • Optionally (currently True) normalizes the signature
  • Calls features.compute() which:
  • Converts signature to correct domain (not needed for time domain)
  • Calls features.computation() which actually runs each featureFunction on the signature. The results are returned in features.featureValues (a list of length domains * features).
  1. An element of allFeatures (a list of lists) is now set equal to features.featureValues (a list). PERHAPS HERE I COULD ELIMINATE DUPLICATE PRE-COMPUTED FEATURES. OR EVEN BETTER, I ADD A NEW FUNCTION TO ONLY ADD THEM HERE.
  1. Each element in allFeatures (a list of lists) is now equal to features.featureValues (a list) for one row/trace.
  2. (text) allLabels are transformed to integer labels using sklearn.preprocessing.LabelEncoder.html
  3. allFeatures are scaled to mean 0 variance 1 using sklearn.preprocessing.StandardScaler
  4. the model is set to RandomForestClassifier(n_estimators=100,criterion='entropy',bootstrap=True, class_weight=None). This can be changed in the config. This means 100 trees. Default criterion is 'gini'. Check all parameters at sklearn.ensemble.RandomForestClassifier.html
  5. Call the fit method of the model, with X=allFeatures, y=allLabels. This is the LEARNING step.
  6. Call the predict method of the model, with allFeatures. This is the TESTING step, and returns the predicted class. HERE I COULD INSTEAD USE predict_proba, which returns predicted class probabilities instead, which I could then weight over multiple channels.
  7. Compute the accuracy score with sklearn.metrics.accuracy_score.html.
  8. Compute (and display) the confusion matrix. See sklearn.metrics.confusion_matrix.html. True labels should be X/rows, predicted labels Y/columns.
  9. Finally, there is a cross-validation step which uses the 'cv' config. This is configured to call a StratifiedShuffleSplit which splits the labelled transients into 30 sets of 50% training and 50% testing. This gives 30 different confusion matrices and accuracy scores.

Results

Results from Paris are at MONTSERRAT/results.csv. These can be compared with results obtained without the extra features I added here.

Clone this wiki locally