Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Amplitude-Magnitude Regression #51

Merged
merged 133 commits into from
Dec 10, 2021
Merged

Conversation

damb
Copy link
Collaborator

@damb damb commented Oct 5, 2021

With this PR scdetect is able to estimate magnitudes based on a template family amplitude-magnitude regression approach (for further information please refer to https://doi.org/10.1029/2019JB017468).

Features and Changes:

  • Provide magnitude processing facilities.
  • Allow the configuration of template families based on a template-families.json configuration file (Template family configuration requirements #50).
  • Provide the both the --templates-family-json path/to/templates-family.json and the --magnitudes-force=0|1 (in accordance to the --amplitudes-force) CLI flags
  • Note that the magnitudes being computed correspond to DataModel::StationMagnitude (which in fact are sensor location magnitudes)
  • Document configuration requirements
  • Fix several bugs when computing MLx amplitudes
  • Prepare the design in order to compute alternative magnitudes (i.e. apart from the regression magnitudes implemented by this PR). Examples for alternatives:
  • Refactor bindings configuration options in order to allow the magnitude types to be computed to be configured on sensor location granularity (regarding sensor location bindings configuration).
  • Refactor config namespace

References: #50

@damb damb added enhancement New feature or request magnitude Refers to magnitude related topics processing Refers to topics w.r.t. data processing labels Oct 5, 2021
@damb damb self-assigned this Oct 5, 2021
@damb damb force-pushed the feature/amplitude-magnitude-regression branch 8 times, most recently from 71f3ad4 to 8744f09 Compare October 15, 2021 15:27
@damb damb force-pushed the feature/amplitude-magnitude-regression branch 2 times, most recently from 73153da to 18bea38 Compare November 9, 2021 16:09
@damb damb force-pushed the feature/amplitude-magnitude-regression branch from 18bea38 to 1e70f3c Compare November 10, 2021 18:05
@damb damb force-pushed the feature/amplitude-magnitude-regression branch from 1e70f3c to b95e5b0 Compare November 29, 2021 17:02
Daniel Armbruster added 13 commits November 29, 2021 17:07
Do not use the Status as in `WaveformProcessor`. Instead wrap the status
into an magnitude processor specific exception.
The purpose of this class is to handle a template family configuration.
A template family configuration is a container for `ReferenceConfig`s
i.e. configuration referencing a template family member. At the time
being a reference configuration may:

  - reference a stream configuration from a detector configuration
    (indirection)
  - reference a sensor location which is NOT used for detection, but
    used anyway for the template family based *amplitude-magnitude*
    regression.
Do not use boilerplate template `AnyFixedSlopeRegressionMagnitude`.
Instead, `std::ratio<>` straight away in
`FixedSlopeRegressionMagnitude`. Therefore,
`FixedSlopeRegressionMagnitude` is the template, now.
Provide dedicated exception.
@damb
Copy link
Collaborator Author

damb commented Dec 6, 2021

Ok, almost done - I know, it took quite a while.

Though, there is still kind of a bug when it comes to computing the magnitude. As a consequence a small unexpected offset is introduced. I'll have a look into it, tomorrow.

@damb
Copy link
Collaborator Author

damb commented Dec 7, 2021

Ok. It should be fixed now.

There is still the remaining issue that the taper length is not included to the amplitude processor's safety time window. I'll fix this as part of another PR.

@damb damb force-pushed the feature/amplitude-magnitude-regression branch 3 times, most recently from e235d22 to 445a12f Compare December 8, 2021 12:59
Daniel Armbruster added 19 commits December 8, 2021 13:05
Trim only if the difference is greater than or equal to a single sample.
Note that it of course matters which data is passed to the
`deconvolveData()` method.

- Compute the trend from the correct time window.
- Apply the response to the correct time window. Note that the response
  is applied to the full dataset in frequency domain.

There is still a remaining issue regarding the preprocessing. That is,
the taper length should be included to the amplitude processor's safety
margin.
Describe template family based amplitude-magnitude regression approach.
Disable magnitude processing. Note that currently, the integration test
facilities do neither test amplitude calculation nor do they test the
magnitude estimation. Though, it would be good if these facilities would
be tested, appriopriately.
Allow magnitudes to be configurable on sensor location granularity.
`calculateAmpltiudes` -> `createAmplitudes`
`calculateMagnitues`  -> `createMagnitudes`
Do not allow the configuration of magnitude types on detector level
granularity. Both amplitude types and magnitude types are going to
configured with bindings (i.e. on sensor location level granularity).

Implement a standalone function in order to extract the sensor location
stream indentifier from the `util::WaveformID`.

Correctly parse and forward magnitude related publish configuration.
When creating `util::WaveformID` an exception is thrown, anyway, if the
parameters supplied are not valid.
@damb damb marked this pull request as ready for review December 10, 2021 12:58
@damb damb merged commit 44c628c into master Dec 10, 2021
@damb damb deleted the feature/amplitude-magnitude-regression branch December 10, 2021 13:02
@damb
Copy link
Collaborator Author

damb commented Dec 10, 2021

@mmesim, ready to go. It's time to give it a try.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request magnitude Refers to magnitude related topics processing Refers to topics w.r.t. data processing
Projects
None yet
1 participant