Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Metrics #92

Merged
merged 23 commits into from Nov 5, 2021
Merged

Refactor Metrics #92

merged 23 commits into from Nov 5, 2021

Conversation

teubert
Copy link
Collaborator

@teubert teubert commented Nov 4, 2021

Refactored existing metrics.

Previous Implementation had metrics entirely in a file called samples. Each metric would work on only one item. Input to metrics would be an array of numbers.

Metrics have now been refactored to include the following changes:

  • Metrics act on uncertain data (or ToEProfile for profile_based metrics)
    • Ability to act on array of numbers or array of dicts kept for backwards compatibility
  • Metrics are no longer all in samples, they've been split based on what they act on (e.g., ToE Metrics, General Metrics, and ToE Profile Metrics)
  • When relevant some metrics have been added as modules for the appropriate data structure (e.g., UncertainData)
  • Added tests

See follow-up #93 which will add an example of alpha-lambda

Note: #91 must be merged before this works.

@teubert teubert changed the title Extension of Metrics Refactor Metrics Nov 4, 2021
@teubert teubert linked an issue Nov 4, 2021 that may be closed by this pull request
@teubert teubert marked this pull request as ready for review November 4, 2021 22:00
@teubert teubert requested a review from kjjarvis November 4, 2021 23:15
Co-authored-by: kjjarvis <55932920+kjjarvis@users.noreply.github.com>
@teubert
Copy link
Collaborator Author

teubert commented Nov 5, 2021

Note for the record: /test-prog-models-released are expected to fail, as the dev branch of prog_algs is not compatible with the currently released version (i.e., they include breaking changes).

@teubert teubert merged commit 76a0ff2 into dev Nov 5, 2021
@teubert teubert deleted the feature/metrics_extension branch November 5, 2021 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update metrics to operate on sample-based and non-sample based results
2 participants