Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Compute classification metrics locally via numpy #651

Draft
wants to merge 30 commits into
base: main
Choose a base branch
from
Draft

Compute classification metrics locally via numpy #651

wants to merge 30 commits into from
+1,955 −553

Conversation

ntlind
Copy link
Collaborator

@ntlind ntlind commented Jul 3, 2024 •
edited
Loading

Improvements

  • Use arrays instead of looping over grouper keys / values
  • Rewrite all metrics in numpy vs sqlalchemy

Benchmarks

GitHub Runner Classification

image

GitHub Runner Object Detection

image

TODOs Before Merging

  • Implement PR curves for classification tasks
  • Implement DetailedPrecisionRecallCurve for both tasks
  • Delete unneeded helper functions. Convert functional tests where necessary
  • Solve pyright issues with pandas

ntlind added 26 commits July 3, 2024 09:13
@ntlind
working through integration tests
d7c8edf
@ntlind
Merge branch 'main' into compute_local_metrics
abf6075
@ntlind
pass functional tests after merge conflicts
18528d6
@ntlind
pass non-ROC integration tests
0334741
@ntlind
add ROCAUC metric
56767df
@ntlind
pass a few more integration tests
d0f0df0
@ntlind
pass integration tests
f1d4b5a
@ntlind
Merge branch 'main' into compute_local_metrics
b0238c3
@ntlind
minor cleanup
a1422ce
@ntlind
update npm test
17dbb00
@ntlind
up benchmarks
2e39ade
@ntlind
update benchmarks
1f4c354
@ntlind
pass first set of AR and AP tests
026094a
@ntlind
add curves
8e9828a
@ntlind
fix OD iou calculation for rasters
3869b27
@ntlind
pass functional tests
9cd1654
@ntlind
fix groundtruths with no predictions
1fa240c
@ntlind
Merge branch 'main' into compute_local_metrics
3e1f438
@ntlind
fix gts in PR output
62a660a
@ntlind
pass more integration tests, edge cases still outstanding
39466db
@ntlind
pass integration tests
b75087b
@ntlind
small benchmarking script changes
dd19fff
@ntlind
clean up and add aggregate OD functions
87930e2
@ntlind
remove deletion
479dab0
@ntlind
adjust benchmarks
66b47b2
@ntlind
update od benchmarks
6dc83bc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@czaloom czaloom czaloom left review comments

@ekorman ekorman Awaiting requested review from ekorman ekorman will be requested when the pull request is marked ready for review ekorman is a code owner

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@ntlind @czaloom