Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] Accuracy from Model.evaluate() is inconsistent with manually computed accuracy #109

Closed
alexander-g opened this issue Nov 16, 2020 · 2 comments · Fixed by #110
Closed

[Bug] Accuracy from Model.evaluate() is inconsistent with manually computed accuracy #109

alexander-g opened this issue Nov 16, 2020 · 2 comments · Fixed by #110
Labels
bug Something isn't working

Comments

@alexander-g
Copy link
Contributor

Describe the bug
The accuracy reported by Model.fit() or Model.evaluate() is different from manually computed via model.predict(x).argmax(-1) == y).mean()

Minimal code to reproduce
Colab Notebook

Expected behavior
Both should be the same.

Library Info
Latest commit in master 19ec87b
@alexander-g alexander-g added the bug Something isn't working label Nov 16, 2020
@cgarciae
Copy link
Collaborator

@alexander-g thanks for reporting!

I will add this as a test case once its solved.

@cgarciae cgarciae mentioned this issue Nov 17, 2020
Merged
@cgarciae
Copy link
Collaborator

cgarciae commented Nov 17, 2020 •
edited

The problem was that it wasn't getting the initial state for the metrics correct so it was getting the wrong numbers after accumulation.

This is fixed in #110 which should be merged after #108.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix metrics error poets-ai/elegy
2 participants
@alexander-g @cgarciae