-
Notifications
You must be signed in to change notification settings - Fork 78
Add initial tests for diagnostic metrics #335
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Codecov ReportAll modified and coverable lines are covered by tests ✅ ❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. The tests and fixtures are generally idiomatic and in line with the rest of bayesflow. I would avoid testing against constant shapes, unless this is explicitly what you want. For example, should the shape of ce["values"] be (3,) or identical to some other shape, e.g. y.shape[1]?
As for disabling the parametrization over the batch size, I think we can turn autouse off on that fixture. If this causes other problems, I will have to make some other changes first, so it should be fine if these tests are run twice for now.
I do not think there is much we can/should do about testing plots. You could verify that the functions return the correct object. But in general, as long as the function runs, we can expect the plotting libraries to do the heavy lifting here.
|
Great, thank you! |
|
I have now added tests for all metrics and most plots. @Kucharssim will add some tests for the model comparison plots soon in another PR. I will merge this PR as soon as all checks are passing. |
I have started writing tests for the diagnostics module, specicially the diagnostic metrics. I have a few questions for @LarsKue before I continue adding more tests.
batch_size2 and 3. I don't want or need to batch_size to play any role here. It's not something I use in these tests nor do I know where it's coming from. How can I deactivate that the tests are run multiple times?