Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

separate scoring from train/test #162

Open
kegl opened this issue Nov 26, 2018 · 0 comments
Open

separate scoring from train/test #162

kegl opened this issue Nov 26, 2018 · 0 comments

Comments

@kegl
Copy link
Contributor

kegl commented Nov 26, 2018

At this time y_pred objects are wrapped into Predictions types, read out from there, and passed to scoring. Two issues: 1) they are not converted to and deconverted from zipped numpy arrays, and 2) scores are not run independently, so if someone created common objects in score, they work with ramp_test_submission but not on the server.

We could keep the second feature, make sure it works in the same way in ramp-engine. But for the first, we should separate scoring from train test, and test that scoring works on saved/loaded predictions. There is a default now, slightly different here and in board, we should make sure that the conversion-deconversion code is the same. I think save/load or convert/deconvert should be implemented as a member class in Predictions, so if someone wants a custom code, they would have to implement a new derived Predictions class.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant