Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unimplemented simplified validation #130

Closed
wlngai opened this issue Jun 29, 2017 · 1 comment
Closed

Unimplemented simplified validation #130

wlngai opened this issue Jun 29, 2017 · 1 comment

Comments

@wlngai
Copy link
Contributor

wlngai commented Jun 29, 2017

A full validation has been deployed on each benchmark run, which is quite heavy and verbose. Sometimes validation dataset might not be available. It will be easier to determine whether a benchmark execution has completed by checking if the output has the correct size.

@wlngai
Copy link
Contributor Author

wlngai commented Jun 29, 2017

Before the validation step, the benchmark runner will first count the number of valid outputs. If the output size does not match expectation, then a completion failure (COM) will be thrown.

@wlngai wlngai closed this as completed Jun 29, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant