Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Peer Review] Reproduction of results of manuscript #11

Closed
cthoyt opened this issue Oct 2, 2020 · 0 comments
Closed

[Peer Review] Reproduction of results of manuscript #11

cthoyt opened this issue Oct 2, 2020 · 0 comments

Comments

@cthoyt
Copy link
Contributor

cthoyt commented Oct 2, 2020

The main research output of this manuscript is obviously the code in this repository. However, to assess the veracity of the claims made in the paper about its goodness in comparison to previous libraries, the code for running the benchmarking experiments, organizing their respective results, and turning them into the figures that appear in the main manuscript and supplemental material should be made available (in a form that is readily useful). I'd go as far as to suggest putting it in a different repository entirely, to mitigate confusion and disorganization.

I appreciate that it's not easy to set up a barrage of different packages written in a variety of languages, and understand that it might take significant time and effort to prepare this code for the same scrutiny as the package itself. I hope you agree that it is necessary to assess the methodology used for benchmarking to give an accurate review.

@RemyLau RemyLau closed this as completed in 35cdd4e Feb 9, 2021
@cthoyt cthoyt changed the title Reproduction of results of manuscript [Peer Review] Reproduction of results of manuscript Aug 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant