Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Review #2 #3

Open
distillpub-reviewers opened this Issue Dec 3, 2018 · 2 comments

Comments

Projects
None yet
3 participants
@distillpub-reviewers
Copy link
Collaborator

commented Dec 3, 2018

The following peer review was solicited as part of the Distill review process.

The reviewer chose to keep anonymity. Distill offers reviewers a choice between anonymous review and offering reviews under their name. Non-anonymous review allows reviewers to get credit for the service them offer to the community.

Distill is grateful to the reviewer for taking the time to review this article.


Labeling figures would have been helpful. Axis labels (even dummy values) on some plots would have been helpful because it's not clear what they refer to on the first pass, such as the first figures in the "Gaussian Process" and "Kernel" sections. It's also not intuitive how to play with the figures.

For diagrams with a shaded portion (first and last figures in article), it's not clear how much "confidence" (standard deviations) they show. Are they showing the mean plus/minus one standard deviation? Two standard deviations?

The definition for regression is not precise (what does "as close as possible" mean?). On the definition of Gaussian Processes, I would say that GPs give a confidence for the predicted value for a given input. GPs can be viewed as a distribution over functions, so it’s not accurate to use the term “predicted function” in the first definition.

For the definition of GPs, I would avoid the use of the term “kernel” and prefer the term “covariance function”. Or include a caveat to avoid confusion between GP kernels and kernel methods (e.g. SVMs).


Distill employs a reviewer worksheet as a help for reviewers.

The first three parts of this worksheet ask reviewers to rate a submission along certain dimensions on a scale from 1 to 5. While the scale meaning is consistently "higher is better", please read the explanations for our expectations for each score—we do not expect even exceptionally good papers to receive a perfect score in every category, and expect most papers to be around a 3 in most categories.

Any concerns or conflicts of interest that you are aware of?: No known conflicts of interest
What type of contributions does this article make?: Explanation of existing results

Advancing the Dialogue Score
How significant are these contributions? 2/5
Outstanding Communication Score
Article Structure 3/5
Writing Style 3/5
Diagram & Interface Style 2/5
Impact of diagrams / interfaces / tools for thought? 2/5
Readability 2/5
Scientific Correctness & Integrity Score
Are claims in the article well supported? 2/5
Does the article critically evaluate its limitations? How easily would a lay person understand them? 2/5
How easy would it be to replicate (or falsify) the results? 3/5
Does the article cite relevant work? 3/5
Does the article exhibit strong intellectual honesty and scientific hygiene? 2/5

@grtlr grtlr added this to the revisions milestone Jan 3, 2019

@RKehlbeck

This comment has been minimized.

Copy link
Collaborator

commented Feb 7, 2019

Thank you very much for taking the time to review our article! We really appreciate the detailed comments and feedback!

We are just wrapping up all the changes. I will reference the corresponding PRs and commits in this issue.

@RKehlbeck

This comment has been minimized.

Copy link
Collaborator

commented Feb 7, 2019

PR #15 adresses

Labeling figures would have been helpful. Axis labels (even dummy values) on some plots would have been helpful because it's not clear what they refer to on the first pass, such as the first figures in the "Gaussian Process" and "Kernel" sections. It's also not intuitive how to play with the figures.

The definition for regression is not precise (what does "as close as possible" mean?). On the definition of Gaussian Processes, I would say that GPs give a confidence for the predicted value for a given input. GPs can be viewed as a distribution over functions, so it’s not accurate to use the term “predicted function” in the first definition.

PR #20 adresses

For diagrams with a shaded portion (first and last figures in article), it's not clear how much "confidence" (standard deviations) they show. Are they showing the mean plus/minus one standard deviation? Two standard deviations?

For the definition of GPs, I would avoid the use of the term “kernel” and prefer the term “covariance function”. Or include a caveat to avoid confusion between GP kernels and kernel methods (e.g. SVMs).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.