[REVIEW]: GraphPPL.jl: A Julia Package for probabilistic model and inference constraints specification

[whedon]

Submitting author: @bvdmitri (Dmitry Bagaev)
Repository: https://github.com/biaslab/GraphPPL.jl
Branch with paper.md (empty if default branch):
Version:
Editor: @matbesancon
Reviewers: @theogf, @sethaxen
Archive: Pending

Status

Status badge code:

HTML: <a href="https://proceedings.juliacon.org/papers/fcc2b8dbec8e463e7d8f4656ffc19c77"><img src="https://proceedings.juliacon.org/papers/fcc2b8dbec8e463e7d8f4656ffc19c77/status.svg"></a>
Markdown: [![status](https://proceedings.juliacon.org/papers/fcc2b8dbec8e463e7d8f4656ffc19c77/status.svg)](https://proceedings.juliacon.org/papers/fcc2b8dbec8e463e7d8f4656ffc19c77)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@theogf & @sethaxen, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

1. Make sure you're logged in to your GitHub account
2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @matbesancon know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Review checklist for @theogf

Conflict of interest

• As the reviewer I confirm that I have read the JuliaCon conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JuliaCon code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Authorship: Has the submitting author (@bvdmitri) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Paper format

• Authors: Does the paper.tex file include a list of authors with their affiliations?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
• Page limit: Is the page limit for extended abstracts respected by the submitted document?

Content

• Context: is the scientific context motivating the work correctly presented?
• Methodology: is the approach taken in the work justified, presented with enough details and reference to reproduce it?
• Results: are the results presented and compared to approaches with similar goals?

Review checklist for @sethaxen

Conflict of interest

• As the reviewer I confirm that I have read the JuliaCon conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JuliaCon code of conduct.

General checks

• Repository: Is the source code for this software available at the repository url?
• License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
• Authorship: Has the submitting author (@bvdmitri) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

• Installation: Does installation proceed as outlined in the documentation?
• Functionality: Have the functional claims of the software been confirmed?
• Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
• Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
• Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
• Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Paper format

• Authors: Does the paper.tex file include a list of authors with their affiliations?
• A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
• Page limit: Is the page limit for extended abstracts respected by the submitted document?

Content

• Context: is the scientific context motivating the work correctly presented?
• Methodology: is the approach taken in the work justified, presented with enough details and reference to reproduce it?
• Results: are the results presented and compared to approaches with similar goals?

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @theogf, @sethaxen it looks like you're currently assigned to review this paper 🎉.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/JuliaCon/proceedings-review) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

1. Set yourself as 'Not watching' https://github.com/JuliaCon/proceedings-review:

2. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

PDF failed to compile for issue #123 with the following error:

 Can't find any papers to compile :-(

[whedon]

Attempting PDF compilation from custom branch paper/julia_proceedings_2. Reticulating splines etc...

[whedon]

PDF failed to compile for issue #123 with the following error:

 Can't find any papers to compile :-(

[bvdmitri]

@whedon generate pdf from branch paper/julia_proceedings_2022

[whedon]

Attempting PDF compilation from custom branch paper/julia_proceedings_2022. Reticulating splines etc...

[whedon]

PDF failed to compile for issue #123 with the following error:

 Can't find any papers to compile :-(

[matbesancon]

@whedon generate pdf from branch paper/julia_proceedings_2022

[whedon]

Attempting PDF compilation from custom branch paper/julia_proceedings_2022. Reticulating splines etc...

[whedon]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[whedon]

👋 @theogf, please update us on how your review is going (this is an automated reminder).

[whedon]

👋 @sethaxen, please update us on how your review is going (this is an automated reminder).

[theogf]

I have started reviewing the abstract and the package and before I go further, there are some major issues for acceptance:

• In the docs, the examples do not work. There is no @model in GraphPPL.jl.
• If it is somewhere else it should be mentioned in the examples (and the docs statements should be corrected)
• There is close to zero test on the main functionality of the package

See generally the missing points in Functionality and Documentation in my checklist.

I have additional comments regarding the text but they can wait.

[bvdmitri]

Hey @theogf , your concerns are valid. That happened because we splitted up (it was quite some time since the initial submission) our repository and now the @model macro lives in the RxInfer repository. So the repository is currently not in a very good shape. The GraphPPL is being refactored by a PhD student of ours to allow more features, tests and be self isolated from the RxInfer. This work, however, will take some time :/

[matbesancon]

@bvdmitri thanks for the response. Given the context, could it be worth submitting the proceeding later, when the package is more mature and stabilized in its new version?

[bvdmitri]

@matbesancon At the current stage I would agree with you, we better shall resubmit the new version. We expect it to be available in summer.

[matbesancon]

I will close this for now with that tag, thanks everyone

[lucaferranti]

@editorialbot reject

[editorialbot]

Paper rejected.