Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW]: DFTK: A Julian approach for simulating electrons in solids #69

Closed
42 tasks done
whedon opened this issue Feb 9, 2021 · 69 comments
Closed
42 tasks done

Comments

@whedon
Copy link
Collaborator

whedon commented Feb 9, 2021

Submitting author: @mfherbst (Michael F. Herbst)
Repository: https://github.com/mfherbst/juliacon2020-proceedings-dftk.git
Version:
Editor: @crstnbr
Reviewer: @MasonProtter, @jagot
Archive: 10.5281/zenodo.4707620

Status

status

Status badge code:

HTML: <a href="https://proceedings.juliacon.org/papers/c74a09582eb08785dc4883f3c733e7a9"><img src="https://proceedings.juliacon.org/papers/c74a09582eb08785dc4883f3c733e7a9/status.svg"></a>
Markdown: [![status](https://proceedings.juliacon.org/papers/c74a09582eb08785dc4883f3c733e7a9/status.svg)](https://proceedings.juliacon.org/papers/c74a09582eb08785dc4883f3c733e7a9)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@MasonProtter & @jagot, please carry out your review in this issue by updating the checklist below. If you cannot edit the checklist please:

  1. Make sure you're logged in to your GitHub account
  2. Be sure to accept the invite at this URL: https://github.com/openjournals/joss-reviews/invitations

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @crstnbr know.

Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest

Review checklist for @MasonProtter

Conflict of interest

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Authorship: Has the submitting author (@mfherbst) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Paper format

  • Authors: Does the paper.tex file include a list of authors with their affiliations?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
  • Page limit: Is the page limit for extended abstracts respected by the submitted document?

Content

  • Context: is the scientific context motivating the work correctly presented?
  • Methodology: is the approach taken in the work justified, presented with enough details and reference to reproduce it?
  • Results: are the results presented and compared to approaches with similar goals?

Review checklist for @jagot

Conflict of interest

Code of Conduct

General checks

  • Repository: Is the source code for this software available at the repository url?
  • License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
  • Authorship: Has the submitting author (@mfherbst) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?

Functionality

  • Installation: Does installation proceed as outlined in the documentation?
  • Functionality: Have the functional claims of the software been confirmed?
  • Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
  • Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
  • Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
  • Automated tests: Are there automated tests or manual steps described so that the function of the software can be verified?
  • Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Paper format

  • Authors: Does the paper.tex file include a list of authors with their affiliations?
  • A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
  • References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?
  • Page limit: Is the page limit for extended abstracts respected by the submitted document?

Content

  • Context: is the scientific context motivating the work correctly presented?
  • Methodology: is the approach taken in the work justified, presented with enough details and reference to reproduce it?
  • Results: are the results presented and compared to approaches with similar goals?
@whedon
Copy link
Collaborator Author

whedon commented Feb 9, 2021

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks. @MasonProtter, @jagot it looks like you're currently assigned to review this paper 🎉.

⚠️ JOSS reduced service mode ⚠️

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

⭐ Important ⭐

If you haven't already, you should seriously consider unsubscribing from GitHub notifications for this (https://github.com/JuliaCon/proceedings-review) repository. As a reviewer, you're probably currently watching this repository which means for GitHub's default behaviour you will receive notifications (emails) for all reviews 😿

To fix this do the following two things:

  1. Set yourself as 'Not watching' https://github.com/JuliaCon/proceedings-review:

watching

  1. You may also like to change your default settings for this watching repositories in your GitHub profile here: https://github.com/settings/notifications

notifications

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

@whedon
Copy link
Collaborator Author

whedon commented Feb 9, 2021

Failed to discover a Statement of need section in paper

@whedon
Copy link
Collaborator Author

whedon commented Feb 9, 2021

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.05 s (236.3 files/s, 62851.6 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
TeX                              8            224            176           2445
Ruby                             1              8              4             45
YAML                             1              1              0             21
Markdown                         1              0              0              2
-------------------------------------------------------------------------------
SUM:                            11            233            180           2513
-------------------------------------------------------------------------------


Statistical information for the repository '8bb4722a9e18d3e1a79961c6' was
gathered on 2021/02/09.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Michael F. Herbst                1            57              0          100.00

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Michael F. Herbst            57          100.0          0.0                7.02

@whedon
Copy link
Collaborator Author

whedon commented Feb 9, 2021

Failed to discover a valid open source license.

@whedon
Copy link
Collaborator Author

whedon commented Feb 9, 2021

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

@whedon
Copy link
Collaborator Author

whedon commented Feb 9, 2021

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1039/d0fd00048e is OK
- 10.1088/1361-648x/abcbdb is OK
- 10.1038/natrevmats.2015.4 is OK
- 10.1038/ncomms11962 is OK
- 10.1021/jacs.7b02120 is OK
- 10.1038/nmat1752 is OK
- 10.1063/1.3148892 is OK
- 10.1021/acscentsci.8b00229 is OK
- 10.1063/1.5144261 is OK
- 10.1063/5.0005082 is OK
- 10.1103/PhysRevB.54.1703 is OK
- 10.1103/PhysRevLett.77.3865 is OK
- 10.1088/1361-648x/aa680e is OK
- 10.1016/j.commatsci.2012.10.028 is OK

MISSING DOIs

- None

INVALID DOIs

- None

@carstenbauer
Copy link
Member

@whedon re-invite @jagot as reviewer

@whedon
Copy link
Collaborator Author

whedon commented Feb 9, 2021

The reviewer already has a pending invite.

@jagot please accept the invite by clicking this link: https://github.com/JuliaCon/proceedings-review/invitations

@carstenbauer
Copy link
Member

@whedon re-invite @MasonProtter as reviewer

@whedon
Copy link
Collaborator Author

whedon commented Feb 9, 2021

@MasonProtter already has access.

@whedon
Copy link
Collaborator Author

whedon commented Feb 23, 2021

👋 @jagot, please update us on how your review is going (this is an automated reminder).

@whedon
Copy link
Collaborator Author

whedon commented Feb 23, 2021

👋 @MasonProtter, please update us on how your review is going (this is an automated reminder).

@jagot
Copy link
Collaborator

jagot commented Mar 13, 2021

@mfherbst I would rewrite

For tackling these aspects multidisciplinary collaboration is essen- tial with mathematicians developing more numerically stable al- gorithms, computer scientists providing high-performance imple- mentations, physicists and chemists designing appropriate models, and application scientists integrating the resulting methods inside a suitable simulation workflow.

as something like

To tackle these aspects, multidisciplinary collaboration with mathematicians developing more numerically stable algorithms, computer scientists providing high-performance implementations, physicists and chemists designing appropriate models, and application scientists integrating the resulting methods inside a suitable simulation workflow is essential.

Also, when I install according to the instructions and try the first tutorial, I get the following warning:

┌ Warning: Package DFTK does not have Plots in its dependencies:- If you have DFTK checked out for development and have
│   added Plots as a dependency but haven't updated your primary
│   environment's manifest file, try `Pkg.resolve()`.
│ - Otherwise you may need to report an issue with DFTK
└ Loading Plots into DFTK from project dependency, future warnings for DFTK are suppressed.

@MasonProtter Until JuliaMolSim/DFTK.jl#418 is fixed, you have to do a little bit of monkey patching to run the tutorial:

julia> function DFTK.pymatgen_lattice(lattice::AbstractArray)
           # Notice: Pymatgen uses rows as lattice vectors, so we unpeel
           # our lattice column by column. The default unit in pymatgen is Ångström
           mg = pyimport("pymatgen.core.lattice")
           bohr_to_A = 1 / austrip(1u"Å")
           mg.Lattice([Array(bohr_to_A .* lattice[:, i]) for i in 1:3])
       end

julia> function DFTK.pymatgen_structure(model_or_lattice, atoms)
           mg = pyimport("pymatgen.core.structure")
           pylattice = DFTK.pymatgen_lattice(model_or_lattice)

           n_species = sum(length(pos) for (spec, pos) in atoms)
           pyspecies = Vector{Int}(undef, n_species)
           pypositions = Array{Vector{Float64}}(undef, n_species)
           ispec = 1
           for (spec, pos) in atoms
               for coord in pos
                   pyspecies[ispec] = spec.Z
                   pypositions[ispec] = Vector{Float64}(coord)
                   ispec = ispec + 1
               end
           end
           @assert ispec == n_species + 1

           mg.Structure(pylattice, pyspecies, pypositions)
       end

@mfherbst
Copy link
Member

@jagot Thanks for the review ... and yes, sorry about the pymatgen issue. That's the second time such a breaking interface change has caused DFTK to stop working properly unfortunately. That's why long-term we definitely want to get rid of it.

Also, when I install according to the instructions and try the first tutorial, I get the following warning:

Thanks. That's because we load Plots using Requires, so in some sense it is a "fake-warning". It seems to be fixed with JuliaMolSim/DFTK.jl#419.

@mfherbst
Copy link
Member

Both things should be fixed in the (latest) DFTK 0.2.7 by the way.

@carstenbauer
Copy link
Member

@MasonProtter Since you haven't commented yet, let me kindly ping you as a reminder :)

@MasonProtter
Copy link
Collaborator

MasonProtter commented Mar 25, 2021

Hi, sorry about this. I will be able to review next week, I've been unexpectedly busy lately, but mid next week I should have time for this.

My apologies.

@carstenbauer
Copy link
Member

@whedon remind @MasonProtter in 2 weeks

@whedon
Copy link
Collaborator Author

whedon commented Apr 1, 2021

Reminder set for @MasonProtter in 2 weeks

@MasonProtter
Copy link
Collaborator

MasonProtter commented Apr 2, 2021

Okay, sorry for all the delays. I really enjoyed reading the paper and exploring the package. This is fantastic work with a slick, powerful user-experience, well written documentation and a ton of features.

A few comments:

When the checklist asks me if the software source code is available at the respository URL, it leads me to https://github.com/mfherbst/juliacon2020-proceedings-dftk rather than https://github.com/JuliaMolSim/DFTK.jl

I would echo @jagot in his suggestion for tweaking the paragraph starting with "For tacking these aspects [...]"

Other than these tiny issues, as someone who is familiar with Monte Carlo and DMRG methods, but not so familiar with DFT methods, I find myself wanting at least a couple sentences either in the documentation or (if possible) in the paper which gives me a bit more context how I should expect this general methodolgy to compare to different techniques. Are there any rough rules of thumb I can use for when I should expect DFT to give inaccurate results relative to 'reality'?

For instance, I have certainly heard in other places that DFT methods tend to not properly model exchange interactions. I see in the examples section discussion of magnetic systems. I'm curious how well do these calculations agree with experiments, monte carlo, or other benchmarks?

I don't need anything super in-depth to give my full approval, but I think it would help me in placing this work in the wider scientific context if there was more comparison to completely different methods, and discussion of potential pitfalls.

@MasonProtter
Copy link
Collaborator

I should also say, if you know of a good reference which discusses the limitations and domains of applicability of these sorts of methods, and you feel that the comments in reference is straightforwardly applicable to your work, then that would also satisfy me!

@mfherbst
Copy link
Member

mfherbst commented Apr 6, 2021

For instance, I have certainly heard in other places that DFT methods tend to not properly model exchange interactions. I see in the examples section discussion of magnetic systems. I'm curious how well do these calculations agree with experiments, monte carlo, or other benchmarks?

Thanks for your review and for this guiding question. The problem with answering it, is that it's really tough to do that in just a couple of sentences and at the same time convey a fair picture. We decided to not go into this, simply because this is more of a software paper and not an application paper after all.

But I agree, that we don't really point to any literature answering this question either, which we should. I amended the paper appropriately and made a PR to the repo. @MasonProtter let me know if that is not sufficient from your point of view.

@mfherbst
Copy link
Member

mfherbst commented Apr 6, 2021

@whedon generate pdf

@whedon
Copy link
Collaborator Author

whedon commented Apr 6, 2021

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

@carstenbauer
Copy link
Member

@whedon accept

@whedon
Copy link
Collaborator Author

whedon commented Apr 23, 2021

No archive DOI set. Exiting...

@carstenbauer
Copy link
Member

carstenbauer commented Apr 23, 2021

@mfherbst Final thing, can you please archive the current state of the DFTK repository, for example using Zenodo and post the associated DOI here (see https://joss.readthedocs.io/en/latest/editing.html#after-reviewers-recommend-acceptance).

Afterwards I can tell whedon to accept the paper!

@mfherbst
Copy link
Member

Well that' done already 😄 ... the most recent commit is a version tag and the zenodo doi is 10.5281/zenodo.4707620.

@carstenbauer
Copy link
Member

@whedon set 10.5281/zenodo.4707620 as archive

@whedon
Copy link
Collaborator Author

whedon commented Apr 23, 2021

OK. 10.5281/zenodo.4707620 is the archive.

@carstenbauer
Copy link
Member

@whedon accept

@whedon
Copy link
Collaborator Author

whedon commented Apr 23, 2021

Attempting dry run of processing paper acceptance...

@whedon
Copy link
Collaborator Author

whedon commented Apr 23, 2021

👋 @JuliaCon/jcon-eics, this paper is ready to be accepted and published.

Check final proof 👉 JuliaCon/proceedings-papers#39

If the paper PDF and Crossref deposit XML look good in JuliaCon/proceedings-papers#39, then you can now move forward with accepting the submission by compiling again with the flag deposit=true e.g.

@whedon accept deposit=true

@whedon
Copy link
Collaborator Author

whedon commented Apr 23, 2021

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1039/d0fd00048e is OK
- 10.1088/1361-648x/abcbdb is OK
- 10.1137/20M1332864 is OK
- 10.1038/natrevmats.2015.4 is OK
- 10.1038/ncomms11962 is OK
- 10.1021/jacs.7b02120 is OK
- 10.1038/nmat1752 is OK
- 10.1063/1.3148892 is OK
- 10.1021/acscentsci.8b00229 is OK
- 10.1063/1.5144261 is OK
- 10.1063/5.0005082 is OK
- 10.1103/PhysRevB.54.1703 is OK
- 10.1103/PhysRevLett.77.3865 is OK
- 10.1088/1361-648x/aa680e is OK
- 10.1016/j.commatsci.2012.10.028 is OK
- 10.1063/1.4704546 is OK
- 10.1002/qua.24259 is OK
- 10.1017/cbo9780511805769 is OK

MISSING DOIs

- None

INVALID DOIs

- None

@carstenbauer
Copy link
Member

@whedon accept deposit=true

@whedon
Copy link
Collaborator Author

whedon commented Apr 23, 2021

I'm sorry @crstnbr, I'm afraid I can't do that. That's something only editor-in-chiefs are allowed to do.

@carstenbauer
Copy link
Member

carstenbauer commented Apr 23, 2021

@vchuravy ☝️

@vchuravy
Copy link

vchuravy commented May 5, 2021

@whedon accept deposit=true

@whedon
Copy link
Collaborator Author

whedon commented May 5, 2021

Doing it live! Attempting automated processing of paper acceptance...

@whedon
Copy link
Collaborator Author

whedon commented May 5, 2021

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JCON! 🚨🚨🚨

Here's what you must now do:

  1. Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.jcon.00069 proceedings-papers#40
  2. Wait a couple of minutes, then verify that the paper DOI resolves https://doi.org/10.21105/jcon.00069
  3. If everything looks good, then close this review issue.
  4. Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

@mfherbst
Copy link
Member

mfherbst commented May 6, 2021

@vchuravy @crstnbr The doi does not seem to work yet. I suppose that is not intended after this time. Anything I can do about that?

@carstenbauer
Copy link
Member

@arfon Do you know what's happening here? Is there anything on our side that we can do to fix this?

@arfon
Copy link
Collaborator

arfon commented May 7, 2021

Hrm, weird. Let me take a look!

@arfon
Copy link
Collaborator

arfon commented May 7, 2021

@whedon accept deposit=true

@whedon
Copy link
Collaborator Author

whedon commented May 7, 2021

Doing it live! Attempting automated processing of paper acceptance...

@arfon
Copy link
Collaborator

arfon commented May 7, 2021

OK, I think Whedon must have been having a bad day. Things look 👍 to me now. Can you verify?

@mfherbst
Copy link
Member

mfherbst commented May 7, 2021

Yes, to me as well, thanks @arfon 👍.

@carstenbauer
Copy link
Member

Looks good! Thanks for the help @arfon!

This completes the publication process. Congratulations @mfherbst!

@whedon
Copy link
Collaborator Author

whedon commented May 7, 2021

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://proceedings.juliacon.org/papers/10.21105/jcon.00069/status.svg)](https://doi.org/10.21105/jcon.00069)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/jcon.00069">
  <img src="https://proceedings.juliacon.org/papers/10.21105/jcon.00069/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://proceedings.juliacon.org/papers/10.21105/jcon.00069/status.svg
   :target: https://doi.org/10.21105/jcon.00069

This is how it will look in your documentation:

DOI

We need your help!

JuliaCon Proceedings is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

@mfherbst
Copy link
Member

mfherbst commented May 7, 2021

Thanks for the reviews @jagot and @MasonProtter and for the editing @crstnbr !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

8 participants