Advertising the atomate2 paper writing

[Zhuoying]

Dear contributors,

We are thrilled to share the commencement of the Atomate2 paper's preparation phase. Our goal is to have the initial draft ready by our upcoming monthly gathering on March 29.

Since Atomate2 is truly a collective endeavor, we want to ensure that every contributor has the opportunity to contribute to the paper (including those who have made minor contributions but would like to do more to qualify for authorship).

Overview

The paper will be split into 5 sections:

1. Introduction & Background (@computron)
2. Atomate2 Design Philosophy & Overview (@utf)
3. Calculators (team)
4. Worklows (team)
5. Conclusion & Future Work (@computron, @utf)

Below, you will find a list of teams categorized according to their workflow and calculator code contributions, compiled to the best of my knowledge. Should there be any oversight or additional workflows you've contributed to that are not mentioned, please do not hesitate to bring them to our attention.

Calculators

• VASP (TBD)
• CP2K (@nwinner)
• ABINIT (@gpetretto, @davidwaroquiers, @VicTrqt, @gmrigna)
• FHI-aims (@tpurcell90, @ansobolev)
• QCHEM (@rdguha1995, @rkingsbury, @rohithsrinivaas)
• Force fields (@matthewkuner @janosh @JaGeo @CompRhys)

We kindly ask each calculator team to prepare the following:

• 300-600 words of text covering implementation, usage notes, and any existing use case/papers using this implementation.
• A summary of the key calculation types implemented.
• No figure needed.

Workflows

• amset (@utf )
• electron phonon (@utf)
• lobster (@JaGeo @naik-aakash)
• elastic (@mjwen @utf @JaGeo)
• harmonic phonon using phonopy (@JaGeo @ab5424)
• anharmonic phonon using hiphive (WIP, @hrushikesh-s @Zhuoying)
• electrode (@jmmshn)
• vasp and cp2k defect (WIP, @jmmshn @nwinner )
• aimd and ffmd (WIP, @gpetretto, @mjwen, @davidwaroquiers,@esoteric-ephemera, @chiang-yuan)
• openMM (WIP, @orionarcher)
• mp (@esoteric-ephemera @Andrew-S-Rosen @janosh)
• magnetic orderings (WIP, @mattmcdermott @mkhorton @guymoore13)
• adsorption (WIP, @itsduowang)
• ferroelectric (WIP, @fraricci)

We kindly ask each workflow team to prepare the following:

• 500-750 words of text covering methodology, usage notes, and any existing use case/papers using this workflow;
• a workflow diagram (will talk more about standardizing the diagram in our next monthly meeting on Mar. 29)
• a "pretty" output collage of plots (from raw data calculated from the workflow).

Welcoming Additional Contributors

There are some contributors to atomate2 who have provided fixes here and there but who have not contributed to major workflows. We want to make it clear that these contributors are still welcome to be included on the atomate2 paper. However, we're suggesting that these people could contribute more to the atomate2 code base, either by fixing an open issue or by adding a new tutorial to the documentation. Currently, we could do with the following tutorials, however, if you have other ideas then please let @utf know.

• How to write your own atomate2 workflow.
• Introduction to the data and document stores.
• An introduction to the input sets framework.
• An introduction to task documents, schemas, and emmet.
• A high level overview of the concepts in atomate2: job/flow makers, input sets, task documents, builders.

If you think you could fit into this category, please reach out to @utf via email and we can chat.

List of tutorial contributors: @QuantumChemist, @ml-evs, @JonathanSchmidt1, @naik-aakash, @JaGeo

Notes

1. We are delighted to hear your thoughts. For example, if you think anyone should be added to the authorship but not currently listed, don't hesitate to let us know.

2. You are encouraged to collaborate within or across teams as you see fit. To simplify the collecting process, we propose nominating the first person listed in each workflow as the team leader by default. However, please feel free to adjust the leadership roles within your teams or to merge different workflow teams if you find it necessary. If any changes occur, we would appreciate a brief notification. I will make ongoing updates to the workflow list based on your feedback for a better track.

Point of contact for accessing collaboration platforms

Overleaf: Please leave your email to @utf for access to our Overleaf project.
Zoom meeting: Please leave your email to @Zhuoying for the Google invite (our next meeting will be 8:30 AM on March 29 PST to discuss jobflow-remote and the paper writing)
Slack channel: Please leave your email to @Andrew-S-Rosen for the Slack invite.

Your contributions are greatly valued. Any of your feedback and thoughts are welcome!

[Andrew-S-Rosen]

I am very excited about this effort and happy to be involved! Apologies for not being able to make the last couple of meetings --- I've had some conflicts.

In any case, @esoteric-ephemera shall I counter-nominate you for the MP workflow lead? 😉 Of course, I am happy to help write.

[Zhuoying]

In any case, @esoteric-ephemera shall I counter-nominate you for the MP workflow lead? 😉 Of course, I am happy to help write.

@Andrew-S-Rosen Thanks for the nomination. I have put @esoteric-ephemera as the first name of the MP workflow.

Hi @mjwen, since @utf and @JaGeo are the lead of a few workflows, are you interested in being the lead of the elastic workflow?

[mjwen]

@Zhuoying Yes, I'd be happy to!

[JaGeo]

Thank you, @Zhuoying . Could you add @ab5424 to the harmonic phonon workflow as well? Thanks! The implemented workflow was based on a previous implementation in his Master thesis.

(To mention shortly, I also only indirectly contributed to the FHI-aims implementation as it starts from the phonon workflow)

[utf]

Thanks very much for putting this message together @Zhuoying. I would just raise two additional points.

Workflows vs Calculators

I think we should make a distinction in the paper between workflows and force/energy calculators. Many of the workflows are agnostic to the choice of calculator (e.g., elastic, phonon, defect etc). I suggest we therefore have a section before workflows providing a short summary for each calculator.

Welcoming additional contributions

There are some contributors to atomate2 who have provided fixes here and there but who have not contributed to major workflows. I want to make it clear that these contributors are still welcome to be included on the atomate2 paper. However, we're suggesting that these people could contribute more to the atomate2 code base, either by fixing an open issue or by adding a new tutorial to the documentation. Currently, I think we could do with the following tutorials, however, if you have other ideas then please let me know.

• How to write your own atomate2 workflow.
• Introduction to the data and document stores.
• An introduction to the input sets framework.
• An introduction to task documents, schemas, and emmet (@rkingsbury)
• A high level overview of the concepts in atomate2: job/flow makers, input sets, task documents, builders (@QuantumChemist)

If you think you could fit into this category, please reach out to me via email and we can chat.

Notes on the contributor list

Looking through the list, @ansobolev has also contributed to the FHI-aims implementation; @CompRhys added the MACE workflow;

[davidwaroquiers]

Very nice to see this issue open! Just adding @VicTrqt, @gmrigna and @gpetretto to the abinit workflows.

[esoteric-ephemera]

Thanks @Andrew-S-Rosen and @Zhuoying! Might also suggest lumping the EOS workflows (WIP waiting on an emmet release) under the elastic flows.

Would we want to put together an examples directory with jupyter / python notebooks showing typical use? Basically would just mean writing a script to reproduce the figure(s) that we add to the paper

[rdguha1995]

Thanks @Zhuoying for nominating me and @rkingsbury. This is a question to both @Zhuoying and @utf . For DFT code contributions, do we have a blueprint similar to the one for workflows? Or do we plan to have a generalized code-agnostic discussion in the paper?

[tpurcell90]

I'm happy to write about the FHI-aims and help out with the phonon workflow parts as well

[matthewkuner]

@JaGeo I wrote up part of the section for forcefields--could you add the relevant parts about phonons/elastic constants to it? We can figure out what kinds of workflow flowcharts we want to do offline (if that works for you)

[utf]

For DFT code contributions, do we have a blueprint similar to the one for workflows?

@rdguha1995 Good question. I think this would be a reasonable starting point:

• 300-600 words of text covering implementation, usage notes, and any existing use case/papers using this code;
• A summary of the key calculation types implemented.
• No figure needed;

I'll update the original message with this information.

[utf]

@matthewkuner @JaGeo @janosh what do you think about putting the force field part in the "Calculators" section rather than "Workflows" section. Since the force fields can be used for many workflows I think this makes sense, but happy to hear your thoughts.

[janosh]

Since the force fields can be used for many workflows I think this makes sense

agreed!

[JaGeo]

@matthewkuner @JaGeo @janosh what do you think about putting the force field part in the "Calculators" section rather than "Workflows" section. Since the force fields can be used for many workflows I think this makes sense, but happy to hear your thoughts.

Fine with me!

[matthewkuner]

@matthewkuner @JaGeo @janosh what do you think about putting the force field part in the "Calculators" section rather than "Workflows" section. Since the force fields can be used for many workflows I think this makes sense, but happy to hear your thoughts.

Sounds good to me

[JaGeo]

@utf @Zhuoying Can we maybe use slack channel or a Team to navigate the communication? I feel this will be otherwise hard to organize. (Especially the individual workflow, code teams)

[Andrew-S-Rosen]

@utf @Zhuoying Can we maybe use slack channel or a Team to navigate the communication? I feel this will be otherwise hard to organize. (Especially the individual workflow, code teams)

Seconded. Plus the GitHub notifications are too much for me... 😅

[tpurcell90]

@utf @Zhuoying Can we maybe use slack channel or a Team to navigate the communication? I feel this will be otherwise hard to organize. (Especially the individual workflow, code teams)

Seconded. Plus the GitHub notifications are too much for me... 😅

I agree with both of these sentiments

[Zhuoying]

@utf @Zhuoying Can we maybe use slack channel or a Team to navigate the communication? I feel this will be otherwise hard to organize. (Especially the individual workflow, code teams)

@JaGeo Sure, it is a good point (always keep a clear organization for efficient communication in such a collaborative project).

@Andrew-S-Rosen, I remember we already have a slack channel called atomate2-working-group created by you a while ago (for monthly meeting purposes). Would you mind using that slack channel (and inviting more contributors to the channel) for the communication?

[Andrew-S-Rosen]

@Andrew-S-Rosen, I remember we already have a slack channel called atomate2-working-group created by you a while ago (for monthly meeting purposes). Would you mind using that slack channel (and inviting more contributors to the channel) for the communication?

I always love when I solved my own problems without remembering 😆 will do!

[Andrew-S-Rosen]

Just added everyone to the Slack (except @ab5424, whose email I need). Feel free to ping @Zhuoying, @utf, or myself to be added if you don't get an email.

[JaGeo]

@Andrew-S-Rosen @ab5424 's mail address is in your mail 😅

[tpurcell90]

@Andrew-S-Rosen I did not get an email (purcellt@arizona.edu)

[QuantumChemist]

Hey everyone :)

@utf, as we agreed, I will work on an atomate2 tutorial for "A high-level overview of the key concepts in atomate2: job/flow makers, input sets, task documents, and builders" to contribute to this paper :)
And could I be added to the slack channel, too, please?

[Andrew-S-Rosen]

@QuantumChemist please send me your email and I'll add you. :)

[ml-evs]

Likewise, discussed with @utf and I will work on some tutorials describing the execution mechanisms (fireworks, JF local, JF remote etc) and would be happy to contribute to the paper. I would also like a slack invite please, hopefully you have my email!

[rkingsbury]

Hi all, thank you for including me in this effort. TBH, I would classify my contributions up to this point as mostly "moral support" to @rdguha1995 plus a few code reviews, so I'm not sure I should qualify as a coauthor (yet). That said, I'd be happy to work with @rdguha1995 on the Q-Chem calculator section and also to develop tutorials on one or both of the following:

• An introduction to the input sets framework.
• An introduction to task documents, schemas, and emmet.

Plus some tutorials on specific Q-Chem topics (although perhaps not relevant for the paper).

@Andrew-S-Rosen can you please add me to the Slack? I either missed the invitation or did not get one. Is this a dedicated workspace or a channel in BerkeleyTheory or MP?

[tpurcell90]

@ansobolev What email should we use to add you to the overleaf paper

[ansobolev]

Hi, sorry for being late, seems that the ping got buried under a bunch of dependabot alerts. Yes, I'd be happy to contribute to FHI-aims implementation of workflows (and also the docs and the paper), @utf can you please add me to the Slack? The e-mail is andrey.n.sobolev@gmail.com.

[computron]

Hi all,

Here is the decision for revised paper structure based on the atomate2 meeting on 3/29/24.

Workflows categorized as "A" would have 2 figures. The first figure details the inputs (both required and optional, like the atomate1 paper) and has a workflow diagram. The second figure details the outputs (an outputs list similar to the inputs, as well as example plots of what you get out of it).

Workflows categorized as "B" will not have figures. However, inputs, workflow description, and outputs will need to part of a text description as well as an overall table.

Paper structure

# Intro
# Design philosophy
# Implementations thus far (calculators and workflows)
# Table of calculators (+1 paragraph per calculator)
# Table of workflows (inputs, outputs) --> these will be detailed in next sections

# Workflows (re-organize by technique (periodic, molecule) --> then A/B within each section)
## Periodic systems
### A
### B
## Molecular systems
### A
### B
## Misc / mixed
### A
### B
# Separate section on MP compatibility 
# Conclusion

A

• LOBSTER
• Harmonic phonons
• FFOpt
• Electrode Discovery
• Point defect
• Classical MD
• AIMD (checkpointing)

B

• Elastic
• Anharmonic phonons
• Ferroelectric
• Materials Project workflows (workflows themselves separated out, not the MP part)
• Magnetic ordering
• Adsorption
• MPMorph
• GW/Excited states

[JaGeo]

Thank you, @computron !
Is there any news on the timeline? This would help us planning the changes (LOBSTER and harmonic phonon part)

[computron]

We want everyone to update their sections by the next meeting (April 26). Overall, I'm expecting it will take until the May meeting to finalize things

@hrushikesh-s can you take care of reorganizing the paper into sections per the discussion?
@hrushikesh-s can you also post an example of the type of diagrams expected for workflows of type A? You can just point to a particular workflow section in Overleaf if it helps

[JaGeo]

So as it turns out I was wrong and option(2) was much easier for a variety of reasons that were not obvious until I actually tried to do it for 10 mins.

I have implemented option (2) in a basic tutorial for blob storage #776, we can probably extend those trivially to mock the other codes.

This is great. I am looking forward to such tutorials.

[JaGeo]

We want everyone to update their sections by the next meeting (April 26). Overall, I'm expecting it will take until the May meeting to finalize things

@hrushikesh-s can you take care of reorganizing the paper into sections per the discussion? @hrushikesh-s can you also post an example of the type of diagrams expected for workflows of type A? You can just point to a particular workflow section in Overleaf if it helps

Thank you!

[hrushikesh-s]

The paper is now reorganized as Periodic/Molecular & Type A/B workflows!

Consider figures/anharmonic_phonon_worflow.pdf & figures/anharmonic_phonon_analysis.pdf as example figures for workflows of type A. I have also uploaded an editable template for the workflow figure in figures/template_workflow_figure.pptx

Also, pls feel free to update Tables 1 & 2 with workflows &/or supported calculators, if I have missed any!

We want everyone to update their sections by the next meeting (April 26). Overall, I'm expecting it will take until the May meeting to finalize things

@hrushikesh-s can you take care of reorganizing the paper into sections per the discussion? @hrushikesh-s can you also post an example of the type of diagrams expected for workflows of type A? You can just point to a particular workflow section in Overleaf if it helps

[rkingsbury]

We want everyone to update their sections by the next meeting (April 26). Overall, I'm expecting it will take until the May meeting to finalize things

Sounds good. Does this timeline also apply for drafting the tutorials?

@rdguha1995 how do you want to approach revising the Q-Chem sections? Is that in your court? My court? Do we need to make/modify figures? Let's coordinate offline.

[jmmshn]

Workflow template:
template_workflow_figure.pptx

Color for dynamic workflow included on the side.

[JaGeo]

@jmmshn thank you! Awesome!

And sorry to all for joining late. The session at the MRS before went over time (finally being in a time zone that made the meeting possible)

[mkhorton]

Also missed due to MRS apologies, but will catch up from Janine.On Apr 26, 2024, at 11:02, J. George ***@***.***> wrote: @jmmshn thank you! Awesome! And sorry to all for joining late. The session at the MRS before went over time (finally being in a time zone that made the meeting possible) —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[computron]

Hi all,

Here are the notes from Friday April 26 meeting:

Needs to be completed!

These are items that were not filled in at the time of needing and need urgently to be filled in

1. MD section (combine AIMD, FFMD, and Classical MD) --> Orion, Guido, Mingjian, David
2. MPMorph - Aaron
3. GW / Excited states - Guido

Needs to be updated

We discussed Table 2 with the checkmarks. This table looks very sparse. Suggest to:

• add a "circle" symbol for things that are on the roadmap / changes we think would be relatively easy to implement
• combine the various force fields into ASE force field calculator
• reorder the table so the most complete columns (e.g., VASP) are first and most sparse columns come later

Suggest @janosh takes the lead or assigns to someone

Needs to be homogenized

We discussed the way figures should be formatted. The general consensus was to use the style of electrode insertion workflow in terms of color scheme / overall layout / level of detail of text. However some refinements to this style were also discussed:

• If there is an overall workflow and a sub-workflow, the overall workflow should be on top and sub-workflows on bottom with a zoom-in effect
• Remove some of the chartjunk on this figure (folded up edges, wavy lines)
• We want the inputs / outputs to be part of the same figure not separate figures. We liked the input/output boxes of the LOBSTER figures along with examples of the output. No need for complex JSON output diagrams however.
• Color scheme can be refined to have clear definitions for hte various colors. As of now - dark blue = concrete individual tasks, light blue=sub workflow, yellow=overall workflow, grayish=optional step. Suggestion to also have a color for dynamic workflow
• @jmmshn was going to send a workflow template for everyone to use in Powerpoint

Minor / TBD

• The word "heterogeneous" in the title is confusing to nearly everyone, title needs refinement

[janosh]

Suggest @janosh takes the lead or assigns to someone

i get nightmares from latex tables! just so you know what you're asking...

jokes aside, i moved the most populated columns to the left (i.e. VASP) and merged the 4 ML columns (CHGNet, M3GNet, MACE, GAP) into 1 titled MLFFs (machine learning force fields), now explained in the caption (adding Nequip to the list).

i added a checkmark for the MLFF column in the equation of state row which is already implemented.
i need input from others what they think would be easy to implement, i.e. where to place circles.
here's what it currently looks like:

rather than order columns by checkmark density, maybe better to order by association e.g. first-principles | ML | MD? i.e. vasp | cp2k | FHI-aims | Abinit | Q-Chem | MLFFs | OpenMM?

[esoteric-ephemera]

Now that there's an MPMorph PR submitted (still WIP but should be ready around submission time), I'd like to nominate @BryantLi-BLI to co-lead the MPMorph write up. Bryant has led much of the rewrite of these workflows. @utf and @Zhuoying, I can send Bryant's contact info over slack

@janosh:

• The EOS workflows are based on a code-agnostic implementation, so those could feasibly have circles for the other codes (you just need to specify relax / optional static makers)
• MPMorph has code agnostic and MLFF-specific implementations in the PR, in addition to the VASP-specific implementation

Should we clarify that the MLFF stuff is now rewritten so that any ASE calculator works in these flows? That's now true for the relax, static, and MD MLFF jobs

AIMD and classical MD might be condensed into just MD (also think we're missing the AIMD implementations)

[gpetretto]

Hi, sorry, I could not make to the meeting, so I have a few questions concerning these points:

1. MD section (combine AIMD, FFMD, and Classical MD) --> Orion, Guido, Mingjian, David

Should these parts just be condensed in a single section? One potential issue is that classical MD is currently in the "Molecular systems" section, while AIMD is in the "Periodic systems" one. Where should the common section stay then?
Since AIMD, FFMD and classical MD have a different workflow structure and different output documents, it would make sense to have multiple workflow diagrams associated with this section. Would that be fine for a single section?

3. GW / Excited states - Guido

Not sure why I was assigned the GW workflow. Is this correct?

[JaGeo]

3. GW / Excited states - Guido

Not sure why I was assigned the GW workflow. Is this correct?

@gpetretto I think we were confused about the responsibilty for the GW workflows. As currently, the VASP and Abinit versions are build, we were not sure who would write it. Thus, it is really rather an open question who should be responsible.

[janosh]

thanks @esoteric-ephemera, used triangles to indicate low implementation effort, circle for workflow-engine combos on atomate2's near-term road map (currently nothing, input welcome!) and combined the AIMD and classical MD rows into a single MD row. i put System = both in that row. the System column overall is low entropy. maybe drop it altogether? or make sure we're not missing any rows that should say "both" instead of "periodic"

[JaGeo]

Just to mention it: LOBSTER could be run for a molecule with VASP as well. Abinit could likely be connected to the workflow as well but this would require some work from our side (a few days to.get lobsterin files for abinit).

[gpetretto]

3. GW / Excited states - Guido

Not sure why I was assigned the GW workflow. Is this correct?

@gpetretto I think we were confused about the responsibilty for the GW workflows. As currently, the VASP and Abinit versions are build, we were not sure who would write it. Thus, it is really rather an open question who should be responsible.

I have checked with @tatha0003, who is the author of the PR for the GW workflows for Abinit (#816). He would be interested to contribute the section for the GW workflow of the paper. Not sure if anyone that wrote the VASP implementation is interested.
If this is fine, I will send the email to add him to overleaf.

[JaGeo]

@naik-aakash has now updated the figures on the LOBSTER and Phonon workflow. If there is already feedback, let us know. Otherwise, we will wait for the next meeting.

[JaGeo]

Hi all,
As the next meeting is coming up:
I just looked at the draft. There is information on the GW calculations in texts on the calculators, but the GW/Excited States Section is still completely empty.

[tatha0003]

Hi @JaGeo I will try to complete the abinit GW and BSE workflow section by the end of this week.

[JaGeo]

Thank you, @tatha0003 ! 😀

[fraricci]

Hi there!

I'd like to update the FE workflow picture. What's the right template to use?
I see at least 3 different ones used so far in figs. 7, 3, 5.

Thanks

[JaGeo]

Hi there!

I'd like to update the FE workflow picture. What's the right template to use? I see at least 3 different ones used so far in figs. 7, 3, 5.

Thanks

@fraricci : #750 (comment) :)

[utf]

Hi everyone, these are the notes from the meeting on the 3rd June.

MD section

• Leave classical MD separate since it focuses on molecules
• @esoteric-ephemera to do final check on removing overlap between AIMD and FFMD descriptions.
• @gpetretto to finalise multi-MD diagram

MPMorph

• Section complete

GW/Excited states

• @tpurcell90 to coordinate incorporating FHI-aims description
• Add VASP GW description. @yanghan234 are you happy to do this?

VASP calculator section

• @esoteric-ephemera and @utf to write

Table 2

• @tpurcell90 to work on converting dashes to ticks.
• Can the leads for other calculations double check if any of the workflows can be converted to triangles (i.e., not currently planned but low-effort to implement): @nwinner, @gpetretto, @davidwaroquiers, @rdguha1995, @janosh

Workflow diagrams

• Many workflow diagrams still not using the updated template.
• Link to template file: https://github.com/materialsproject/atomate2/files/15133898/template_workflow_figure.pptx
• Figure leads to update:
  • QChem geometry optimisation (@rdguha1995, @rkingsbury, @rohithsrinivaas)
  • Multistep MD (@gpetretto)
  • Anharmonic phonon workflow (@hrushikesh-s)
  • Magnetic ordering workflow (@mattmcdermott)
  • Adsorption (@itsduowang)
  • Frequency flattening (@rohithsrinivaas)
  • MPMorph (@esoteric-ephemera)

Timeframe

• Deadline for completed draft to share with all co-authors is 26 July.

[yanghan234]

Add VASP GW description. @yanghan234 are you happy to do this?

Sure, glad to contribute!