Merge branch 'main' into main

.github/workflows/docs.yml

@@ -4,8 +4,6 @@ on:
   workflow_dispatch:
   push:
     branches: [main]
-  pull_request:
-    branches: [main]
 
 jobs:
   build-docs:
@@ -34,6 +32,7 @@ jobs:
         run: sphinx-build docs docs_build
 
       - name: Deploy
+        if: github.repository_owner == 'materialsproject' && github.event.workflow_run.conclusion == 'success' && github.event.workflow_run.event == 'push' 
         uses: peaceiris/actions-gh-pages@v3
         with:
           deploy_key: ${{ secrets.ACTIONS_DEPLOY_KEY }}

.pre-commit-config.yaml

@@ -3,7 +3,7 @@ default_language_version:
 exclude: '^src/atomate2/vasp/schemas/calc_types/|^.github/'
 repos:
 - repo: https://github.com/charliermarsh/ruff-pre-commit
-  rev: v0.0.277
+  rev: v0.0.284
   hooks:
   - id: ruff
     args: [--fix]
@@ -16,11 +16,11 @@ repos:
   - id: end-of-file-fixer
   - id: trailing-whitespace
 - repo: https://github.com/psf/black
-  rev: 23.3.0
+  rev: 23.7.0
   hooks:
   - id: black
 - repo: https://github.com/asottile/blacken-docs
-  rev: 1.14.0
+  rev: 1.15.0
   hooks:
   - id: blacken-docs
     additional_dependencies: [black]
@@ -46,7 +46,7 @@ repos:
   - id: rst-directive-colons
   - id: rst-inline-touching-normal
 - repo: https://github.com/pre-commit/mirrors-mypy
-  rev: v1.4.1
+  rev: v1.5.0
   hooks:
   - id: mypy
     files: ^src/

docs/dev/workflow_tutorial.md

@@ -0,0 +1,63 @@
+# How to Develop a new workflow for `atomate2`
+
+## Anatomy of an `atomate2` computational workflow (i.e., what do I need to write?)
+
+Every `atomate2` workflow is an instance of jobflow's `Flow ` class, which is a collection of Job and/or other `Flow` objects. So your end goal is to produce a `Flow `.
+
+In the context of computational materials science, `Flow ` objects are most easily created by a `Maker`, which contains a factory method make() that produces a `Flow `, given certain inputs. Typically, the input to `Maker`.make() includes atomic coordinate information in the form of a `pymatgen` `Structure` or `Molecule` object. So the basic signature looks like this:
+
+```python
+class ExampleMaker(Maker):
+    def make(self, coordinates: Structure) -> Flow:
+        # take the input coordinates and return a `Flow `
+        return Flow(...)
+```
+
+The `Maker` class usually contains most of the calculation parameters and other settings that are required to set up the calculation in the correct way. Much of this logic can be written like normal python functions and then turned into a `Job` via the `@job` decorator.
+
+One common task encountered in almost any materials science calculation is writing calculation input files to disk so they can be executed by the underlying software (e.g., VASP, Q-Chem, CP2K, etc.). This is preferably done via a `pymatgen` `InputSet` class. `InputSet` is essentially a dict-like container that specifies the files that need to be written, and their contents. Similarly to the way that `Maker` classes generate `Flow`s, `InputSet`s are most easily created by `InputGenerator` classes. `InputGenerator`
+have a method `get_input_set()` that typically takes atomic coordinates (e.g., a `Structure` or `Molecule` object) and produce an `InputSet`, e.g.,
+
+```python
+class ExampleInputGenerator(InputGenerator):
+    def get_input_set(self, coordinates: Structure) -> InputSet:
+        # take the input coordinates, determine appropriate
+        # input file contents, and return an `InputSet`
+        return InputSet(...)
+```
+
+`pymatgen` already contains `InputSet` for many common codes, so when developing a workflow `Maker` it is convenient to use the `InputGenerator` / `InputSet` to prepare your files. This is done in `atomate2` by making the `InputGenerator` a class parameter, e.g.,
+
+**TODO - the code block below needs refinement. Not exactly sure how write_inputs() fits into a`Job`**
+
+```python
+class ExampleMaker(Maker):
+    input_set_generator: ExampleInputGenerator = field(
+        default_factory=ExampleInputGenerator
+    )
+
+    def make(self, coordinates: Structure) -> Flow:
+        # create an`InputSet`
+        input_set = self.input_set_generator.get_input_set(coordinates)
+        # write the input files
+        input_set.write_inputs()
+        return Flow(...)
+```
+
+Finally, most `atomate2` workflows return structured output in the form of "Task Documents". Task documents are instances of `emmet`'s `BaseTaskDocument` class (similarly to a `python` `@dataclass`) that define schemas for storing calculation outputs. `emmet` already contains calculation schemas for codes utilized by the Materials Project (e.g., VASP, Q-Chem, FEFF) as well as a number of schemas for code-agnostic structural and molecular information (for example, the `MaterialsDoc` is a schema for solid material calculation data). `atomate2` can also interpret output generated by [`cclib`](https://cclib.github.io/), which is able to parse the output of many additional codes.
+
+**TODO - extend code block above to illustrate TaskDoc usage**
+
+In summary, a new `atomate2` workflow consists of the following components:
+ - A `Maker` that actually generates the workflow
+ - One or more `Job` and/or `Flow ` classes that define the discrete steps in the workflow
+ - (optionally) an `InputGenerator` that produces a `pymatgen` `InputSet` for writing calculation input files
+ - (optionally) a `TaskDocument` that defines a schema for storing the output data
+
+## Where do I put my code?
+
+Because of the distributed design of the MP Software Ecosystem, writing a complete new workflow may involve making contributions to more than one GitHub repository. The following guidelines should help you understand where to put your contribution.
+
+ - All workflow code (`Job`, `Flow `, `Maker`) belongs in `atomate2`
+ - `InputSet` and `InputGenerator` code belongs in `pymatgen`. However, if you need to create these classes from scratch (i.e., you are working with a code that is not already supported in`pymatgen`), then it is recommended to include them in `atomate2` at first to facilitate rapid iteration. Once mature, they can be moved to `pymatgen` or to a `pymatgen` [addon package](https://pymatgen.org/addons).
+ - `TaskDocument` schemas should generally be developed in `atomate2` alongside the workflow code. We recommend that you first check emmet to see if there is an existing schema that matches what you need. If so, you can import it. If not, check [`cclib`](https://cclib.github.io/). `cclib` output can be imported via [`atomate2.common.schemas.TaskDocument`](https://github.com/materialsproject/atomate2/blob/main/src/atomate2/common/schemas/cclib.py). If neither code has what you need, then new schemas should be developed within `atomate2` (or `cclib`).

pyproject.toml

@@ -41,9 +41,9 @@ amset = ["amset>=0.4.15", "pydash"]
 cclib = ["cclib"]
 mp = ["mp-api>=0.27.5"]
 phonons = ["phonopy>=1.10.8", "seekpath"]
-lobster = ["lobsterpy"]
+lobster = ["lobsterpy>=0.3.0"]
 defects = ["dscribe>=1.2.0", "pymatgen-analysis-defects>=2022.11.30"]
-forcefields = ["chgnet==0.2.0", "matgl==0.7.1"]
+forcefields = ["chgnet==0.2.0", "matgl==0.8.2", "quippy-ase==0.9.14"]
 docs = [
     "FireWorks==2.0.3",
     "autodoc_pydantic==1.9.0",
@@ -64,17 +64,18 @@ strict = [
     "click==8.1.6",
     "custodian==2023.7.22",
     "dscribe==2.0.1",
-    "emmet-core==0.60.1",
+    "emmet-core==0.64.4",
     "jobflow==0.1.11",
-    "lobsterpy==0.2.9",
-    "matgl==0.7.1",
-    "monty==2023.5.8",
+    "lobsterpy==0.3.0",
+    "matgl==0.8.2",
+    "monty==2023.8.8",
     "mp-api==0.33.3",
     "numpy",
     "phonopy==2.20.0",
     "pydantic==1.10.9",
-    "pymatgen-analysis-defects==2023.7.24",
-    "pymatgen==2023.7.17",
+    "pymatgen-analysis-defects==2023.7.31",
+    "pymatgen==2023.8.10",
+    "quippy-ase==0.9.14",
     "seekpath==2.1.0",
     "typing-extensions==4.7.1",
 ]
@@ -171,19 +172,23 @@ select = [
     "YTT",  # flake8-2020
 ]
 ignore = [
-    "PD011",  # pandas-use-of-dot-values
-    "PLR",    # pylint-refactor
-    "PT004",  # pytest-missing-fixture-name-underscore
-    "PT006",  # pytest-parametrize-names-wrong-type
-    "RUF013", # implicit-optional
-    # TODO remove PT011, pytest.raises() should always check err msg
+    "PD011",   # pandas-use-of-dot-values
     "PERF203", # try-except-in-loop
-    "PT011",   # pytest-raises-too-broad
-    "PT013",   # pytest-incorrect-pytest-import
+    "PLR",     # pylint-refactor
+    "PT004",   # pytest-missing-fixture-name-underscore
+    "PT006",   # pytest-parametrize-names-wrong-type
+    "RUF013",  # implicit-optional
+    # TODO remove PT011, pytest.raises() should always check err msg
+    "PT011", # pytest-raises-too-broad
+    "PT013", # pytest-incorrect-pytest-import
 ]
 pydocstyle.convention = "numpy"
 isort.known-first-party = ["atomate2"]
 
 [tool.ruff.per-file-ignores]
 "__init__.py" = ["F401"]
 "**/tests/*" = ["D"]
+# flake8-type-checking (TCH): things inside TYPE_CHECKING aren't available
+#     at runtime and so can't be used by pydantic models
+# flake8-future-annotations (FA): future annotations only work in pydantic models in python 3.10+
+"**/schemas/*" = ["FA", "TCH"]

src/atomate2/vasp/jobs/phonons.py → src/atomate2/common/jobs/phonons.py

@@ -2,29 +2,33 @@
 
 from __future__ import annotations
 
+import contextlib
 import logging
 from dataclasses import dataclass, field
 from typing import TYPE_CHECKING
 
 from jobflow import Flow, Response, job
 from phonopy import Phonopy
 from phonopy.units import VaspToTHz
+from pymatgen.core import Structure
 from pymatgen.io.phonopy import get_phonopy_structure, get_pmg_structure
 from pymatgen.phonon.bandstructure import PhononBandStructureSymmLine
 from pymatgen.phonon.dos import PhononDos
 from pymatgen.transformations.advanced_transformations import (
     CubicSupercellTransformation,
 )
 
+from atomate2.common.schemas.phonons import ForceConstants, PhononBSDOSDoc
 from atomate2.vasp.jobs.base import BaseVaspMaker
-from atomate2.vasp.schemas.phonons import PhononBSDOSDoc
 from atomate2.vasp.sets.core import StaticSetGenerator
 
 if TYPE_CHECKING:
+    from pathlib import Path
+
     import numpy as np
     from emmet.core.math import Matrix3D
-    from pymatgen.core import Structure
 
+    from atomate2.forcefields.jobs import ForceFieldStaticMaker
     from atomate2.vasp.sets.base import VaspInputGenerator
 
 logger = logging.getLogger(__name__)
@@ -122,7 +126,7 @@ def get_supercell_size(
     return transformation.transformation_matrix.tolist()
 
 
-@job
+@job(data=[Structure])
 def generate_phonon_displacements(
     structure: Structure,
     supercell_matrix: np.array,
@@ -183,7 +187,10 @@ def generate_phonon_displacements(
     return [get_pmg_structure(cell) for cell in supercells]
 
 
-@job(output_schema=PhononBSDOSDoc, data=[PhononDos, PhononBandStructureSymmLine])
+@job(
+    output_schema=PhononBSDOSDoc,
+    data=[PhononDos, PhononBandStructureSymmLine, ForceConstants],
+)
 def generate_frequencies_eigenvectors(
     structure: Structure,
     supercell_matrix: np.array,
@@ -249,12 +256,13 @@ def generate_frequencies_eigenvectors(
     )
 
 
-@job
+@job(data=["forces", "displaced_structures"])
 def run_phonon_displacements(
     displacements,
     structure: Structure,
     supercell_matrix,
-    phonon_maker: BaseVaspMaker = None,
+    phonon_maker: BaseVaspMaker | ForceFieldStaticMaker = None,
+    prev_vasp_dir: str | Path = None,
 ):
     """
     Run phonon displacements.
@@ -265,11 +273,13 @@ def run_phonon_displacements(
     ----------
     displacements
     structure: Structure object
-        Fully optimized structure used for phonon computations
+        Fully optimized structure used for phonon computations.
     supercell_matrix: Matrix3D
         supercell matrix for meta data
     phonon_maker : .BaseVaspMaker
         A VaspMaker to use to generate the elastic relaxation jobs.
+    prev_vasp_dir : str or Path or None
+        A previous vasp calculation directory to use for copying outputs.
     """
     if phonon_maker is None:
         phonon_maker = PhononDisplacementMaker()
@@ -279,11 +289,13 @@ def run_phonon_displacements(
         "forces": [],
         "uuids": [],
         "dirs": [],
-        "displaced_structures": [],
     }
 
     for i, displacement in enumerate(displacements):
-        phonon_job = phonon_maker.make(displacement)
+        if prev_vasp_dir is not None:
+            phonon_job = phonon_maker.make(displacement, prev_vasp_dir=prev_vasp_dir)
+        else:
+            phonon_job = phonon_maker.make(displacement)
         phonon_job.append_name(f" {i + 1}/{len(displacements)}")
 
         # we will add some meta data
@@ -293,15 +305,17 @@ def run_phonon_displacements(
             "supercell_matrix": supercell_matrix,
             "displaced_structure": displacement,
         }
-        phonon_job.update_maker_kwargs(
-            {"_set": {"write_additional_data->phonon_info:json": info}}, dict_mod=True
-        )
+        with contextlib.suppress(Exception):
+            phonon_job.update_maker_kwargs(
+                {"_set": {"write_additional_data->phonon_info:json": info}},
+                dict_mod=True,
+            )
+
         phonon_jobs.append(phonon_job)
         outputs["displacement_number"].append(i)
         outputs["uuids"].append(phonon_job.output.uuid)
         outputs["dirs"].append(phonon_job.output.dir_name)
         outputs["forces"].append(phonon_job.output.output.forces)
-        outputs["displaced_structures"].append(displacement)
 
     displacement_flow = Flow(phonon_jobs, outputs)
     return Response(replace=displacement_flow)
@@ -341,10 +355,9 @@ class PhononDisplacementMaker(BaseVaspMaker):
     """
 
     name: str = "phonon static"
-
     input_set_generator: VaspInputGenerator = field(
         default_factory=lambda: StaticSetGenerator(
-            user_kpoints_settings={"grid_density": 7000},
+            user_kpoints_settings={"reciprocal_density": 100},
             user_incar_settings={
                 "IBRION": 2,
                 "ISIF": 3,
@@ -355,7 +368,7 @@ class PhononDisplacementMaker(BaseVaspMaker):
                 "ALGO": "Normal",
                 "NSW": 0,
                 "LCHARG": False,
-                "ISMEAR": 0,
             },
+            auto_ispin=True,
         )
     )

src/atomate2/common/schemas/cclib.py

@@ -222,7 +222,7 @@ def from_logfile(
 
         # Calculate any properties
         if analysis:
-            if type(analysis) == str:
+            if isinstance(analysis, str):
                 analysis = [analysis]
             analysis = [a.lower() for a in analysis]