Skip to content

Commit

Permalink
Replace querying notebook with shorter section
Browse files Browse the repository at this point in the history
  • Loading branch information
@mbercx
mbercx committed May 27, 2021
1 parent d44722a commit 4485a80
Showing 1 changed file with 134 additions and 16 deletions.
150 changes: 134 additions & 16 deletions docs/source/sections/data.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -217,35 +217,153 @@ Before we continue, let us delete these paths:
$ verdi group delete -f tutorial/gga/pbe
$ verdi group delete -f tutorial/gga/pbesol
Querying for data
-----------------
Querying the database
---------------------

For this part of the tutorial, we will move to interacting with AiiDA using a Jupyter notebook, which you will be able to run in your browser.
Download the notebook using:
As you will use AiiDA to run your calculations, the database that stores all the data and the provenance will quickly grow to be very large.
To help you find the needle that you might be looking for in this big haystack, we need an efficient search tool.
AiiDA provides a tool to do exactly this: the ``QueryBuilder``.
The ``QueryBuilder`` acts as the gatekeeper to your database, to whom you can ask questions about its contents (also referred to as queries), by specifying what are looking for.
In this final part of the tutorial, we will show an short demo on how to use the ``QueryBuilder`` to make these queries and understand/use the results.

Let's have another look at the groups we've imported from the archive above, using the ``-C`` option so we also get a count of the number of nodes:

.. code-block:: console
wget https://aiida-tutorials.readthedocs.io/en/tutorial-2020-intro-week/_downloads/855b61cc17925901d193ca02a45c878a/querybuilder-tutorial.ipynb
$ verdi group list --count
Info: to show groups of all types, use the `-a/--all` option.
PK Label Type string User Node count
---- --------------- ------------- --------------- ------------
5 tutorial_pbesol core aiida@localhost 57
6 tutorial_lda core aiida@localhost 57
7 tutorial_pbe core aiida@localhost 57
The notebook will show you how the ``QueryBuilder`` can be used to query your database for specific data.
It will demonstrate certain concepts and then ask you to use those to perform certain queries on your own database.
Some of these question cells will have partial solutions that you will have to complete.
On the AiiDAlab cluster, simply use the file manager to navigate to the notebook and open it.
When running this tutorial on the Quantum Mobile, simply start the notebook server with:
Each group contains a different set of 57 ``PwCalculation`` nodes (one for every different perovskite structure), organized according to the functional which was used in the calculation (LDA, PBE and PBEsol) .
Imagine you want to use this data to understand the influence of the functional on the magnetization of the structure.
Let's *build* a query that helps us investigate this question.
Start the ``verdi shell``, and load the ``StructureData`` and ``PwCalculation`` classes:

.. code-block:: console
.. code-block:: ipython
In [1]: StructureData = DataFactory('structure')
...: PwCalculation = CalculationFactory('quantumespresso.pw')
We start every query by creating an instance of the ``QueryBuilder`` class:

.. code-block:: ipython
In [2]: qb = QueryBuilder()
To build a query, we *append* entities (nodes, groups, ...) to the query.
Let's build the query for one of the groups - say, ``tutorial_pbesol`` - step by step to help understand the process.
We first append the ``Group`` to our ``QueryBuilder`` instance:

.. code-block:: ipython
In [3]: qb.append(Group, filters={'label': 'tutorial_pbesol'}, tag='group');
Let's explain the different arguments used in this call of the ``append()`` method:

* The first *positional* argument is the ``Group`` class, preloaded in the ``verdi shell``.
* The first *keyword* argument is ``filters``, here we *filter* for the group with ``label`` equal to ``tutorial_pbesol``.
* The second *keyword* argument is ``tag``.
This is a reference we will use to indicate *relationships* between nodes in future ``append()`` calls (as seen below).

Next, we'll look for all the ``PwCalculations`` in this group:

.. code-block:: ipython
In [4]: qb.append(PwCalculation, with_group='group', tag='pw');
Here, we use the ``'group'`` tag we created in the previous step to query for ``PwCalculation``'s in the ``tutorial_pbesol`` group using the ``with_group`` *relationship* argument.
Moreover, we once again *tag* this append step of our query with ``pw``.
Let's have a look at how many ``PwCalculation`` nodes we have in the ``tutorial_pbesol`` group:

.. code-block:: ipython
In [5]: qb.count()
Out[5]: 57
Great, now let's figure out which structures are magnetic!
Of course, the information we are interested in are the structures and their absolute magnetization, which we'll query for in the final two steps.
First, we'll *append* the ``StructureData`` to the query:

.. code-block:: ipython
In [6]: qb.append(StructureData, with_outgoing='pw', project='extras.formula');
In this step, we've used the ``with_outgoing`` relationship to look for structures that have an *outgoing* link to the ``PwCalculations`` referenced with the ``pw`` tag.
That means that from the ``PwCalculation``'s perspective, the ``StructureData`` is an input.
We also use the ``project`` keyword argument to *project* the formula of the structure, which has been conveniently stored in the ``extras`` of these ``StructureData`` nodes for the purpose of this tutorial.
By *projecting* the formula, it will be a part of the results of our query.
Try looking at the results of the *current* query using ``qb.all()``:

.. code-block:: ipython
In [7]: qb.all()
The final ``append()`` call puts using *relationships*, *filters* and *projections* together.
Here we are looking for the ``output_parameters`` ``Dict`` nodes, which are outputs of the ``PwCalculation`` nodes.
However, we are only interested in structures for which the ``absolute_magnetization`` is larger than zero:

.. code-block:: ipython
In [8]: qb.append(
...: Dict, with_incoming='pw', filters={'attributes.absolute_magnetization': {'>': 0.0}},
...: project='attributes.absolute_magnetization'
...: );
Let's go over the arguments again:

* The first *positional* argument tells the ``QueryBuilder`` we want to append ``Dict`` nodes to our query.
* ``with_incoming`` indicates there is an incoming link from a ``PwCalculation``, referenced by the ``'pw'`` tag.
* We're ``filter``-ing for magnetic structures, i.e. with ``absolute_magnetization`` above zero.
* Finally, we ``project`` the absolute magnetization so it is added to the list of our results for *each* query result.

$ jupyter notebook
Our query is now complete!
Let's have a look at the results:

And likewise use the file manager to open the Jupyter notebook.
.. code-block:: ipython
In [9]: qb.all()
Out[9]:
[['LaMnO3', 3.5],
['MnO3Sr', 3.15],
['CoO3Sr', 2.42],
['FeLaO3', 3.11],
['CoLaO3', 1.13],
['NiO3Sr', 0.77],
['FeO3Sr', 3.38]]
You can see that we've found 7 magnetic structures for the calculations in the ``tutorial_pbesol`` group, along with their formulas and magnetizations.
We've set up a script (:download:`demo_query.py <include/snippets/demo_query.py>`) that performs a similar query to obtain the magnetization and smearing energy for all results in the three groups, and then postprocess the data to visualize it.
You can find it in the dropdown panel below:

.. dropdown:: **Query demo script**

.. literalinclude:: include/snippets/demo_query.py

Download it using ``wget``:

Once you have finished the notebook, you can download a notebook with the solutions using:
.. code-block:: console
$ wget https://aiida-tutorials.readthedocs.io/en/tutorial-2020-bigmap-lab/_downloads/6773ba4cad0c046e468d13e15186cdd8/demo_query.py
and use ``verdi run`` to execute it:

.. code-block:: console
wget https://aiida-tutorials.readthedocs.io/en/tutorial-2020-intro-week/_downloads/e3aaf77cf1730bdd699fe43ab9cfd5d8/querybuilder-solutions.ipynb
$ verdi run demo_query.py
The resulting plot should look like the one shown in :numref:`BIGMAP_2020_Query_demo`.

.. _BIGMAP_2020_Query_demo:
.. figure:: include/images/demo_query.png
:width: 100%
:align: center

Comparison of the absolute magnetization and smearing energy of the cell of the perovskite structures, calculated with different functionals.

However, try not to use them at first!

What next?
----------
Expand Down

0 comments on commit 4485a80

Please sign in to comment.