Exabyte API Examples

Contents of this Repository

Below, we list the contents of this repository, in roughly the order that a user might want to go through it in order to learn how our API works.

Folder	Notebook	Description
Examples/System	Get Authentication Params	Demonstrates how to programatically find your user ID and access token, which is to authenticate for many portions of the Exabyte API.
Examples/Workflow	Get Workflows	Walks through how to query the Exabyte API to programatically search for workflows. In this example, we search for workflows that calculate the total energy of a material.
Examples/Material	Get Materials by Formula	Shows how queries can be made to search for materials stored on your account by their formula. In this example, we search for a system containing Si.
Examples/Material	Create Material	Gives an overview of how materials can be generated in JSON format and uploaded to your user account. In this example, we create an FCC Si crystal and upload it.
Examples/Material	Import Materials from Materials Project	Demonstrates how materials can be imported from Materials Project, if their Materials Project ID is known. In this example, we import monoclinic and hexagonal SiGe cells.
Examples/Material	Import Materials from Poscar	Provides an example of how materials can be imported directly from Poscar files (a common chemical file format best-known for its use in VASP). In this example, we import the unit cell of SiGe.
Examples/Job	Create and Submit Job	Shows how to use the Exabyte API to create jobs and run them on our cluster. In this example, we run a DFT calculation to get the total energy of an FCC Si unit cell using Quantum Espresso.
Examples/Job	Get File from Job	Guides you through using the Exabyte API to query for a list of files produced by a job, describes the metadata assigned to each file, and ends by demonstrating how to download any remote file generated by a job to the local disk.
Examples/Job	Run Simulations and Extract Properties	Leads you through the process of copying a bank workflow to your account and using it to automatically calculate the properties of multiple materials. In this example, we determine the band gap of Si and Ge.
Examples/Job	ML - Train Model Predict Properties	Walks you through automated dataset generation and the training/prediction of material properties using machine learning. In this example, we calculate the band gaps of Si and SiGe, and using various materials properties as descriptors, train a model to predict their band gaps. Finally, we use this trained model to predict the band gap of Ge.

Setup

NOTE: tested with python version 3.8.6, please assert that the virtual environment is created with it.

Follow the steps below in order to setup and view the Jupyter notebooks:

0. Install git-lfs [3] in order to get access to the source code and notebook files.

1. Clone repository:

   git clone git@github.com:Exabyte-io/exabyte-api-examples.git

   In case for some reason git-lfs was not installed at the time of cloning, the files can be pulled after installing git-lfs, through git lfs pull.

   Related to this, please be aware that as the .ipynb and .poscar files are stored on git-lfs, they are not part of the zip archive downloaded through GitHub's web interface.

2. Install virtualenv using pip if not already present:

   pip install virtualenv

3. Create virtual environment and install required packages:

   cd exabyte-api-examples
   virtualenv .env
   source .env/bin/activate
   pip install --no-deps -r requirements.txt

4. Run Jupyter and open a notebook in a browser. In order for the post-save hook feature to work properly, one must launch their Jupyter Notebook environment in the folder that contains the file config.py, which is the examples folder shown below:

   cd examples
   jupyter lab --config=config.py

Usage

In order to run or edit the examples:

1. Assert an existing Exabyte.io account. Examples require an account to run. New users can register here to obtain one.

2. Open settings and adjust it to provide the API authentication parameters. See the corresponding example to learn how to obtain the authentication parameters.

3. Open the desired example notebook, adjust it as necessary and run.

Contribute

This is an open-source repository and we welcome contributions for other use cases. The original set of examples is only meant to demonstrate the capabilities and can be extended.

We suggest forking this repository and introducing the adjustments there. The changes in the fork can further be considered for merging into this repository as it is commonly used on Github. This process is explained in more details elsewhere online [4].

If you would like to add new examples or adjust existing ones, please consider the following:

1. Put examples into the corresponding directories by domain.

2. Walk the readers through the examples by providing step-by-step explanation similar to this.

3. We use post-save hooks to automatically convert notebooks to python scripts. See config file for more information. In order to facilitate code review, we exclude notebook sources from version control and store them in Git LFS [3]. Please follow this convention.

Links

1. Exabyte.io RESTful API, description in the online documentation: link
2. Jupyter.org, official website: link
3. Git Large File Storage, official website: link
4. GitHub Standard Fork & Pull Request Workflow, online explanation: link