Merge pull request #70 from Exabyte-io/bugfix/SOF-5578

Bugfix/sof 5578

.gitattributes

@@ -11,3 +11,4 @@ examples/material/get_materials_by_formula.ipynb    !filter !diff !merge text
 examples/material/import_materials_from_materialsproject.ipynb  !filter !diff !merge text
 examples/material/import_materials_from_poscar.ipynb    !filter !diff !merge text
 examples/system/get_authentication_params.ipynb !filter !diff !merge text
+images/*.png filter=lfs diff=lfs merge=lfs -text

.github/workflows/check_links.yml

@@ -4,12 +4,13 @@ on: [push, pull_request]
 
 jobs:
   check-links:
+    # Do not run this check if a PR is created from the same repo.
+    if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name != github.repository
     runs-on: ubuntu-latest
     timeout-minutes: 10
     steps:
-    - uses: actions/checkout@v1
-    - name: Link Checker
-      id: lc
-      uses: peter-evans/link-checker@v1
-    - name: Fail if there were link errors
-      run: exit ${{ steps.lc.outputs.exit_code }}
+      - uses: actions/checkout@v3
+      - name: Link Checker
+        uses: lycheeverse/lychee-action@v1.8.0
+        with:
+          fail: true

.lycheeignore

@@ -0,0 +1,2 @@
+https://pubs.acs.org/doi/abs/10.1021/acs.nanolett.8b00670
+https://advances.sciencemag.org/content/5/9/eaax5101.abstract

README.md

@@ -6,22 +6,22 @@ Below, we list the contents of this repository, in roughly the order that a user
 
 | Folder            | Notebook                                | Description |
 | ------------------|-----------------------------------------| ----------- |
-| [Examples/System](examples/system/)     | [Get Authentication Params](examples/system/get_authentication_params.ipynb)               | Demonstrates how to programatically find your user ID and access token, which is to [authenticate](https://docs.exabyte.io/rest-api/authentication/) for many portions of the Exabyte API.
-| [Examples/Workflow](examples/workflow/) | [Get Workflows](examples/workflow/get_workflows.ipynb)                           | Walks through how to [query](https://docs.exabyte.io/rest-api/query-structure/) the Exabyte API to programatically search for workflows. In this example, we search for workflows that calculate the total energy of a material.
-| [Examples/Material](examples/material/) | [Get Materials by Formula](examples/material/get_materials_by_formula.ipynb)                | Shows how [queries](https://docs.exabyte.io/rest-api/query-structure/) can be made to search for materials stored on your account by their formula. In this example, we search for a system containing Si.
-| [Examples/Material](examples/material/) | [Create Material](examples/material/create_material.ipynb)                         | Gives an overview of how materials can be generated in [JSON format](https://docs.exabyte.io/materials/data/) and uploaded to your user account. In this example, we create an FCC Si crystal and upload it.
+| [Examples/System](examples/system/)     | [Get Authentication Params](examples/system/get_authentication_params.ipynb)               | Demonstrates how to programatically find your user ID and access token, which is to [authenticate](https://docs.mat3ra.com/rest-api/authentication/) for many portions of the Exabyte API.
+| [Examples/Workflow](examples/workflow/) | [Get Workflows](examples/workflow/get_workflows.ipynb)                           | Walks through how to [query](https://docs.mat3ra.com/rest-api/query-structure/) the Exabyte API to programatically search for workflows. In this example, we search for workflows that calculate the total energy of a material.
+| [Examples/Material](examples/material/) | [Get Materials by Formula](examples/material/get_materials_by_formula.ipynb)                | Shows how [queries](https://docs.mat3ra.com/rest-api/query-structure/) can be made to search for materials stored on your account by their formula. In this example, we search for a system containing Si.
+| [Examples/Material](examples/material/) | [Create Material](examples/material/create_material.ipynb)                         | Gives an overview of how materials can be generated in [JSON format](https://docs.mat3ra.com/materials/data/) and uploaded to your user account. In this example, we create an FCC Si crystal and upload it.
 | [Examples/Material](examples/material/) | [Import Materials from Materials Project](examples/material/import_materials_from_materialsproject.ipynb) | Demonstrates how materials can be imported from [Materials Project](https://materialsproject.org/about), if their Materials Project ID is known. In this example, we import monoclinic and hexagonal SiGe cells.
 | [Examples/Material](examples/material/) | [Import Materials from Poscar](examples/material/import_materials_from_poscar.ipynb)            | Provides an example of how materials can be imported directly from Poscar files (a common chemical file format best-known [for its use in VASP](https://www.vasp.at/wiki/index.php/Input)). In this example, we import the unit cell of SiGe.
-| [Examples/Job](examples/job/)            | [Create and Submit Job](examples/job/create_and_submit_job.ipynb)                   | Shows how to use the Exabyte API to [create jobs](https://docs.exabyte.io/jobs/data/) and run them on our cluster. In this example, we run a DFT calculation to get the total energy of an FCC Si unit cell using Quantum Espresso.
+| [Examples/Job](examples/job/)            | [Create and Submit Job](examples/job/create_and_submit_job.ipynb)                   | Shows how to use the Exabyte API to [create jobs](https://docs.mat3ra.com/jobs/data/) and run them on our cluster. In this example, we run a DFT calculation to get the total energy of an FCC Si unit cell using Quantum Espresso.
 | [Examples/Job](examples/job/)            | [Get File from Job](examples/job/get-file-from-job.ipynb) | Guides you through using the Exabyte API to query for a list of files produced by a job, describes the metadata assigned to each file, and ends by demonstrating how to download any remote file generated by a job to the local disk.
-| [Examples/Job](examples/job/)            | [Run Simulations and Extract Properties](examples/job/run-simulations-and-extract-properties.ipynb)  | Leads you through the process of copying a [bank workflow](https://docs.exabyte.io/workflows/bank/) to your account and using it to automatically calculate the [properties](https://docs.exabyte.io/properties/overview/) of multiple materials. In this example, we determine the [band gap](https://docs.exabyte.io/properties-directory/non-scalar/band-gaps/) of Si and Ge.
-| [Examples/Job](examples/job/)            | [ML - Train Model Predict Properties](examples/job/ml-train-model-predict-properties.ipynb)     | Walks you through automated dataset generation and the training/prediction of material properties using [machine learning](https://docs.exabyte.io/software-directory/overview/#machine-learning). In this example, we calculate the band gaps of Si and SiGe, and using various materials properties as descriptors, train a model to predict their band gaps. Finally, we use this trained model to predict the band gap of Ge.
+| [Examples/Job](examples/job/)            | [Run Simulations and Extract Properties](examples/job/run-simulations-and-extract-properties.ipynb)  | Leads you through the process of copying a [bank workflow](https://docs.mat3ra.com/workflows/bank/) to your account and using it to automatically calculate the [properties](https://docs.mat3ra.com/properties/overview/) of multiple materials. In this example, we determine the [band gap](https://docs.mat3ra.com/properties-directory/non-scalar/band-gaps/) of Si and Ge.
+| [Examples/Job](examples/job/)            | [ML - Train Model Predict Properties](examples/job/ml-train-model-predict-properties.ipynb)     | Walks you through automated dataset generation and the training/prediction of material properties using [machine learning](https://docs.mat3ra.com/software-directory/overview/#machine-learning). In this example, we calculate the band gaps of Si and SiGe, and using various materials properties as descriptors, train a model to predict their band gaps. Finally, we use this trained model to predict the band gap of Ge.
 
 
 
 ## Setup
 
-NOTE: tested with python version 3.8.6, please assert that the virtual environment is created with it.
+NOTE: tested with Python version 3.8.6, please assert that the virtual environment is created with it.
 
 Follow the steps below in order to setup and view the Jupyter notebooks:
 
@@ -30,7 +30,7 @@ Follow the steps below in order to setup and view the Jupyter notebooks:
 1. Clone repository:
 
     ```bash
-    git clone git@github.com:Exabyte-io/exabyte-api-examples.git
+    git clone git@github.com:Exabyte-io/api-examples.git
     ```
 
     In case for some reason git-lfs was not installed at the time of cloning, the files can be pulled after installing git-lfs, through `git lfs pull`.
@@ -46,7 +46,7 @@ Follow the steps below in order to setup and view the Jupyter notebooks:
 3. Create virtual environment and install required packages:
 
     ```bash
-    cd exabyte-api-examples
+    cd api-examples
     virtualenv .env
     source .env/bin/activate
     pip install --no-deps -r requirements.txt
@@ -63,19 +63,23 @@ Follow the steps below in order to setup and view the Jupyter notebooks:
 
 In order to run or edit the examples:
 
-1. Assert an existing Exabyte.io account. Examples require an account to run. New users can register [here](https://platform.exabyte.io/register) to obtain one.
+1. Assert an existing Exabyte.io account. Examples require an account to run. New users can register [here](https://platform.mat3ra.com/register) to obtain one.
 
 2. Open [settings](examples/settings.json) and adjust it to provide the API authentication parameters. See the [corresponding example](examples/system/get_authentication_params.ipynb) to learn how to obtain the authentication parameters.
 
-3. Open the desired example notebook, adjust it as necessary and run.
+3. Open the desired example notebook, adjust it as necessary and run. One can speed up the notebooks execution after running the [Get Authentication Params](examples/system/get_authentication_params.ipynb) one by reusing the kernel from the first notebook.
+
+  <img src="images/reusable-kernel.png" width="250px" />
+
+NOTE: The Materials Project API key should be obtained from https://legacy.materialsproject.org/open.
 
 
 ## Contribute
 
 This is an open-source repository and we welcome contributions for other use cases. The original set of examples is only meant to demonstrate the capabilities and can be extended.
 
 We suggest forking this repository and introducing the adjustments there. The changes in the fork can further be considered for merging into this repository as it is commonly used on Github. This process is explained in more details elsewhere online [[4](#links)].
- 
+
 If you would like to add new examples or adjust existing ones, please consider the following:
 
 1. Put examples into the corresponding directories by domain.
@@ -86,7 +90,7 @@ If you would like to add new examples or adjust existing ones, please consider t
 
 ## Links
 
-1. Exabyte.io RESTful API, description in the online documentation: [link](https://docs.exabyte.io/rest-api/overview/)
-2. Jupyter.org, official website: [link](http://jupyter.org/)
+1. Exabyte.io RESTful API, description in the online documentation: [link](https://docs.mat3ra.com/rest-api/overview/)
+2. Jupyter.org, official website: [link](https://jupyter.org/)
 3. Git Large File Storage, official website: [link](https://git-lfs.github.com/)
-4. GitHub Standard Fork & Pull Request Workflow, online explanation: [link](https://gist.github.com/Chaser324/ce0505fbed06b947d962) 
+4. GitHub Standard Fork & Pull Request Workflow, online explanation: [link](https://gist.github.com/Chaser324/ce0505fbed06b947d962)

examples/settings.py

@@ -36,5 +36,5 @@
 PORT = 443
 SECURE = True
 VERSION = "2018-10-01"
-HOST = "platform.exabyte.io"
+HOST = "platform.mat3ra.com"
 ENDPOINT_ARGS = [HOST, PORT, ACCOUNT_ID, AUTH_TOKEN, VERSION, SECURE]

examples/utils/initialize_settings.py

@@ -6,10 +6,21 @@
 # If using Jupyter, these variables can be left to their default values in the code cell, but the user
 # should change these values in the settings.json file located in the examples folder.
 
-environment_variables_config = {'ACCOUNT_ID': ACCOUNT_ID,
-                                'AUTH_TOKEN': AUTH_TOKEN,
-                                'MATERIALS_PROJECT_API_KEY': MATERIALS_PROJECT_API_KEY,
-                                'ORGANIZATION_ID': ORGANIZATION_ID}
+import glob
+import os
+import socket
+import sys
+
+import requests
+from IPython import get_ipython
+
+environment_variables_config = {
+    "ACCOUNT_ID": ACCOUNT_ID,  # noqa F821
+    "AUTH_TOKEN": AUTH_TOKEN,  # noqa F821
+    "MATERIALS_PROJECT_API_KEY": MATERIALS_PROJECT_API_KEY,  # noqa F821
+    "ORGANIZATION_ID": ORGANIZATION_ID,  # noqa F821
+}
+
 
 def set_notebook_environment(environment_variables_config):
     """
@@ -24,15 +35,30 @@ def set_notebook_environment(environment_variables_config):
     Return:
         None
     """
-    notebook_environment = environment_variables_config['notebook_environment']
-    if notebook_environment == 'Colab':
+    notebook_environment = environment_variables_config["notebook_environment"]
+    if notebook_environment == "Colab":
         from utils.colab import setup_colab_runtime_environment
+
         setup_colab_runtime_environment(environment_variables_config)
     else:
         from utils.generic import ensure_packages_are_installed
+
         ensure_packages_are_installed(notebook_environment)
 
 
+def get_notebook_name():
+    """
+    Get the name of a currently running notebook in Google Colab.
+    Args:
+        None
+    Return:
+        filename
+    """
+    ip = socket.gethostbyname(socket.gethostname())  # 172.28.0.12
+    filename = requests.get(f"http://{ip}:9000/api/sessions").json()[0]["name"]
+    return filename
+
+
 def execute():
     """
     Main execution function. This function determines and sets the runtime environment
@@ -43,22 +69,25 @@ def execute():
         None
     """
 
-    if 'is_setup_executed' not in os.environ:
-        if 'google.colab' in str(get_ipython()):
-            environment_variables_config.update({'notebook_environment': 'Colab'})
-            get_ipython().system('git clone https://github.com/Exabyte-io/exabyte-api-examples.git')
-            from google.colab import _message
-            notebook_name = _message.blocking_request('get_ipynb')['ipynb']['metadata']['colab']['name']
-            notebook_path = glob.glob('**/'+notebook_name, recursive=True)[0][0:-len(notebook_name)]
-            os.chdir(notebook_path) # go to the folder in the repo where one would be if this was in local Jupyter
-        elif 'ZMQInteractiveShell' in str(get_ipython()):
-            environment_variables_config.update({'notebook_environment': 'Jupyter'})
+    if "is_setup_executed" not in os.environ:
+        ip = get_ipython()
+        if "google.colab" in str(ip):
+            branch = os.getenv("GIT_BRANCH", "dev")  # a way to inject a branch of interest from Colab if run via a PR.
+            environment_variables_config.update({"notebook_environment": "Colab"})
+            ip.system(f"git clone --depth=1 --single-branch -b {branch} https://github.com/Exabyte-io/api-examples.git")
+            notebook_name = get_notebook_name()
+            notebook_path = glob.glob(f"**/{notebook_name}", recursive=True)[0][0 : -len(notebook_name)]
+            os.chdir(notebook_path)  # go to the folder in the repo where one would be if this was in local Jupyter
+        elif "ZMQInteractiveShell" in str(ip):
+            environment_variables_config.update({"notebook_environment": "Jupyter"})
         else:
-            environment_variables_config.update({'notebook_environment': ''})
+            environment_variables_config.update({"notebook_environment": ""})
 
-        module_path = os.path.abspath(os.path.join('..'))
-        if module_path not in sys.path: sys.path.append(module_path)
+        module_path = os.path.abspath(os.path.join(".."))
+        if module_path not in sys.path:
+            sys.path.append(module_path)
         set_notebook_environment(environment_variables_config)
 
+
 execute()
-os.environ.update({'is_setup_executed': 'True'})
+os.environ.update({"is_setup_executed": "True"})