Merge pull request #119 from Materials-Consortia/ml-evs/fixes

A few fixes related to usage as a library

.github/workflows/deps_lint.yml

@@ -47,7 +47,7 @@ jobs:
       run: |
         python -m pip install --upgrade pip
         pip install -U setuptools
-        pip install .[dev]
+        pip install -e .[dev]
 
     - name: Run pre-commit
       run: |

MANIFEST.in

@@ -1 +1,3 @@
-recursive-include *.txt *.g *.py *.ini
+recursive-include optimade/server/ *.ini *.cfg *.json
+recursive-include optimade/server/data *
+recursive-include optimade/grammar *.g

README.md

@@ -2,7 +2,7 @@
 
 | Latest release | Build status | Activity |
 |:--------------:|:------------:|:--------:|
-| [![PyPI Version](https://img.shields.io/pypi/v/optimade?logo=pypi)](https://pypi.org/project/optimade/) [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/optimade?logo=python)](https://pypi.org/project/optimade/) [![OPTiMaDe](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Materials-Consortia/optimade-python-tools/master/.ci/optimade-version.json&logo=json)](https://github.com/Materials-Consortia/OPTiMaDe/) | [![Build Status](https://img.shields.io/github/workflow/status/Materials-Consortia/optimade-python-tools/Dependency,%20linting,%20and%20OpenAPI%20validation/gh_action_linter?logo=github)](https://github.com/Materials-Consortia/optimade-python-tools/actions?query=branch%3Amaster+) [![codecov](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/branch/master/graph/badge.svg)](https://codecov.io/gh/Materials-Consortia/optimade-python-tools)[![Heroku](https://heroku-badge.herokuapp.com/?app=optimade&root=optimade/info)](https://optimade.herokuapp.com/optimade/info) | [![Commit Activity](https://img.shields.io/github/commit-activity/m/Materials-Consortia/optimade-python-tools?logo=github)](https://github.com/Materials-Consortia/optimade-python-tools/pulse) |
+| [![PyPI Version](https://img.shields.io/pypi/v/optimade?logo=pypi)](https://pypi.org/project/optimade/)<br>[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/optimade?logo=python)](https://pypi.org/project/optimade/)<br>[![OPTiMaDe](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/Materials-Consortia/optimade-python-tools/master/.ci/optimade-version.json&logo=json)](https://github.com/Materials-Consortia/OPTiMaDe/) | [![Build Status](https://img.shields.io/github/workflow/status/Materials-Consortia/optimade-python-tools/Dependency,%20linting,%20and%20OpenAPI%20validation/gh_action_linter?logo=github)](https://github.com/Materials-Consortia/optimade-python-tools/actions?query=branch%3Amaster+)<br>[![codecov](https://codecov.io/gh/Materials-Consortia/optimade-python-tools/branch/master/graph/badge.svg)](https://codecov.io/gh/Materials-Consortia/optimade-python-tools)<br>[![Heroku](https://heroku-badge.herokuapp.com/?app=optimade&root=optimade/info)](https://optimade.herokuapp.com/optimade/info) | [![Commit Activity](https://img.shields.io/github/commit-activity/m/Materials-Consortia/optimade-python-tools?logo=github)](https://github.com/Materials-Consortia/optimade-python-tools/pulse) |
 
 The aim of OPTiMaDe is to develop a common API, compliant with the [JSON API 1.0](http://jsonapi.org/format/1.0/) specification.
 This is to enable interoperability among databases that contain calculated properties of existing and hypothetical materials.
@@ -16,14 +16,25 @@ Both the OPTiMaDe specification and this repository are **under development**.
 ## Installation (Index Meta-Database)
 
 This package may be used to setup and run an [OPTiMaDe index meta-database](https://github.com/Materials-Consortia/OPTiMaDe/blob/develop/optimade.rst#index-meta-database).
-Install the package via `pip install optimade[server]` and change the file `server.cfg` found in the root of the package.
+Install the package via `pip install optimade[server]` and change the file [`server.cfg`](server.cfg) found in the root of the package.
+
+The `server.cfg` file serves paths to a server runtime configuration file (either an `ini` or `json` file, see the [`config.ini` file](optimade/server/config.ini) for an example) and an index `/links`-endpoint data file.
+The paths must be relative from your current working directory, where your `server.cfg` is located, or they must be absolute paths.
 
-The `server.cfg` file serves paths to a server config file (either an `ini` or `json` file, see the `./optimade/server/config.ini` for an example) and an index `/links`-endpoint data file.
 The index meta-database is set up to populate a `mongomock` in-memory database with resources from a static `json` file containing the `child` resources you, as a database provider, want to serve under this index meta-database.
 
 Running the index meta-database is then as simple as writing `./run.sh index` in a terminal from the root of this package.
 You can find it at the base URL: [`http://localhost:5001/index/optimade/`](http://localhost:5001/index/optimade/).
 
+_Note_: `server.cfg` is loaded from the current working directory, from where you run `run.sh`.
+E.g., if you have installed `optimade` on a Linux machine at `/home/USERNAME/optimade/optimade-python-tools` and you run the following:
+
+```shell
+:~$ ./optimade/optimade-python-tools/run.sh index
+```
+
+Then you need `server.cfg` to be located in your home folder containing either relative paths from its current location or absolute paths.
+
 ## Development installation & Contributing
 
 Full installation instructions and contribution guidelines can be found in [CONTRIBUTING](CONTRIBUTING.md).

optimade/server/config.py

@@ -11,12 +11,12 @@ class NoFallback(Exception):
 class Config:
     """Base class for loading config files and its parameters"""
 
-    index_links_path: Path = Path("./optimade/server/index_links.json")
-    _path: Path = Path("./optimade/server/config.ini")
+    index_links_path: Path = Path(__file__).parent.joinpath("index_links.json")
+    _path: Path = Path(__file__).parent.joinpath("config.ini")
 
     def __init__(self, server_cfg: Path = None):
         self._server = (
-            Path(__file__).resolve().parent.parent.parent.joinpath("server.cfg")
+            Path().resolve().joinpath("server.cfg")
             if server_cfg is None
             else server_cfg
         )
@@ -38,9 +38,9 @@ def __getattr__(self, name: str) -> Any:
 
     def _create_server_config(self):
         """Create 'server.cfg' in top-package dir from 'server_template.cfg' if it does not exist"""
-        import shutil
+        import shutil  # pylint: disable=import-outside-toplevel
 
-        server_cfg_template = self._server.parent.joinpath("server_template.cfg")
+        server_cfg_template = Path(__file__).parent.joinpath("server_template.cfg")
         shutil.copyfile(server_cfg_template, self._server)
 
     def _load_server_config(self):
@@ -66,7 +66,7 @@ def _load_server_config(self):
             )
 
         if not self.index_links_path.exists():
-            from warnings import warn
+            from warnings import warn  # pylint: disable=import-outside-toplevel
 
             warn(
                 f'Cannot resolve {self.index_links_path}. Check the index_links.json file exists. Note, "~" is not allowed.'

optimade/server/entry_collections.py

@@ -124,20 +124,20 @@ def find(
         for doc in self.collection.find(**criteria):
             results.append(self.resource_cls(**self.resource_mapper.map_back(doc)))
 
+        nresults_now = len(results)
         if isinstance(params, EntryListingQueryParams):
-            nresults_now = len(results)
             criteria_nolimit = criteria.copy()
             criteria_nolimit.pop("limit", None)
             data_returned = self.count(**criteria_nolimit)
             more_data_available = nresults_now < data_returned
         else:
             # SingleEntryQueryParams, e.g., /structures/{entry_id}
-            data_returned = 1
+            data_returned = nresults_now
             more_data_available = False
-            if len(results) > 1:
+            if nresults_now > 1:
                 raise HTTPException(
                     status_code=404,
-                    detail=f"Instead of a single entry, {len(results)} entries were found",
+                    detail=f"Instead of a single entry, {nresults_now} entries were found",
                 )
             results = results[0] if results else None
 

optimade/server/main.py

@@ -33,7 +33,7 @@
     "references": Path(__file__).resolve().parent.joinpath("data/test_references.json"),
     "links": Path(__file__).resolve().parent.joinpath("data/test_links.json"),
 }
-if not CONFIG.use_real_mongo and (path.exists() for path in test_paths.values()):
+if not CONFIG.use_real_mongo and all(path.exists() for path in test_paths.values()):
     import bson.json_util
     from .routers import ENTRY_COLLECTIONS
 
@@ -89,7 +89,12 @@ def load_entries(endpoint_name: str, endpoint_collection: MongoCollection):
 
 def update_schema(app):
     """Update OpenAPI schema in file 'local_openapi.json'"""
-    with open("openapi/local_openapi.json", "w") as f:
+    local_openapi = (
+        Path(__file__)
+        .resolve()
+        .parent.parent.parent.joinpath("openapi/local_openapi.json")
+    )
+    with open(local_openapi, "w") as f:
         json.dump(app.openapi(), f, indent=2)
 
 

optimade/server/main_index.py

@@ -1,4 +1,5 @@
 import json
+from pathlib import Path
 
 from pydantic import ValidationError
 from fastapi import FastAPI
@@ -69,7 +70,12 @@
 
 def update_schema(app):
     """Update OpenAPI schema in file 'local_index_openapi.json'"""
-    with open("openapi/local_index_openapi.json", "w") as f:
+    local_openapi = (
+        Path(__file__)
+        .resolve()
+        .parent.parent.parent.joinpath("openapi/local_index_openapi.json")
+    )
+    with open(local_openapi, "w") as f:
         json.dump(app.openapi(), f, indent=2)
 
 

setup.py

@@ -31,7 +31,6 @@
     keywords="optimade jsonapi materials",
     include_package_data=True,
     packages=find_packages(),
-    package_data={"": ["*.lark"]},
     classifiers=[
         "Development Status :: 3 - Alpha",
         "Programming Language :: Python :: 3",

tests/server/test_server.py

@@ -189,6 +189,20 @@ def test_structures_endpoint_data(self):
         )
 
 
+class MissingSingleStructureEndpointTests(EndpointTests, unittest.TestCase):
+
+    test_id = "mpf_random_string_that_is_not_in_test_data"
+    request_str = f"/structures/{test_id}"
+    response_cls = StructureResponseOne
+
+    def test_structures_endpoint_data(self):
+        self.assertTrue("data" in self.json_response)
+        self.assertTrue("meta" in self.json_response)
+        self.assertEqual(self.json_response["data"], None)
+        self.assertEqual(self.json_response["meta"]["data_returned"], 0)
+        self.assertEqual(self.json_response["meta"]["more_data_available"], False)
+
+
 class SingleStructureWithRelationshipsTests(EndpointTests, unittest.TestCase):
 
     test_id = "mpf_1"