Index meta-database #103
Index meta-database #103
Conversation
Codecov Report
@@ Coverage Diff @@
## master #103 +/- ##
==========================================
+ Coverage 85.02% 86.16% +1.13%
==========================================
Files 44 47 +3
Lines 2371 2508 +137
==========================================
+ Hits 2016 2161 +145
+ Misses 355 347 -8
Continue to review full report at Codecov.
|
CasperWA
force-pushed
the
index_meta_db
branch
2 times, most recently
from
8c7ff42
to
190ad0f
Compare
There was a problem hiding this comment.
This is looking good @CasperWA , very minor thing below.
For the sake of my understanding, could you just clarify why the index meta-db needs to be run through a separate web server? Is it just to allow the main app to run above the base url?
| # raise ValueError( | ||
| # f"'{endpoint}' must be listed in available_endpoints to be valid" | ||
| # ) | ||
| # return v |
There was a problem hiding this comment.
Why has this been removed? If it was intentional can we remove the commented out version
There was a problem hiding this comment.
The issue is class-inheritance.
This validator is run for sub-classes as well, as was intended, but it doesn't register and recognize the instance of the sub-class as that: An instance of the sub-class. Instead it thinks it's an instance of the parent class and fails.
In the pydantic docs it's writen that this can be avoided by passing the check_fields=False to the validator. But this is for pydantic v1, and didn't seem to work for me when I tested it here.
So I have commented it out, since it should be able to work for pydantic v1.
I don't know if this makes this PR blocked until we have upgraded or not, but it definitely not desireable to have a commented out model validator.
Several reasons. Finally, I thought it was nice to be able to start both servers at the same time, testing that the linking/cross-server referencing works. |
CasperWA
force-pushed
the
index_meta_db
branch
from
55497f2
to
4c658a2
Compare
Thanks for the explanation, certainly makes sense for a reference implementation. I'll accept this once we're sure other blocking PRs are dealt with. |
In my opinion, this separation also works towards using this repository to directly run an index meta-database. In any case, I think this should/can be merged whenever, preferrably before #110. But maybe we want to publish a version after this has been merged (i.e. merge #107)? |
By calling `./run.sh index`, the index meta-database will be run at port 5001. This means one can run both servers on the same machine.
server_template.cfg is a template for a server.cfg file. server.cfg is equivalent to an .ini file and can be read using ConfigParser(). The point of server.cfg is to have a central place to write server-specific paths to possibly changing configuration files. At this point, it is used to direct the config.py towards the config-file, whatever format it may be in, as well as pointing to the index_links.json file that contains all resource objects to be added under the index meta-database's /links endpoint. If a server.cfg file does not exist upon loading CONFIG from config.py, it will be created as a copy of server_template.cfg.
Warn if index_links.json cannot be found. Raise if config-file cannot be found.
Add comparison tests for index_openapi.json. Update pre-commit to perform tests as well.
Add test file for index meta-db test server, based on test_server.py
This is mainly to avoid creating the `server.cfg` file automatically even when it is not needed.
Also install package .[dev] for pre-commit Actions job.
CasperWA
force-pushed
the
index_meta_db
branch
from
b66f81e
to
b411bcc
Compare
Closes #100
Easily implement an OPTiMaDe Index meta-database using this repository.
Start
Start the index meta-database by writing:
This runs an
uvicornserver athttp://localhost:5001/index/optimade, making it possible to open a new terminal and also run the "standard" OPTiMaDe server (athttp://localhost:5000/optimade).Setup
The file
/optimade/server/index_links_jsoncan be modified to list all known OPTiMaDe implementations by a provider to be listed under the index meta-database's/linksendpoint.EDIT: By request from @ltalirz, I have added support for a
server.cfgfile in the top-package directory.This is also added to
.gitignore, so if anyone forks the repository and wished to contribute back some updates, the file will not be overwritten. Instead, aserver_template.cfgfile exists that should never be changed in forks, but used as a template for their ownserver.cfgfile. This is due to the fact that if aserver.cfgfile is indeed not found, one will be created as a copy ofserver_template.cfg. This is needed for tests etc.The point of
server.cfgis to relay paths to server configuration files. So it currently has the paths for theconfig.iniorconfig.jsonfile and theindex_links.jsonfile.Note that the paths at this point MUST be relative, with respect to the location ofserver.cfg. This may be updated, with a check if the paths given inserver.cfgare absolute or not, before setting them inCONFIG.The paths can be relative, but MUST be so from the location of
server.cfg. They may also be absolute. They may, however, never include "~", since thePath-class cannot resolve this.Missing
local_index_openapi.jsonwith the fileindex_openapi.json.