Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test mandatory queries in validator #205

Merged
merged 3 commits into from Mar 6, 2020
Merged

Conversation

ml-evs
Copy link
Member

@ml-evs ml-evs commented Mar 4, 2020

This PR currently adds a mechanism for testing mandatory query syntax in the validator. It expects nothing in return (but if it receives data, it validates it) but does checks that an error is not returned.

Some thoughts:

  • At one point we spoke about keeping a centralized list of example queries, did this ever go anywhere? It would be nice to take these directly from the spec... (EDIT: this is now handled in Validate all non-optional :filter: examples from the spec #213)
  • The validator needs to be able to handle "optional features", I can either add that to this PR, or make another one... (EDIT: this is for a future PR...)

@ml-evs ml-evs force-pushed the ml-evs/validator_updates branch 3 times, most recently from 04c4be8 to 6fe4da3 Compare March 4, 2020 18:01
@ml-evs ml-evs changed the title Test mandatory queries in validator [WIP] Test mandatory queries in validator Mar 4, 2020
@ml-evs
Copy link
Member Author

ml-evs commented Mar 4, 2020

For some reason, my changes work when calling the validator as a script, but not in the CI. Will investigate later, this is WIP until then.

@codecov
Copy link

codecov bot commented Mar 5, 2020

Codecov Report

Merging #205 into master will increase coverage by 0.06%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #205      +/-   ##
==========================================
+ Coverage   87.08%   87.15%   +0.06%     
==========================================
  Files          42       42              
  Lines        1882     1892      +10     
==========================================
+ Hits         1639     1649      +10     
  Misses        243      243              
Flag Coverage Δ
#unittests 87.15% <100.00%> (+0.06%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3dff907...dec4e2e. Read the comment docs.

@ml-evs ml-evs changed the title [WIP] Test mandatory queries in validator Test mandatory queries in validator Mar 5, 2020
@ml-evs ml-evs requested a review from CasperWA March 5, 2020 17:24
@ml-evs ml-evs requested a review from shyamd March 5, 2020 21:30
@CasperWA
Copy link
Member

CasperWA commented Mar 6, 2020

Some thoughts:

  • At one point we spoke about keeping a centralized list of example queries, did this ever go anywhere? It would be nice to take these directly from the spec...

Not yet, I believe the issue you're thinking of is #80

  • The validator needs to be able to handle "optional features", I can either add that to this PR, or make another one...

Agreed - already the as type is nice, but it should be on a more fundamental level.

Copy link
Member

@CasperWA CasperWA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good; merge whenever.
We're starting to get a lot of hardcoding, but I guess it's difficult to avoid for validators and testing, but maybe it's time to start thinking about cutting this file (and the main class) into more modular pieces?

@ml-evs
Copy link
Member Author

ml-evs commented Mar 6, 2020

Seems good; merge whenever.
We're starting to get a lot of hardcoding, but I guess it's difficult to avoid for validators and testing, but maybe it's time to start thinking about cutting this file (and the main class) into more modular pieces?

Agreed, I'm trying to do this more in #213.

@ml-evs ml-evs merged commit 9a0c1fb into master Mar 6, 2020
@CasperWA CasperWA mentioned this pull request Mar 6, 2020
CasperWA added a commit that referenced this pull request Mar 6, 2020
Up to v0.6.0.

**New features**:
- GitHub Action validator that runs `optimade_validator` for a locally running OPTiMaDe server (#191, @CasperWA, tested by @ml-evs)
- Support filter queries for `HAS ALL`, `HAS ANY` and `HAS ONLY` and value lists on MongoDB backends (#173, @ml-evs)
  Note: `OPERATOR` use in value lists are still _not_ supported.
- Debug mode. Start the server in debug mode to enable `debug` log-level in `uvicorn` and get a full Python traceback in the JSON response (#190, @CasperWA)
- Add testing of mandatory `filter` queries to `optimade_validator` (#205, @ml-evs)

**Updates**:
- Allow Cross-Origin requests from anywhere (`*`), i.e., enable CORS for both servers (#194, @CasperWA)
- Updates to models (correct misspelling, more transparent model class naming, streamline models with respect to python class inheritance, update field descriptions) (#195, @CasperWA)
- API change: Rename `optimade.models.toplevel.py` to `optimade.models.responses.py` (#195, @CasperWA)
- Update dependencies to newest versions (as of 02.03.2020) (#202, #196, @CasperWA)
- Move imports from `starlette` to `fastapi`, where possible (#202, @ml-evs)
- Remove custom middleware to redirect slashed URLs in favor of `starlette` implementation (#202, @ml-evs)
- CI tests are now performed with a real MongoDB in the backend. CI tests are also performed with a `mongomock` backend for the tests in `server/test_middleware.py`, `server/test_server_validation.py`, and `server/test_config.py` (#196, @ml-evs, additional testing by @CasperWA )
- Remove `/optimade` from base URLs. This was especially important for the OpenAPI schema (#201, #216, @CasperWA, @ml-evs)
- Check integrity of query part of the raw URL using a custom middleware (#209, @CasperWA)
- New `optimade/server/exceptions.py` to contain custom `HttpException`s, moved `BadRequest` here (#209, @CasperWA)
- Pattern and regex testing for `data.available_api_versions` parts in `/info` endpoint and fix tests for the same (#211, @CasperWA)
- Restructure import of test data for regular server (#212, @shyamd)

**Bug fixes**:
- New retrieval URL for Materials-Consortia's list of providers (#187, @CasperWA)
- Skip local `HAS ONLY` tests with a `mongomock` backend, since v3.19.0 does not support these (#206, @ml-evs)
- Resource ID's can now contain slashes (`/`) (#183, @ml-evs, @CasperWA)
- Remove _valid_ version part of base URL in `meta.query.representation` (#201, @CasperWA)
@CasperWA CasperWA deleted the ml-evs/validator_updates branch March 9, 2020 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants