Update to pydantic v1 #110

CasperWA · 2019-12-08T17:14:42Z

Fixes #101
Fixes #111

So far, this PR fixes the server so that it runs, pass all tests (locally) as well as the optimade-validator.

The whole of the server and all models should still be run through to check what else should be upgraded. E.g., introduce root validators, utilize new constrained types and more.

CasperWA · 2019-12-08T17:32:55Z

This may supersede the old PR #56 or reimplement the commits/ideas from there, providing full or co-authorship to @ltalirz (when these concepts have been implemented in this PR).

ml-evs · 2019-12-10T15:09:48Z

Happy with this so far, thanks @CasperWA. Anything I can do to help? Is it worth trying to do #104 here too, or in a separate PR?

CasperWA · 2019-12-10T16:10:49Z

Happy with this so far, thanks @CasperWA. Anything I can do to help? Is it worth trying to do #104 here too, or in a separate PR?

This should be in a separate PR in my opinion.
However, you are very welcome to help in moving to v1.2 here 👍

CasperWA · 2019-12-11T16:56:46Z

Latest commit fixes #111, re-instating the commented-out validator.

ml-evs

Cant see any issues with this, thanks @CasperWA!

CasperWA · 2019-12-22T17:16:21Z

Cant see any issues with this, thanks @CasperWA!

Cheers. I still want to go through all validators, see what can be updated according to the new v1 of pydantic, i.e., using root-validators, etc.
I think I will do this when bored in the holidays here. So I'll go through this page to see what we should check and perhaps update.
But we can also do this in another PR? I don't know. Getting this in as is, will see all of our tests starting to pass again...

CasperWA · 2019-12-22T21:13:07Z

I am having a hard time understanding why the tests related to relationships are currently failing. It seems the loading of the test structures into the mongomock db is failling to store the relationships field. At least that's the conclusion I have arrived at when printing doc in the map_back function.

CasperWA · 2019-12-22T22:52:46Z

I am having a hard time understanding why the tests related to relationships are currently failing. It seems the loading of the test structures into the mongomock db is failling to store the relationships field. At least that's the conclusion I have arrived at when printing doc in the map_back function.

I have pin-pointed it to the root_validator of the Attributes model under optimade.models.jsonapi. If this is returned to its previous state, all tests run fine (locally). But I have a hard time understanding why this is the case.

CasperWA · 2019-12-22T23:25:30Z

I am having a hard time understanding why the tests related to relationships are currently failing. It seems the loading of the test structures into the mongomock db is failling to store the relationships field. At least that's the conclusion I have arrived at when printing doc in the map_back function.

I have pin-pointed it to the root_validator of the Attributes model under optimade.models.jsonapi. If this is returned to its previous state, all tests run fine (locally). But I have a hard time understanding why this is the case.

Figured it out. It was a matter of hard-coding top-level non-attributes fields in _parse_params in the collection class: It should not be hard-coded ;)

CasperWA · 2019-12-22T23:28:47Z

@ml-evs I think this is ready for review + merge. I suggest any further pydantic "upgrades" can be put into separate PRs.

ml-evs · 2019-12-23T21:15:51Z

Any idea why pre-commit is failing for the openapi generation? I can't match the diff we get from the CI... (seems to be happening to a few PRs)

CasperWA · 2019-12-23T21:43:44Z

Any idea why pre-commit is failing for the openapi generation? I can't match the diff we get from the CI... (seems to be happening to a few PRs)

No idea. It is weird. But as long as the openapi test is not failing this should be fine. We can always choose to remove the check from the pre-commit, however, I do like it for commits.

ltalirz · 2020-01-06T14:57:55Z

@CasperWA Can you fix the conflicts so that we can get this merged?

Remove UrlStr validator for meta -> query -> representation. Set dependency for pydantic to ~=1.2

Update info for json_diff.sh message

Include minor corrections. Add validator for AvailableApiVersion.version.

CasperWA · 2020-01-07T10:54:58Z

@CasperWA Can you fix the conflicts so that we can get this merged?

Hereby done. The pre-commit is still failing, specifically the json-diff check, but this is irrelevant, since the openapi test succeeds, and this is essentially what we are testing.
I am not sure why this is failing on GH Actions (it is not failing locally). But in any case, the important one of these two checks (json-diff under pre-commit vs. openapi) is the openapi check.

ml-evs

I get the same behavior (i.e. it works locally) so happy to merge this and chalk it up to one of life's mysteries...

Thanks again @CasperWA!

ltalirz · 2020-01-07T14:56:25Z

Thanks for checking @ml-evs !
Good to go then!

CasperWA requested review from ltalirz and ml-evs December 8, 2019 17:14

CasperWA mentioned this pull request Dec 10, 2019

Index meta-database #103

Merged

3 tasks

CasperWA force-pushed the fix_101_pydantic_v1 branch from a47401c to a81b9b4 Compare December 11, 2019 16:50

ml-evs previously approved these changes Dec 22, 2019

View reviewed changes

CasperWA dismissed ml-evs’s stale review via 36d5ae9 December 22, 2019 19:58

CasperWA requested a review from ml-evs December 22, 2019 23:30

ltalirz mentioned this pull request Dec 24, 2019

Checklist for OPTiMaDe v0.10.1 #29

Closed

16 tasks

CasperWA added 10 commits January 7, 2020 10:42

UrlStr -> AnyUrl OR AnyHttpUrl

65e0164

Schema -> Field. For Optional: ... -> None

23dfaf3

Validator's whole -> each_item

ddcae8c

skip_defaults -> exclude_unset

5d7ba13

Remove UrlStr validator for meta -> query -> representation. Set dependency for pydantic to ~=1.2

Fix validators for pydantic v1

5aca71c

Update pydantic to 1.2 in requirements.txt

e69eb30

Re-instate validator for BaseInfoEndpoint

f4382cd

Update info for json_diff.sh message

Introduce root_validator

370f460

Include minor corrections. Add validator for AvailableApiVersion.version.

Check for None for illegal, existing fields

969f80c

Reroute top-level fields for docs

a44b548

Project all doc information

b8ab5b8

CasperWA force-pushed the fix_101_pydantic_v1 branch from 7c224a0 to b8ab5b8 Compare January 7, 2020 09:42

CasperWA changed the title ~~Update to pydantic v1.2~~ Update to pydantic v1 Jan 7, 2020

ml-evs approved these changes Jan 7, 2020

View reviewed changes

ltalirz approved these changes Jan 7, 2020

View reviewed changes

ml-evs merged commit a252260 into Materials-Consortia:master Jan 7, 2020

CasperWA deleted the fix_101_pydantic_v1 branch January 8, 2020 09:39

CasperWA mentioned this pull request Jan 8, 2020

Update to pydantic v1 aiidateam/aiida-optimade#27

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to pydantic v1 #110

Update to pydantic v1 #110

CasperWA commented Dec 8, 2019 •

edited

CasperWA commented Dec 8, 2019 •

edited

ml-evs commented Dec 10, 2019

CasperWA commented Dec 10, 2019

CasperWA commented Dec 11, 2019

ml-evs left a comment

CasperWA commented Dec 22, 2019 •

edited

CasperWA commented Dec 22, 2019

CasperWA commented Dec 22, 2019

CasperWA commented Dec 22, 2019 •

edited

CasperWA commented Dec 22, 2019

ml-evs commented Dec 23, 2019

CasperWA commented Dec 23, 2019

ltalirz commented Jan 6, 2020

CasperWA commented Jan 7, 2020

ml-evs left a comment

ltalirz commented Jan 7, 2020

Update to pydantic v1 #110

Update to pydantic v1 #110

Conversation

CasperWA commented Dec 8, 2019 • edited

CasperWA commented Dec 8, 2019 • edited

ml-evs commented Dec 10, 2019

CasperWA commented Dec 10, 2019

CasperWA commented Dec 11, 2019

ml-evs left a comment

Choose a reason for hiding this comment

CasperWA commented Dec 22, 2019 • edited

CasperWA commented Dec 22, 2019

CasperWA commented Dec 22, 2019

CasperWA commented Dec 22, 2019 • edited

CasperWA commented Dec 22, 2019

ml-evs commented Dec 23, 2019

CasperWA commented Dec 23, 2019

ltalirz commented Jan 6, 2020

CasperWA commented Jan 7, 2020

ml-evs left a comment

Choose a reason for hiding this comment

ltalirz commented Jan 7, 2020

CasperWA commented Dec 8, 2019 •

edited

CasperWA commented Dec 8, 2019 •

edited

CasperWA commented Dec 22, 2019 •

edited

CasperWA commented Dec 22, 2019 •

edited