Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add relationships functionality #91

Merged
merged 28 commits into from Nov 27, 2019
Merged

Conversation

ml-evs
Copy link
Member

@ml-evs ml-evs commented Nov 21, 2019

Closes #71.

  • Add subclassed relationships to include references key as in example spec 6.6.
  • Populate included key in response with the relationships attached to each entry inside the collection.
  • Actually make extra requests to populate included key in response with the reference's attributes, and validate them with reference models in the response.

A few notes/queries:

  • I'm not entirely clear how we should approach populating the included key: should it be an additional request/request per reference in the server code, hardcoded to only look in the references endpoint? Similarly, to adhere to the JSON API spec, we'd need to provide endpoints such as /structures/mpf_1/relationships/references/author, which I can't see us implementing...
  • I've had to change the JSON API implementation for relationship data from sets to lists as BaseResource is not hashable. As a result, the server code then has to loop over the relationships->references->data field to filter out duplicates based on ID.
  • How should we handle validating that the relationship returned by the structure actually exists? JSON warnings?
  • Do we want to support returning relationships as links rather than data (JSON API allows both/either,

@ml-evs ml-evs added enhancement New feature or request help wanted Extra attention is needed labels Nov 21, 2019
@ml-evs ml-evs added this to In progress in Road to optimade-python-tools 1.0 via automation Nov 21, 2019
@ml-evs ml-evs changed the title [WIP] Add functionality relationships [WIP] Add relationships functionality Nov 21, 2019
@codecov
Copy link

codecov bot commented Nov 21, 2019

Codecov Report

Merging #91 into master will increase coverage by 0.67%.
The diff coverage is 92.8%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #91      +/-   ##
==========================================
+ Coverage   85.06%   85.74%   +0.67%     
==========================================
  Files          38       38              
  Lines        2210     2315     +105     
==========================================
+ Hits         1880     1985     +105     
  Misses        330      330
Impacted Files Coverage Δ
optimade/models/links.py 86.95% <0%> (-3.96%) ⬇️
optimade/models/optimade_json.py 86.2% <0%> (ø) ⬆️
optimade/server/main.py 78.65% <100%> (+0.24%) ⬆️
optimade/models/tests/test_models.py 100% <100%> (ø) ⬆️
optimade/models/entries.py 100% <100%> (ø) ⬆️
optimade/server/utils.py 73.45% <100%> (+9.16%) ⬆️
optimade/server/tests/test_server.py 90.2% <100%> (+1.5%) ⬆️
optimade/server/mappers/structures.py 93.54% <100%> (+0.21%) ⬆️
optimade/validator/validator_model_patches.py 100% <100%> (ø) ⬆️
...ade/filtertransformers/tests/test_elasticsearch.py 80.64% <100%> (ø) ⬆️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 18181e2...b9290a5. Read the comment docs.

Copy link
Member

@CasperWA CasperWA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good @ml-evs.

I think the only solution is to, indeed, do a second query for all collected relationships.
As I can see now, this is what is missing?

optimade/models/entries.py Outdated Show resolved Hide resolved
optimade/models/tests/test_models.py Show resolved Hide resolved
optimade/models/toplevel.py Outdated Show resolved Hide resolved
optimade/server/entry_collections.py Outdated Show resolved Hide resolved
optimade/server/tests/test_structures.json Outdated Show resolved Hide resolved
@ltalirz ltalirz mentioned this pull request Nov 25, 2019
16 tasks
- Something goes wrong when validating and where AssertionErrors are raised which
pollutes all future data to be validated
optimade/server/entry_collections.py Outdated Show resolved Hide resolved
optimade/server/entry_collections.py Outdated Show resolved Hide resolved
Copy link
Member

@CasperWA CasperWA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't gone through your get_included_relationships function in detail, but it seems legit :)

optimade/server/main.py Outdated Show resolved Hide resolved
optimade/server/utils.py Show resolved Hide resolved
optimade/server/utils.py Outdated Show resolved Hide resolved
optimade/server/entry_collections.py Outdated Show resolved Hide resolved
optimade/server/entry_collections.py Outdated Show resolved Hide resolved
optimade/server/utils.py Show resolved Hide resolved
ml-evs and others added 4 commits November 27, 2019 11:45
Co-Authored-By: Casper Welzel Andersen <43357585+CasperWA@users.noreply.github.com>
Co-Authored-By: Casper Welzel Andersen <43357585+CasperWA@users.noreply.github.com>
@ml-evs ml-evs marked this pull request as ready for review November 27, 2019 12:05
@ml-evs
Copy link
Member Author

ml-evs commented Nov 27, 2019

I think this is basically ready now, the only thing that's not being done is returning a warning if the requested reference was not found under the references endpoint.

Also changed all but one `assert` statements in tests.
The only remaining `assert` statement is in a "real" pytest class.
CasperWA
CasperWA previously approved these changes Nov 27, 2019
Copy link
Member

@CasperWA CasperWA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to be in a state, where it can be merged in and we can call it "version 1" of adding relationships functionality.
Depending on how included and the include parameter will be defined finally in the spec, we will need to update this accordingly.

Thanks a bunch @ml-evs !

optimade/models/references.py Show resolved Hide resolved
optimade/models/toplevel.py Outdated Show resolved Hide resolved
optimade/server/entry_collections.py Outdated Show resolved Hide resolved
@CasperWA CasperWA changed the title [WIP] Add relationships functionality Add relationships functionality Nov 27, 2019
CasperWA
CasperWA previously approved these changes Nov 27, 2019
@ml-evs
Copy link
Member Author

ml-evs commented Nov 27, 2019

Seems to be in a state, where it can be merged in and we can call it "version 1" of adding relationships functionality.
Depending on how included and the include parameter will be defined finally in the spec, we will need to update this accordingly.

Thanks a bunch @ml-evs !

Yes, I came back to comment that this will currently populate included with any linked structures as well as references, which will probably need to be modified.

@ml-evs ml-evs requested a review from CasperWA November 27, 2019 13:50
@CasperWA CasperWA merged commit ed84012 into master Nov 27, 2019
Road to optimade-python-tools 1.0 automation moved this from In progress to Done Nov 27, 2019
@CasperWA CasperWA deleted the close_#71_add_relationships branch November 27, 2019 19:10
@CasperWA
Copy link
Member

Thanks @ml-evs !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Development

Successfully merging this pull request may close these issues.

Implement relationships in server
2 participants