Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Validation of 'structures' type crashes #397

Closed
merkys opened this issue Jul 10, 2020 · 4 comments · Fixed by #395
Closed

Validation of 'structures' type crashes #397

merkys opened this issue Jul 10, 2020 · 4 comments · Fixed by #395
Assignees
@ml-evs
Labels
validator Related to the OPTIMADE validator

Comments

@merkys
Copy link
Member

merkys commented Jul 10, 2020

I am trying to validate COD implementation (in development) using optimade-python-tools v0.9.8. I experience a crash, although I assume the validator should be crash-free, even in dealing with weirdest responses:

$ optimade_validator https://www.crystallography.net/cod-test/optimade/structures --as_type structures
.Traceback (most recent call last):
  File "/home/andrius/.local/lib/python3.6/site-packages/optimade/validator/__init__.py", line 90, in validate
    validator.main()
  File "/home/andrius/.local/lib/python3.6/site-packages/optimade/validator/validator.py", line 360, in main
    self.test_as_type()
  File "/home/andrius/.local/lib/python3.6/site-packages/optimade/validator/validator.py", line 472, in test_as_type
    self.deserialize_response(response, self.as_type_cls)
  File "/home/andrius/.local/lib/python3.6/site-packages/optimade/validator/validator.py", line 174, in wrapper
    result, msg = test_fn(validator, *args, **kwargs)
  File "/home/andrius/.local/lib/python3.6/site-packages/optimade/validator/validator.py", line 558, in deserialize_response
    response_cls(**response.json()),
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/main.py", line 336, in __init__
    values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/main.py", line 887, in validate_model
    v_, errors_ = field.validate(value, values, loc=field.alias, cls=cls_)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/fields.py", line 560, in validate
    v, errors = self._validate_sequence_like(v, values, loc, cls)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/fields.py", line 591, in _validate_sequence_like
    r, ee = self._validate_singleton(v_, values, v_loc, cls)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/fields.py", line 697, in _validate_singleton
    value, error = field.validate(v, values, loc=loc, cls=cls)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/fields.py", line 549, in validate
    v, errors = self._validate_singleton(v, values, loc, cls)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/fields.py", line 704, in _validate_singleton
    return self._apply_validators(v, values, loc, cls, self.validators)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/fields.py", line 711, in _apply_validators
    v = validator(cls, v, values, self, self.model_config)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/class_validators.py", line 313, in <lambda>
    return lambda cls, v, values, field, config: validator(v)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/main.py", line 573, in validate
    return cls(**value)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/main.py", line 336, in __init__
    values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/main.py", line 887, in validate_model
    v_, errors_ = field.validate(value, values, loc=field.alias, cls=cls_)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/fields.py", line 549, in validate
    v, errors = self._validate_singleton(v, values, loc, cls)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/fields.py", line 704, in _validate_singleton
    return self._apply_validators(v, values, loc, cls, self.validators)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/fields.py", line 711, in _apply_validators
    v = validator(cls, v, values, self, self.model_config)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/class_validators.py", line 313, in <lambda>
    return lambda cls, v, values, field, config: validator(v)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/main.py", line 573, in validate
    return cls(**value)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/main.py", line 336, in __init__
    values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/main.py", line 887, in validate_model
    v_, errors_ = field.validate(value, values, loc=field.alias, cls=cls_)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/fields.py", line 549, in validate
    v, errors = self._validate_singleton(v, values, loc, cls)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/fields.py", line 704, in _validate_singleton
    return self._apply_validators(v, values, loc, cls, self.validators)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/fields.py", line 711, in _apply_validators
    v = validator(cls, v, values, self, self.model_config)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/class_validators.py", line 313, in <lambda>
    return lambda cls, v, values, field, config: validator(v)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/main.py", line 573, in validate
    return cls(**value)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/main.py", line 336, in __init__
    values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/main.py", line 887, in validate_model
    v_, errors_ = field.validate(value, values, loc=field.alias, cls=cls_)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/fields.py", line 563, in validate
    v, errors = self._apply_validators(v, values, loc, cls, self.post_validators)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/fields.py", line 711, in _apply_validators
    v = validator(cls, v, values, self, self.model_config)
  File "/home/andrius/.local/lib/python3.6/site-packages/pydantic/class_validators.py", line 280, in <lambda>
    return lambda cls, v, values, field, config: validator(cls, v)
  File "/home/andrius/.local/lib/python3.6/site-packages/optimade/models/entries.py", line 23, in check_rel_type
    if hasattr(cls, "_req_type") and any(obj.type != cls._req_type for obj in data):
  File "/home/andrius/.local/lib/python3.6/site-packages/optimade/models/entries.py", line 23, in <genexpr>
    if hasattr(cls, "_req_type") and any(obj.type != cls._req_type for obj in data):
AttributeError: 'tuple' object has no attribute 'type'
@merkys merkys added the validator Related to the OPTIMADE validator label Jul 10, 2020
@ml-evs ml-evs self-assigned this Jul 10, 2020
@ml-evs
Copy link
Member

ml-evs commented Jul 10, 2020

Will look into this too!

ml-evs added a commit that referenced this issue Jul 10, 2020
@ml-evs
Validate relationships data as a list before testing type (closes #397)
6ffc74c
@ml-evs
Copy link
Member

ml-evs commented Jul 10, 2020 •
edited

I've fixed the validator crash, but I'm not entirely sure what to do about the actual validation... the problem here is that your example structure contains a responds with a relationship to just one other structure as below:

      "relationships": {
        "references": {
          "data": {
            "id": "1000000",
            "type": "references"
          }

whereas we are expecting data to be a list:

      "relationships": {
        "references": {
          "data": [
            {
                "id": "1000000",
                "type": "references"
            }
          ]

Following the spec and relevant part of JSON:API, it looks like we should allow both. Do you agree with this summary @merkys?

ml-evs added a commit that referenced this issue Jul 10, 2020
@ml-evs
Validate relationships data as a list before testing type (closes #397)
2b4c3bd
@ml-evs ml-evs mentioned this issue Jul 10, 2020
Merged
@merkys
Copy link
Member Author

merkys commented Jul 10, 2020

Following the spec and relevant part of JSON:API, it looks like we should allow both. Do you agree with this summary @merkys?

Yes, I have the same opinion.

ml-evs added a commit that referenced this issue Jul 16, 2020
@ml-evs
Validate relationships data as a list before testing type (closes #397)
a3b420a
ml-evs added a commit that referenced this issue Jul 17, 2020
@ml-evs @CasperWA @merkys
Fix some validator-specific crashes (#395)
93e50b1 
* Give validator patched `included` a default value to make it really optional (closes #393)

* Validate relationships data as a list before testing type (closes #397)

* Improve --verbosity help string for validator (closes #396)

* Add concept of InternalErrors to validator

These errors should be handled differently to ValidationError and
ResponseError as they indicate a problem with the validator itself.

* Added validator flags --fail_fast and --skip_optional_tests

* Improvements to pagination validation

- Allow links objects to be passed in pagination
- Enforced maximum recursion depth (5) for pagination tests

* Reduce set of example queries from the specification, pending future improvements (#357)

* Improve error message for single entry endpoint if deserialization fails

* Implemented suggestions from code review

Co-authored-by: Casper Welzel Andersen <CasperWA@users.noreply.github.com>

* Renamed entrypoint optimade_validator->optimade-validator

* Flip arg options the "standard" way around

Use -t for --as-type (instead of -a).

Co-authored-by: Casper Welzel Andersen <casper.andersen@epfl.ch>
Co-authored-by: Andrius Merkys <andrius.merkys@gmail.com>
@merkys merkys mentioned this issue Jul 22, 2020
Closed
@merkys
Copy link
Member Author

merkys commented Jul 22, 2020

I confirm this is fixed as of v0.10.0.

@merkys merkys mentioned this issue Jul 24, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ml-evs ml-evs

Labels
validator Related to the OPTIMADE validator
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix some validator-specific crashes Materials-Consortia/optimade-python-tools
2 participants
@ml-evs @merkys