Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data Quality - Test case props should be unique #3217

Closed
pmbrull opened this issue Mar 7, 2022 · 0 comments · Fixed by #3220
Closed

Data Quality - Test case props should be unique #3217

pmbrull opened this issue Mar 7, 2022 · 0 comments · Fixed by #3220
Assignees
Labels
bug Something isn't working P0 Highest priority

Comments

@pmbrull
Copy link
Collaborator

pmbrull commented Mar 7, 2022

Affected module
Ingestion

Describe the bug
If we try to parse a TestCase pydantic model by passing a JSON, if there are multiple models with the same property names, we cannot properly parse this.

To Reproduce

obj = {'config': {'maxValue': '', 'minValue': '6'},
      'columnTestType': 'columnValueLengthsToBeBetween'}

from metadata.generated.schema.tests.columnTest import ColumnTestCase

ColumnTestCase.parse_obj(obj)

and we'll get

ValidationError: 14 validation errors for ColumnTestCase
config -> maxValue
  extra fields not permitted (type=value_error.extra)
config -> minValue
  extra fields not permitted (type=value_error.extra)
config -> maxValue
  extra fields not permitted (type=value_error.extra)
config -> minValue
  extra fields not permitted (type=value_error.extra)
config -> regex
  field required (type=value_error.missing)
config -> maxValue
  extra fields not permitted (type=value_error.extra)
config -> minValue
  extra fields not permitted (type=value_error.extra)
config -> forbiddenValues
  field required (type=value_error.missing)
config -> maxValue
  extra fields not permitted (type=value_error.extra)
config -> minValue
  extra fields not permitted (type=value_error.extra)
config -> maxValue
  value is not a valid integer (type=type_error.integer)
config -> maxValue
  extra fields not permitted (type=value_error.extra)
config -> minValue
  extra fields not permitted (type=value_error.extra)
config -> maxValue
  value is not a valid integer (type=type_error.integer)

Expected behavior
Test definitions should have unique naming in their properties to make sure pydantic properly identifies each class

Version:

  • OS: [e.g. iOS]
  • Python version:
  • OpenMetadata version: [e.g. 0.8]
  • OpenMetadata Ingestion package version: [e.g. openmetadata-ingestion[docker]==XYZ]

Additional context
Add any other context about the problem here.

@pmbrull pmbrull self-assigned this Mar 7, 2022
@pmbrull pmbrull added bug Something isn't working P0 Highest priority labels Mar 7, 2022
pmbrull added a commit that referenced this issue Mar 7, 2022
Fix #3217 - Test case props should be unique (#3220)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working P0 Highest priority
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant