Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Import feature doesn't properly validate conformance rules if output is a struct #1954

Closed
Zejnilovic opened this issue Oct 22, 2021 · 2 comments · Fixed by #1971
Closed

Import feature doesn't properly validate conformance rules if output is a struct #1954

Zejnilovic opened this issue Oct 22, 2021 · 2 comments · Fixed by #1971
Assignees
@Zejnilovic
Labels
bug Something isn't working Menas Menas component affected priority: high Critical to the health of the project

Comments

@Zejnilovic
Copy link
Contributor

Zejnilovic commented Oct 22, 2021 •
edited

Describe the bug

When importing a dataset having conformance rules that have structs as outputs, the validation does not add subfields to the evolving schema. This fails the validation if the output subfields are used in subsequent conformance rules

To Reproduce

Steps to reproduce the behaviour OR commands run:

  1. Have a dataset
  2. Have Mapping conformance rule which output a struct
  3. Have any other rule using a subfield from output struct of the 2nd step
  4. Try importing this Dataset

Expected behaviour

Import passes

Additional context

Check Rules -> Mapping, Single Column, Coalesce
@Zejnilovic Zejnilovic added bug Something isn't working Menas Menas component affected priority: high Critical to the health of the project labels Oct 22, 2021
@Zejnilovic
Copy link
Contributor Author

Since this will have to introduce a Mapping table entity into the DatasetService, we will also validate the targetAttribute

@Zejnilovic Zejnilovic self-assigned this Nov 2, 2021
Zejnilovic added a commit that referenced this issue Nov 3, 2021
@Zejnilovic
#1954 Fix mt conformance rule import validation ignoring structs
3c99983
@Zejnilovic Zejnilovic mentioned this issue Nov 3, 2021
Merged
Zejnilovic added a commit that referenced this issue Nov 3, 2021
@Zejnilovic
#1954 Fix mt conformance rule import validation ignoring structs
e76e8ba
dk1844 added a commit that referenced this issue Nov 5, 2021
@dk1844
#1954 testing case added
20e68e6
dk1844 added a commit that referenced this issue Nov 5, 2021
@dk1844
#1954 testing cases added, implementation of mapping-prefix-replaceme…
b14424b 
…nts broken down into multiple methods to make it readable
Zejnilovic pushed a commit that referenced this issue Nov 8, 2021
@dk1844
#1954 fix import validation mt rule suggestions
feffb56
Zejnilovic added a commit that referenced this issue Nov 15, 2021
@Zejnilovic @dk1844
#1954 Fix mt conformance rule import validation ignoring structs
95d760f 
Co-authored-by: Daniel K <dk1844@gmail.com>
@Zejnilovic
Copy link
Contributor Author

Release Notes:
When importing a dataset a mapping table conformance rule can now output a struct and will be properly validated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Zejnilovic Zejnilovic

Labels
bug Something isn't working Menas Menas component affected priority: high Critical to the health of the project
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

1954 Fix mt conformance rule import validation ignoring structs AbsaOSS/enceladus
1 participant
@Zejnilovic