Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL variable substitution fails when result is a double #53

Closed
phpisciuneri opened this issue Jan 14, 2021 · 0 comments · Fixed by #54
Closed

SQL variable substitution fails when result is a double #53

phpisciuneri opened this issue Jan 14, 2021 · 0 comments · Fixed by #54
Assignees
Labels
bug Something isn't working

Comments

@phpisciuneri
Copy link
Contributor

Observed, for example, when combined with ColumnSumCheck. See example config below:

---
numKeyCols: 2
numErrorsToReport: 1
detailedErrors: true

vars:
  - name: MAX_AGE
    sql: SELECT CAST(MAX(age) AS DOUBLE) FROM census_income.adult

outputs:
  - filename: report.json
    append: false

email:
  smtpHost: smtp.example.com
  subject: Data Validation Summary
  from: data-validator-no-reply@example.com
  to:
    - person1@example.com

tables:
  - db: census_income
    table: adult
    checks:
      - type: columnSumCheck
        column: age
        minValue: $MAX_AGE
        inclusive: true

yields:

...
...
21/01/14 09:12:01 ERROR JsonUtils$: Unimplemented dataType 'double' in column: CAST(max(age) AS DOUBLE) Please report this as a bug.
21/01/14 09:12:01 INFO ValidatorConfig: substituteVariables()
21/01/14 09:12:02 INFO Substitutable$class: Substituting Json minValue Json: "$MAX_AGE" with `null`
...
...
21/01/14 09:12:02 ERROR ColumnSumCheck$$anonfun$configCheck$1: 'minValue' defined but type is not a Number, is: Null
21/01/14 09:12:02 ERROR ValidatorTable$$anonfun$1: ConfigCheck failed for HiveTable:`census_income.adult`
...
...
@phpisciuneri phpisciuneri self-assigned this Jan 14, 2021
@phpisciuneri phpisciuneri added the bug Something isn't working label Jan 14, 2021
@phpisciuneri phpisciuneri linked a pull request Jan 14, 2021 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant