Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing variable in freshness check should throw an error or at least warning #1200

Closed
vijaykiran opened this issue Mar 28, 2022 · 1 comment · Fixed by #1304
Closed

Missing variable in freshness check should throw an error or at least warning #1200

vijaykiran opened this issue Mar 28, 2022 · 1 comment · Fixed by #1304
Assignees
Labels
bug Something isn't working

Comments

@vijaykiran
Copy link
Contributor

running this without scan_execution_date being available as a variable

checks for fact_internet_sales:
  - freshness using ship_date with scan_execution_date < 12h45

it uses current scan run time instead of complaining about missing variable

@github-actions github-actions bot added this to New in Soda Core Roadmap Mar 28, 2022
@tombaeyens tombaeyens added the bug Something isn't working label Mar 31, 2022
@tombaeyens tombaeyens moved this from New to Bugs in Soda Core Roadmap Mar 31, 2022
@vijaykiran vijaykiran moved this from Bugs to Sprint Backlog in Soda Core Roadmap Apr 19, 2022
@vijaykiran vijaykiran moved this from Sprint Backlog to Bugs in Soda Core Roadmap Apr 19, 2022
@tombaeyens tombaeyens moved this from Bugs to Sprint Backlog in Soda Core Roadmap Apr 21, 2022
@m1n0 m1n0 self-assigned this Apr 27, 2022
m1n0 added a commit that referenced this issue Apr 27, 2022
m1n0 added a commit that referenced this issue Apr 27, 2022
@m1n0
Copy link
Contributor

m1n0 commented Apr 27, 2022

Linked PR removes the fallback to current time behaviour and fails the check early.

$ cat checks.yml
checks for sodatest_customers_b7580920:
  - freshness using ts with scan_execution_date < 1d

$ soda scan -d postgres -c configuration.yml checks.yml
Soda Core 3.0.0b9
Could not parse variable scan_execution_date as a timestamp: variable not found
Scan summary:
1/1 check FAILED:
    sodatest_customers_b7580920 in postgres
      freshness using ts with scan_execution_date < 1d [FAILED]
1 errors.
Oops! 1 failures. 0 warnings. 1 error. 0 pass.
ERRORS:
Could not parse variable scan_execution_date as a timestamp: variable not found

@m1n0 m1n0 moved this from Sprint Backlog to In progress in Soda Core Roadmap Apr 27, 2022
@m1n0 m1n0 moved this from In progress to Done in Soda Core Roadmap Apr 28, 2022
@m1n0 m1n0 moved this from Done to In progress in Soda Core Roadmap Apr 30, 2022
Soda Core Roadmap automation moved this from In progress to Done May 2, 2022
vijaykiran added a commit that referenced this issue May 2, 2022
* Fail freshness if variable is missing.

Fix #1200

Co-authored-by: Vijay Kiran <mail@vijaykiran.com>
linderttobias pushed a commit to linderttobias/soda-core that referenced this issue Sep 14, 2022
* Fail freshness if variable is missing.

Fix sodadata#1200

Co-authored-by: Vijay Kiran <mail@vijaykiran.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

3 participants