Skip to content

Add support for variable substitution for minNumRows in rowCount check #58

@phpisciuneri

Description

@phpisciuneri

We should be able to support the following use case or something analogous to it:

---
numKeyCols: 2
numErrorsToReport: 1
detailedErrors: true

vars:
  - name: NUM_ROWS
    value: 1000
  
tables:
  - db: census_income
    table: adult
    checks:
      - type: rowCount
        minNumRows: $NUM_ROWS

Currently, trying the above configuration yields a fairly non-descriptive DecodingFailure

21/01/15 12:24:03 ERROR Main$: Failed to parse config file 'issue.yaml, {}
DecodingFailure(Attempt to decode value on failed cursor, List(DownField(parquetFile), DownArray, DownField(tables)))

It is noted in the documentation that this is not currently supported.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions