This repository has been archived by the owner on Jul 3, 2023. It is now read-only.
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Adds ability to specify applies_to in data quality decorator.
This is *very* rough. The idea is that we should be able to choose one of the following modes: 1. Apply a validator to every final nod ein a subdag of a decorated function 2. Apply a validator to a specific node within the subdag of a decorated function This basically allows (1) which is the default, but also (2) if using the applies_to keyword. Note that this only works if its in the final subdag (I.E. a sync), and not in the middle. We should add that but it'll be a little bit of a change. Nothing we can't make backwards compatible, we just might need to crawl back a little further in our layered API -- E.G. use subdag transformer rather than node transformer. Either way, this shows that we can ddo what we want without too many modifications. Note that this is not tested, just a proof of concept.
- Loading branch information