Skip to content
This repository has been archived by the owner on May 5, 2022. It is now read-only.

Gracefully handle source regressions #772

Open
pnoll1 opened this issue Jul 26, 2020 · 3 comments
Open

Gracefully handle source regressions #772

pnoll1 opened this issue Jul 26, 2020 · 3 comments

Comments

@pnoll1
Copy link

pnoll1 commented Jul 26, 2020

Machine accepts bad data from source making it the last successful run and what gets packaged in the data downloads.

Example:Franklin County WA  28,619 addresses to 37

Marking the run as failed and returning the previously cached result would be much better for data consumers.

@iandees
Copy link
Member

iandees commented Jul 26, 2020

Machine doesn't know that it's bad data, but it could check to see if the row count changed significantly and flag it as an error. I think the system that @ingalls is working on to replace machine should provide for this.

@pnoll1
Copy link
Author

pnoll1 commented Jul 27, 2020

Is there a roadmap or public code to track progress on this?

@ingalls
Copy link
Member

ingalls commented Jul 27, 2020

Yes, you can see the new processing system here: https://github.com/openaddresses/batch

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants