New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What's the purpose of having four input files? #176

Open
bsweger opened this Issue Sep 9, 2015 · 1 comment

Comments

Projects
None yet
1 participant
@bsweger
Contributor

bsweger commented Sep 9, 2015

Logging this great architecture question that emerged from an agency conversation last week:

What's the purpose of having four input files that will be ingested and validated/standardized centrally? In our case, we'd prefer to create a single file and do the validations and final conversion before submitting the data. We want to be as close to the source data as possible when running validations/conversions because that shortens the loop for making fixes.

For example, the person who sends the data to Treasury is likely not the person who can fix validation errors. So if we don't know about problems until after the data is submitted, the errors have to get reported back to the submitter, who in turn will have to report them to the people who will fix them. This means a longer lag time and the possibility of problems getting lost in translation.

@bsweger bsweger added the question label Sep 9, 2015

@bsweger

This comment has been minimized.

Show comment
Hide comment
@bsweger

bsweger Sep 9, 2015

Contributor

Adding my own thoughts to this great question...

One way to strike a balance between agencies that want to run validations/conversions "closer to the metal" and agencies that may not have the resources to do this would be to expose the broker's functionality via API/web service.

This would allow agencies to invoke data validations and/or conversions as needed and get the results back. Other agencies could submit via the website if that works better for them. But everyone is subject to the same set of validations, etc.

Bonus: what about a write API that would handle automated data submission (which could also address user story #151)?

Contributor

bsweger commented Sep 9, 2015

Adding my own thoughts to this great question...

One way to strike a balance between agencies that want to run validations/conversions "closer to the metal" and agencies that may not have the resources to do this would be to expose the broker's functionality via API/web service.

This would allow agencies to invoke data validations and/or conversions as needed and get the results back. Other agencies could submit via the website if that works better for them. But everyone is subject to the same set of validations, etc.

Bonus: what about a write API that would handle automated data submission (which could also address user story #151)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment