Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Input file format and structure validation #39

Closed
cimendes opened this issue Nov 11, 2020 · 3 comments
Closed

Input file format and structure validation #39

cimendes opened this issue Nov 11, 2020 · 3 comments

Comments

@cimendes
Copy link
Member

In each parser, I suggest adding a validation for the expected file format (tsv, txt, json..) and structure (fields present), giving a clean exit when it fails.

@fmaguire
Copy link
Member

We are kinda limited in how we can do this because the parsers are designed to handle streaming data.

The validity is inherently checked by the putting it in the hamronised format and raising an error if there is an issue, I don't really think wrapping that in try except to give a cleaner error message really adds much (just obfuscation) but if you think that's better it can be easily done.

@cimendes
Copy link
Member Author

I would like to get some more opinios on this. Indeed it adds an obfuscation but I think a custom error message would be helful for a user as the general key error doesn't actually explain much. Personally I would prefer a more informative error message but I'm good with what you decide on!

@fmaguire
Copy link
Member

fmaguire commented Mar 23, 2021

Created a new issue with a plan to improve this (adding a debug flag for full traceback and logging library to handle the different levels.)
Will close and redirect to #56

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants