Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

datastreams: better exception handling #183

Open
Tracked by #332
slint opened this issue Apr 12, 2022 · 1 comment
Open
Tracked by #332

datastreams: better exception handling #183

slint opened this issue Apr 12, 2022 · 1 comment

Comments

@slint
Copy link
Member

slint commented Apr 12, 2022

Readers/transformers/writers can have unexpected errors, e.g. because of malformed data or missed edge cases. We should probably have a more "forgiving" strategy that allows not completely failing on generic errors, but still preserving any context-aware or useful errors that could help with debugging/fixing the input data.

Example:

This line is quite error prone if something is malformed, makes me wonder if we should have a global try/except so the datastream.process() wouldnt fail 🤔

Originally posted by @ppanero in #167 (comment)

@slint slint changed the title datastreams: btter exception handling datastreams: better exception handling Apr 12, 2022
@ppanero
Copy link
Member

ppanero commented May 17, 2023

Required for Zenodo migration:

Some grants failed to be processed by the datastreams code. We need to add error logging logic to find these and address separately (in invenio-vocabularies)

might be related to #221

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants