Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate_batch function for debugging batch calls #358

Open
dougbrn opened this issue Jan 24, 2024 · 1 comment
Open

Validate_batch function for debugging batch calls #358

dougbrn opened this issue Jan 24, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@dougbrn
Copy link
Collaborator

dougbrn commented Jan 24, 2024

Currently, batch fails if any lightcurve passed to the analysis function fails. This all or nothing behavior creates some extreme user-unfriendliness at scale, where a single nan can crash a full computation at the very end. Dask error logging is very unsupportive for this as well, as it really doesn't give enough to go on to debug properly.

This motivates a potential debug function, that does batch but guarantees that the run will execute, but the result is all object and/or source rows that failed the computation. This would allow for users to easily key into the cause of the issue between their batch function and the lightcurves and filter/adjust as needed.

@dougbrn dougbrn added the enhancement New feature or request label Jan 24, 2024
@dougbrn
Copy link
Collaborator Author

dougbrn commented Jan 24, 2024

This is related to #354, and the problem generating this may be solved by the result of it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant