Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more checks to postprocessing pipelines #10

Open
andreyzhd opened this issue Dec 19, 2022 · 2 comments
Open

Add more checks to postprocessing pipelines #10

andreyzhd opened this issue Dec 19, 2022 · 2 comments

Comments

@andreyzhd
Copy link
Collaborator

Currently postprocessing pipelines sometimes fail, for all or some of the videos, without notifying the user. Add the checks that the expected number of videos has been created.

@jjnurminen
Copy link
Collaborator

I guess you are referring to the Nexus bug where it will no longer produce the videos until it's been restarted?
Yea, it would be good to detect this condition. On the other hand, the "postprocessing pipelines" mechanism is intended to be more general, i.e. it should be able to run any operations that require processed data. Checking that videos have been created would only apply to our specific use case of making overlay videos.
One possible solution would be to create a Nexus pipeline that checks whether videos have been properly created, and add that to our postprocessing pipelines?

@jjnurminen
Copy link
Collaborator

The Nexus pipeline for checking the videos could be written in Python. Alternatively, maybe gaitutils should directly allow Python scripts in preprocessing/postprocessing operations. Perhaps you could e.g. have a rule that if the name of an operation starts with "PY:", it will be interpreted as a Python script to run, instead of a Nexus pipeline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants