Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dag-level, class-level and task-level hooks #27

Closed
edublancas opened this issue Mar 7, 2020 · 2 comments
Closed

dag-level, class-level and task-level hooks #27

edublancas opened this issue Mar 7, 2020 · 2 comments

Comments

@edublancas
Copy link
Contributor

edublancas commented Mar 7, 2020

Currently, on_render, on_finish and on_failure hooks can only be set at the task level. making impossible to set the same hook for many tasks at once.

One use case for this is static analysis, an on_render hook could be set for all NotebookRunner tasks to do static analysis in the source code to detect potential issues before executing.

Potential API (similar to how dag.clients work):

from ploomber.tasks import PythonCallable

t = PythonCallable(...)

t.on_render = task_level_on_render
dag.class_on_render[PythonCallable] = class_level_on_render
dag.on_render = dag_level_on_render
@edublancas
Copy link
Contributor Author

Also take into account that tasks can be executed via an executor or directly using task.build, should dag-level hooks be executed when using task.build?

@edublancas
Copy link
Contributor Author

class-level hooks discarded. DAG-level hooks already implemetned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant