Add a function to ad-hoc materialize a table for interactive debugging #117
Conversation
| if table.name is None: | ||
| table.name = task.name + "_" + suffix | ||
| elif table.name.endswith("%%"): | ||
| table.name = table.name[:-2] + suffix |
There was a problem hiding this comment.
I think it would be nice to code share this name mangling with the original code that materializes tables. It will be more maintainable then.
There was a problem hiding this comment.
Changed in 971ee05
| debug_suffix: str | None = None, | ||
| flag_task_debug_tainted: bool = True, | ||
| keep_table_name: bool = True, | ||
| replace_table: bool = True, |
There was a problem hiding this comment.
The naming of replace_table might be optimized. Some dataframe materializations use if_exists="replace". This is a crude enum technique. However, people might be familiar with it. And we could raise an exception for all other string values.
Another alternative would be to rename it drop_if_exists. This would be closest to what this feature actually does and thus makes it easier to understand error messages reported by the database backend.
There was a problem hiding this comment.
We now use drop_if_exists: 971ee05
… a table from a task for interactive debugging
nicolasmueller
force-pushed
the
debug_materialize
branch
from
155ffdc
to
971ee05
Compare
Add pipedag.debug.materialize_table, a function to ad-hoc materialize a table from a task for interactive debugging
Checklist
CHANGELOG.rstentryRelated issue: #116