Make dbt runs expandable in the run view #30146
Description
What's the use case?
When executing a set of dbt models in dagster, the run view groups them altogether. This is because each block in the run view is a single step. I think it would be incredibly useful if an exception for dbt assets could be made. Right now we need to sift through the stdout and event stream to see when assets start and complete.
For reference, the view currently looks like this:
If this was done the benefits would be:
- Finding bottlenecks in model execution
- Understanding the points at which the execution is constrained by the thread count
- Easily spotting models that are taking much longer to execute
- Validating the execution order of models
- Selecting each model block could filter the stdout for logs/events only pertaining to that model
Ideas of implementation
To clearly illustrate to the end user that the blocks they are seeing in the view are not asset steps/ops, but are rather dbt models, I propose keeping the current implementation, but make dbt asset executions expandable. By clicking an expand arrow on the dbt block, faux steps are shown for each asset. The step blocks could be outlined in DBTs signature orange colour to show they are dbt models.
UI isn't my forte, but something like the above. Looking at it, maybe not the DBT outline colour, perhaps the dbt logo at the beginning or end of the block?
Additional information
This idea could be generalized to multi-assets as well, relying on the user to yield an asset start event, as well as the usual asset materialization event.
The start and end times of the assets could be used to populate an expandable block for the multi asset step. Expanding it would show the cascade of timings for the assets materialized as part of the multi asset definition.
Message from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.