You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To make it easier to track dbt's resource usage, an event summarizing resource consumption should be fired at the end of an invocation, and include important resource usage statistics.
At a minimum, this should include the peak memory usage in bytes, as it would appear in top or the Windows Task Manager. While we are implementing this event, however, we should look at including CPU usage (with user/kernel breakdown), and perhaps some block I/O stats.
Because Python does not have a standard mechanism for collecting this information, we will have to consider a library like pstools or limit support to specific OSes (for example, by using the /proc filesystem on linux).
Acceptance Criteria
After executing a command, dbt fires an INFO-level event called ResourceReport.
The ResourceReport event contains a property called process_mem_max_rss, a 64-bit integer representing the peak memory usage during the command execution, in bytes.
The text was updated successfully, but these errors were encountered:
github-actionsbot
changed the title
[Applied State] Fire ResourceReport Event at End of Run
[CT-2950] [Applied State] Fire ResourceReport Event at End of Run
Aug 8, 2023
To make it easier to track dbt's resource usage, an event summarizing resource consumption should be fired at the end of an invocation, and include important resource usage statistics.
At a minimum, this should include the peak memory usage in bytes, as it would appear in top or the Windows Task Manager. While we are implementing this event, however, we should look at including CPU usage (with user/kernel breakdown), and perhaps some block I/O stats.
Because Python does not have a standard mechanism for collecting this information, we will have to consider a library like pstools or limit support to specific OSes (for example, by using the /proc filesystem on linux).
Acceptance Criteria
The text was updated successfully, but these errors were encountered: