[Feature][framework] record subtask execution time

[klesh]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

Currently, we have task.spendSeconds to record the execution time of a task.
However, the execution time of subtask is missing. It would be beneficial to have this information recorded.

Use case

This information can help identify the bottleneck of a pipeline

1. For developers: to improve the performance
2. For users: to skip some unneeded subtasks

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[keon94]

I should be able to pick this one up.

[keon94]

@klesh how do you want the duration recorded?

1. A new table for subtasks - i.e. _devlake_subtasks (I think this is what you want)
2. Reuse the tasks table, and just write the time info into the subtasks Json
3. Just simply log the elapsed time after a subtask is done (write nothing to DB)

[klesh]

Yes, you are correct. I would prefer the First option because it would be easier for us to analyze in the future. Say we like to know which subtasks ran slower than others.

…

On 22/07/08 04:19PM, Keon Amini wrote: @klesh how do you want the duration recorded? 1. A new table for subtasks - i.e. _devlake_subtasks (I think this is what you want) 2. Reuse the tasks table, and just write the time info into the subtasks Json 3. Just simply log the elapsed time after a subtask is done (write nothing to DB) -- Reply to this email directly or view it on GitHub: #2433 (comment) You are receiving this because you were mentioned. Message ID: ***@***.***>