### Motivation Support profiling of workload for different frameworks. ### References [vllm](https://docs.vllm.ai/en/latest/usage/security/#unprotected-endpoints-no-api-key-required) [sglang](https://docs.sglang.io/docs/developer_guide/benchmark_and_profiling#using-/start_profile-endpoint) ### Proposed Solution Goal is to enable profiling for a phase by a boolean flag, and have the client send a start/end profile request around the requests. ### Alternatives Considered _No response_ ### Additional Context _No response_
Motivation
Support profiling of workload for different frameworks.
References
vllm
sglang
Proposed Solution
Goal is to enable profiling for a phase by a boolean flag, and have the client send a start/end profile request around the requests.
Alternatives Considered
No response
Additional Context
No response