# Add fmperf Library and Update Dependencies #42
Conversation
Signed-off-by: Chen Wang <Chen.Wang1@ibm.com>
k8s-ci-robot
added
the
cncf-cla: yes
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: wangchen615 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
k8s-ci-robot
added
the
size/XXL
There was a problem hiding this comment.
Thanks for sending this out! Having the ability to deploy the model server and benchmark with different configurations makes sense. It would be good to get this working with the inference-perf library and clean up additional logic like report generation that is handled separately by inference-perf.
|
|
||
| from kubernetes import client | ||
|
|
||
| from fmperf.ModelSpecs import ModelSpec, TGISModelSpec, vLLMModelSpec |
There was a problem hiding this comment.
Should we call this library something else instead of fmperf? Maybe a name that makes it clear that it simplifies deployment of model server and the benchmarking tool?
There was a problem hiding this comment.
@achandrasekar , what would be the good library name?
| from fmperf.Cluster import DeployedModel | ||
|
|
||
|
|
||
| class WorkloadSpec: |
There was a problem hiding this comment.
Can we have this deploy the inference-perf tool instead?
There was a problem hiding this comment.
Will take a look at your tool. Thanks, @achandrasekar
| pd.set_option("future.no_silent_downcasting", True) | ||
|
|
||
|
|
||
| def parse_results(results, print_df=False, print_csv=False): |
There was a problem hiding this comment.
Would be good to replace this with the reportgen in inference-perf.
k8s-ci-robot
added
the
needs-rebase
|
PR needs rebase. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
This PR adds the fmperf library and updates project dependencies to support it.
Changes
Added
fmperflibrary with its core components:Cluster.py: Kubernetes cluster managementModelSpecs.py: Model specification handlingWorkloadSpecs.py: Workload configurationutils/: Utility modules for benchmarking, logging, and data processingUpdated project dependencies in
pyproject.toml:pandas>=2.2.0for data processingkubernetes>=29.0.0for cluster managementpyyaml>=6.0.1for configuration handlingFixed code quality issues:
__all__exports for better module organizationFeatures
Testing
Notes