Fixed the priority of parameters defined in register curl cmd vs model-config.yaml

[lxning]

Description

Please read our CONTRIBUTING.md prior to creating your first pull request.

Please include a summary of the feature or issue being fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes #(issue)
#2851

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

Feature/Issue validation/testing

Please describe the Unit or Integration tests that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

Except regression test, a manual test for GPU deviceIds is added b/w it requires multi-GPU host.

• model-config.yaml

cat model_store/mnist/model-config.yaml
# TorchServe frontend parameters
minWorkers: 4
maxWorkers: 4
maxBatchDelay: 100
responseTimeout: 1200
deviceType: "gpu"
deviceIds: [1,3]

• GPU status

nvidia-smi
Wed Dec 20 02:05:05 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.54.03              Driver Version: 535.54.03    CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA A10G                    On  | 00000000:00:1B.0 Off |                    0 |
|  0%   15C    P8              16W / 300W |      5MiB / 23028MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   1  NVIDIA A10G                    On  | 00000000:00:1C.0 Off |                    0 |
|  0%   22C    P0              57W / 300W |    550MiB / 23028MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   2  NVIDIA A10G                    On  | 00000000:00:1D.0 Off |                    0 |
|  0%   16C    P8              16W / 300W |      5MiB / 23028MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   3  NVIDIA A10G                    On  | 00000000:00:1E.0 Off |                    0 |
|  0%   20C    P0              56W / 300W |    550MiB / 23028MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    1   N/A  N/A    389913      C   /opt/conda/envs/py38/bin/python3.8          270MiB |
|    1   N/A  N/A    389916      C   /opt/conda/envs/py38/bin/python3.8          270MiB |
|    3   N/A  N/A    389914      C   /opt/conda/envs/py38/bin/python3.8          270MiB |
|    3   N/A  N/A    389915      C   /opt/conda/envs/py38/bin/python3.8          270MiB |
+---------------------------------------------------------------------------------------+

Checklist:

• Did you have fun?
• Have you added tests that prove your fix is effective or that this feature works?
• Has code been commented, particularly in hard-to-understand areas?
• Have you made corresponding changes to the documentation?

[agunapal]

nit: Do we need this? Can't we achieve the same with json?

[lxning]

According to the description, it is fastest json lib. I'm thinking if we need apply orjson in our handler for json input.

[agunapal]

LGTM