Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add non-batching model support to grpc_image_client.py #126

Merged
merged 4 commits into from Jul 8, 2022
Merged

Add non-batching model support to grpc_image_client.py #126

merged 4 commits into from Jul 8, 2022

Conversation

kthui
Copy link
Contributor

@kthui kthui commented Jul 7, 2022 •
edited

Fix grpc_image_client.py to support models with/without batching, and make sure it works with the densenet_onnx example that does not have batching (vs inception_graphdef).

The PR extends fixes from https://github.com/triton-inference-server/client/pull/103/files

@kthui kthui marked this pull request as draft July 7, 2022 16:45
@kthui
Copy link
Contributor Author

kthui commented Jul 7, 2022

densenet_onnx:
grpc_image_client.py:

root@tritonserver_qa:/opt/tritonserver/qa/L0_grpc# python3 ../clients/grpc_image_client.py -m densenet_onnx -c 3 -s INCEPTION ../images/mug.jpg
../clients/grpc_image_client.py:168: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
  resized_img = sample_img.resize((w, h), Image.BILINEAR)
Image '../images/mug.jpg':
    13.915174 (504) = COFFEE MUG
    12.018237 (968) = CUP
    9.839118 (967) = ESPRESSO
PASS

vs image_client.py, as reference:

root@tritonserver_qa:/opt/tritonserver/qa/L0_grpc# python3 ../clients/image_client.py -m densenet_onnx -c 3 -s INCEPTION ../images/mug.jpg
../clients/image_client.py:154: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
  resized_img = sample_img.resize((w, h), Image.BILINEAR)
Request 1, batch size 1
    13.915174 (504) = COFFEE MUG
    12.018237 (968) = CUP
    9.839118 (967) = ESPRESSO
PASS

@kthui
Copy link
Contributor Author

kthui commented Jul 7, 2022

inception_graphdef:
grpc_image_client.py:

root@tritonserver_qa:/opt/tritonserver/qa/L0_grpc# python3 ../clients/grpc_image_client.py -m inception_graphdef -c 3 -s INCEPTION ../images/mug.jpg
../clients/grpc_image_client.py:168: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
  resized_img = sample_img.resize((w, h), Image.BILINEAR)
Image '../images/mug.jpg':
    0.826934 (505) = COFFEE MUG
    0.124097 (969) = CUP
    0.002272 (900) = WATER JUG
PASS

vs image_client.py, as reference:

root@tritonserver_qa:/opt/tritonserver/qa/L0_grpc# python3 ../clients/image_client.py -m inception_graphdef -c 3 -s INCEPTION ../images/mug.jpg
../clients/image_client.py:154: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
  resized_img = sample_img.resize((w, h), Image.BILINEAR)
Request 1, batch size 1
    0.826933 (505) = COFFEE MUG
    0.124097 (969) = CUP
    0.002272 (900) = WATER JUG
PASS

@kthui kthui marked this pull request as ready for review July 7, 2022 17:12
rmccorm4
rmccorm4 previously approved these changes Jul 7, 2022
View reviewed changes
CoderHam
CoderHam suggested changes Jul 7, 2022
View reviewed changes
src/python/examples/grpc_image_client.py Outdated
metadata_response, config_response.config)

supports_batching = max_batch_size > 0
if not supports_batching and FLAGS.batch_size != 1:
print("ERROR: This model doesn't support batching.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raise Exception instead of print + exit

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

@kthui kthui requested review from CoderHam and rmccorm4 July 7, 2022 22:03
tanmayv25
tanmayv25 previously approved these changes Jul 7, 2022
View reviewed changes
rmccorm4
rmccorm4 previously approved these changes Jul 8, 2022
View reviewed changes
CoderHam
CoderHam suggested changes Jul 8, 2022
View reviewed changes
src/python/examples/grpc_image_client.py Outdated
metadata_response, config_response.config)

supports_batching = max_batch_size > 0
if not supports_batching and FLAGS.batch_size != 1:
raise Exception("ERROR: This model doesn't support batching.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't need "ERROR:"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

@kthui kthui dismissed stale reviews from rmccorm4 and tanmayv25 via ba56f18 July 8, 2022 21:46
@kthui kthui requested a review from CoderHam July 8, 2022 21:47
@kthui kthui merged commit e20ac64 into main Jul 8, 2022
@rmccorm4 rmccorm4 deleted the jacky-fix-grpc-image-client-batching branch July 8, 2022 23:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@CoderHam CoderHam CoderHam approved these changes

@tanmayv25 tanmayv25 tanmayv25 left review comments

@rmccorm4 rmccorm4 rmccorm4 left review comments

Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@kthui @tanmayv25 @CoderHam @rmccorm4