Trouble deploying Yolov5 model in Torch Serve environment

[atinesh-s]

I am trying to deploy the Yolov5 model in the Torch Serve environment but on sending the below request I am not getting any response, the request seems to get stuck

$ curl http://127.0.0.1:8080/predictions/yolov5x -T /home/atinesh/Desktop/COCO_val2014_000000562557.jpg

I have followed all the steps as mentioned in the README.md and I was able to successfully deploy the container

Note: I had to make some changes in Dockerfile and docker run command to make it work

Deployed model is running properly, on checking the model health I am getting below response

$ curl "http://localhost:8081/models/yolov5x"
[
  {
    "modelName": "yolov5x",
    "modelVersion": "1.0",
    "modelUrl": "yolov5x.mar",
    "runtime": "python",
    "minWorkers": 6,
    "maxWorkers": 6,
    "batchSize": 1,
    "maxBatchDelay": 100,
    "loadedAtStartup": true,
    "workers": [
      {
        "id": "9000",
        "startTime": "2021-11-11T09:25:16.121Z",
        "status": "READY",
        "memoryUsage": 0,
        "pid": 907,
        "gpu": false,
        "gpuUsage": "N/A"
      },
      {
        "id": "9001",
        "startTime": "2021-11-11T09:25:16.123Z",
        "status": "READY",
        "memoryUsage": 0,
        "pid": 902,
        "gpu": false,
        "gpuUsage": "N/A"
      },
      {
        "id": "9002",
        "startTime": "2021-11-11T09:25:16.123Z",
        "status": "READY",
        "memoryUsage": 0,
        "pid": 897,
        "gpu": false,
        "gpuUsage": "N/A"
      },
      {
        "id": "9003",
        "startTime": "2021-11-11T09:25:16.124Z",
        "status": "READY",
        "memoryUsage": 0,
        "pid": 882,
        "gpu": false,
        "gpuUsage": "N/A"
      },
      {
        "id": "9004",
        "startTime": "2021-11-11T09:25:16.124Z",
        "status": "READY",
        "memoryUsage": 0,
        "pid": 892,
        "gpu": false,
        "gpuUsage": "N/A"
      },
      {
        "id": "9005",
        "startTime": "2021-11-11T09:25:16.127Z",
        "status": "READY",
        "memoryUsage": 0,
        "pid": 887,
        "gpu": false,
        "gpuUsage": "N/A"
      }
    ]
  }
]

Yolov5 (XLarge) model is trained on custom COCO dataset to detect 2 objects person & bicycle, below is the link of the trained model file

yolov5x.pt

Image used for Inference: COCO_val2014_000000562557.jpg

I have tried two different handler files handler #1 provided by you @louisoutin and handler #2 provided by @joek13 but the same issue persists

It seems like latest Yolov5 code is creating some problem