YOLOv5 pruned_quant-aggressive_94 exception #226
Comments
|
Hi @SkalskiP thanks for reporting this issue. Could you help us a little more to get enough details? Which script are you using? I tried to replicate your situation using I checked this on 0.9.1 and the nightly build and couldn't replicate it unfortunately. |
|
Hello @mgoin I actually run some demo version of my app: from deepsparse import compile_model
from src.deepsparse_utils import modify_yolo_onnx_input_shape, postprocess_nms
import numpy as np
import cv2
from typing import Tuple
MODEL_PATH = 'data/model.onnx'
IMAGE_PATH = 'data/image-1.jpeg'
BATCH_SIZE = 1
INPUT_RESOLUTION = (800, 800)
def preprocess_image(image: np.ndarray, image_size: Tuple[int, int] = (640, 640)) -> np.ndarray:
image_resized = cv2.resize(image, image_size)
image_transposed = image_resized[:, :, ::-1].transpose(2, 0, 1)
image_wrapped = image_transposed[np.newaxis, ...]
return np.ascontiguousarray(image_wrapped)
model_path, _ = modify_yolo_onnx_input_shape(MODEL_PATH, INPUT_RESOLUTION)
engine = compile_model(model_path, batch_size=BATCH_SIZE)
print(f"Engine info: {engine}")
image = cv2.imread(IMAGE_PATH)
batch = preprocess_image(image=image, image_size=INPUT_RESOLUTION)
outputs = engine.run([batch])[0]
results = postprocess_nms(outputs)[0]
print(results) |
|
Is it possible that the presence of other python dependencies is causing this problem? I have quite a few other packages installed in this python environment? I am trying to put together some sort of replication path for you guys outside of my application, but so far I have not been able to isolate the problem. Any suggestions are welcome. |
|
Hi @mgoin, I am facing a similar issue I have deepsparse-nightly==0.10.0.20211208 installed. The engine runs fine when using the original image size for yolo, however, it fails with the above stack trace when I modify the model input size using |
|
I tested and it looks like it is the same issue as in @prasanth-pivotchain case. When I'm using default model without changing input size, code runs correctly. |
|
Hello, @mgoin we spent time today to create some path of reproduction for you. Here is a small repo with Docker image and a simple Python script. Everything is described in README.md. https://github.com/SkalskiP/deepsparse-yolov5-minimal-example It fails when we run it on |
|
Hi there @SkalskiP thanks so much for all of the details and help! I was able to successfully recreate the error now and we are working on a fix. I will keep you posted when we have an update available on nightly to test. Thanks again. |
|
We are meeting again! I'm very glad you were able to reproduce the bug. We put a lot of energy into reproducing it. I know it's a little early, but do you happen to have an ETA for this fix? |
|
Thanks for joining the community and your effort 🙂 |
|
It is pure pleasure. A few months ago I heard about what you are doing and I must say I am very impressed. A few months ago I heard about what you are doing and I must say I am very impressed. And I have to say that since I was assigned the task of optimizing our jobs (using Deepsparse among other things), I'm constantly learning something new. Fascinating subject. 🧐 Let me know when it's ready here or on Slack, and I'll be happy to test the fix on our infrastructure. |
|
Hi @SkalskiP and @prasanth-pivotchain, thanks for waiting over the holidays! I'm happy to share a fix for this issue on our latest nightly build which can be found here: https://pypi.org/project/deepsparse-nightly/0.10.0.20211227/ Here is how I validated that this issue on this build: |
|
Hi @mgoin, I'm off until January 4th/5th, but I will certainly test if it has solved the problems we originally encountered. I'll let you know in about a week or so. Thank you 🙏 |
|
Hi @mgoin, Thank you for your quick response. However, the issue still persists in my case. I am going to wait for the hotfix release. |
|
Sorry to hear that @prasanth-pivotchain , is it the same issue as before now? We might need more information in order to deal with your problem for the next release if it is still present. I noticed your problem occurs on an avx2 machine I tried running a similar setup on the new build with an image shape of 800x800 It seems to run fine for the sparse FP32 and sparse INT8 yolov5s, so I'm not sure what else to try. Did you use a different image shape perhaps? |
|
Hi, @mgoin. I'll be back at work in 2 days. I'll let you know what I found ;) |
|
Yes @mgoin, I am using image shape (320, 320) on an avx2 only machine. I will make a clean install and post the stack trace for you. |
|
@prasanth-pivotchain I was able to recreate a failure using image shape (320, 320). Thanks for the additional info that should be enough to start working on a fix! |
|
Hi there @prasanth-pivotchain we have an updated nightly build that should address your issue if you'd like to try it. Let me know if you get the chance, otherwise it will be in our next release (0.10) in a few weeks. Thanks for your help finding and reporting this! |
|
Hi @mgoin, everything is working as expected. Thanks for the help. |
|
Hello @prasanth-pivotchain I am going to go ahead and close this thread out with your confirmation. But if you have more comments, please re-open and we'd love to chat. Lastly, if you have not starred our deepsparse repo already, and you feel inclined, please do! Thank you in advance for your support! https://github.com/neuralmagic/deepsparse/ Best, Jeannie / Neural Magic |
Describe the bug
I was trying to run demo code with YOLOv5 pruned_quant-aggressive_94 model on
g4dn.x2largeand encountered this exception.Stack trace
Environment
Is there any chance you could help me out to debug that issue?
The text was updated successfully, but these errors were encountered: