-
-
Notifications
You must be signed in to change notification settings - Fork 349
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to use DLA to build engine #27
Comments
Hi, edit
and add these lines in config_infer_primary.txt file (in [property] section):
Note: edit these lines according to:
I don't have Xavier board to test. Please tell me if it works. |
It can run on Xavier, but seems some layer convert to DLA unsuccessfully. I use tegrastats and jtop for seeing effects. The effect is as same as original. Building YOLO network complete 0:02:48.599896399 13265 0x7f20002300 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1748> [UID = 1]: serialize cuda engine to file: /opt/nvidia/deepstream/deepstream-5.0/sources/yolo/model_b1_dla0_fp16.engine successfully 0:02:48.925054321 13265 0x7f20002300 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary_gie> [UID 1]: Load new model:/opt/nvidia/deepstream/deepstream-5.0/sources/yolo/config_infer_primary.txt sucessfully |
Some layers can't run in DLA core. It's a TensorRT issue. I think it will be implemented in future releases of TensorRT/DeepStream SDK. |
what could be the reason of the following error trying to run on Xavier NX
|
Did you followed these steps? You need to recompile nvdsinfer_custom_impl_Yolo too. |
@marcoslucianops yes !, it seems that for batch-size=1 i am able to use dla, but on batch size >= 2 it doesn't work.
|
it this fix updated? because the function prototype nvinfer1::ICudaEngine Yolo::createEngine (nvinfer1::IBuilder builder) doesn't match with nvinfer1::ICudaEngine Yolo::createEngine (nvinfer1::IBuilder builder, nvinfer1::IBuilderConfig* config) |
I don't know if it will work on DLA because I don't have board to test. Can you check please? |
i am checking but it can't compile because the method in the new version has a different prototype, so maybe it's updated and the fix you provided time ago wont' work anymore? |
Can you test only adding these lines in config_infer_primary.txt file (in [property] section):
|
seems to ignore it, in fact it use model_b2_gpu0_fp16.engine Deserialize yoloLayer plugin: yolo_93 |
Can you send the output when the model is building? |
in fact the engine fails building, i deleted the .engine file, and i got, with batch_size = 2 the following output ERROR: Deserialize engine failed because file path: /home/aaeon/sdcard/fogsphere-edge/fogsphere-engine-py/model_b2_gpu0_fp16.engine open error Loading pre-trained weights
(0) conv_silu 3 x 640 x 640 32 x 320 x 320 3584 WARNING: [TRT]: route_3: Concatenation on DLA requires at least two inputs. ERROR: [TRT]: 2: [nvdlaUtils.cpp::getInputDesc::176] Error Code 2: Internal Error (Assertion idx < num failed.Index is out of range of valid number of input tensors.) |
@willosonico, it's an issue in TensorRT: https://forums.developer.nvidia.com/t/error-while-building-engine-on-tensorrt8-0-2/202978/3. |
https://forums.developer.nvidia.com/t/how-to-use-dla-in-deepstream-yolov5/161550/25
Hi @marcoslucianops ,
I used deepstream-yolov4, and I check out the engine. That been build on GPU. I saw the article before. How do I fix following program for building DLA engine?
DeepStream-Yolo/native/nvdsinfer_custom_impl_Yolo/yolo.cpp
Lines 74 to 81 in 470ed82
The text was updated successfully, but these errors were encountered: