This repository has been archived by the owner on Nov 17, 2023. It is now read-only.
Segmentation fault during inference with quantization #14022
Labels
Quantization
Issues/Feature Requests related to Quantization
Note: Providing complete information in the most concise form is the best way to get help. This issue template serves as the checklist for essential information to most of the technical issues and bug reports. For non-technical issues and feature requests, feel free to present the information in what you believe is the best form.
For Q & A and discussion, please start a discussion thread at https://discuss.mxnet.io
Description
I got "Segmentation fault: 11" error when I ran the following command,
"python imagenet_inference.py --symbol-file=./model/imagenet1k-inception-bn-quantized-5batches-naive-symbol.json --param-file=./model/imagenet1k-inception-bn-quantized-0000.params --rgb-mean=123.68,116.779,103.939 --num-skipped-batches=50 --batch-size=64 --num-inference-batches=500 --dataset=./data/val_256_q90.rec --ctx=cpu --data-nthreads=1"
Please note: the model was quantized using "imagenet_gen_qsym.py" instead of "imagenet_gen_qsym_mkldnn.py"
Environment info (Required)
Package used (Python/R/Scala/Julia):
I am using Python
For Scala user, please provide:
java -version
)mvn -version
)scala -version
)For R user, please provide R
sessionInfo()
:Build info (Required if built from source)
Compiler (gcc/clang/mingw/visual studio):
MXNet commit hash:
(Paste the output of
git rev-parse HEAD
here.)Build config:
(Paste the content of config.mk, or the build command.)
Error Message:
(Paste the complete error message, including stack trace.)
INFO:logger:batch size = 64 for inference
INFO:logger:rgb_mean = 123.68,116.779,103.939
INFO:logger:rgb_std = 1,1,1
INFO:logger:label_name = softmax_label
INFO:logger:Input data shape = (3, 224, 224)
INFO:logger:Dataset for inference: ./data/val_256_q90.rec
[14:01:27] src/io/iter_image_recordio_2.cc:170: ImageRecordIOParser2: ./data/val_256_q90.rec, use 1 threads for decoding..
INFO:logger:Loading symbol from file /work/projects/qat/incubator-mxnet-master/example/quantization/./model/imagenet1k-inception-
bn-quantized-5batches-naive-symbol.json
INFO:logger:Loading params from file /work/projects/qat/incubator-mxnet-master/example/quantization/./model/imagenet1k-inception-
bn-quantized-0000.params
INFO:logger:Skipping the first 50 batches
INFO:logger:Running model ./model/imagenet1k-inception-bn-quantized-5batches-naive-symbol.json for inference
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_pooling
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_pooling
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_pooling
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_pooling
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_pooling
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_pooling
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_pooling
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_pooling
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_pooling
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_pooling
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_pooling
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_pooling
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_conv
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_pooling
[14:01:34] src/executor/attach_op_execs_pass.cc:335: Neither FCompute nor FComputeEx registered _contrib_quantized_fully_connected
Segmentation fault: 11
Stack trace returned 10 entries:
[bt] (0) /work/anaconda3/envs/py3/lib/python3.7/site-packages/mxnet/libmxnet.so(+0x381822) [0x7fa65e7af822]
[bt] (1) /work/anaconda3/envs/py3/lib/python3.7/site-packages/mxnet/libmxnet.so(+0x368d87e) [0x7fa661abb87e]
[bt] (2) /lib/x86_64-linux-gnu/libc.so.6(+0x3ef20) [0x7fa6c57fbf20]
[bt] (3) /work/anaconda3/envs/py3/lib/python3.7/site-packages/mxnet/libmxnet.so(+0x2f3bcbe) [0x7fa661369cbe]
[bt] (4) /work/anaconda3/envs/py3/lib/python3.7/site-packages/mxnet/libmxnet.so(+0x2f3c661) [0x7fa66136a661]
[bt] (5) /work/anaconda3/envs/py3/lib/python3.7/site-packages/mxnet/libmxnet.so(+0x2f5c3d6) [0x7fa66138a3d6]
[bt] (6) /work/anaconda3/envs/py3/lib/python3.7/site-packages/mxnet/libmxnet.so(+0x2f63574) [0x7fa661391574]
[bt] (7) /work/anaconda3/envs/py3/lib/python3.7/site-packages/mxnet/libmxnet.so(+0x2f63c64) [0x7fa661391c64]
[bt] (8) /work/anaconda3/envs/py3/lib/python3.7/site-packages/mxnet/libmxnet.so(MXExecutorSimpleBind+0x2260) [0x7fa6612ed940]
[bt] (9) /work/anaconda3/envs/py3/lib/python3.7/lib-dynload/../../libffi.so.6(ffi_call_unix64+0x4c) [0x7fa6c3f63ec0]
Minimum reproducible example
(If you are using your own code, please provide a short script that reproduces the error. Otherwise, please provide link to the existing example.)
https://github.com/apache/incubator-mxnet/tree/master/example/quantization
Steps to reproduce
(Paste the commands you ran that produced the error.)
(Note: This command worked fine)
(Note: This command stopped and returned error)
What have you tried to solve it?
The text was updated successfully, but these errors were encountered: