Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NNTOOL: Error when 'open *.onnx' #370

Closed
ChenMathilda opened this issue Feb 12, 2023 · 3 comments
Closed

NNTOOL: Error when 'open *.onnx' #370

ChenMathilda opened this issue Feb 12, 2023 · 3 comments

Comments

@ChenMathilda
Copy link

ChenMathilda commented Feb 12, 2023 •
edited

Hi, I tried to use open command in nntool to open the onnx model generated by the parameters trained by pulp-dronet-v2, but I encountered the following error:

(NNT) open model_gapflow.onnx 
open - opening graph file model_gapflow.onnx load_quantization = False
onnx - unable to determine batch dimension. if the graph fails to import properly set it to 1 or a variable.
Traceback (most recent call last):
  File "/home/chennaiting/anaconda3/envs/gap_sdk3.9/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/chennaiting/anaconda3/envs/gap_sdk3.9/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/chennaiting/anaconda3/envs/gap_sdk3.9/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/interpreter/commands/open.py", line 118, in do_open
    self.__open_graph(args)
  File "/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/interpreter/commands/open.py", line 92, in __open_graph
    G = create_graph(graph_file, opts=opts)
  File "/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/importer/importer.py", line 52, in create_graph
    graph = importer.create_graph(filename, opts)
  File "/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/importer/onnx/onnx.py", line 54, in create_graph
    G = self._import_onnx_model(G, model.graph, opset_import, opts)
  File "/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/importer/onnx/onnx.py", line 68, in _import_onnx_model
    self._import_nodes(G, graph, handlers, all_nodes, outputs, opts)
  File "/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/importer/onnx/onnx.py", line 169, in _import_nodes
    params = handler.handle(OnnxNode(node), all_nodes=all_nodes, vars_dict=vars_dict,
  File "/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/importer/onnx/handlers/handler.py", line 63, in handle
    raise BackendIsNotSupposedToImplementIt(
onnx.backend.test.runner.BackendIsNotSupposedToImplementIt: Constant version 13 is not implemented.
EXCEPTION of type 'BackendIsNotSupposedToImplementIt' occurred with message: 'Constant version 13 is not implemented.'

Here is the network dronet_v2_gapflow
Here is the .pth file pulp_dronet_v2_10.pth
Here is the .onnx file model_gapflow.onnx

And my environment are listed:
python: 3.8
gap_sdk: 3.9.1

I was able to run the open command successfully before, but I don't know if I made some changes to the gap_sdk environment that caused this problem.
In addition, I noticed that there is a bn module in .pth file, but there is no such module in the generated onnx model.This is different from the onnx model in the official pulp-dronet-v2 is as follow: :model_original.onnx
image

This question has been bothering me for a long time. I look forward to your reply!
@sousoux
Copy link
Contributor

sousoux commented Feb 12, 2023

Your SDK is very old. Please update it. ONNX Constant operator is supported in opset version 13 in the latest SDK.

(https://github.com/GreenWaves-Technologies/gap_sdk/blob/master/tools/nntool/nntool/importer/onnx/handlers/backend/constant.py)

@sousoux sousoux closed this as completed Feb 12, 2023
@ChenMathilda
Copy link
Author

ChenMathilda commented Feb 14, 2023 •
edited

``> Your SDK is very old. Please update it. ONNX Constant operator is supported in opset version 13 in the latest SDK.

(https://github.com/GreenWaves-Technologies/gap_sdk/blob/master/tools/nntool/nntool/importer/onnx/handlers/backend/constant.py)

Hi @sousoux,
I changed gap_sdk version into 4.22.0, and can run 'open *.onnx' successfully!
But when I tried to runmake clean all run platform=gvsoc under path /pulp-dronet-v2/gapflow, I met this error:

(gap_sdk)
# chennaiting @ seu-netsi in ~/pulp-dronet/pulp-dronet-v2/gapflow on git:main x [20:05:39]
$ make clean all run platform=gvsoc
APP_SRCS... main.c BUILD_MODEL_SQ8BIT/networkKernels.c /home/chennaiting/pulp-dronet/gap_sdk/libs/gap_lib/img_io/ImgIO.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Activation_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Bias_Linear_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Pooling_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_DW_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_MatAlgebra_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_SoftMax_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_AT_Misc.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/RNN_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/kernels/norm_transpose.c /home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/kernels/copy.c
APP_CFLAGS... -g -O3 -w -mno-memcpy -fno-tree-loop-distribute-patterns -I. -I/home/chennaiting/pulp-dronet/gap_sdk/libs/gap_lib/include -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/Emulation -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8 -I/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/kernels -IBUILD_MODEL_SQ8BIT -DAT_MODEL_PREFIX=network -DAT_INPUT_HEIGHT=200 -DAT_INPUT_WIDTH=200 -DAT_INPUT_COLORS=1 -DSTACK_SIZE=6096 -DSLAVE_STACK_SIZE=1024 -DAT_IMAGE=/home/chennaiting/pulp-dronet/pulp-dronet-v2/gapflow/images/frame_2.pgm -DPERF -DMODEL_ID= -DFREQ_FC=250 -DFREQ_CL=175 -DAT_CONSTRUCT=networkCNN_Construct -DAT_DESTRUCT=networkCNN_Destruct -DAT_CNN=networkCNN -DAT_L3_ADDR=network_L3_Flash
sudo rm -f BUILD_MODEL_SQ8BIT/GenTile
sudo rm -f -rf BUILD_MODEL_SQ8BIT
sudo rm -f -rf nntool_output
**make: *** No rule to make target '/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/generators/nntool_extra_generators.c', needed by 'BUILD_MODEL_SQ8BIT/GenTile'.  Stop.**

According to the error report, I deleted 55 lines of code in pulp-dronet-v2/gapflow/common/model_decl.mk

image
It do make some progresses, but still encountered error:

# chennaiting @ seu-netsi in ~/pulp/pulp-dronet/pulp-dronet-v2/gapflow on git:master x [19:08:43]
make clean all run platform=gvsoc
APP_SRCS... main.c BUILD_MODEL_SQ8BIT/networkKernels.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/libs/gap_lib/img_io/ImgIO.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Activation_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Bias_Linear_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Pooling_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_DW_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_MatAlgebra_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_SoftMax_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_AT_Misc.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/RNN_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels/norm_transpose.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels/copy.c  
APP_CFLAGS... -g -O3 -w -mno-memcpy -fno-tree-loop-distribute-patterns -I. -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/libs/gap_lib/include -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Emulation -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8 -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels -IBUILD_MODEL_SQ8BIT -DAT_MODEL_PREFIX=network -DAT_INPUT_HEIGHT=200 -DAT_INPUT_WIDTH=200 -DAT_INPUT_COLORS=1 -DSTACK_SIZE=6096 -DSLAVE_STACK_SIZE=1024 -DAT_IMAGE=/home/chennaiting/pulp/pulp-dronet/pulp-dronet-v2/gapflow/images/frame_2.pgm -DPERF -DMODEL_ID= -DFREQ_FC=250 -DFREQ_CL=175 -DAT_CONSTRUCT=networkCNN_Construct -DAT_DESTRUCT=networkCNN_Destruct -DAT_CNN=networkCNN -DAT_L3_ADDR=network_L3_Flash
rm -f BUILD_MODEL_SQ8BIT/GenTile
rm -f -rf BUILD_MODEL_SQ8BIT
rm -f -rf nntool_output 
mkdir BUILD_MODEL_SQ8BIT
cp nntool_input/models_onnx/model_original.onnx BUILD_MODEL_SQ8BIT/network.onnx
echo "GENERATING NNTOOL STATE FILE"
GENERATING NNTOOL STATE FILE
echo BUILD_MODEL_SQ8BIT
BUILD_MODEL_SQ8BIT
nntool -s nntool_input/nntool_scripts/nntool_script_deployment BUILD_MODEL_SQ8BIT/network.onnx -q
settings - set log level to INFO
log_level - was: 'INFO'
now: 'INFO'
open - opening graph file BUILD_MODEL_SQ8BIT/network.onnx load_quantization = True
__init__ - unable to determine batch dimension. if the graph fails to import properly set it to 1 or a variable.
nngraph - update graph dimensions
nngraph - update graph dimensions
debug - was: False
now: True
adjust_order - adding transposes to correct tensor order for AT kernels
eliminate_transposes - no transposes to eliminate found
nngraph - update graph dimensions
nngraph - update graph dimensions
eliminate_transposes - no further transpose sequences found
nngraph - adjusted order
nngraph - update graph dimensions
move_node_up - Node _sig_Sigmoid cannot be moved
fuse_gap_convs - fusing nodes _layer1_Conv,_pool_MaxPool into _layer1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock1_conv1_Conv,_resBlock1_relu1_Clip into _resBlock1_conv1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock1_conv2_Conv,_resBlock1_relu2_Clip into _resBlock1_conv2_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock1_bypass_Conv,_resBlock1_relu3_Clip into _resBlock1_bypass_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock2_conv1_Conv,_resBlock2_relu1_Clip into _resBlock2_conv1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock2_conv2_Conv,_resBlock2_relu2_Clip into _resBlock2_conv2_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock2_bypass_Conv,_resBlock2_relu3_Clip into _resBlock2_bypass_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock3_conv1_Conv,_resBlock3_relu1_Clip into _resBlock3_conv1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock3_conv2_Conv,_resBlock3_relu2_Clip into _resBlock3_conv2_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock3_bypass_Conv,_resBlock3_relu3_Clip into _resBlock3_bypass_Conv_fusion
matcher - ++ fusion fuse_gap_convs modified graph
slice_to_split - replaced slice nodes _Gather,_Gather_1 with split node _Gather_split
matcher - ++ fusion slice_to_split modified graph
insert_copies - inserting copy between _Gather_reshape0:0 and output_1:0
matcher - ++ fusion insert_copies modified graph
remove_reshapes_before_linear - removing unnecessary reshape before linear _Flatten
matcher - ++ fusion remove_reshapes_before_linear modified graph
fuse_op_activation - fusing nodes _resBlock3_Add,_relu_Clip
matcher - ++ fusion fuse_op_activation_scale8 modified graph
move_node_up - Node _sig_Sigmoid cannot be moved
matcher - ++ fusion scaled_match_group modified graph
move_node_up - Node _sig_Sigmoid cannot be moved
adjust_order - adding transposes to correct tensor order for AT kernels
eliminate_transposes - no transposes to eliminate found
nngraph - update graph dimensions
nngraph - update graph dimensions
eliminate_transposes - no further transpose sequences found
nngraph - adjusted order
input_norm_func - was: ''
now: 'x:x/255'
Traceback (most recent call last):
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/aquant.py", line 99, in do_aquant
    data = [import_data(input_file, **input_args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/aquant.py", line 99, in <listcomp>
    data = [import_data(input_file, **input_args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/utils/data_importer.py", line 171, in import_data
    return import_image_data(filename, **kwargs)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/utils/data_importer.py", line 134, in import_image_data
    return postprocess(img_in, height, width, channels, **kwargs)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/utils/data_importer.py", line 86, in postprocess
    img_in = np.array(img_in, dtype=np.float)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/numpy/__init__.py", line 305, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'float'.
`np.float` was a deprecated alias for the builtin `float`. To avoid this error in existing code, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
EXCEPTION of type 'AttributeError' occurred with message: 'module 'numpy' has no attribute 'float'.
`np.float` was a deprecated alias for the builtin `float`. To avoid this error in existing code, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations'
Traceback (most recent call last):
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/qshow.py", line 41, in do_qshow
    self._check_quantized()
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/nntool_shell_base.py", line 271, in _check_quantized
    raise GraphNotReadyException(CHECK_QUANTIZED_ERROR)
nntool.interpreter.nntool_shell_base.GraphNotReadyException: 
The opened graph must be quantized to use this command. Run the aquant command.

EXCEPTION of type 'GraphNotReadyException' occurred with message: '
The opened graph must be quantized to use this command. Run the aquant command.
'
nngraph - update graph dimensions
inserted image formatter after node input_1 withformat bw8 and normalization shift_int8
l3_ram_ext_managed - was: False
now: True
default_input_exec_location - was: 'AT_MEM_L2'
now: 'AT_MEM_L3_HRAM'
graph_produce_node_names - was: False
now: True
graph_reorder_constant_in - was: True
now: True
graph_produce_operinfos - was: False
now: True
graph_monitor_cycles - was: False
now: True
Traceback (most recent call last):
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/save_state.py", line 49, in do_save_state
    self._check_quantized()
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/nntool_shell_base.py", line 271, in _check_quantized
    raise GraphNotReadyException(CHECK_QUANTIZED_ERROR)
nntool.interpreter.nntool_shell_base.GraphNotReadyException: 
The opened graph must be quantized to use this command. Run the aquant command.

EXCEPTION of type 'GraphNotReadyException' occurred with message: '
The opened graph must be quantized to use this command. Run the aquant command.
'
echo "GENERATING AUTOTILER MODEL"
GENERATING AUTOTILER MODEL
nntool -g -M nntool_output -m networkModel.c -T nntool_output/tensors BUILD_MODEL_SQ8BIT/network.json
Traceback (most recent call last):
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/scripts/nntool", line 107, in <module>
    main()
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/scripts/nntool", line 89, in main
    mod.generate_code(args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/generator.py", line 72, in generate_code
    nntool_shell.load_state_file(args.graph_file)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/nntool_shell_base.py", line 374, in load_state_file
    with open(filepath) as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'BUILD_MODEL_SQ8BIT/network.json'
make: *** [common/model_rules.mk:49: nntool_output/networkModel.c] Error 1

According to debug, error reporting seems to be generated in the process of quantification:
image
It seems that the error is caused by the numpy version. My numpy==1.24.2.
So I changed my numpy into numpy==1.23.0, and it works! But still met error:

echo "COMPILING AUTOTILER MODEL"
COMPILING AUTOTILER MODEL
gcc -g -o BUILD_MODEL_SQ8BIT/GenTile -I. -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Emulation -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8 -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/generators -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8 -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators/CNN_Generator_Util.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/CNN_Generators_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/RNN_Generators_SQ8.c nntool_output/networkModel.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler/LibTile.a  
In file included from /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_BasicKernels_SQ8.h:21,
                 from /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/RNN_Generators_SQ8.h:21,
                 from /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/RNN_Generators_SQ8.c:21:
/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/../CNN_Libraries/CNN_Copy.h:23:10: fatal error: CNN_FloatType.h: No such file or directory
   23 | #include "CNN_FloatType.h"
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [common/model_rules.mk:56: BUILD_MODEL_SQ8BIT/GenTile] Error 1

I think this problem is caused by the large span of changing gap_sdkv3.9 to gap_sdkv4.22 (some file location may already changed). I wonder if there are any solutions? Or I can just modify the pulp-dronet's makefile by my own?

@ChenMathilda ChenMathilda mentioned this issue Feb 15, 2023
Open
@ChenMathilda
Copy link
Author

``> Your SDK is very old. Please update it. ONNX Constant operator is supported in opset version 13 in the latest SDK.

(https://github.com/GreenWaves-Technologies/gap_sdk/blob/master/tools/nntool/nntool/importer/onnx/handlers/backend/constant.py)

Hi @sousoux, I changed gap_sdk version into 4.22.0, and can run 'open *.onnx' successfully! But when I tried to runmake clean all run platform=gvsoc under path /pulp-dronet-v2/gapflow, I met this error:

(gap_sdk)
# chennaiting @ seu-netsi in ~/pulp-dronet/pulp-dronet-v2/gapflow on git:main x [20:05:39]
$ make clean all run platform=gvsoc
APP_SRCS... main.c BUILD_MODEL_SQ8BIT/networkKernels.c /home/chennaiting/pulp-dronet/gap_sdk/libs/gap_lib/img_io/ImgIO.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Activation_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Bias_Linear_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Pooling_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_DW_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_MatAlgebra_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_SoftMax_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_AT_Misc.c /home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/RNN_SQ8.c /home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/kernels/norm_transpose.c /home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/kernels/copy.c
APP_CFLAGS... -g -O3 -w -mno-memcpy -fno-tree-loop-distribute-patterns -I. -I/home/chennaiting/pulp-dronet/gap_sdk/libs/gap_lib/include -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/Emulation -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries -I/home/chennaiting/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8 -I/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/kernels -IBUILD_MODEL_SQ8BIT -DAT_MODEL_PREFIX=network -DAT_INPUT_HEIGHT=200 -DAT_INPUT_WIDTH=200 -DAT_INPUT_COLORS=1 -DSTACK_SIZE=6096 -DSLAVE_STACK_SIZE=1024 -DAT_IMAGE=/home/chennaiting/pulp-dronet/pulp-dronet-v2/gapflow/images/frame_2.pgm -DPERF -DMODEL_ID= -DFREQ_FC=250 -DFREQ_CL=175 -DAT_CONSTRUCT=networkCNN_Construct -DAT_DESTRUCT=networkCNN_Destruct -DAT_CNN=networkCNN -DAT_L3_ADDR=network_L3_Flash
sudo rm -f BUILD_MODEL_SQ8BIT/GenTile
sudo rm -f -rf BUILD_MODEL_SQ8BIT
sudo rm -f -rf nntool_output
**make: *** No rule to make target '/home/chennaiting/pulp-dronet/gap_sdk/tools/nntool/autotiler/generators/nntool_extra_generators.c', needed by 'BUILD_MODEL_SQ8BIT/GenTile'.  Stop.**

According to the error report, I deleted 55 lines of code in pulp-dronet-v2/gapflow/common/model_decl.mk

image It do make some progresses, but still encountered error:

# chennaiting @ seu-netsi in ~/pulp/pulp-dronet/pulp-dronet-v2/gapflow on git:master x [19:08:43]
make clean all run platform=gvsoc
APP_SRCS... main.c BUILD_MODEL_SQ8BIT/networkKernels.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/libs/gap_lib/img_io/ImgIO.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Activation_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Bias_Linear_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Pooling_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_Conv_DW_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_MatAlgebra_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_SoftMax_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_AT_Misc.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/RNN_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels/norm_transpose.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels/copy.c  
APP_CFLAGS... -g -O3 -w -mno-memcpy -fno-tree-loop-distribute-patterns -I. -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/libs/gap_lib/include -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Emulation -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8 -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels -IBUILD_MODEL_SQ8BIT -DAT_MODEL_PREFIX=network -DAT_INPUT_HEIGHT=200 -DAT_INPUT_WIDTH=200 -DAT_INPUT_COLORS=1 -DSTACK_SIZE=6096 -DSLAVE_STACK_SIZE=1024 -DAT_IMAGE=/home/chennaiting/pulp/pulp-dronet/pulp-dronet-v2/gapflow/images/frame_2.pgm -DPERF -DMODEL_ID= -DFREQ_FC=250 -DFREQ_CL=175 -DAT_CONSTRUCT=networkCNN_Construct -DAT_DESTRUCT=networkCNN_Destruct -DAT_CNN=networkCNN -DAT_L3_ADDR=network_L3_Flash
rm -f BUILD_MODEL_SQ8BIT/GenTile
rm -f -rf BUILD_MODEL_SQ8BIT
rm -f -rf nntool_output 
mkdir BUILD_MODEL_SQ8BIT
cp nntool_input/models_onnx/model_original.onnx BUILD_MODEL_SQ8BIT/network.onnx
echo "GENERATING NNTOOL STATE FILE"
GENERATING NNTOOL STATE FILE
echo BUILD_MODEL_SQ8BIT
BUILD_MODEL_SQ8BIT
nntool -s nntool_input/nntool_scripts/nntool_script_deployment BUILD_MODEL_SQ8BIT/network.onnx -q
settings - set log level to INFO
log_level - was: 'INFO'
now: 'INFO'
open - opening graph file BUILD_MODEL_SQ8BIT/network.onnx load_quantization = True
__init__ - unable to determine batch dimension. if the graph fails to import properly set it to 1 or a variable.
nngraph - update graph dimensions
nngraph - update graph dimensions
debug - was: False
now: True
adjust_order - adding transposes to correct tensor order for AT kernels
eliminate_transposes - no transposes to eliminate found
nngraph - update graph dimensions
nngraph - update graph dimensions
eliminate_transposes - no further transpose sequences found
nngraph - adjusted order
nngraph - update graph dimensions
move_node_up - Node _sig_Sigmoid cannot be moved
fuse_gap_convs - fusing nodes _layer1_Conv,_pool_MaxPool into _layer1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock1_conv1_Conv,_resBlock1_relu1_Clip into _resBlock1_conv1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock1_conv2_Conv,_resBlock1_relu2_Clip into _resBlock1_conv2_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock1_bypass_Conv,_resBlock1_relu3_Clip into _resBlock1_bypass_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock2_conv1_Conv,_resBlock2_relu1_Clip into _resBlock2_conv1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock2_conv2_Conv,_resBlock2_relu2_Clip into _resBlock2_conv2_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock2_bypass_Conv,_resBlock2_relu3_Clip into _resBlock2_bypass_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock3_conv1_Conv,_resBlock3_relu1_Clip into _resBlock3_conv1_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock3_conv2_Conv,_resBlock3_relu2_Clip into _resBlock3_conv2_Conv_fusion
fuse_gap_convs - fusing nodes _resBlock3_bypass_Conv,_resBlock3_relu3_Clip into _resBlock3_bypass_Conv_fusion
matcher - ++ fusion fuse_gap_convs modified graph
slice_to_split - replaced slice nodes _Gather,_Gather_1 with split node _Gather_split
matcher - ++ fusion slice_to_split modified graph
insert_copies - inserting copy between _Gather_reshape0:0 and output_1:0
matcher - ++ fusion insert_copies modified graph
remove_reshapes_before_linear - removing unnecessary reshape before linear _Flatten
matcher - ++ fusion remove_reshapes_before_linear modified graph
fuse_op_activation - fusing nodes _resBlock3_Add,_relu_Clip
matcher - ++ fusion fuse_op_activation_scale8 modified graph
move_node_up - Node _sig_Sigmoid cannot be moved
matcher - ++ fusion scaled_match_group modified graph
move_node_up - Node _sig_Sigmoid cannot be moved
adjust_order - adding transposes to correct tensor order for AT kernels
eliminate_transposes - no transposes to eliminate found
nngraph - update graph dimensions
nngraph - update graph dimensions
eliminate_transposes - no further transpose sequences found
nngraph - adjusted order
input_norm_func - was: ''
now: 'x:x/255'
Traceback (most recent call last):
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/aquant.py", line 99, in do_aquant
    data = [import_data(input_file, **input_args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/aquant.py", line 99, in <listcomp>
    data = [import_data(input_file, **input_args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/utils/data_importer.py", line 171, in import_data
    return import_image_data(filename, **kwargs)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/utils/data_importer.py", line 134, in import_image_data
    return postprocess(img_in, height, width, channels, **kwargs)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/utils/data_importer.py", line 86, in postprocess
    img_in = np.array(img_in, dtype=np.float)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/numpy/__init__.py", line 305, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'float'.
`np.float` was a deprecated alias for the builtin `float`. To avoid this error in existing code, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
EXCEPTION of type 'AttributeError' occurred with message: 'module 'numpy' has no attribute 'float'.
`np.float` was a deprecated alias for the builtin `float`. To avoid this error in existing code, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations'
Traceback (most recent call last):
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/qshow.py", line 41, in do_qshow
    self._check_quantized()
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/nntool_shell_base.py", line 271, in _check_quantized
    raise GraphNotReadyException(CHECK_QUANTIZED_ERROR)
nntool.interpreter.nntool_shell_base.GraphNotReadyException: 
The opened graph must be quantized to use this command. Run the aquant command.

EXCEPTION of type 'GraphNotReadyException' occurred with message: '
The opened graph must be quantized to use this command. Run the aquant command.
'
nngraph - update graph dimensions
inserted image formatter after node input_1 withformat bw8 and normalization shift_int8
l3_ram_ext_managed - was: False
now: True
default_input_exec_location - was: 'AT_MEM_L2'
now: 'AT_MEM_L3_HRAM'
graph_produce_node_names - was: False
now: True
graph_reorder_constant_in - was: True
now: True
graph_produce_operinfos - was: False
now: True
graph_monitor_cycles - was: False
now: True
Traceback (most recent call last):
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 1661, in onecmd_plus_hooks
    stop = self.onecmd(statement, add_to_history=add_to_history)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/cmd2.py", line 2081, in onecmd
    stop = func(statement)
  File "/home/chennaiting/anaconda3/envs/gap_sdk_4.22/lib/python3.8/site-packages/cmd2/decorators.py", line 223, in cmd_wrapper
    return func(cmd2_app, args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/commands/save_state.py", line 49, in do_save_state
    self._check_quantized()
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/nntool_shell_base.py", line 271, in _check_quantized
    raise GraphNotReadyException(CHECK_QUANTIZED_ERROR)
nntool.interpreter.nntool_shell_base.GraphNotReadyException: 
The opened graph must be quantized to use this command. Run the aquant command.

EXCEPTION of type 'GraphNotReadyException' occurred with message: '
The opened graph must be quantized to use this command. Run the aquant command.
'
echo "GENERATING AUTOTILER MODEL"
GENERATING AUTOTILER MODEL
nntool -g -M nntool_output -m networkModel.c -T nntool_output/tensors BUILD_MODEL_SQ8BIT/network.json
Traceback (most recent call last):
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/scripts/nntool", line 107, in <module>
    main()
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/scripts/nntool", line 89, in main
    mod.generate_code(args)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/generator.py", line 72, in generate_code
    nntool_shell.load_state_file(args.graph_file)
  File "/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool/nntool/interpreter/nntool_shell_base.py", line 374, in load_state_file
    with open(filepath) as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'BUILD_MODEL_SQ8BIT/network.json'
make: *** [common/model_rules.mk:49: nntool_output/networkModel.c] Error 1

According to debug, error reporting seems to be generated in the process of quantification: image It seems that the error is caused by the numpy version. My numpy==1.24.2. So I changed my numpy into numpy==1.23.0, and it works! But still met error:

echo "COMPILING AUTOTILER MODEL"
COMPILING AUTOTILER MODEL
gcc -g -o BUILD_MODEL_SQ8BIT/GenTile -I. -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Emulation -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8 -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/generators -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8 -I/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/nntool//autotiler/kernels /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators/CNN_Generator_Util.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/CNN_Generators_SQ8.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/RNN_Generators_SQ8.c nntool_output/networkModel.c /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/Autotiler/LibTile.a  
In file included from /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/CNN_BasicKernels_SQ8.h:21,
                 from /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/RNN_Generators_SQ8.h:21,
                 from /home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Generators_SQ8/RNN_Generators_SQ8.c:21:
/home/chennaiting/pulp/pulp-dronet/gap_sdk/tools/autotiler_v3/CNN_Libraries_SQ8/../CNN_Libraries/CNN_Copy.h:23:10: fatal error: CNN_FloatType.h: No such file or directory
   23 | #include "CNN_FloatType.h"
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [common/model_rules.mk:56: BUILD_MODEL_SQ8BIT/GenTile] Error 1

I think this problem is caused by the large span of changing gap_sdkv3.9 to gap_sdkv4.22 (some file location may already changed). I wonder if there are any solutions? Or I can just modify the pulp-dronet's makefile by my own?

I try to solve this problem by my own and do make some progresses.
If you have any suggestiones, pls tell me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sousoux @ChenMathilda