Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nv_full runtime test failed #143

Open
guodonglai opened this issue Jun 10, 2019 · 8 comments
Open

nv_full runtime test failed #143

guodonglai opened this issue Jun 10, 2019 · 8 comments

Comments

@guodonglai
Copy link

I followed the virtualplatform instruction guide but when I try one of the sanity test, it failed.

Here is the log file:

./nvdla_runtime --loadable ../../regression/flatbufs/kmd/CONV/CONV_D_L0_0_larg

e_fbuf
creating new runtime context...
Emulator starting
submitting tasks...
[ 62.871762] Enter:dla_read_network_config
[ 62.875498] Exit:dla_read_network_config status=0
[ 62.888309] Enter: dla_initiate_processors
[ 62.903849] Enter: dla_submit_operation
[ 62.905443] Prepare Convolution operation index 0 ROI 0 dep_count 1
[ 62.908877] Enter: dla_prepare_operation
[ 62.911547] processor:Convolution group:0, rdma_group:0 available
[ 62.913815] Enter: dla_read_config
[ 62.915589] Exit: dla_read_config
[ 62.920740] Exit: dla_prepare_operation status=0
[ 62.923508] Enter: dla_program_operation
[ 62.928722] Program Convolution operation index 0 ROI 0 Group[0]
[ 62.941093] no desc get due to index==-1
[ 62.944967] no desc get due to index==-1
[ 62.952742] no desc get due to index==-1
[ 62.966997] no desc get due to index==-1
[ 62.968714] no desc get due to index==-1
[ 62.971225] no desc get due to index==-1
[ 62.972885] Enter: dla_op_programmed
[ 62.979211] Update dependency operation index 1 ROI 0 DEP_COUNT=1
[ 62.982929] enable SDP in dla_update_dependency as depdency are resolved
[ 62.988152] Enter: dla_enable_operation
[ 62.989812] exit dla_enable_operation without actual enable due to processor hasn't been programmed
[ 62.996153] Exit: dla_enable_operation status=0
[ 63.007180] Exit: dla_op_programmed
[ 63.008845] Exit: dla_program_operation status=0
[ 63.011889] Exit: dla_submit_operation
[ 63.013824] Enter: dla_dequeue_operation
[ 63.016482] exit Convolution as there's no further operation
[ 63.022989] Exit: dla_dequeue_operation
[ 63.027468] Enter: dla_submit_operation
[ 63.029075] Prepare SDP operation index 1 ROI 0 dep_count 0
[ 63.039124] Enter: dla_prepare_operation
[ 63.041107] processor:SDP group:0, rdma_group:0 available
[ 63.043659] Enter: dla_read_config
[ 63.045195] Exit: dla_read_config
[ 63.047137] Exit: dla_prepare_operation status=0
[ 63.048941] Enter: dla_program_operation
[ 63.049646] Program SDP operation index 1 ROI 0 Group[0]
[ 63.065550] no desc get due to index==-1
[ 63.072565] no desc get due to index==-1
[ 63.076414] no desc get due to index==-1
[ 63.087045] no desc get due to index==-1
[ 63.088770] no desc get due to index==-1
[ 63.093519] no desc get due to index==-1
[ 63.096387] Enter: dla_op_programmed
[ 63.099191] Exit: dla_op_programmed
[ 63.104234] Exit: dla_program_operation status=0
[ 63.107019] Enter: dla_enable_operation
[ 63.112596] Enable SDP operation index 1 ROI 0
[ 63.118555] Enter: dla_op_enabled
[ 63.120020] Update dependency operation index 0 ROI 0 DEP_COUNT=1
[ 63.137489] enable Convolution in dla_update_dependency as depdency are resolved
[ 63.141712] Enter: dla_enable_operation
[ 63.144328] Enable Convolution operation index 0 ROI 0
[ 63.148278] Enter: dla_op_enabled
[ 63.149705] Exit: dla_op_enabled
[ 63.152031] Exit: dla_enable_operation status=0
[ 63.153873] Exit: dla_op_enabled
[ 63.155912] Exit: dla_enable_operation status=0
[ 63.159603] Exit: dla_submit_operation
[ 63.161200] Enter: dla_dequeue_operation
[ 63.163713] exit SDP as there's no further operation
[ 63.168822] Exit: dla_dequeue_operation
[ 63.171264] Exit: dla_initiate_processors status=0

Info: NV_NVDLA_csc.cpp: 647:NV_NVDLA_csc::SendDataToMacSequencerDirectConvCommon, invalid configuration csc_entries_, actual value is 0xF, it shall be 0x7.

Info: NV_NVDLA_cdma.cpp: 1265:NV_NVDLA_cdma::DirectConvDataResponseSequencerCommon, invalid configuration cdma_entries_, set value is 0xF, it shall be 0x7FF.

Info: NV_NVDLA_cdma.cpp: 1272:NV_NVDLA_cdma::DirectConvDataResponseSequencerCommon, invalid configuration of cdma_line_packed_, cdma_line_packed_ is 1, line_stride == ATOM_CUBE_SIZE * cube_width is 0.

Info: NV_NVDLA_cdma.cpp: 2073:If weight uncompressed, all cdma2sc_wt should have been sent

Any ideas how to fix this?

@guodonglai
Copy link
Author

This is the latest version of compiler and I am using nv_full as provided in the hardware/specs.

@prasshantg
Copy link
Collaborator

@guodonglai
Copy link
Author

I tried the CONV_D_L0_0_fbuf with opendla_1.ko and five.pgm on the image you provided. The result is still the same.

@guodonglai
Copy link
Author

I have also tried to test it on the image I generated using buildroot, but it is still showing the same info:
Info: NV_NVDLA_cdma.cpp: 1265:NV_NVDLA_cdma::DirectConvDataResponseSequencerCommon, invalid configuration cdma_entries_, set value is 0xF, it shall be 0x7FF.
So where exactly does the problem come from? From Cmodel or from the loadable provided in sw folder?

@shazib-summar
Copy link

shazib-summar commented Oct 6, 2019

You should be using https://github.com/nvdla/sw/blob/master/regression/flatbufs/kmd/CONV/CONV_D_L0_0_fbuf for nv_full

Which file should I use for nv_small @prasshantg ?

@gitosu67
Copy link

@killerzula For nv_small you should be using https://github.com/nvdla/sw/blob/master/regression/flatbufs/kmd/CONV/CONV_D_L0_0_small_fbuf

@ICermanRan
Copy link

You should be using https://github.com/nvdla/sw/blob/master/regression/flatbufs/kmd/CONV/CONV_D_L0_0_fbuf for nv_full
I meet the same problem,when I build and compile vp HW_PROJECT without nv_small,such as nv_small_256 or nv_large,the QEMU will directly quit,and faile.
The log file as:
creating new runtime context...
Emulator starting
dlaimg height: 28 x 28 x 1: LS: 896 SS: 0 Size: 25088
submitting tasks...
[ 49.212281] Enter:dla_read_network_config
[ 49.218963] Exit:dla_read_network_config status=0
[ 49.219117] Enter: dla_initiate_processors
[ 49.219451] Enter: dla_submit_operation
[ 49.219660] Prepare Convolution operation index 0 ROI 0 dep_count 1
[ 49.220188] Enter: dla_prepare_operation
[ 49.225101] processor:Convolution group:0, rdma_group:0 available
[ 49.226196] Enter: dla_read_config
[ 49.226997] Exit: dla_read_config
[ 49.227370] Exit: dla_prepare_operation status=0
[ 49.228722] Enter: dla_program_operation
[ 49.229118] Program Convolution operation index 0 ROI 0 Group[0]
[ 49.250708] no desc get due to index==-1
[ 49.250906] no desc get due to index==-1
[ 49.250996] no desc get due to index==-1
[ 49.251109] no desc get due to index==-1
[ 49.251201] no desc get due to index==-1
[ 49.251317] Enter: dla_op_programmed
[ 49.251435] Update dependency operation index 3 ROI 0 DEP_COUNT=3
[ 49.251573] Update dependency operation index 1 ROI 0 DEP_COUNT=1
[ 49.251700] enable SDP in dla_update_dependency as depdency are resolved
[ 49.251863] Enter: dla_enable_operation
[ 49.251967] exit dla_enable_operation without actual enable due to processor hasn't been programmed
[ 49.252199] Exit: dla_enable_operation status=0
[ 49.252320] Exit: dla_op_programmed
[ 49.252408] Exit: dla_program_operation status=0
[ 49.252495] Exit: dla_submit_operation
[ 49.252574] Enter: dla_dequeue_operation
[ 49.252715] Dequeue op from Convolution processor, index=3 ROI=0
[ 49.252887] Enter: dla_submit_operation
[ 49.252991] Prepare Convolution operation index 3 ROI 0 dep_count 2
[ 49.254260] Enter: dla_prepare_operation
[ 49.256650] processor:Convolution group:1, rdma_group:0 available
[ 49.256761] Enter: dla_read_config
[ 49.256861] Exit: dla_read_config
[ 49.256931] Exit: dla_prepare_operation status=0
[ 49.257049] Enter: dla_program_operation
[ 49.258710] Program Convolution operation index 3 ROI 0 Group[1]
[ 49.262756] no desc get due to index==-1
[ 49.262897] no desc get due to index==-1
[ 49.262978] no desc get due to index==-1
[ 49.263073] no desc get due to index==-1
[ 49.263321] no desc get due to index==-1
[ 49.263426] Enter: dla_op_programmed
[ 49.263539] Update dependency operation index 6 ROI 0 DEP_COUNT=3
[ 49.263671] Update dependency operation index 4 ROI 0 DEP_COUNT=2
[ 49.263802] Exit: dla_op_programmed
[ 49.263850] Exit: dla_program_operation status=0
[ 49.263894] Exit: dla_submit_operation
[ 49.263957] Exit: dla_dequeue_operation
[ 49.264004] Enter: dla_submit_operation
[ 49.264043] Prepare SDP operation index 1 ROI 0 dep_count 0
[ 49.264191] Enter: dla_prepare_operation
[ 49.264578] processor:SDP group:0, rdma_group:0 available
[ 49.264705] Enter: dla_read_config
[ 49.264843] Exit: dla_read_config
[ 49.264953] Exit: dla_prepare_operation status=0
[ 49.265061] Enter: dla_program_operation
[ 49.266137] Program SDP operation index 1 ROI 0 Group[0]
[ 49.268374] no desc get due to index==-1
[ 49.268485] no desc get due to index==-1
[ 49.268539] no desc get due to index==-1
[ 49.268661] no desc get due to index==-1
[ 49.268786] Enter: dla_op_programmed
[ 49.268826] Update dependency operation index 4 ROI 0 DEP_COUNT=1
[ 49.268988] enable SDP in dla_update_dependency as depdency are resolved
[ 49.269898] Enter: dla_enable_operation
[ 49.270140] exit dla_enable_operation without actual enable due to processor hasn't been programmed
[ 49.270314] Exit: dla_enable_operation status=0
[ 49.270417] Exit: dla_op_programmed
[ 49.270533] Exit: dla_program_operation status=0
[ 49.270762] Enter: dla_enable_operation
[ 49.270859] Enable SDP operation index 1 ROI 0
[ 49.272428] Enter: dla_op_enabled
[ 49.272524] Update dependency operation index 0 ROI 0 DEP_COUNT=1
[ 49.272668] enable Convolution in dla_update_dependency as depdency are resolved
[ 49.273001] Enter: dla_enable_operation
[ 49.274491] Enable Convolution operation index 0 ROI 0
[ 49.276213] Enter: dla_op_enabled
[ 49.276299] Exit: dla_op_enabled
[ 49.276373] Exit: dla_enable_operation status=0
[ 49.276496] Exit: dla_op_enabled
[ 49.276597] Exit: dla_enable_operation status=0
[ 49.276707] Exit: dla_submit_operation
[ 49.276787] Enter: dla_dequeue_operation
[ 49.276876] Dequeue op from SDP processor, index=4 ROI=0
[ 49.277000] Enter: dla_submit_operation
[ 49.277088] Prepare SDP operation index 4 ROI 0 dep_count 0
[ 49.277948] Enter: dla_prepare_operation

Fatal: NV_NVDLA_csc.cpp: 659:NV_NVDLA_csc::SendDataToMacSequencerDirectConvCommon, invalid configuration csc_entries_, actual value is 0xD, it shall be 0x1B.
In file: ../cmod/csc/NV_NVDLA_csc.cpp:659
In process: nvdla.nvdla_core.nvdla_core.csc.DataLoadSequenceThread @ 49100 ms
Aborted (core dumped)
Any ideas how to fix this? How should I use https://github.com/nvdla/sw/blob/master/regression/flatbufs/kmd/CONV/CONV_D_L0_0_fbuf for nv_full? change kmd?
Waitting for your response! QAQ

@long771
Copy link

long771 commented Mar 26, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants