Does the NVDLA backend have a plan to support INT8 configuration?

[shgoupf]

Thanks for supporting NVDLA backend, great work!

I've tried ONNC for generating NVDLA loadable, everything looks good that I can get the desired loadable file from the ONNX alexnet models. Currently the loadable generated by ONNC is in FP16 (HALF) data type, which is supposed to be used with nv_full configuration (https://github.com/nvdla/hw, branch nvdlav1).

However, many works in NVDLA community is based on the nv_large or nv_small configuration (https://github.com/nvdla/hw, branch master). These two configs are based on INT8 for data types (weights, input/output tensors, etc.), and they differ from nv_full not only in data precision, but also in the hardware supported engines, for example there is not RUBIK in nv_large and nv_small.

So my question is, do we have a plan to support INT8 for NVDLA? Possibly it is already supported but I didn't find the knob to enable it, anyone can show me how to enable that? (I tried to go through the docs and the help info of onnc, no clue until now ... )

[weifen]

Thanks for your positive feedback. ONNC does support nv_small and other configurations. However, it is under commercial license and not open sourced at this moment. You are welcome to hack ONNC to support nv_small. You are right that the NVDLA community focus on the nv_small version. The original nvdla_compiler does not support nv_small either. It took quiet an effort to port ONNC to nv_small. The ONNC compiler for nv_small is able to compile 6 ONNX zoo models and those loadable are runnable on virtual platform and the FPGA platform.

[shgoupf]

Great thanks for the feedback. Actually I believe nv_large, which is the same as nv_small but has 2048 MACs (nv_small has 64 MACs), is one of another worth-a-try configuration. We can have nv_large running on IBM CAPI system (https://github.com/shgoupf/snap/tree/nvdla). nv_large has 40%~50% LUT and BRAM utilization on Xilinx VU9P FPGA.

We are also working on a compiler to support nv_large and we decided to base our work on this ONNC.

Will post result to the community when things are ready.

[shgoupf]

Question answered, closing ...

[LeiWang1999]

@weifen Hi, is that commercial license is avaliable now? If so, how can we got it..