TFLite got Operator not supported #133
Comments
|
Hi @ilous12 this is really good work, thank you. I think this is the first time someone has tried to run deeplabv3+ via TfLite on ArmNN, and actually the missing functionalilty is not that bad: MUL, ADD, RESIZE_BILINEAR, BATCH_TO_SPACE_ND, SPACE_TO_BATCH_ND and SUB are all supported by ArmNN, and just need adding to the TfLite parser. @brunomorishita actually added some of these recently, and his code is available in the development branch at https://review.mlplatform.org/#/admin/projects/ml/armnn |
|
ARG_MAX recently got added to Compute Library but hasn't been integrated into ArmNN yet. CAST is a bit harder, it depends what is actually happening in the model. We don't have this functionality in ArmNN yet, and it might not be present in Compute Library either. Then we have the ParseReshape issue - I think this is a limitation just of the TfLite parser, as ArmNN can now handle all kinds of reshapes. So some of this is "easy" to fix in ArmNN, some is harder. deeplab v3 is a network that we are trying to support, but I can't confirm a timeline for it. Are you in a position to help add support? Many thanks, |
|
@MatthewARM |
|
Hi @ilous12 if you are willing to help, a really good first step would be to get the latest master Arm NN and Compute Library and try your test again, that will give us a good picture of the remaining work. The links to the master branches can be found on our new developer website here: https://mlplatform.org/contributing/ |
|
ok. I will try tomorrow. |
|
Hi @MatthewARM I got a result. see below.
my tflite file was below input/output
my code what's #89. Is there verbose mode? |
|
Hi, @MatthewARM did you have any update related tensorflowlite lite? |
|
Thanks @ilous12 we'll try those steps and see. |
|
Hi @ilous12 , Recently I push some commits adding support for some of the operations in deeplab v3 model for tflite. The operations I pushed were not merged in the development branch yet, so you'll have to get my patches. They are available at: Please let me know if this works for you. |
|
thanks @brunomorishita How can I get your patches? can you guide? |
|
|
I downloaded your patches |
|
Thanks. guys. you did. |
|
Hi. guys. Thanks @brunomorishita I tried next steps and I saw invalid semantic label |
|
Hi @ilous12 where you have " engineConfig_->device_ = armnn::Compute::GpuAcc;" it's probably worth a quick try with CpuRef to see if the output from our reference (non-accelerated) implementation is different. That will help track down the problem. |
|
Unfortunately It didn't work. did you check my code have no problem? |
|
I thought my code looks fine, But I want to check that most likely something is going wrong with handling the input buffer or output. |
|
At armnnwrapper.cc_:143 you have |
|
You were right. I fixed. I try again. |
|
We tried test but not enough. our code src.zip
|
|
@ilous12 might not be exactly the same, if ArmNN does rounding differently to TfLite, but should be very close. |
|
Hi @ilous12 I can't see anything else obviously wrong with the code so I don't know why you would get the wrong result. The only thing I can suggest is that in the latest master branch (at https://review.mlplatform.org/#/admin/projects/ml/armnn) we have added a 'debug' flag to Actually, if you haven't tried already, you should try with the latest master code anyway, just in case this is a bug that we've fixed without realising it. Good luck, |
|
@ilous12 this is a floating-point model, isn't it? Not quantised? I'm just trying to figure out what could be going wrong. |
|
Hi @oms1226 we should be getting something very close to Tensorflow Lite's answer - just small differences sometimes due to different arithmetic implementations. As you are seeing something very different, it looks like there is perhaps a bug. If you can figure out which layer in the network is producing the wrong output, that would be very helpful. Many thanks, |
|
Thank @MatthewARM. |
|
Hi @oms1226 when you run with m_Debug set to true, it will print all tensor values on standard output, so you can capture them in a file for debugging. By the way, changing the default value is overkill - it would be more usual to set it in your application with something like: |
|
I feel so sorry for @MatthewARM . |
|
Hi, @MatthewARM https://drive.google.com/open?id=1wJ4RGXJWllWXPj2vyawVDojApB9E3jKF I'm checking a layer "sub_7", I saw ArmNN does rounding differently to TfLite, but should be very close. On Tensorflow Lite On Armnn Is it right? |
|
Hi, @MatthewARM @brunomorishita |
|
Hi @MatthewARM |
|
Hello @ilous12 , |
|
@ilous12 if the debug flag isn't working on Android, can you at least print the numbers coming out of your network and compare them to TfLite? At the moment I can't figure out if what we're seeing is a bug in ArmNN or some sort of numerical precision issue. |
|
By the way, the debug flag eventually causes the code to get called in src/backends/reference/workloads/Debug.cpp which uses std::cout. If std::cout isn;t working, maybe you could hack in something that works. I'm sorry I don't really know much about what debug / printing features are available from an Android application. |
|
Hi @kevmay01 @MatthewARM |
|
@MatthewARM |
|
Hi. @MatthewARM @kevmay01
|
|
Hello @ilous12 However if you train and convert a model that only uses dilation values of 1 you should see the same results between Arm NN and Tensorflow Lite. I converted your frozen_inference_graph.pb which you attached here to a tflite file and was able to get the same results running on Arm NN and Tf Lite. |
|
Thanks for reply. I understood your comment. I will try to train. after I will check, I share you results |
|
Hi @kevmay01 |
|
I converted the file you attached to this ticket: frozen_inference_graph.pb tflite_convert You will need to train a new model I think and figure out how to convert and optimize it to make a similar model to the official deeplabv3_257_mv_gpu.tflite file, but only using default dilation values. |
|
@ilous12 I think the information you need regarding creating a model with dilation set to 1 is in this ticket: tensorflow/tensorflow#26474 There is also an example attached to that ticket of an identical deeplabv3_257_mv_gpu.tflite, but with dilation values set to 1. |
|
Hi @kevmay01 . Do you have a shchdule to work convolution layers with dilation values of 2 and 4? Or not? |
|
Hi @MatthewARM @kevmay01 But I got some problems. You will see below. Tensorflow Lite Result Armnn Result Composite image (tflite + armnn) I think the shape of the result is a little different, can you check? |
|
Thanks @ilous12 I see what you mean about the output being different. I'll try to get someone to look at whether this indicates a bug but I'm not sure whether that will be this week or later. Have you tried with the Arm NN CpuRef backend? That doesn't use Compute Library so it eliminates one possible source of errors. |
|
Hi @MatthewARM I got a result and I will se belows CpuRef Gpu We have a crash issue to run cpuAcc on Release 19.02. You will se belows com.test.sample#00 pc 00000000004c184c /data/app/com.test.sample-IsGLQ5XgkmORLc4rtHSI7w==/lib/arm64/libarmnn.so (arm_compute::NEScaleKernel::scale_nhwc(arm_compute::Window const&)+792) <---- This point |
|
Hi @MatthewARM We measured elapsed times and I noticed delayed API. You will see belows. Can you check this problem? |
|
Hi @MatthewARM When I use "optimizerOptions.m_Debug = true" with GpuAcc on 19.02, I got a crash. armnn/src/armnn/LoadedNetwork.cpp:192: const armnn::IWorkloadFactory &armnn::LoadedNetwork::GetWorkloadFactory(const armnn::Layer &) const: assertion "(IWorkloadFactory::IsLayerSupported(layer, {}, reasonIfUnsupported))&&("Factory does not support layer")" failed Thanks. |
|
Hi @MatthewARM
Can you check "Up Scale" function by resizeBilinear op? We attached output files. |
|
Hi @ilous12 sorry for the long delay in replying. The only question I know how to answer is this one:
I expect that the 2.3 seconds here is spent compiling OpenCL kernels for the GpuAcc backend. We're looking into various ways to cache those kernels so that it doesn't have to be done every time you run your program, but it's a hard problem. I'm sorry but I don't yet have an answer for your other problems. |
|
Hi @ilous12 the support for dilation in DepthwiseConvolution has been merged to master so hopefully your original model will now work! On the 'upscale' issue, do you happen to know which resize method is in use? Should be one of: Many thanks, |
|
Many thanks to @brunomorishita for contributing the dilation support |
|
Hi @ilous12 we have just fixed the crash with NEScale in this patch https://review.mlplatform.org/#/c/ml/ComputeLibrary/+/1141/ which is now on the master branch. |
|
thanks guys. We will check soon. |
hi, I tried to apply deeplabv3+.
I built on Deeplab and I got frozen.pb [frozen_inference_graph.pb.zip]
When I converted, I got [deeplab_257_quantized.tflite.zip]
Finally, I ran on android (Samsung Note8, supported OpenCL + NEON) with below code
and I got below error. I understood supported ops. Is that problem?
The text was updated successfully, but these errors were encountered: