[Bug] tvm fp16 much more slower than float32

[lucasjinreal]

I have a simple MLP model which have 3 layer fcs, I using te to build it manually and specified the dtype in each layer. the exported lib inference much more slower than float32.....

and the exported .so size is twice bigger than float32. How could this be possible?

[masahi]

Without HW support for fp16, fp16 compute will be emulated via software. So it is not surprising that perf is worse and the generated code is bigger than fp32.

Please post basic usage questions to the forum.

[lucasjinreal]

@masahi thanks. I am using iPhone12 realphone, which obisouly support fp16. But the generated .a size, bigger than float32, speed slower.

[masahi]

You need to tell TVM that your target has native fp16, otherwise we can't generate fp16 code. See my forum post answer.

[lucasjinreal]

@masahi thanks, I have gened the lib with llvm config:

llvm arm64 -mtriple=iphoneos-apple-darwin

And all te build operations using dtype = fp16, the inference memory actually shrined down, but the .a size and speed are not expected. So it actually gened fp16 .so, but somehow the .a is bigger which I can not figure out, it doesn't make sense.

And the speed slower.

Does there any examples on how to exactly gen fp16 .so or .a on iPhone 12?