Compile FP16 type Model

[workdd]

Hello, I am a student who is very interested in AWS Inferentia. Through the Neuron document, I learned that if model in FP32 are compiled, they are cast as BP16 and if model in FP16 are compiled, they are cast as FP16. Therefore, I want to compile model in FP16 into neuron sdk to know performance and accuracy. But the way I can get the model in FP16 is to convert the model with TFlite. Is there a way to compile the TFlite model? As far as I know, only model in pb(protobuf) format can be compiled.

[aws-zejdaj]

@ manchann thanks for you interest in Inferentia! One minor correction: the FP16 models stay in FP16, no cast is involved.
TFlite models are generally for edge devices, they may not have sufficient workload for ML accelerator such as Inferentia. Do you have a specific model in mind?
To use Neuron SDK you can try to export TFlite to pb as mentioned eg in https://stackoverflow.com/questions/67229953/is-it-possible-to-convert-tflite-to-pb

[workdd]

Thank you for your reply. I understood more because you pointed out the part related to FP16.
According to the link, TFlite to pb seems very difficult at the moment.
Then I'll have to get the FP16 pb models to experiment. Thank you ! :)