Float16 Support (#3794)

[myungjoo]

Q1: should we support it or not?

(IF Q1==yes)
Q2: should this be "binary16"? (arbitrary binary value that we do NOT care its semantics) or should we treat it as floating point numbers?

If Q2 == float, it means that potentially, tensor_transform will do some arithmetics on it.
If Q2 == binary, it means that tensor_transform won't be able to touch it.

(IF Q2==float)
Q3: Is there a general consensus on its format that we can follow?

[anyj0527]

In https://www.tensorflow.org/lite/performance/model_optimization, there are some guide on how to use fp16, int16 quantize tflite models. But all examples keep the input / output as float32 or uint8 and do quantize parameters INSIDE a model.

I've never seen 16bit input / output models yet, but it could be appeared in the future.

[myungjoo]

There are hw accelerators using FP16. (NXP)

[designe]

I think that fp16 quantization is relatively easier to convert than 8-bit integer quantization, so it will be a necessary feature .

I have a fp16 quantize model. but it was failed to run.
below is the log in version of 2.1.1.0-0~202207041214~ubuntu20.04.1

Unimplemented data type FLOAT16 (1) in tensor
** (python3:3075844): CRITICAL **: 10:33:03.476: Failed to construct interpreter
** (python3:3075844): CRITICAL **: 10:33:03.476: Failed to load model (TensorFlow-lite interpreter->loadModel() has returned -2. Please check if the model

Can I use fp16 quantize model with build option -DFLOAT16_SUPPORT now?

[designe]

Oh, I checked #3784 issue now.

[myungjoo]

Uh.. and FP16 data are NOT quantized data (same as FP32 data are not quantized data.)

[designe]

I thought float16 support was the basis on which the FP16 quantization model could run.
Sorry if I didn't understand properly!

[myungjoo]

Quantization = Integer. (practically, w/ common exponents)
FP16 = Floating point 16 = not integers (individual exponents)
If FP16 can be called quantized, FP32 can be, too. :)

[myungjoo]

Anyway. FP16 support is merged.