Densify ?

[geaxgx]

1. Ubuntu 18.04

2. OS Architecture x86_64

3. OpenVINO e.g. 2021.4.582

9. Download URL for .tflite IR model https://github.com/google/mediapipe/blob/master/mediapipe/modules/pose_detection/pose_detection.tflite

Hi @PINTO0309 !
New mediapipe version 0.8.6 comes with new models for Blazepose (that's a never ending story :-)
The size of the pose detection model (link above) has been significantly reduced (from ~7.5MB to ~3MB) but unfortunately the model is using a layer named Densify that is not implemented in tflite2tensorflow. I guess it is a relatively new layer. When trying to visualize its data in Netron, I get an "Invalid tensor data size" message.

Do you think Densify can be easily implemented in your tools ? Note that it is not something I am eagerly waiting for since I can do without it by using the previous version of the pose detection model.

[PINTO0309]

After analyzing the TensorFlow implementation, it seems that I need to implement the following additional steps in the following order. It's going to take some time.

1. https://www.tensorflow.org/api_docs/python/tf/sparse/SparseTensor
2. https://www.tensorflow.org/api_docs/python/tf/sparse/to_dense
3. https://github.com/tensorflow/tensorflow/blob/5d442828288614d57062f77d8af5d5b090b21469/tensorflow/lite/kernels/internal/reference/densify.h#L29-L43
4. https://github.com/tensorflow/tensorflow/blob/a6b6df2de94421ecdb1baa97aca3ffca74ee04ad/tensorflow/lite/tools/optimize/sparsity/format_converter.cc#L215-L256

template <typename T>
inline void Densify(const TfLiteSparsity* sparsity,
                    const RuntimeShape& input_shape, const T* input_data,
                    const RuntimeShape& output_shape, T* output_data,
                    TfLiteContext* context) {
  const int dims_count = output_shape.DimensionsCount();
  std::vector<int> vector_shape(dims_count);
  for (int i = 0; i < dims_count; i++) {
    vector_shape[i] = output_shape.Dims(i);
  }

  tflite::optimize::sparsity::FormatConverter<T> converter(vector_shape, *sparsity);
  converter.SparseToDense(input_data, output_shape.FlatSize(), output_data, context);
}

[geaxgx]

Thanks @PINTO0309 ! As I said before, take your time. I can play with the previous version.

I guess Densify allows a smaller size of the model on disk, but probably not in memory :-)

[MrNeither]

@geaxgx I get the same problem. Could you please give a link to the previous version of the model, which can be converted with tflite2tensorflow?

[MrNeither]

@geaxgx Sorry for the trouble, it's just in another committee there is)

[PINTO0309]

https://github.com/PINTO0309/PINTO_model_zoo/tree/main/053_BlazePose

[PINTO0309]

Fixes: f032b31

This is an experimental implementation at the moment, so it is not well tested.

• Before
  https://github.com/google/mediapipe/blob/master/mediapipe/modules/pose_detection/pose_detection.tflite

• After
  https://drive.google.com/file/d/1mu8Qmkq28QMURiWre2-BnP08gYcyLwFs/view?usp=sharing

tflite2tensorflow v1.11.7 https://github.com/PINTO0309/tflite2tensorflow/releases/tag/v1.11.7

[PINTO0309]

Commited. TFLite Float32/Float16, EdgeTPU, ONNX, OpenVINO IR, Myriad Blob, TF-TRT, TFJS, CoreML.
https://github.com/PINTO0309/PINTO_model_zoo/tree/main/053_BlazePose/20_densify_pose_detection

[geaxgx]

Thanks @PINTO0309 !