Issue running posenet benchmark

[saliknadeem]

Hi,

I tried running the benchmark for posenet which is giving the following error:

ValueError: Input 0 of layer "model" is incompatible with the layer: expected shape=(None, 30, 68), found shape=(None, 30, 52)

Please let me know what I can do about this.

Thank you!

[simoneangarano]

Hi @saliknadeem ,
What configuration are you using (e.g. DATASET, MODEL_SIZE, ...)?
I'm unable to reproduce your issue; could you please explain it better? Are you using the latest version of mpose (1.0.26)?
Thank you!

[saliknadeem]

Hi @simoneangarano,
Thank you for your response.
I am using mpose 1.0.26 and using the default config file with just posenet as the dataset and I get the same error for all model sizes.

Here is the error for reference:

----- Start Split 1 ----

- Fold 1
Epoch 1/350
Traceback (most recent call last):
  File "main.py", line 42, in <module>
    trainer.do_benchmark()
  File "/home/salik/AcT/utils/trainer.py", line 160, in do_benchmark
    acc, bal_acc = self.do_training()
  File "/home/salik/AcT/utils/trainer.py", line 120, in do_training
    history = self.model.fit(self.ds_train,
  File "/home/salik/anaconda3/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/salik/anaconda3/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 1147, in autograph_handler
    raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:

    File "/home/salik/anaconda3/lib/python3.8/site-packages/keras/engine/training.py", line 1021, in train_function  *
        return step_function(self, iterator)
    File "/home/salik/anaconda3/lib/python3.8/site-packages/keras/engine/training.py", line 1010, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/home/salik/anaconda3/lib/python3.8/site-packages/keras/engine/training.py", line 1000, in run_step  **
        outputs = model.train_step(data)
    File "/home/salik/anaconda3/lib/python3.8/site-packages/keras/engine/training.py", line 859, in train_step
        y_pred = self(x, training=True)
    File "/home/salik/anaconda3/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
        raise e.with_traceback(filtered_tb) from None
    File "/home/salik/anaconda3/lib/python3.8/site-packages/keras/engine/input_spec.py", line 264, in assert_input_compatibility
        raise ValueError(f'Input {input_index} of layer "{layer_name}" is '

    ValueError: Input 0 of layer "model" is incompatible with the layer: expected shape=(None, 30, 68), found shape=(None, 30, 52)

Edit: It works perfectly fine with openpose for all model sizes. But I have another problem there, the larger models (Large and Base) are performing worse than the small model. They are getting outputs at around 88-89%

[simoneangarano]

Hi @saliknadeem ,
I'm sorry. I realized the config file still referred to an older code version. I pushed the new config.yaml, and now it should be ok!
As for your second question, please be sure to use the same configuration described in the paper and test on all the random splits to get the correct result! The accuracy we reported is the average among all validation splits, so the results may differ if you run the benchmark on a single split.
Let me know!

[saliknadeem]

Thank you @simoneangarano for the prompt response.

I will try running all the benchmarks again with the new commit and report back if I face some issues.

Thanks!

[saliknadeem]

Hi @simoneangarano,
So I ran a few benchmarks, everything is running fine now. But I had 2 issues:

1- Posenet in the config.yml file says 13 keypoints, even though in the paper it says it has 17 keypoints, so I'm wondering why it's different.
2- For some reason the performance is still not similar to the paper results. I am trying all 3 splits with 10 fold as explained in the paper with the default hyperparameters. I have also tried more epochs and tried to modify other hyperparameters but still the results are a below 90% for all sizes.

Thank you!

[simoneangarano]

Hi @saliknadeem ,

1 - The 13-keypoints configuration has been added after the paper's publication to further speed up inference and use the same number of keypoints for openpose and posenet. If you want to use the paper version, go to utils/data.py in load_mpose() and delete the dataset.reduce_keypoints() line. This will get you the desired shape. You also should change the config.yaml file accordingly.

2 - I will check as soon as possible and let you know!

Thanks,
Simone.

[saliknadeem]

Hi @simoneangarano,
Thank you for that information, this is helpful.

Also just to further add some more feedback. I tried running inference on the three pretrained models and here is the results I got:

--------------------------------------------
RUNNING INFERENCE ON pretrained/AcT_large_1_0.h5
MODEL_SIZE: large, DATASET: openpose
--------------------------------------------
6/6 [==============================] - 3s 224ms/step - loss: 4.9475 - accuracy: 0.1001
Accuracy Test: 0.10010463744401932 <> Balanced Accuracy: 0.10416016535558978

--------------------------------------------
RUNNING INFERENCE ON pretrained/AcT_micro_1_0.h5
MODEL_SIZE: micro, DATASET: openpose
--------------------------------------------
6/6 [==============================] - 2s 63ms/step - loss: 0.9695 - accuracy: 0.8856
Accuracy Test: 0.8855947256088257 <> Balanced Accuracy: 0.8302190257215626

--------------------------------------------
RUNNING INFERENCE ON pretrained/AcT_base_1_0.h5
MODEL_SIZE: base, DATASET: openpose
--------------------------------------------
Traceback (most recent call last):
  File "main.py", line 51, in <module>
    trainer.do_inference()
  File "/home/salik/AcT/utils/trainer.py", line 188, in do_inference
    self.model.load_weights(self.config['LOAD_MODEL'])
  File "/home/salik/anaconda3/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/home/salik/anaconda3/lib/python3.8/site-packages/keras/backend.py", line 4019, in batch_set_value
    x.assign(np.asarray(value, dtype=dtype_numpy(x)))
ValueError: Cannot assign value to variable ' dense_36/kernel:0': Shape mismatch.The variable shape (39, 192), and the assigned value shape (54, 192) are incompatible.

The micro is somewhat similar to the training results I'm getting, the large one is way off and the base model is getting an error.

Thank you so much for taking the time out and helping me understand your research better!

Best,
Salik

[simoneangarano]

Hi @saliknadeem ,
I deeply checked the code and found out the problem was in the data. I wonder why the latest versions of mpose yield slightly worse results. I will investigate that, and in the meantime, you can reproduce the results from the paper by specifying "openpose_legacy" as the dataset. Please also pull updated versions of both AcT and mpose. Let me know if you need any help!
Simone

[saliknadeem]

Hi @simoneangarano,

Thank you so much for all your help, and sorry for the late reply. I was able to reproduce the results with the latest changes. I just had to modify things just a little bit, had to change data folder to temp data_folder=f'tmp/openpose_legacy/{split}/' and use 'y_train.npy' with a smaller y but otherwise it worked perfectly!

Thank you!