Keypoints to check for New training ?

[angenieux]

Hi, thanks for this huge work!!

My intend is to test your solution on a JETSON NANO (inference only)-->
Your pre-trained model runs at 3.4 FPS with realtime webcam video input (224x224, mobilenet_v2) on my device = Great

I've prepared my own training set of images based on video, but the movie rate is 25FPS (not 10 FPS).
As you used a 5 Frames sliding windows, what are the keypoints to check in the code :
-For training ?
-For Inference on nano?

Thanks

[felixchenfy]

@angenieux Hi, the size of the sliding window is defined in this script src/mylib/feature_proc.py.

Its constructor is shown below. You can see the parameter FEATURE_T_LEN=5, which is the window size:

class FeatureGenerator(object):
    def __init__(self, config_add_noise=False):
        self.reset()
        self.FEATURE_T_LEN = 5
        self.config_add_noise = config_add_noise

This class is used for extracting features in both training and inference. So this FEATURE_T_LEN is the only thing you need to change.

[angenieux]

HI !

I would like to evaluate the prediction results regarding the number of frames used for the temporal windows.

Does the FEATURE_T_LEN values must be the same for training AND inference ?

I have trained my data with FEATURE_T_LEN=5.
When i try to make predictions (python3 run_detector.py --source webcam) with another value (example FEATURE_T_LEN=3) I get the following issue :

Traceback (most recent call last):python3 run_detector.py --source webcam

File "run_detector.py", line 339, in
dict_id2label = multipeople_classifier.classify(dict_id2skeleton)
File "run_detector.py", line 281, in classify
id2label[id] = classifier.predict(skeleton) # predict label
File "/home/nico/Realtime-Action-Recognition/src/mylib/action_classifier.py", line 64, in predict
curr_scores = self.model.predict_proba(features)[0]
File "/home/nico/Realtime-Action-Recognition/src/mylib/action_classifier.py", line 170, in predict_proba
Y_probs = self.clf.predict_proba(self.pca.transform(X))
File "/usr/local/lib/python3.6/dist-packages/sklearn/decomposition/base.py", line 129, in transform
X = X - self.mean_
ValueError: operands could not be broadcast together with shapes (1,170) (314,)

Thanks for your help

[felixchenfy]

@angenieux Hi, the training and inference must use the same length of feature vector. So the FEATURE_T_LEN must be the same.

[angenieux]

Hi
Thanks for your reply :)

One last question.
1- I Have a 25 FPS video input
2- My inference capabilities with jetson nano is around 5 FPS
3- I would like to track an action duration < 1s (0.5s to 2 s)

Can you give me any advice regarding the FEATURE_T_LEN value to use for training and inference ?

best regards

[felixchenfy]

@angenieux Hi.

First of all, since your inference speed is 5 fps, you need to lower down the video input to 5 fps (no matter how fast the original video streaming is). You can do this either by triggering the video frame at 5 fps, or manually sampling the video frame at 5 fps.

As for the FEATURE_T_LEN, since your shortest action is only 0.5s, which corresponding to only 0.5s*5fps=2.5 frame. So you may use 2 frame for FEATURE_T_LEN.

[felixchenfy]

@angenieux Hi, I just refactored the code to make it more readable, as well as making the API easier to use. Just for your information. Thanks for the support!

The FEATURE_T_LEN is now renamed as window_size and is set in the configuration file config/config.yaml. All other parameters and intput/output settings are also in this yaml file.