adding postprocessing improvements

[aaroncherian]

No description provided.

[jonmatthis]

I'm was about ready to approve your branch, but when I tested with a recording I made while sitting down, it worked but returned a bunch of wonky data for the legs and whatnot

I think that's also the source of all the mean of an empty slice warning that @philip_q mentioned above

Questions;

• Is this a new behavior that comes from skellyforge? Or did the old post_processing methods also have this issue?

• If so, can you check for the (likely sloppy) method we used to fix it in the previous code and think about re-implementing it in either skellyforge or freemocap? (If it's a sloppy method, consider something new, like a reprojection_error filter (see below)

Possible causes/fixes

• it's possible that mediapipe is still assigning my legs a confidence value greater than 0.5 even if it doesn't ever seem them. Could be fixed by making the medipaipe_confidence threshold more restrictive (i.e. put the default at .7 instead of .5, though I know that caused issues before...)

• Could be fixed by implementing some "reprojection error filter" (either in skellyforge or freemocap) that replaces wackadoo 3d data points with nans. In order to avoid the "over filtering" issue we ran into a while ago, we could could be made to be very permissive, say only filtering dottos that are greater than 5-10*the standard deviation of the rest of the projection error data

[jonmatthis]

Apparently that issue mentioned in the previous review has been a problem for a while so we can approve this PR and work on fixing that issue in the future!

[aaroncherian]

I'm was about ready to approve your branch, but when I tested with a recording I made while sitting down, it worked but returned a bunch of wonky data for the legs and whatnot

I think that's also the source of all the mean of an empty slice warning that @philip_q mentioned above

Questions;

• Is this a new behavior that comes from skellyforge? Or did the old post_processing methods also have this issue?
• If so, can you check for the (likely sloppy) method we used to fix it in the previous code and think about re-implementing it in either skellyforge or freemocap? (If it's a sloppy method, consider something new, like a reprojection_error filter (see below)

Possible causes/fixes

• it's possible that mediapipe is still assigning my legs a confidence value greater than 0.5 even if it doesn't ever seem them. Could be fixed by making the medipaipe_confidence threshold more restrictive (i.e. put the default at .7 instead of .5, though I know that caused issues before...)
• Could be fixed by implementing some "reprojection error filter" (either in skellyforge or freemocap) that replaces wackadoo 3d data points with nans. In order to avoid the "over filtering" issue we ran into a while ago, we could could be made to be very permissive, say only filtering dottos that are greater than 5-10*the standard deviation of the rest of the projection error data

So I think the main underlying issue here is one we had talked about awhile ago actually - which is that the interpolation/filtering was mainly built for our research purposes, where we assumed a full body view would be used. Essentially, the Butterworth Filter won't work if there are any NaN values - so the interpolation function does two things

1. it interpolates between the data it can see, and then
2. replaces any NaN values left over with mean values of the array at that time (in our research recordings, this is usually at the start and end of the recording when the person is walking into/out of frame)

Both of these are needed to get through the butterworth filtering.

However, when you have a 'sitting at your desk' recording - where the legs are never seen, this causes the second step to go a little haywire in filling in all of those NaN values. This is also what causes the 'mean of an empty slice' error.

So basically - the interpolation/filter functions were made to work on 'whole body' skeletons, since that's what we needed it for in research, and if we want it to work for something like a 'sitting at a desk' recording, we would need a different/smarter way about determining what joints get fed into interpolating/filtering.