V1.1

New additions:

1. Rotating camera, change parameter to add this
2. Linear interpolation instead of relying on just kalman prediction
3. Able to handle more configs
4. Fix issue with rotation and parenting, can change setting in yaml file

Issues

1. The parenting of object is very dependent on the SMPL pose. So if the SMPL pose is wrong, the object's transformations are not very accurate and may look even worse. Reason for this is the first frame the object is detected to be attached to a joint (distance check) it will use that offset. But if the initial pose causes the joint in that frame to be at an inaccurate position, then the offset will be inaccurate.
2. Not being able to detect fast moving objects for videos with low frame rate. The application heavily relies on object detector and tracking for rendering objects. Without those it is hard to accurately render the object and attach it to the correct humans. Using a 60fps video with 4k resolution seemed to make the detection and tracking more accurate
3. When the camera rotates, keypoint attachment issues is more prominent. It is hard to tell depth based on a monocular video, which makes it hard to tell which hand is interacting with the ball

V1.0

What this release can do

1. Able to render the ball at the correct position
2. Able to 'attach' to the correct hand

This release has some issues

When using this release, make sure the Renderer repo is at ver1.0 tag commit.

broken attachment

• Calculations are wrong for the quaternions resulting in very little rotation or weird rotations)
• For attachment, the translation offset is using the 2D joints2D_img_coord and not the joints 3D position

Use Kalman prediction algorithm for interpolation
Object detector is not able to detect fast-moving objects and Deepsort is not able to match certain objects to previously detected objects, resulting in the same object having different IDs. Since in this release we made it so deepsort will 'interpolate' objects by not destroying them for a few frames, it results in numerous objects spawning.

Videos uploaded

See videos uploaded to showcase issue

PassBallTwoPeople: Passing ball between two people. Slow motion. Can see kalman prediction working in certain frames where the ball movement is not as accurate

ThrowingBallBetweenTwoPeople: A 3d render of two people throwing and catching a ball between each other. However, can see the ball disappears multiple time and even multiple balls being spawned. Reason is the ball is hard to track when moving at fast speeds, the object tracker is not able to associate the current ball detected with previous's frame balls. So it tries to assign a new ID, but still update and predict the previous ball's position via kalman prediction.

ThrowingBallBetweenTwoPeople_objTrackingAndDetection: It is showcasing the objects detected and tracked in ThrowingBallBetweenTwoPeople with bounding boxes. The blue box is the last position detected by the object detector, green boxes are the positions tracked and predicted by deepsort. Black boxes are detected objects with too little frames and are removed.