pose estimation accuracy

[wleisenr]

Search before asking

• I have searched the HUB issues and discussions and found no similar questions.

Question

We are experiencing some pose estimation accuracy issues particularly when the arm crosses past the head during a baseball pitching motion. See attached screenshots for example of the issue. Any recommendations for how to improve this? We have added some interpolation based on confidence which helps but this is so far off that it doesn't remediate the problem completely.
dvs.zip

Additional

No response

[github-actions]

👋 Hello @wleisenr, thank you for raising an issue about Ultralytics HUB 🚀! Please visit our HUB Docs to learn more:

• Quickstart. Start training and deploying YOLO models with HUB in seconds.
• Datasets: Preparing and Uploading. Learn how to prepare and upload your datasets to HUB in YOLO format.
• Projects: Creating and Managing. Group your models into projects for improved organization.
• Models: Training and Exporting. Train YOLOv5 and YOLOv8 models on your custom datasets and export them to various formats for deployment.
• Integrations. Explore different integration options for your trained models, such as TensorFlow, ONNX, OpenVINO, CoreML, and PaddlePaddle.
• Ultralytics HUB App. Learn about the Ultralytics App for iOS and Android, which allows you to run models directly on your mobile device.
  • iOS. Learn about YOLO CoreML models accelerated on Apple's Neural Engine on iPhones and iPads.
  • Android. Explore TFLite acceleration on mobile devices.
• Inference API. Understand how to use the Inference API for running your trained models in the cloud to generate predictions.

If this is a 🐛 Bug Report, please provide screenshots and steps to reproduce your problem to help us get started working on a fix.

If this is a ❓ Question, please provide as much information as possible, including dataset, model, environment details etc. so that we might provide the most helpful response.

We try to respond to all issues as promptly as possible. Thank you for your patience!

[pderrenger]

Hello! Thanks for reaching out with your issue on pose estimation accuracy during dynamic motions like a baseball pitch. 🚀

It sounds like the model might be struggling with occlusions and fast movements typical in sports actions. Here are a couple of suggestions that might help improve the accuracy:

1. Data Augmentation: If not already done, consider augmenting your training dataset with more examples of similar poses and motions, especially where limbs overlap or move rapidly.

2. Model Fine-tuning: If possible, fine-tune the model on a dataset that includes more sports actions, particularly baseball pitching, to help the model better learn these specific movements.

3. Post-processing: Since you've tried interpolation, consider also implementing more advanced smoothing techniques that can handle sudden changes in pose estimation, like Kalman filters or moving average filters.

4. Increase Model Complexity: If computational resources allow, using a more complex model might capture dynamics better.

Here's a quick example of how you might implement a simple moving average for smoothing:

import numpy as np

def smooth_pose(predictions, window_size=3):
    return np.convolve(predictions, np.ones(window_size)/window_size, mode='valid')

# Example usage with dummy data
predicted_poses = np.array([10, 12, 15, 20, 28, 18, 15, 14, 13, 12])
smoothed_poses = smooth_pose(predicted_poses)
print(smoothed_poses)

Hope this helps! Let us know how it goes or if you have further questions.

[github-actions]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐