[Doc] Add rtmlib examples

projects/rtmo/README.md

@@ -13,6 +13,16 @@ RTMO is a one-stage pose estimation model that achieves performance comparable t
 python demo/inferencer_demo.py $IMAGE --pose2d rtmo --vis-out-dir vis_results
 ```
 
+**rtmlib demo**
+
+[rtmlib](https://github.com/Tau-J/rtmlib/tree/main) provides simple and easy-to-use API for inference with RTMPose models.
+
+- Support OpenCV/ONNXRuntime/OpenVINO inference and does not require Pytorch or MMCV.
+- Super user-friendly API for inference and visualization.
+- Support both CPU and GPU inference.
+- Automatically download onnx models from OpenMMLab model zoo.
+- Support all series of RTMPose models (RTMPose, DWPose, RTMO, RTMW etc.)
+
 ## 📜 Introduction
 
 Real-time multi-person pose estimation presents significant challenges in balancing speed and precision. While two-stage top-down methods slow down as the number of people in the image increases, existing one-stage methods often fail to simultaneously deliver high accuracy and real-time performance. This paper introduces RTMO, a one-stage pose estimation framework that seamlessly integrates coordinate classification by representing keypoints using dual 1-D heatmaps within the YOLO architecture, achieving accuracy comparable to top-down methods while maintaining high speed. We propose a dynamic coordinate classifier and a tailored loss function for heatmap learning, specifically designed to address the incompatibilities between coordinate classification and dense prediction models. RTMO outperforms state-of-the-art one-stage pose estimators, achieving 1.1% higher AP on COCO while operating about 9 times faster with the same backbone. Our largest model, RTMO-l, attains 74.8% AP on COCO val2017 and 141 FPS on a single V100 GPU, demonstrating its efficiency and accuracy.

projects/rtmpose/README.md

@@ -91,6 +91,7 @@ ______________________________________________________________________
   |   s   |   72.2   |  200+   |  710+   |
   |   m   |   75.8   |   90+   |  430+   |
   |   l   |   76.5   |   50+   |  280+   |
+  | l-384 |   78.3   |    -    |  160+   |
 
 - 🛠️ **Easy to deploy**
 
@@ -460,13 +461,24 @@ We also provide the ImageNet classification pre-trained weights of the CSPNeXt b
 
 ## 😎 Get Started [🔝](#-table-of-contents)
 
-We provide two appoaches to try RTMPose:
+We provide following appoaches to try RTMPose:
 
 - [Online RTMPose Demo](https://openxlab.org.cn/apps/detail/mmpose/RTMPose)
 - [Examples](https://github.com/open-mmlab/mmpose/tree/dev-1.x/projects/rtmpose/examples/onnxruntime) based on Python and ONNXRuntime (without mmcv)
+- [rtmlib](https://github.com/Tau-J/rtmlib/tree/main) （without mmcv, pytorch）
 - MMPose demo scripts (based on Pytorch)
 - Pre-compiled MMDeploy SDK (Recommended, 6-10 times faster)
 
+## rtmlib
+
+[rtmlib](https://github.com/Tau-J/rtmlib/tree/main) provides simple and easy-to-use API for inference with RTMPose models.
+
+- Support OpenCV/ONNXRuntime/OpenVINO inference and does not require Pytorch or MMCV.
+- Super user-friendly API for inference and visualization.
+- Support both CPU and GPU inference.
+- Automatically download onnx models from OpenMMLab model zoo.
+- Support all series of RTMPose models (RTMPose, DWPose, RTMO, RTMW etc.)
+
 ### MMPose demo scripts
 
 MMPose provides demo scripts to conduct [inference with existing models](https://mmpose.readthedocs.io/en/latest/user_guides/inference.html).

projects/rtmpose/README_CN.md

@@ -87,6 +87,7 @@ ______________________________________________________________________
   |   s   |   72.2   |  200+   |  710+   |
   |   m   |   75.8   |   90+   |  430+   |
   |   l   |   76.5   |   50+   |  280+   |
+  | l-384 |   78.3   |    -    |  160+   |
 
 - 🛠️ **易部署**
 
@@ -451,13 +452,24 @@ RTMPose 是一个长期优化迭代的项目，致力于业务场景下的高性
 
 ## 😎 快速尝试 [🔝](#-table-of-contents)
 
-我们提供了两种途径来让用户尝试 RTMPose 模型：
+我们提供了多种途径来让用户尝试 RTMPose 模型：
 
 - [在线 RTMPose Demo](https://openxlab.org.cn/apps/detail/mmpose/RTMPose)
 - [Examples](https://github.com/open-mmlab/mmpose/tree/dev-1.x/projects/rtmpose/examples/onnxruntime) 基于 Python 和 ONNXRuntime （无 MMCV 依赖）
+- [rtmlib 推理库](https://github.com/Tau-J/rtmlib/tree/main) （无 MMCV、Pytorch 依赖）
 - MMPose demo 脚本 （基于 Pytorch）
 - MMDeploy SDK 预编译包 （推荐，速度提升6-10倍）
 
+### rtmlib 推理库
+
+[rtmlib](https://github.com/Tau-J/rtmlib/tree/main) 提供了开箱即用的 RTMPose 全系列官方及衍生模型的推理：
+
+- 无需安装 mmcv，mmengine，mmpose 等一系列训练库，无需 pytorch 环境，有 opencv 就能推理
+- 超级友好简洁的推理和可视化接口
+- 支持 CPU 和 GPU 推理
+- 自动下载和缓存 onnx 模型
+- 支持 RTMPose 全系列官方及衍生模型：RTMPose，DWPose，RTMO，RTMW etc.
+
 ### MMPose demo 脚本
 
 通过 MMPose 提供的 demo 脚本可以基于 Pytorch 快速进行[模型推理](https://mmpose.readthedocs.io/en/latest/user_guides/inference.html)和效果验证。

projects/rtmpose/examples/README.md

@@ -16,3 +16,9 @@ This example shows how to run RTMPose inference with ONNXRuntime in Python.
 PoseTracker Android Demo Prototype based on mmdeploy.
 
 - [Original Repository](https://github.com/hanrui1sensetime/PoseTracker-Android-Prototype)
+
+### 4. rtmlib
+
+rtmlib is a super lightweight library to conduct pose estimation based on RTMPose models WITHOUT any dependencies like mmcv, mmpose, mmdet, etc.
+
+- [Original Repository](https://github.com/Tau-J/rtmlib/tree/main)

projects/rtmpose/examples/rtmlib/README.md

@@ -0,0 +1,124 @@
+# rtmlib
+
+![demo](https://github.com/Tau-J/rtmlib/assets/13503330/b7e8ce8b-3134-43cf-bba6-d81656897289)
+
+rtmlib is a super lightweight library to conduct pose estimation based on [RTMPose](https://github.com/open-mmlab/mmpose/tree/dev-1.x/projects/rtmpose) models **WITHOUT** any dependencies like mmcv, mmpose, mmdet, etc.
+
+Basically, rtmlib only requires these dependencies:
+
+- numpy
+- opencv-python
+- opencv-contrib-python
+- onnxruntime
+
+Optionally, you can use other common backends like opencv, onnxruntime, openvino, tensorrt to accelerate the inference process.
+
+- For openvino users, please add the path `<your python path>\envs\<your env name>\Lib\site-packages\openvino\libs` into your environment path.
+
+## Installation
+
+- install from pypi:
+
+```shell
+pip install rtmlib -i https://pypi.org/simple
+```
+
+- install from source code:
+
+```shell
+git clone https://github.com/Tau-J/rtmlib.git
+cd rtmlib
+
+pip install -r requirements.txt
+
+pip install -e .
+
+# [optional]
+# pip install onnxruntime-gpu
+# pip install openvino
+
+```
+
+## Quick Start
+
+Run `webui.py`:
+
+```shell
+# Please make sure you have installed gradio
+# pip install gradio
+
+python webui.py
+```
+
+![image](https://github.com/Tau-J/rtmlib/assets/13503330/49ef11a1-a1b5-4a20-a2e1-d49f8be6a25d)
+
+Here is also a simple demo to show how to use rtmlib to conduct pose estimation on a single image.
+
+```python
+import cv2
+
+from rtmlib import Wholebody, draw_skeleton
+
+device = 'cpu'  # cpu, cuda
+backend = 'onnxruntime'  # opencv, onnxruntime, openvino
+img = cv2.imread('./demo.jpg')
+
+openpose_skeleton = False  # True for openpose-style, False for mmpose-style
+
+wholebody = Wholebody(to_openpose=openpose_skeleton,
+                      mode='balanced',  # 'performance', 'lightweight', 'balanced'. Default: 'balanced'
+                      backend=backend, device=device)
+
+keypoints, scores = wholebody(img)
+
+# visualize
+
+# if you want to use black background instead of original image,
+# img_show = np.zeros(img_show.shape, dtype=np.uint8)
+
+img_show = draw_skeleton(img_show, keypoints, scores, kpt_thr=0.5)
+
+
+cv2.imshow('img', img_show)
+cv2.waitKey()
+```
+
+### Visualization
+
+|                                            MMPose-style                                             |                                            OpenPose-style                                             |
+| :-------------------------------------------------------------------------------------------------: | :---------------------------------------------------------------------------------------------------: |
+| <img width="357" alt="result" src="https://github.com/Tau-J/rtmlib/assets/13503330/c9e6fbaa-00f0-4961-ac87-d881edca778b"> | <img width="357" alt="result" src="https://github.com/Tau-J/rtmlib/assets/13503330/9afc996a-59e6-4200-a655-59dae10b46c4"> |
+| <img width="357" alt="result" src="https://github.com/Tau-J/rtmlib/assets/13503330/b12e5f60-fec0-42a1-b7b6-365e93894fb1"> | <img width="357" alt="result" src="https://github.com/Tau-J/rtmlib/assets/13503330/5acf7431-6ef0-44a8-ae52-9d8c8cb988c9"> |
+| <img width="357" alt="result" src="https://github.com/Tau-J/rtmlib/assets/13503330/091b8ce3-32d5-463b-9f41-5c683afa7a11"> | <img width="357" alt="result" src="https://github.com/Tau-J/rtmlib/assets/13503330/4ffc7be1-50d6-44ff-8c6b-22ea8975aad4"> |
+| <img width="357" alt="result" src="https://github.com/Tau-J/rtmlib/assets/13503330/6fddfc14-7519-42eb-a7a4-98bf5441f324"> | <img width="357" alt="result" src="https://github.com/Tau-J/rtmlib/assets/13503330/2523e568-e0c3-4c2e-8e54-d1a67100c537"> |
+
+### Citation
+
+```
+@misc{rtmlib,
+  title={rtmlib},
+  author={Jiang, Tao},
+  year={2023},
+  howpublished = {\url{https://github.com/Tau-J/rtmlib}},
+}
+
+@misc{jiang2023,
+  doi = {10.48550/ARXIV.2303.07399},
+  url = {https://arxiv.org/abs/2303.07399},
+  author = {Jiang, Tao and Lu, Peng and Zhang, Li and Ma, Ningsheng and Han, Rui and Lyu, Chengqi and Li, Yining and Chen, Kai},
+  keywords = {Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences},
+  title = {RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose},
+  publisher = {arXiv},
+  year = {2023},
+  copyright = {Creative Commons Attribution 4.0 International}
+}
+
+@misc{lu2023rtmo,
+      title={{RTMO}: Towards High-Performance One-Stage Real-Time Multi-Person Pose Estimation},
+      author={Peng Lu and Tao Jiang and Yining Li and Xiangtai Li and Kai Chen and Wenming Yang},
+      year={2023},
+      eprint={2312.07526},
+      archivePrefix={arXiv},
+      primaryClass={cs.CV}
+}
+```