Information about Face Output

[RiRyuichi]

Dear Rajivo,

Thanks for your code and troubleshooting, now I can run with Intel RealSense camera.
Also I edit run.launch file as shown below-
<arg name="openpose_args" value="--face"/>
to show my face keypoints. But I need some help.

I want to output coordinates of face keypoints. Maybe I need to edit
text = [bodyPart.pixel for person in msg.persons for bodyPart in person.bodyParts] in echo.py file, but I don't know how.

Could you give some advice please?

[ravijo]

@RiRyuuichi

I am glad to see your message. Thank you.

Yes, you are right. You need to edit the following line:

ros_openpose/scripts/echo.py

Line 14 in c6f5ce7

text = [bodyPart.pixel for person in msg.persons for bodyPart in person.bodyParts]

However, first, you should check the /frame topic to confirm that face key points are there. I have not used it with face key points, but some users did in the past.

Next, you should figure out the mapping of face key points in the topic. For example, please see the below mapping for complete body tracking:

ros_openpose/scripts/visualizer.py

Lines 43 to 60 in c6f5ce7

	See the link below to get the id of each joint as defined in Kinect v2
	src: https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/output.md#keypoint-ordering
	Result for BODY_25 (25 body parts consisting of COCO + foot)
	const std::map<unsigned int, std::string> POSE_BODY_25_BODY_PARTS {
	{ 0, "Nose"}, {13, "LKnee"}
	{ 1, "Neck"}, {14, "LAnkle"}
	{ 2, "RShoulder"}, {15, "REye"}
	{ 3, "RElbow"}, {16, "LEye"}
	{ 4, "RWrist"}, {17, "REar"}
	{ 5, "LShoulder"}, {18, "LEar"}
	{ 6, "LElbow"}, {19, "LBigToe"}
	{ 7, "LWrist"}, {20, "LSmallToe"}
	{ 8, "MidHip"}, {21, "LHeel"}
	{ 9, "RHip"}, {22, "RBigToe"}
	{10, "RKnee"}, {23, "RSmallToe"}
	{11, "RAnkle"}, {24, "RHeel"}
	{12, "LHip"}, {25, "Background"}

Finally, you can use conditional statement to print only face key points based on the index.

Let me know how it goes!

Cheers.

[RiRyuichi]

Dear Rajivo,

Thank you for the prompt reply.
I checked the /frame topic and it repeatedly showed :

header: 
  seq: 69
  stamp: 
    secs: 1667797310
    nsecs: 784934479
  frame_id: "camera_color_optical_frame"
persons: 
  - 
    bodyParts: 
      - 
        score: 0.79394453764
        pixel: 
          x: 444.27545166
          y: 210.642166138
        point: 
          x: 0.121169157326
          y: -0.0344984829426
          z: 0.658000051975
      - 
        score: 0.56282800436
        pixel: 
          x: 403.805603027
          y: 326.811676025
        point: 
          x: 0.0890130102634
          y: 0.102006264031
          z: 0.75100004673
      - 
        score: 0.327622026205
        pixel: 
          x: 262.836120605
          y: 341.170684814
        point: 
          x: -0.0823214277625
          y: 0.119004175067
          z: 0.748000025749
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.444403111935
        pixel: 
          x: 540.85559082
          y: 308.534851074
        point: 
          x: 0.282483309507
          y: 0.0880472958088
          z: 0.82900005579
      - 
        score: 0.237627774477
        pixel: 
          x: 590.471923828
          y: 454.699707031
        point: 
          x: 0.312954962254
          y: 0.254904001951
          z: 0.743000030518
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.848506808281
        pixel: 
          x: 412.987182617
          y: 180.612442017
        point: 
          x: 0.159630015492
          y: -0.0722588449717
          z: 0.686000049114
      - 
        score: 0.839394986629
        pixel: 
          x: 474.21484375
          y: 175.371795654
        point: 
          x: 0.159628242254
          y: -0.0751760751009
          z: 0.686000049114
      - 
        score: 0.814783334732
        pixel: 
          x: 350.308013916
          y: 187.127853394
        point: 
          x: 0.0234853327274
          y: -0.0669053867459
          z: 0.739000022411
      - 
        score: 0.484164863825
        pixel: 
          x: 497.76574707
          y: 183.194992065
        point: 
          x: 0.185554116964
          y: -0.0663822293282
          z: 0.685000061989
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
      - 
        score: 0.0
        pixel: 
          x: 0.0
          y: 0.0
        point: 
          x: -1.91599595547
          y: -1.40698289871
          z: 3.57300019264
    leftHandParts: []
    rightHandParts: []
---

It looks like only showed bodyParts key points, and support for leftHandParts and rightHandParts. But not support for face key points...
Also I checked visualizer.py file, there are some contents about bodyParts, leftHandParts and rightHandParts, but nothing about face parts.
Should I add something in visualizer.py file?

[ravijo]

@RiRyuuichi

Thank you very much. As I said, I never used --face flag. Therefore, it could be possible that it is not implemented. But some users have used it previously for face as well.

Please see Person definition, i.e., Person.msg below:

ros_openpose/msg/Person.msg

Lines 1 to 5 in c6f5ce7

	# A person has some body parts. That is why we have created
	# an array of body parts.
	BodyPart[] bodyParts
	BodyPart[] leftHandParts
	BodyPart[] rightHandParts

You should add one more attribute say BodyPart[] faceParts and then fill this field. You can easily take help from the existing code. Please see how leftHandParts and rightHandParts are filled.

Let me know your progress.

[RiRyuichi]

Dear Rajivo,

Thank you for your information.
I edited Person.msg, rosOpenposeAsync.cpp and rosOpenposeSync.cpp files, and run with
text = [facePart.pixel for person in msg.persons for facePart in person.faceParts] in run.lauch.
Now I can output coordinates of face keypoints as shown below-

[INFO] [1667808734.583089]: [x: 309.166168213
y: 184.518478394, x: 311.559112549
y: 203.063949585, x: 315.14855957
y: 218.019973755, x: 320.532745361
y: 233.574249268, x: 328.309875488
y: 246.735549927, x: 338.479980469
y: 260.49508667, x: 349.846557617
y: 270.66519165, x: 364.204315186
y: 279.638793945, x: 379.758575439
y: 280.237030029, x: 393.518127441
y: 275.45111084, x: 403.688232422
y: 265.879241943, x: 412.063598633
y: 255.110916138, x: 419.84072876
y: 241.949615479, x: 424.028411865
y: 228.788314819, x: 427.019622803
y: 215.62701416, x: 428.216094971
y: 200.67098999, x: 427.019622803
y: 186.313201904, x: 331.301086426
y: 164.776535034, x: 340.872924805
y: 158.794128418, x: 353.435974121
y: 155.80291748, x: 364.204315186
y: 159.392364502, x: 374.97265625
y: 164.17829895, x: 388.732208252
y: 164.17829895, x: 397.705810547
y: 158.794128418, x: 407.277679443
y: 155.80291748, x: 416.849517822
y: 157.597640991, x: 421.635467529
y: 165.374771118, x: 382.749786377
y: 179.13432312, x: 382.749786377
y: 186.911453247, x: 382.749786377
y: 194.09034729, x: 383.348022461
y: 203.063949585, x: 371.981445312
y: 217.421737671, x: 377.365631104
y: 218.019973755, x: 382.749786377
y: 218.618225098, x: 387.535736084
y: 218.019973755, x: 392.321655273
y: 217.421737671, x: 344.462371826
y: 180.330795288, x: 350.444793701
y: 179.732559204, x: 358.820159912
y: 179.732559204, x: 364.204315186
y: 183.92024231, x: 358.820159912
y: 183.92024231, x: 349.846557617
y: 183.92024231, x: 396.509338379
y: 183.322006226, x: 402.491760254
y: 179.732559204, x: 408.474151611
y: 179.732559204, x: 415.05480957
y: 180.929046631, x: 409.072387695
y: 183.92024231, x: 402.491760254
y: 183.92024231, x: 363.606079102
y: 241.351379395, x: 369.588500977
y: 232.975997925, x: 377.963867188
y: 228.190078735, x: 382.749786377
y: 230.58303833, x: 388.133972168
y: 228.190078735, x: 394.116363525
y: 232.975997925, x: 398.304046631
y: 241.351379395, x: 393.518127441
y: 246.735549927, x: 388.133972168
y: 250.324981689, x: 382.749786377
y: 250.324981689, x: 377.963867188
y: 250.324981689, x: 368.990264893
y: 246.735549927, x: 368.391998291
y: 241.351379395, x: 377.963867188
y: 237.761932373, x: 382.749786377
y: 237.761932373, x: 387.535736084
y: 237.761932373, x: 395.911102295
y: 240.753128052, x: 387.535736084
y: 237.761932373, x: 382.749786377
y: 238.958404541, x: 377.963867188
y: 238.360168457, x: 354.034240723
y: 180.929046631, x: 405.482940674
y: 180.330795288]

The accuracy of the numerical value has not been confirmed yet, but I think the mapping of face key points in the topic will just like be official format.

Thank you very much! :-)

[ravijo]

@RiRyuuichi

I am glad to see your speedy progress. Well done!!!

May I request you create a pull request to share your code? In this way, we can merge it with this repository. This helpful information can be made available to others.

Cheers!

[RiRyuichi]

Dear Rajivo,

I tried to create a pull request to share codes. I'm sorry that I'm a github beginner...
Is this alright?

[ravijo]

Thanks @RiRyuichi

It is perfect 👍

The pull request is merged. afecab6

Thanks again for the great contribution.