How get Iris_depth

[k0hh2]

How can I get Iris_depth?

And how do I run it at the same time as holistic tracking?

[ehsanwwe]

((distance between iris
+ distance between 2 ears 
+ distance between 2 noise
) / 3)
- normal value of the formula  when some one stand front the camera )

do you want other thing ?

[k0hh2]

Thanks.
I'm looking for a way to get const float left_iris_depth and right_iris_depth in a holistic scene.

iris_to_render_data_calculator.cc 188

  if (cc->Inputs().HasTag(kLeftIrisDepthTag) &&
      !cc->Inputs().Tag(kLeftIrisDepthTag).IsEmpty()) {
    const float left_iris_depth =
        cc->Inputs().Tag(kLeftIrisDepthTag).Get<float>();
    if (!std::isinf(left_iris_depth)) {
      line = "Left : ";
      absl::StrAppend(&line, ":", std::round(left_iris_depth / 10), " cm");
      lines.emplace_back(line);
    }
  }
  if (cc->Inputs().HasTag(kRightIrisDepthTag) &&
      !cc->Inputs().Tag(kRightIrisDepthTag).IsEmpty()) {
    const float right_iris_depth =
        cc->Inputs().Tag(kRightIrisDepthTag).Get<float>();
    if (!std::isinf(right_iris_depth)) {
      line = "Right : ";
      absl::StrAppend(&line, ":", std::round(right_iris_depth / 10), " cm");
      lines.emplace_back(line);
    }
  }

[ehsanwwe]

for left eye distance between landmark index 33 to 133
and for right eye distance between landmark index 362 to 363

[homuler]

@k0hh2 It looks like you are looking at iris_to_render_data_calculator.cc, but it is actually implemented in iris_to_depth_calculator.cc.

Currently, the sample app does not support the calculation of iris depths.
To calculate them, you need to implement it in C# or build native libraries with IrisToDepthCalculator and modify the graph config to use it.
Instructions on how to run Iris Tracking with Holistic can be found in #322 (comment).

[k0hh2]

@homuler
So that's it. Thank you very much.
Are there any plans to implement depth in the sample app in the future?
I will try it myself, but I find it difficult.

I know how to perform iris tracking holistic. thank you.

[k0hh2]

for left eye distance between landmark index 33 to 133 and for right eye distance between landmark index 362 to 363

I'm not sure, is this a way to calculate iris_depth from the landmark? thank you.

[homuler]

Are there any plans to implement depth in the sample app in the future?

If you want, I'll treat this issue as a feature request.
But it's not so difficult as it looks.

If we know the FoV and the focal length of the camera (these values need to be given beforehand or calculated somehow), we can calculate the focal length in pixels, since we know the resolution of the image.
Based on the landmark data, we can calculate the diameter of the iris in pixels, but if we make a reasonable assumption about the actual diameter of the iris (MediaPipe assumes that the average size of a human iris is 11.8mm), we can calculate millimeters per pixel (e.g. 11.8 / iris_in_pixels).

Finally, if you consider the focal length in pixels, you can calculate the distance from the lens to the iris (i.e. focal_length_in_pixels * mms_per_pixel).

[k0hh2]

@homuler
I want you to treat this feature as a feature request 🙏

Thank you for explaining in detail.
You can find the diameter of the iris from iris_landmark and know the distance to the camera from there.
I'm not sure about the FoV and focal length of the camera due to lack of study.

[Morgane-Tilak]

Hi,

First of all, thank you very much for this plugin.

I am trying to get the Iris depth using your plugin. I can launch your project on device and on Unity Editor and I create an empty project with your plugin to understand most of how the iris tracking works.

However, I am using the event OnFaceLandmarksWithIrisOutput which give me OutputEventArgs<NormalizedLandmarkList>. I don't really understand what this list of values means ?
If I understand well your explanation here, I need to calculate the diameter of the iris in pixels based on the landmark data. I think OnFaceLandmarksWithIrisOutput and the landmark data are related but I am stuck at this point.

Can you give me some help ?

[Morgane-Tilak]

Hello,

I figured everything out and want to share if anyone needs help.

First, we get a list of landmark positions for the face and iris detected in the OnFaceLandmarksWithIrisOutput event. In order to split up the large list of landmarks, we know that the first 468 values are for the face. Then there are 5 values for the first (left) iris, and the next 5 values are for the second iris. In the MediaPipe source code we can find these values here: Annotation\FaceLandmarkListWithIrisAnnotation

To calculate the depth with these landmarks, I do these following steps:

• We get the image size from the webcamtexture.

• We calculate the width and length of the iris using our landmarks and multiply them by the image size to convert them to mm. The Iris Diameter is calculated by summing these values and dividing by 2 to get their average.

• For each eye, we calculate the depth:

1. Thanks to the Unity method Vector2.Distance(), we calculate the distance between the iris center and the image center.
2. We calculate the distance between the displayed eye and the focal point with the Pythagorean theorem: sqrt(distance_eye_image^2 + focal_length^2).
3. MediaPipe assumes that the average size of the human iris is 11.8mm. With this assumption, we calculate the ratio mm per px.

The distance iris/camera in mm is the distance iris/camera in pixels calculated with Pythagore above, multiplied by the ratio mm per px.

I would like to thank you again for your API and your explanations. It allowed me to do what I wanted to do.