Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JS Mediapipe face recognition crashes badly #3950

Closed
edge7 opened this issue Dec 24, 2022 · 7 comments
Closed

JS Mediapipe face recognition crashes badly #3950

edge7 opened this issue Dec 24, 2022 · 7 comments
Assignees
@ayushgdev
Labels
legacy:face detection Issues related to Face Detection platform:javascript MediaPipe Javascript issues type:docs-bug Doc issues for any errors or broken links or rendering

Comments

@edge7
Copy link

edge7 commented Dec 24, 2022

System information

  • Os Linux ed7-laptop 5.14.0-1027-oem #30-Ubuntu SMP Mon Mar 7 15:00:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
  • Programming language: Javascript
  • Chrome version: Google Chrome 108.0.5359.124

Describe the current behavior:
When trying to run the example taken from the official website:

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/camera_utils/camera_utils.js" crossorigin="anonymous"></script>
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/control_utils/control_utils.js" crossorigin="anonymous"></script>
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/drawing_utils/drawing_utils.js" crossorigin="anonymous"></script>
  <script src="https://cdn.jsdelivr.net/npm/@mediapipe/face_detection/face_detection.js" crossorigin="anonymous"></script>

</head>

<body>
  <div class="container">
    <video class="input_video"></video>
    <canvas class="output_canvas" width="1280px" height="720px"></canvas>
  </div>
</body>
</html>
<script type="module">
const videoElement = document.getElementsByClassName('input_video')[0];
const canvasElement = document.getElementsByClassName('output_canvas')[0];
const canvasCtx = canvasElement.getContext('2d');

function onResults(results) {
  // Draw the overlays.
  canvasCtx.save();
  canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);
  canvasCtx.drawImage(
      results.image, 0, 0, canvasElement.width, canvasElement.height);
  if (results.detections.length > 0) {
    drawingUtils.drawRectangle(
        canvasCtx, results.detections[0].boundingBox,
        {color: 'blue', lineWidth: 4, fillColor: '#00000000'});
    drawingUtils.drawLandmarks(canvasCtx, results.detections[0].landmarks, {
      color: 'red',
      radius: 5,
    });
  }
  canvasCtx.restore();
}

const faceDetection = new FaceDetection({locateFile: (file) => {
  return `https://cdn.jsdelivr.net/npm/@mediapipe/face_detection@0.4/${file}`;
}});
faceDetection.setOptions({
  modelSelection: 0,
  minDetectionConfidence: 0.5
});
faceDetection.onResults(onResults);

const camera = new Camera(videoElement, {
  onFrame: async () => {
    await faceDetection.send({image: videoElement});
  },
  width: 1280,
  height: 720
});
camera.start();
</script>

I get the following crash:
image

Describe the expected behavior:
It should work, I guess.

Thanks
@edge7 edge7 added the type:support General questions label Dec 24, 2022
@edge7 edge7 changed the title Mediapipe face recognition crashes badly JS Mediapipe face recognition crashes badly Dec 24, 2022
@ayushgdev ayushgdev self-assigned this Dec 26, 2022
@ayushgdev ayushgdev added platform:javascript MediaPipe Javascript issues legacy:face detection Issues related to Face Detection labels Dec 26, 2022
@ayushgdev
Copy link
Collaborator

Hi @edge7
Thanks for pointing out this issue. We can notice the documentation is not up to date and that is why you are facing this issue.
Please do two changes for the code to work.

  1. Remove modelSelection option and add model: 'short', option to faceDetection.setOptions(). This will give Reference error for drawingUtils variable.
  2. Add const drawingUtils = window; before function onResults(){} to define the drawingUtils variable

We will update the documentation code with the changes shortly.

@ayushgdev ayushgdev added stat:awaiting response Waiting for user response type:docs-bug Doc issues for any errors or broken links or rendering and removed type:support General questions labels Dec 26, 2022
@esinanturan
Copy link

@ayushgdev Is project dead or something ? I am checking documentations but there isn't example codes for given image examples and there is no source code for javascript API but only a npm package which was updated 1 year ago. I guess project having some issues to have contributor support.

@ayushgdev
Copy link
Collaborator

Hi @esinanturan
For the Javascript API, we have released the Typescript type definition files which should give an idea on the possible options and APIs. For full source code release, please check the comment here.
Rest assured, the project is certainly not dead and we are working on actually streamlining the build process for the solutions. Hence, though you might not see many features being added, small important improvements are on the way.

@esinanturan
Copy link

@ayushgdev Thank you for your response. I am very interested in the library but seeing not enough resources making me feel bad to achieve what I am looking for which is the given example of face filters: https://mediapipe.dev/images/face_mesh_ar_effects.gif
It seems mediapipe doing great job for that purpose but I don't know how to achieve it by the given documentation.
Any code examples for such result ?

@ayushgdev
Copy link
Collaborator

Javascript FaceEffect is not supported at the moment. As you can find here in the documentation, the face effect example targets are only available for Android and iOS.

@esinanturan
Copy link

I understand. Thank you for clarification. I hope there will be a support for that soon.

@edge7
Copy link
Author

edge7 commented Jan 4, 2023

I confirm that like this works:

const faceDetection = new FaceDetection({locateFile: (file) => {
  return `https://cdn.jsdelivr.net/npm/@mediapipe/face_detection@0.4/${file}`;
}});
faceDetection.onResults(onResults);
faceDetection.setOptions({
  model: 'short'
});

@edge7 edge7 closed this as completed Jan 4, 2023
@ayushgdev ayushgdev mentioned this issue Jan 11, 2023
Closed
@kuaashish kuaashish removed the stat:awaiting response Waiting for user response label Feb 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ayushgdev ayushgdev

Labels
legacy:face detection Issues related to Face Detection platform:javascript MediaPipe Javascript issues type:docs-bug Doc issues for any errors or broken links or rendering
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@edge7 @esinanturan @kuaashish @ayushgdev