Skip to content

danielzotti/fake-3d-website-next

Repository files navigation

Fake 3D Website

A 3D mode for a website using webcam and face detection

Dependencies

  • NextJs 14
  • TensorflowJs
  • MediaPipe Face Detection

Notes

Project idea

  • Create a hide & seek website where the user can enable 3D mode using the webcam in order to find hidden things
    • A famous 3D painting?
  • Alternative point of views:
    • Far away (frame?)
    • Very close (Keyhole?)
  • Recreate famous paintings in 3D

Project features

  • Enable webcam
    • Read webcam stream
    • Show video on page
    • Once the video is loaded, show "Enable 3D" button
  • Enable 3D
    • Load the model
    • Start face detection
    • Read right eye position
    • Calculate distance from the center
    • Apply translation coords to child elements (Box3D)
  • Disable 3D
    • Stop face detection
    • Stop video
    • Position content in the basic position (0,0)
  • Disable webcam
    • Hide video & Reset Stream
  • Enable mouse
    • Listen to mouse move events (change x and y position)
    • Listen to mouse wheel event (zoom in/out)
  • Disable mouse
    • Remove listeners (mouse move, mouse wheel)

TODO

  • Add local mediapipe detector solutionPath
  • Cache model using Service Worker (do not download model every time)
  • Add React Context in order to simplify the set of x and y point
  • Add/Improve depth using both eyes (it works only calculating the horizontal distance)
  • Create a more graphical example (Magritte paintings)
  • Add threshold/interpolation for face detection (view position buffer)
  • Make z (zoom) work not just if eyes are horizontal
  • Select webcam (in case of multiple webcams)
  • Improve UI
  • Improve performance (lighter images?)
  • Create an example of a real "standard" website with 3D mode

Thanks to