Skip to content
3D virtual puppet remote controlled via smartphone
Branch: master
Clone or download

Latest commit

Fetching latest commit…
Cannot retrieve the latest commit at this time.


Type Name Latest commit message Commit time
Failed to load latest commit information.


3D virtual puppet remote controlled via smartphone.

Puppets Screenshot




A puppet consisting of 10 body elements (head, torso, shoulders, arms, thighs and legs) and the proper joints (to connect the body elements and the control bar) gets rendered in the browser via WebGL (using ThreeJS).

To achieve realism it applies a physics engine (oimo.js). Every time the puppet gets rendered the physics thread calculates where each element is positioned as well as the element's orientation.

In order to remote control the puppet a WebRTC bridge is used (Peer.js helps the two clients find each other).

Once the connection is established the smartphone's gyroscope sends it's data over and the puppet's control bar mimics the smartphone's orientation.


npm install
npm run build
npm run server
npm run peerjs-server
npm run stutter-server
  • Then open up a chrome browser in your computer to http://localhost:8000/public/theater/ it will show a QR code and a URL.
  • Open the URL in a mobile device (right now it only works with Android).
  • Once the synching gets done you will see a 3D virtual puppet rendered in the computer and it will respond accordingly to your smartphone's movements in real time.
  • Using the Theater id you can open up more theaters to broadcast and render the puppet independently at different browsers or tabs (...public/theater/#/?t=theater_id_here ).
  • Added a new section: control-bar. This sections renders the control bar as it mimics the smarthpone's orientation.

iOS support

  • Clone this repo:
  • Deploy an app to any device. The app should request the file at public/ios/index.js (this repo) and it will have full WebRTC support.
  • Please set the correct hostname or ip address in the config.js file.

Future Features

  • Tests
  • Linter

Tech Stack

  • JavaScript
  • React
  • node
  • WebRTC
  • WebGL
  • ThreeJS
  • oimo.js



You can’t perform that action at this time.