Skip to content
A collection of web experiments designed to allude to the artistic possibilities of using PoseNet (running on tensorflow.js) to create a relationship between movement and machine 🕺
Branch: master
Clone or download
Maya Man
Latest commit 876842b Jul 12, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
assets initial commit Jul 1, 2019
dist fixed flipped keypoint names Jul 12, 2019
docs
sketches fixed flipped keypoint names Jul 12, 2019
.babelrc initial commit Jul 1, 2019
.gitignore
CONTRIBUTING.md initial commit Jul 1, 2019
LICENSE initial commit Jul 1, 2019
README.md initial commit Jul 1, 2019
favicon.ico initial commit Jul 1, 2019
header.png initial commit Jul 1, 2019
index.html initial commit Jul 1, 2019
package.json updating to the new version of tfjs and posenet Jul 10, 2019
style.css initial commit Jul 1, 2019
webpack.config.js initial commit Jul 1, 2019

README.md

PoseNet Sketchbook

PoseNet Sketchbook

A collection of web experiments designed to allude to the artistic possibilities of using PoseNet (running on tensorflow.js) to create a relationship between movement and machine.

CHECK OUT THE LIVE DEMOS.

These prototypes are the thought starters that kicked off the process of creating Body, Movement, Language: A.I. Sketches with Bill T. Jones, a creative collaboration between The Google Creative Lab and New York Live Arts. Preparing for the first workshop with Bill and the Bill T. Jones/Arnie Zane Company, we wanted to provide examples of the range of interactions that PoseNet can enable, so we experimented with lots of different ways of using the model. This is essentially the raw starter sketchbook that asks: what can you do when you combine movement and machine?

This is not a library or code repository that intends to evolve. Instead, it is an archive of Body, Movement, Language’s beginnings. You can use this collection of code as a starting point to create your own wacky or wild or just plain useful PoseNet experiments.

To read more about the journey from PoseNet Sketchbook to Body, Movement, Language in the blog post here.

Install and Run

First, clone or download this repository. For more information on how to do this, check out this article.

Next, Make sure you are in the project folder: All of the commands below should be run from your terminal on your machine.

cd posenet-sketches

Install dependencies:

yarn

To watch files for changes, and launch a dev server:

yarn watch

The server should be running at localhost:1234.

File structure

All sketches use PoseDetection.js, a wrapper class I created to handle the PoseNet data.

Each individual sketch is hosted in the 'sketches' folder.

  • index.html:
  • style.css: Styling for the sketch.
  • assets/: The thumbnail, gif, and any additional assets used in the sketch.
  • js/: The soure files.
    • main.js: Set up the camera, load the video, and initialize Pose Detection and the sketch.
    • sketch.js: This is where the ~ magic ~ happens. Some functions to note:
      • setup: Initializes the the canvas width and height.
      • initSketchGui: Sets up the GUI elements that will affect the sketch and adds them to the GUI structure.
      • draw: Looping at 60 fps. Renders and updates elements on canvas with each call.

The Sketches

Basic

How does PoseNet interpret your pose?

basic demo gif

Movement Multiplier

How might we allow past motion to linger?

movement multiplier gif

Text Trailer

How might movement history affect text on screen?

text trailer gif

Shape Shifter

How might movement be translated and abstracted into new forms?

shape shifter gif

Collage Creator

How might a variety of elements collage to recreate a figure on screen?

collage creator gif

Body Transcriber

How might spoken words manifest on screen in relation to the body?

body transcriber gif

Color Mapper

How might body poisition be used as a controller?

color mapper gif

Image Mapper

How might body position surface and highlight content?

image mapper gif

Audio Controller

How might body position manipulate an audio experience?

audio controller gif

Contributors

Built by Maya Man at the Google Creative Lab.

Notes

This is not an official Google product.

We encourage open sourcing projects as a way of learning from each other. Please respect our and other creators’ rights, including copyright and trademark rights when present when sharing these works and creating derivative work. If you want more info on Google's policy, you can find it here. To contribute to the project, please refer to the contributing document in this repository.

You can’t perform that action at this time.