Skip to content

noahlevenson/stealing-ur-feelings

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

88 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

stealing ur feelings

Stealing Ur Feelings

🧭 Table of contents

❓ What is Stealing Ur Feelings?

https://stealingurfeelin.gs ⬅️ ⬅️ ⬅️

Meet the new AI that knows you better than you know yourself.

Stealing Ur Feelings is a deep learning-powered AR experience which analyzes your facial reactions to reveal the dangers of Big Tech's emotional surveillance programs. Using the AI techniques described in corporate patents, Stealing Ur Feelings learns your deepest secrets just by analyzing your face.

It won the 2020 Webby Award for Best Immersive Documentary.

The project premiered on the internet on September 23, 2019.

It world premiered at the 2019 Tribeca Film Festival and was exhibited at the Tate Modern. It was profiled by the Museum of the Moving Image, Scientific American, Engadget, CBC News, and many more; it made the front page of Hacker News; MIT selected it for inclusion in DocuBase, a curated database of people and projects transforming documentary in the digital age.

Stealing Ur Feelings was somewhat notoriously plagiarized by the Financial Times; this event was chronicled in an article for The American Prospect.

The project began life as an application for Mozilla's 2018 awards for art and advocacy exploring artificial intelligence.

📰 Press and recognition

Scientific American

Museum of the Moving Image

Hacker News

Engadget

CBC News

Let's Go with Sabrina Marandola on CBC Radio One / .mp3

MIT DocuBase

Adweek

It's Nice That

Forbes

El Pais

Report on Rai3

Business Insider Italy

El Nacional

ProPakistani

Boing Boing

Mozilla

🎥 Festival and exhibition history

2019 Tribeca Film Festival

2019 Camden International Film Festival

2019 Montreal International Documentary Festival

2019 Open City Documentary Festival

Tate Exchange at the Tate Modern

The Glass Room SF presented by Tactical Tech

👀 Tech notes

  • The code is a mess. It required a lot of trial and error over many months just to figure out how to tell the story. Consequently, the version of Stealing Ur Feelings which premiered at the Tribeca Film Festival was programmed over 6 frantic weeks between March and April 2019. That version was so unoptimized that it simply wouldn't run on average hardware. Optimization began in June 2019, and there wasn't enough time to improve most of the earlier bad decisions. The indecipherable comments have been preserved.

  • Different versions of the project have used different computer vision engines. Journalists often asked what emotion recognition models we used; the answer changed depending on the day they viewed the project. We built an excessively complicated custom engine on top of TensorFlow that wound up getting scrapped. At one point we managed to compile Dlib for the browser via the WebAssembly toolchain, and we got its multithreaded facial landmark detector working with HTML5 canvas by mapping native pthreads to WebWorkers. You can see that experiment here. The version of Stealing Ur Feelings published in September 2019 uses face-api.js.

  • Stealing Ur Feelings uses a custom system for frame-accurate video synchronization. It's based on embedding encoded information in the overscan area of the video, a la datacasting. The purely arithmetic approaches to frame-accurate sync are interesting, but the effects of floating point approximation error means that none of them work at the level of precision we required.

  • On Chrome, Stealing Ur Feelings deploys a zero-copy hack using OffscreenCanvas to run the computer vision engine on a separate thread. During development, Firefox's implementation of OffscreenCanvas was incomplete. The result is that Stealing Ur Feelings runs much smoother on Chrome. (Another result: there have been OffscreenCanvas regressions in Chrome stable releases which have caused Stealing Ur Feelings to segfault the browser.)

  • There's a secret physical installation mode. At the Tribeca Film Festival premiere, Stealing Ur Feelings was installed under a big television hanging from the ceiling which broadcast the AI-extracted "secrets" of whoever was experiencing the project at that moment. The client and server to support this functionality are located in the /installation directory. For the premiere, we tuned the political bias algorithm to classify absolutely everyone as a far-right conservative Republican. It was fun to watch journalists freak out when their secret political affiliation was revealed on the jumbotron.

💾 Archive

⚡ Changelog

06/01/2021 The American Prospect article is published, detailing plagiarism by The Financial Times

05/26/2021 We're featured in Slanted Magazine #37, the AI issue

05/19/2020 We won the 2020 Webby Award for Best Immersive Documentary

01/31/2020 Named one of the Top 50 XR Experiences of 2019 by Forbes

01/15/2020 Feature in El Pais

11/22/2019 Featured on Let's Go with Sabrina Marandola on CBC Radio One

11/18/2019 Profiled by CBC News

11/12/2019 Featured on Report on Italy's national public broadcaster Rai3

11/03/2019 Feature in It's Nice That

10/21/2019 We're coming to Montreal International Documentary Festival and The Glass Room SF

09/23/2019 Internet premiere

09/14/2019 We're coming to Tate Exchange at the Tate Modern

08/19/2019 We're coming to Camden International Film Festival in Maine

08/14/2019 MIT selected us for inclusion in DocuBase

08/10/2019 We're coming to Open City Documentary Festival in London

05/07/2019 Profiles by the Museum of the Moving Image and Engadget

03/06/2019 We've been selected to premiere at the 2019 Tribeca Film Festival!

01/02/2019 Interactive tech demo v2.0 is live!

10/24/2018 We won

07/12/2018 Submitted full application to Mozilla

07/08/2018 The interactive tech demo is live!

07/06/2018 Registered domain name: stealingurfeelin.gs

06/29/2018 Original funding concept accepted by Mozilla - we've been invited to submit a full application!