Skip to content

A Chrome extension that adds a 3d photo effect to instagram pages.

Notifications You must be signed in to change notification settings

florstone/instagram-3d-photo

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

A chrome extension that adds depth parallax (an effect similar to Facebook 3D photos) on images from instagram profile pages.

It uses 3d-photo-inpainting running in Colab (free GPU) and Cloud pubsub/storage for communication.

More info: https://twitter.com/cyrildiagne/status/1251920177782042624

Demo

Original Paper: 3D Photography using Context-aware Layered Depth Inpainting Meng-Li Shih, Shih-Yang Su, Johannes Kopf, Jia-Bin Huang

⚠️ Word of caution!

🧪 This project is highly experimental and requires a strong knowledge of the Google Cloud Platform to setup & run.

Architecture

architecture

This extension works by communicating with a Colab notebook running as GPU worker to compute the 3D inpainting videos.

For this extension to work, there are 2 other applications that must be running:

  1. The Colab notebook. It must be openned in another tab with the last cell running.
  2. The proxy service. It must be running (either remotely or on your computer) to receive HTTP requests from the extension and turn them into PubSub messages that the Colab listens to.

Setup the GCP Project

Create a GCP project

First create a project on the Google Cloud Platform. Then install the Cloud SDK and initialize it in your local machine with gcloud init.

Alternatively, you can also use Cloud Shell.

In a terminal window, set the local default project:

export PROJECT_ID=<your gcp project id>
gcloud config set project $PROJECT_ID

Create a service account for colab to be able to access cloud pubsub & storage.

It will create a insta3d-colab-key.json file that you must upload on the Colab to authenticate the notebook with your project.

export COLAB_SA=insta3d-colab
export COLAB_KEY_FILE=./insta3d-colab-key.json
./scripts/create_colab_key.sh

Create the Pubsub topic and subscription

export TOPIC_NAME=insta3d
./scripts/create_pubsub.sh

Enable CORS on your bucket

For the extension to be able to load the images generated by the colab, the CORS must be enabled on a Cloud Storage Bucket.

One way to do so is to create a file cors.json

[
  {
    "origin": ["*"],
    "responseHeader": ["Content-Type"],
    "method": ["GET"],
    "maxAgeSeconds": 3600
  }
]

And apply it to your bucket by running:

export BUCKET_NAME=<your-bucket-name>
gsutil cors set cors.json gs://$BUCKET_NAME

/!\ This will enable the CORS on the entire bucket.

Run the proxy service locally

(Recommended) Create a virtual environment

virtualenv venv
source venv/bin/activate

Install the dependencies:

pip install -r proxy/requirements.txt

Start the proxy:

export GOOGLE_APPLICATION_CREDENTIALS=$(pwd)/insta3d-colab-key.json
python proxy/main.py

Run the Colab

Open the notebook in Colab and follow the instructions. Make sure you've activated a GPU Runtime.

Load the extension & enjoy

Phew, when all that is done, it's time to load the extension and profit!

  • Navigate to chrome://extensions
  • Click the Load unpacked button
  • Select the "extension" folder from this repo.

It should add a new icon in your extensions bar that you can use to trigger the extension. Keep an eye on the console to check for eventual errors.

Changes with the original 3d-photo-inpainting

  • Update argument.yml to use custom path + smaller size + custom fps/duration
  • Update straight-line and circle paths in utils.py
  • Set ffmpeg to create 1 keyframe per frame in mesh.py

Thanks and acknowledgements

3d-photo-inpainting The original paper, code and notebook.

@derek-xia for the help improving the documentation.

About

A Chrome extension that adds a 3d photo effect to instagram pages.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Jupyter Notebook 50.8%
  • JavaScript 23.9%
  • Python 13.0%
  • Shell 11.1%
  • CSS 1.2%