Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
src
 
 
 
 
 
 
 
 
 
 

Sg2im Web Demo

This is my attempt at making a web based demo of Google's sg2im, an end-to-end neural network model that converts a scene graph into a human recognizable image. My hope is to use their pretrained model to allow users to generate images themselves.

This web app will be built with React.js and Flask. My hope is that users can input potential objects (ie. sky, grass, shape) and their orders of relation so that they can create realistic images from the pre-trained model on the fly. This repo will contain the code for running the React.js frontend, this other repo contains the Flask service for serving the generated photos.

From their paper this is a visual example of how it works:

alt text

Things to do

  1. Use React components to save users objects and orders of relation (Done)
    • Function to save objects inputed from user to global scene graph ✔️
    • Function to add new objection relation rule components ✔️
    • Save object relations to global scene graph state ✔️
    • Create function to give users unique ID
  2. Write a Flask endpoint to receive scene graphs from React front end & write Ajax function to pass info (In Progress)
    • Rewrite some of the sg2im Python functions for better handling to generate image ✔️
    • Create Flask endpoint to receive JSON data via HTTP requests ✔️
    • Write GET request to retrieve image from API based on user ID
    • Add Zappa tooling to deploy serverless to AWS Lambda

Local demo instructions

To deploy the frontend, open a terminal and do the following:

git clone https://github.com/jtloong/sg2im-demo.git
cd sg2im-demo
npm start

If you want to have the full application, you'll need to clone the Flask API here and follow the demo instructions there.

Relevant links for the project

About

A web based demo of Google's sg2im neural network to convert a scene graph to a photorealistic image.

Resources

Releases

No releases published

Packages

No packages published