Texture synthesis with image quilting
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bing
quiltlib
README.md
bing.js
bundle.js
index.html
script.js
style.css
textures.js
textures.js.mem
worker.js we doin it now Nov 12, 2016

README.md

Inspiration

We can take a texture like this:

Lemon texture

... and a potato like this:

Potato

... and we can create a lemon potato:

Lemon potato

Examples

What it does

It uses a machine learning / energy minimization algorithm in order to a model a transposition of a texture onto another natural image.

Our web demo lets users take pictures from their webcams and retexture them using a number of default options.

Web interface

tl;dr it copies the texture from one picture onto another

How we built it

We wrote the entire algorithm from the ground-up in C++, including neighbor search data structures, energy minimization routines, and marshaling code. We compiled the C++ code using emscripten so that the entire algorithm can be ran in native asm.js JavaScript on the client side.

Based on the initial ideas presented by Efros and Freeman.

Challenges we ran into

Debugging memory faults in emscripten compiled JavaScript is really hard! We had to write a light-weight version of Valgrind (and libVEX) to track down a few nasty bugs. The algorithm itself was also pretty tricky, requiring us to gain a little bit more mathematical knowledge than we had before.

Accomplishments that we're proud of

It works! And we have some neat images that really show how everything works.

Compare

Team member Sofiya pictured, comparing actual synthesis to an overlay / soft light blending.

What we learned

So, so much about low-level optimizations and image processing!

What's next for Quilty

Expanding it beyond just the client, doing video texture transfer.