Skip to content
Code for the paper "A Similarity Measure for Material Appearance" presented in SIGGRAPH 2019 and published in ACM Transactions on Graphics (TOG).
Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
__media__
data
.gitignore
LICENSE
README.md
get_embs.py
model.py
plot_similar.py
plot_umap.py
utils.py

README.md

A Similarity Measure for Material Appearance

Project page Paper Conference Journal

Abstract

We present a model to measure the similarity in appearance between different materials, which correlates with human similarity judgments. We first create a database of 9,000 rendered images depicting objects with varying materials, shape and illumination. We then gather data on perceived similarity from crowdsourced experiments; our analysis of over 114,840 answers suggests that indeed a shared perception of appearance similarity exists. We feed this data to a deep learning architecture with a novel loss function, which learns a feature space for materials that correlates with such perceived appearance similarity. Our evaluation shows that our model outperforms existing metrics. Last, we demonstrate several applications enabled by our metric, including appearance-based search for material suggestions, database visualization, clustering and summarization, and gamut mapping.

Setting it up

Note that this has been tested using python 3.7

Dependencies

First, clone and install dependencies

# clone project   
git clone https://github.com/mlagunas/material-appearance-similarity.git   

cd material-appearance-similarity 
pip install scipy numpy matplotlib umap-learn Pillow
# install pytorch/torchvision (https://pytorch.org)

Get model pretrained weights

How to run the code

Getting image feature vectors

Next, get the feature vectors for some images. First, modify paths inside get_embs.py.

...
weights_path = 'data/model_best.pth.tar'
imgs_path = 'data/havran1_stpeters_256x256_LDR'
embs_path = 'data/embs.mat' # we will store the obtained feature vectors in this path
...

Then, get the feature vectors for the downloaded images

python3 get_embs.py    

Get similar images

We can obtain similar images to a given reference using the previously computed feature vectors. First, set the path and necesary variables in plot_similar.py. This will store the 5 more similar images to the reference according to our metric in the path data/nickel.

...
embs_path = 'data/embs.mat'  # /mat file with the embeddings
n_close_elems = 5  # number of close elements to find
reference_img = 'data/havran1_stpeters_256x256_LDR/nickel.png'
do_unit_norm = False # normalize feature vectors to have unit norm
...

Generate UMAP plot

We can visualize the feature vectors generated for the images using dimensionality reduction algorithms like UMAP. First we set the path of the feature vectors inside plot_umap.py.

...
embs_path = 'data/embs.mat'
do_unit_norm = False # normalize feature vectors to have unit norm
...

To generate the plot we run:

python3 plot_umap.py

Citation

If you found this code useful please cite our work as:

@article{lagunas2019similarity,
    author = {Lagunas, Manuel and Malpica, Sandra and Serrano, Ana and
    Garces, Elena and Gutierrez, Diego and Masia, Belen},
    title = {A Similarity Measure for Material Appearance},
    journal = {ACM Transactions on Graphics (SIGGRAPH 2019)},
    volume = {38},
    number = {4},
    year = {2019}
}
You can’t perform that action at this time.