Complementary Factorization towards Outfit Compatibility Modeling

An official implementation of the paper on Complementary Factorization towards Outfit Compatibility Modeling

Authors

Tianyu Su, Xuemeng Song, Na Zheng, Weili Guan, Yan Li, Liqiang Nie

Links

• Paper: Paper
• Demo / Video: Video Introduction
• Code Repository: GitHub

---

Table of Contents

• Updates
• Introduction
• Highlights
• Method / Framework
• Project Structure
• Installation
• Checkpoints / Models
• Dataset / Benchmark
• Usage
• Citation
• License

---

Updates

• [10/2021] Initial release
• [10/2021] Release paper at ACM MM 2021
• [10/2021] Release code

---

Introduction

This project is the official implementation of the paper Complementary Factorization towards Outfit Compatibility Modeling.

• What problem does the paper solve: Outfit compatibility modeling
• What is the core idea of the method: Using complementary factorization to model outfit compatibility
• What are the features compared to existing methods: Through complementary factorization, more effectively capture compatibility relationships between clothing items
• What does this repository provide:
  • Training code
  • Inference code
  • Pretrained model weights
  • Data processing instructions
  • Evaluation scripts

Example Description

We present Complementary Factorization, a framework for outfit compatibility modeling.
Our method addresses the problem of outfit compatibility by introducing complementary factorization approach.
This repository provides the official implementation, pretrained checkpoints, and evaluation scripts.

---

Highlights

• Supports outfit compatibility modeling task
• Provides training, inference, and evaluation scripts
• Provides pretrained model weights
• Suitable for paper reproduction, project demonstration, and follow-up research

---

Method / Framework

Framework Figure

Figure 1. Overall framework of Complementary Factorization for Outfit Compatibility Modeling.

---

Project Structure

.
├── assets/                # Images, framework diagrams
├── data/                  # Data directory
├── inference/             # Inference scripts
├── models/                # Model definitions
├── utils/                 # Utility functions
├── README.md              # Project documentation
├── config.py              # Configuration file
├── main.py                # Main training script

---

Installation

Python Environment by Conda

1. create a conda env: conda create -n ocm-cf python=3.8
2. install packages:

• PyTorch. e.g.

  conda install pytorch==1.6.0 torchvision==0.7.0 cudatoolkit=10.1 -c pytorch

• pytorch_geometric. e.g.

  CUDA=cu101
  TORCH=1.6.0
  pip install torch-scatter -f https://pytorch-geometric.com/whl/torch-${TORCH}+${CUDA}.html
  pip install torch-sparse -f https://pytorch-geometric.com/whl/torch-${TORCH}+${CUDA}.html
  pip install torch-geometric

• conda install scikit-learn tensorboard

---

Checkpoints / Models

• Pre-trained models: Model Link or HF hub

After downloading, please place the files in the following directory:

checkpoints/
├── disjoint_best.pt
└── nondisjoint_best.pt

---

Dataset / Benchmark

• Dataset: Polyvore OR polyvore-dataset-github

Data organization:

data/
└── polyvore_outfits
    ├── disjoint
    ├── images
    ├── maryland_polyvore_hardneg
    ├── nondisjoint
    └── retrieval

---

Usage

Inference

cd inference
bash inference_all_tasks.sh ${gpu_id}

Training

For the Polyvore Outfits dataset:

python main.py --polyvore-split nondisjoint

For the Polyvore Outfits-D dataset:

python main.py --polyvore-split disjoint

---

Citation

@inproceedings{743aced44b004a3dac16da3feb57edbd,
title = "Complementary Factorization towards Outfit Compatibility Modeling",
author = "Tianyu Su and Xuemeng Song and Na Zheng and Weili Guan and Yan Li and Liqiang Nie",
note = "Publisher Copyright: {\textcopyright} 2021 ACM.; 29th ACM International Conference on Multimedia, MM 2021 ; Conference date: 20-10-2021 Through 24-10-2021",
year = "2021",
doi = "10.1145/3474085.3475537",
series = "MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia",
publisher = "Association for Computing Machinery, Inc",
pages = "4073--4081",
booktitle = "MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia",
}

---

License

This project is released under the MIT License.