Tag My Outfit

Overview

This project implements a gRPC service to classify clothing parts within a set of categories, as well as predict their attributes, in a given image.

Model Description

The prediction model is the Visual Semantic Attention Model (VSAM), a compact framework with guided attention for multi-label classification in the fashion domain. This model is supervised by automatic pose extraction creating a discriminative feature space.

The code made available through this service allows performing inference with the VSAM model pre-trained on the DeepFashion dataset. The provided model weights were obtained by training VSAM on the training set and validating on the validation set of the previous dataset.

Technologies

• Python

• Keras

• TensorFlow

• Scikit-learn

• SciPy

• NumPy

• gRPC

• Docker

Usage

The project can be deployed with docker. The [docker image](insert link here) can be obtained from docker hub. A docker-compose file is provided for an easy deployment with default configurations.

In order to achieve that, execute the following steps:

• Install docker and docker compose by following the respective page's instructions (Note: Docker compose might be automatically installed with docker)

• Download the docker-compose file from the github repository

• Start the server (The image will be automatically downloaded):

$ docker-compose up

Developers

Requirements

• Python 3.6+

• Python packages

Installing

The installation steps are as follows:

• Install python by following the respective page's instructions

• Clone the github repository (Specific versions are marked by tags):

$ git clone https://github.com/sipg-isr/tag_my_outfit_server.git

• Install the necessary packages by running the following command in the project's root directory:

$ pip install -r requirements.txt

• Download the weights file and place it in the model folder with the name weights.h5

Running

With the required software installed, we can start the server with the following command, executed in the project's root directory:

$ python src/server.py

Testing

The project tests require the unittest package. All the following commands should be executed inside the project's root directory.

Unit testing

These tests do not need the server running and only verifies if key project components are functional:

$ python -m unittest discover -s "tests/unit/" -p "*.py"

Integration testing

These tests need the server running and will create clients to exhaustively test the server functionality:

$ python -m unittest discover -s "tests/integration/" -p "*.py"

Load testing

These tests also require that the server is running, and will check its performance and correctness in a load scenario by sending multiple requests simultaneously. Not every combination of message parameters is tested since Integration Tests will cover all cases:

$ python -m unittest discover -s "tests/load/" -p "*.py"

License

This project is released under the MIT License.

Citing

@Inproceedings{QuintinoFerreira2019,
  author    = {Quintino Ferreira, Beatriz and Costeira, J.P and Sousa, Ricardo G. and Gui, Liang-Yan and Gomes, João P.},
  title     = {{Pose Guided Attention for Multi-label Fashion Image Classification}},
  booktitle = {IEEE International Conference on Computer Vision (ICCV) Workshop on Computer Vision for Fashion, Art and Design},
  year      = {2019} ,
}