Skip to content

Dataset and codes for the paper "Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training".

License

Notifications You must be signed in to change notification settings

syuqings/Fashion-MMT

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training

This repository contains the Fashion-MMT dataset and PyTorch implementation of our paper Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training (ACMMM 2021 Oral).

Requirements

  • Python 3.6
  • Java 15.0.2
  • PyTorch 1.1
  • numpy, tqdm, h5py, scipy, six

Fashion-MMT Dataset

Annotations

Annotations of Fashion-MMT(C) and Fashion-MMT(L) datasets can be downloaded from BaiduNetdisk (code: i55n).

JSON Format:
[
  {
    "id": int,
    "split": str,
    "en": str,
    "zh": str,
    "images": [str],
    "category": str,
    "attr": [str]
  }
]

The images can be downloaded from the url https://n.nordstrommedia.com/id/sr3/image_name with image_names.

Features

We also provide the image features of ResNet101 pretrained on ImageNet and finetuned on Fashion-MMT at BaiduNetdisk (code: i55n)(~57G). Decompress and merge the downloaded features into one folder:

$ cat resnet101.finetune.tar.gz* | tar -xz

Training & Inference

Start training

  1. Pre-train the model with three pre-training tasks
$ CUDA_VISIBLE_DEVICES=0 python train.py ../results/pretrain/model.json ../results/pretrain/path.json --is_train
  1. Fine-tune the model to MMT
$ CUDA_VISIBLE_DEVICES=0 python train.py ../results/finetune/model.json ../results/finetune/path.json --is_train --resume_file ../results/pretrain/model/step.*.th

Evaluation

$ CUDA_VISIBLE_DEVICES=0 python train.py ../results/finetune/model.json ../results/finetune/path.json --eval_set val --resume_file ../results/finetune/model/step.*.th

Reference

If you find this repo helpful, please consider citing:

@inproceedings{song2021FashionMMT,
  title={Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training},
  author={Song, Yuqing and Chen, Shizhe and Jin, Qin and Luo, Wei and Xie, Jun and Huang, Fei},
  booktitle={Proceedings of the 29th {ACM} International Conference on Multimedia},
  year={2021}
}

About

Dataset and codes for the paper "Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training".

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages