FastAD

This repo is a version of FastAD, which is designed based on Deep Crossing. This work have been published in WWW19 Learning Fast Matching Models from Weak Annotations

This work is done in Microsoft Ads team. For More information, please contact jianjzh@microsoft.com.

Introduction

Query and Ads Keyword Relevance is one of major metric of Bing Ads to control quality and user experience.

Query is the sentence user search in Bing. Ads Keyword is the sentence advertiser bided in Bing Ads.

The labels are generated by Bing Ads labeler to measure if the query and ads keyword are relevant in semantic level.

Data

Training set: 100K pairs with Binary Label, sampled in Bing Ads QKRel training set.

Dev set : 10K pairs with Binary Label, sampled in Bing Ads QKRel training set.

Test set : 95K pairs with Binary Label, Bing Ads QK_Rel test set.

Log set : 500K pairs without Label, sampled in Bing Ads impression log.

Knowledge Distillation Process

1. Using training set fine tune MT-DNN as the Teacher Model.
2. Scoring the log set / training set via Teacher Model to get Soft Label.
3. Using C-DSMM_KD to learn from log set via Soft Label as the Student Model.
4. Fine-tuning the Student Model with training set via Soft/Hard Label (Label-aware weights).

How to run

This code needed to be run in docker.

1. Pull docker

docker pull allenlao/pytorch-mt-dnn:v0.11

2. Run docker

docker run -it --rm -p 0.0.0.0:6006:6006 --runtime nvidia allenlao/pytorch-mt-dnn:v0.11 bash

BibTeX

@inproceedings{li2019learning,
  title={Learning Fast Matching Models from Weak Annotations},
  author={Li, Xue and Luo, Zhipeng and Sun, Hao and Zhang, Jianjin and Han, Weihao and Chu, Xianqi and Zhang, Liangjie and Zhang, Qi},
  booktitle={The World Wide Web Conference},
  pages={2985--2991},
  year={2019}
}