CS-F-LTR

This is the MindSpore implementation of CS-F-LTR in the following paper.

ICDE 2021: An Efficient Approach for Cross-Silo Federated Learning to Rank

The paper is located in ./ICDE21-wang.pdf

Contents

• CS-F-LTR Description
• Dataset
• Directory
• Quick Start
• Script Description
  • Mapper
  • Builder
  • Server
  • Operations

CS-F-LTR-description

CS-F-LTR is a noval framework named Cross-Silo Federated Learning-to-Rank.

Dataset

Dataset used: MS MARCO Ranking dataset

• Dataset partition：
  • 4 parties
  • 200 queries and 36,400 documents per party
  • 1000 terms per document
  • Train
    • 28,000 instances per party
  • Test
    • 32,000 instances per party

Directory

• raw docs are split into txt in ./data/docs
• raw queries are split into txt in ./data/queries
• all term have been numbered and the dictionary is in ./data

Quick Start

  bash scripts/run.sh

Script Description

Mapper

• transfer all raw docs queries and top100 into mapper
• the docs and queries and relevance docs and score are all in ./data/mapper{TOP_NUM}

Builder

based on the data transferred by mapper, build {FED_NUM} federations

Server

this script helps to exchange most relevance features and upgrade each federation then each federation will gen features

Operations

python3 dictionary.py (optional)
python3 mapper.py
python3 builder.py -b 1
python3 builder.py -b 2 -f 0
python3 builder.py -b 2 -f 1
python3 builder.py -b 2 -f 2
python3 builder.py -b 2 -f 3
python3 server.py -f 0
python3 server.py -f 1
python3 server.py -f 2
python3 server.py -f 3