Skip to content

ccj5351/crowdsourcing

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Pairwise Ranking Aggregation by Non-interactive Crowdsourcing with Budget Constraints

If you find this repository helpful, please cite our paper Pairwise Ranking Aggregation by Non-interactive Crowdsourcing with Budget Constraints.

@inproceedings{cai2017pairwise,
  title={Pairwise Ranking Aggregation by Non-interactive Crowdsourcing with Budget Constraints},
  author={Cai, Changjiang and Sun, Haipei and Dong, Boxiang and Zhang, Bo and Wang, Ting and Wang, Hui},
  booktitle={2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS)},
  pages={2567--2568},
  year={2017},
  organization={IEEE}
}


Code Usage:

The main() function is provided in the main_SATD.cpp file. The experiments in our ICDCS 2017 paper are finished by this function.

Input Parameters:

Note that all the parameters are string variables, which will be automatically changed into appropriate variables (like, string to double via function stod(), string to int via stoi(), etc.).

  • baseAddress, an directory storing the task assignment graphs, each of which is saved in a "txt" file. E.g., = E:/OpenCVProjects_CCJ/CrowdSourcing2/ICDE-2017/GT/GT-v100/. Please pay attention the last "/" which is necessary.

  • k, finding top-k paths using TA or SA. E.g., k = 3, meaning to return top-3 optimum paths.

  • vertexNum, number of the vertex, this value can be set any initial value, since it be assigned later. E.g., vertexNum = 20;

  • d, degree of the graph, this value can be set any initial value, since it be assigned later. E.g., d = 5.

  • T, temperature used in the Simulation Annealing algorithm. E.g., T = 50000.

  • coolRate, the rate of cooling, still used in the Simulation Annealing algorithm. E.g., coolRate = 0.95.

  • IterationNum, the predefined iteration times for Simulation Annealing algorithm, E.g., IterationNum = 1000.

  • ExperimentTimes, meaning how many times we will do the experiments for getting an averaged result. E.g, IterationNum = 10.

  • distribuion_type, there are two types of distribution used to generate the simulation dataset considering the variety of the workers' qualities. distribuion_type = 0, means Gaussian distribution; distribuion_type = 1, means uniform distribution.

  • permuStartingIdx, please set this value as 1. Actually it does not work during the experiments, it is not deleted just because I do not want to change the order of parameters too much.

  • workerNum, the number of workers for each pairwise comparison, that is, how many workers will work on the pairwise comparison (i.e., one edge). E.g., workerNum = 20, or 50.

  • ratio, selection ratio, different selection ratio means different degree in the task assignment graph. This value can be set any initial value, since it be assigned later, e.g., ratio = 0.5.

  • SA_flag, different methods to find an initial path for Simulation Annealing algorithm. SA_flag = 2 is used in our methods. The different values are explained in the function of TSPalgorithm::Run() located in the file of TSPalgorithm.cpp.

  • isDisplay, a boolean value used to print or not print information. Set it 0 (i.e., false) for disabling the printing.

  • tasks_ratio, the ratio of all the pairwise comparison tasks, e.g, ratio = 0.1, that means each worker will pick at random up to 500 * 0.1 = 50 pairwise comparison tasks to vote, given, like, 500 total pairwise comparison tasks.

  • Worker_quality, Gaussian variance to control the worker's quality.

About

Pairwise Ranking Aggregation by Non-interactive Crowdsourcing with Budget Constraints

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages