Skip to content

ccj5351/crowdsourcing

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
src
 
 
 
 
 
 
 
 

Pairwise Ranking Aggregation by Non-interactive Crowdsourcing with Budget Constraints

If you find this repository helpful, please cite our paper Pairwise Ranking Aggregation by Non-interactive Crowdsourcing with Budget Constraints.

@inproceedings{cai2017pairwise,
  title={Pairwise Ranking Aggregation by Non-interactive Crowdsourcing with Budget Constraints},
  author={Cai, Changjiang and Sun, Haipei and Dong, Boxiang and Zhang, Bo and Wang, Ting and Wang, Hui},
  booktitle={2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS)},
  pages={2567--2568},
  year={2017},
  organization={IEEE}
}


Code Usage:

The main() function is provided in the main_SATD.cpp file. The experiments in our ICDCS 2017 paper are finished by this function.

Input Parameters:

Note that all the parameters are string variables, which will be automatically changed into appropriate variables (like, string to double via function stod(), string to int via stoi(), etc.).

  • baseAddress, an directory storing the task assignment graphs, each of which is saved in a "txt" file. E.g., = E:/OpenCVProjects_CCJ/CrowdSourcing2/ICDE-2017/GT/GT-v100/. Please pay attention the last "/" which is necessary.

  • k, finding top-k paths using TA or SA. E.g., k = 3, meaning to return top-3 optimum paths.

  • vertexNum, number of the vertex, this value can be set any initial value, since it be assigned later. E.g., vertexNum = 20;

  • d, degree of the graph, this value can be set any initial value, since it be assigned later. E.g., d = 5.

  • T, temperature used in the Simulation Annealing algorithm. E.g., T = 50000.

  • coolRate, the rate of cooling, still used in the Simulation Annealing algorithm. E.g., coolRate = 0.95.

  • IterationNum, the predefined iteration times for Simulation Annealing algorithm, E.g., IterationNum = 1000.

  • ExperimentTimes, meaning how many times we will do the experiments for getting an averaged result. E.g, IterationNum = 10.

  • distribuion_type, there are two types of distribution used to generate the simulation dataset considering the variety of the workers' qualities. distribuion_type = 0, means Gaussian distribution; distribuion_type = 1, means uniform distribution.

  • permuStartingIdx, please set this value as 1. Actually it does not work during the experiments, it is not deleted just because I do not want to change the order of parameters too much.

  • workerNum, the number of workers for each pairwise comparison, that is, how many workers will work on the pairwise comparison (i.e., one edge). E.g., workerNum = 20, or 50.

  • ratio, selection ratio, different selection ratio means different degree in the task assignment graph. This value can be set any initial value, since it be assigned later, e.g., ratio = 0.5.

  • SA_flag, different methods to find an initial path for Simulation Annealing algorithm. SA_flag = 2 is used in our methods. The different values are explained in the function of TSPalgorithm::Run() located in the file of TSPalgorithm.cpp.

  • isDisplay, a boolean value used to print or not print information. Set it 0 (i.e., false) for disabling the printing.

  • tasks_ratio, the ratio of all the pairwise comparison tasks, e.g, ratio = 0.1, that means each worker will pick at random up to 500 * 0.1 = 50 pairwise comparison tasks to vote, given, like, 500 total pairwise comparison tasks.

  • Worker_quality, Gaussian variance to control the worker's quality.

About

Pairwise Ranking Aggregation by Non-interactive Crowdsourcing with Budget Constraints

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages