crowdkit.metrics.workers.accuracy_on_aggregates.accuracy_on_aggregates.md

accuracy_on_aggregates

crowdkit.metrics.workers.accuracy_on_aggregates.accuracy_on_aggregates | Source code

accuracy_on_aggregates(
    answers: DataFrame,
    aggregator: Optional[BaseClassificationAggregator] = ...,
    aggregates: Optional[Series] = None,
    by: Optional[str] = None
)

Accuracy on aggregates: a fraction of worker's answers that match the aggregated one.

Parameters Description

Parameters	Type	Description
answers	DataFrame	a data frame containing task, worker and label columns.
aggregator	Optional[BaseClassificationAggregator]	aggregation algorithm. default: MajorityVote
aggregates	Optional[Series]	aggregated answers for provided tasks.
by	Optional[str]	if set, returns accuracies for every worker in provided data frame. Otherwise, returns an average accuracy of all workers.
Returns	-	Union[float, pd.Series]