Extending positives_not_captured_by_blocking_rules_scored_as_zero to other evaluation methods

[Mike Edwards (medwar99)]

The positives_not_captured_by_blocking_rules_scored_as_zero flag allows a user to compare:

• False: Discriminative performance of the FS model as a binary classifier (all labelled pairs are scored and compared to the threshold)
• True (default): Performance of the entire linkage pipeline including the blocking rule filtering (all labelled pairs are scored, then those not hit by blocking are set to zero, flagging missed true-match labels as FN)

There are the following main evaluation methods:

• linker.evaluation.accuracy_analysis_from_labels_table()
• linker.evaluation.accuracy_analysis_from_labels_column()
• linker.evaluation.prediction_errors_from_labels_table()
• linker.evaluation.prediction_errors_from_labels_column()

Currently the capability to toggle the blocking_rule_scored_as_zero behaviour is only present when using accuracy_analysis_from_labels_column() as it passes the argument through to truth_space_table_from_labels_column(), which itself passes it through to truth_space_table_from_labels_with_predictions_sqls(). In all cases it defaults to True.

• accuracy_analysis_from_labels_table() does not accept this parameter.
• truth_space_table_from_labels_table() does not accept this parameter.
• truth_space_table_from_labels_table() calls truth_space_table_from_labels_with_predictions_sqls(), using the default positives_not_captured_by_blocking_rules_scored_as_zero=True

This brought me to a few questions:

1. Is there a reason this would only be beneficial in the label_column case? Having a good quality label column is tricky, and clerical review often produces pairwise labels.
2. Is it worth extending this capability to accuracy_analysis_from_labels_table()?
3. Similarly, is it worth adding this to the prediction errors...() methods? Currently the FN/FP always reflect model-scoring here, not blocking-exclusion effects.
4. Given they tell us different things, is it best to divide this behaviour into different, documented methods? (probably not, to avoid bloat)