Enable salting for EM training

[RobinL]

This PR enables the use of salting for EM training addressing #1817

This was previously arbitrarily denied on the basis it probably wasn't needed, but there's no real reason to disable it it, since it's opt-in (by default salting is not used)

I thought this may help with parallelisation in duckdb, but it's actually fully parallelised already. I have checked and salting doesn't make it any slower, but not really any faster either. It's almost fully parallelised already because

• First we create the comparison vectors as a table once (not in parallel)
• Then we used this table repeatedly to iterate the EM parameters. Because this is a large table with millions of rows, aggregations on it are parrallelised by default
• These aggregations comprise most of the work

test code

from IPython.display import display
import time
from splink.datasets import splink_datasets
from splink.duckdb.blocking_rule_library import block_on
from splink.duckdb.comparison_library import (
    exact_match,
    levenshtein_at_thresholds,
)
from splink.duckdb.linker import DuckDBLinker

df = splink_datasets.historical_50k
df.head()

settings = {
    "probability_two_random_records_match": 0.01,
    "link_type": "dedupe_only",
    "blocking_rules_to_generate_predictions": [
        block_on(["first_name"]),
        block_on(["surname"]),
    ],
    "comparisons": [
        levenshtein_at_thresholds("first_name", 2),
        exact_match("surname"),
        exact_match("dob"),
        exact_match("birth_place"),
        exact_match("postcode_fake"),
    ],
    "retain_intermediate_calculation_columns": True,
    "max_iterations": 10,
    "em_convergence": 0.01,
}


linker = DuckDBLinker(df, settings)


br = block_on("first_name")

start_time = time.time()
linker.estimate_parameters_using_expectation_maximisation(br)
end_time = time.time()
print(f"Time taken: {end_time - start_time:.1f} seconds")

linker = DuckDBLinker(df, settings)
br = block_on("first_name", salting_partitions=12)

start_time = time.time()
linker.estimate_parameters_using_expectation_maximisation(br)
end_time = time.time()
print(f"Time taken: {end_time - start_time:.1f} seconds")

[ADBond]

Great, makes sense 👍