Record Linkage using MySQL

[coommark]

First I apologize for having to ask a question here.

The question is, is there an example of record linkage MySQL (large dataset)? I have tried about six times to convert the record linkage example to MySQL without success.

Our specific usage scenario is:

1. We have a large dataset which we have deduped successfully using dedupe (about 500k records).
2. However, every other week, new messy data is acquired, which we dedupe separately using our trained settings. Now we need to link these two canonical dataset, the first being about 500k records and the second being maybe 1k records. Now as you can see, we cannot use CSV to link such large data (or can we?) So we're desperate for a sample MySQL record linkage.

I am fairly new to Python, started learning it only so I can use this library (Dedupe), but I am not yet that good to understand the nuances of this awesome lib and be able to create the needed MySQL record linkage.

PLEASE HELP!

[mehrrsaa]

I think this may help you out:

https://github.com/dedupeio/address-matching/blob/sqlclass/address_matching.py

But you would have to change whatever is necessary to make it work, depending on your mysql version, use-case, etc.

There is no ready-made solution, you need to spend time on it.

[fgregg]

Hi, we don't have an full example for that yet. Sorry.

Duplicate of dedupeio/dedupe-examples#23