A Python package that allows the user to fuzzy match two pandas dataframes based on one or more common fields.
sqlite3's Full Text Search to find potential matches.
It then uses probabilistic record linkage to score matches.
Finally it outputs a list of the matches it has found and associated score.
pip install fuzzymatcher
Note that you will need a build of sqlite which includes FTS4. This seems to be widely included by default, but otherwise see here.
See examples.ipynb for examples of usage and the output.
You can run these examples interactively here.
Suppose you have a table called
df_left which looks like this:
And you want to link it to a table
df_right that looks like this:
|1||Havering London Boro|
|2||Sir Fynwy - Monmouthshire|
|3||Knowsley District (B)|
|4||Charnwood District (B)|
You can write:
import fuzzymatcher fuzzymatcher.fuzzy_left_join(df_left, df_right, left_on = "ons_name", right_on = "os_name")
And you'll get:
|0.133371||Monmouthshire||Sir Fynwy - Monmouthshire|
|0.102473||Havering||Havering London Boro|
|0.155775||Knowsley||Knowsley District (B)|
|0.155775||Charnwood||Charnwood District (B)|