Benefit of full word versus abbreviation?

[mmagoffin-sd]

I came across the following suggestion in https://moj-analytical-services.github.io/splink/demos/tutorials/01_Prerequisites.html:
"Replace abbreviations with full words (e.g., standardize "St." and "Street" to "Street")."
Curious why the directionality matters here for the splink model, i.e. how is replacing any instances st > street better than replacing any instances of street > st?
Our current cleaning process transforms full words to abbreviations, the opposite of this suggestion, thus why I am hoping to understand better!
My thought had been by abbreviating standard words like street, place, circle, etc for addresses, the name of the street main, sunset, G, 123, ... would have more sway on comparisons and lead to more accurate matches.

[Pradip Kumar Prajapati (pradipNP)]

Directionality matters because you are choosing which collisions to create, not because Splink scores “Street” as more informative than “St.”

What goes wrong if you abbreviate

St. is overloaded. Expanding it to Street is lossy in the other direction too, but collapsing everything to St merges distinct tokens:

original	after expand	after abbreviate
10 Main Street	10 Main Street	10 Main St
10 Main St.	10 Main Street	10 Main St
10 Main Saint	10 Main Saint	10 Main St

The first two rows should match. Expanding them both to Street does that. Abbreviating them both to St also does that — and silently merges Saint into the same token. That is the usual reason the tutorial says expand, not compress.

The same pattern shows up with Rd/Road/Rd., N/North/Nth, Mt/Mount/Mt.. Abbreviations are a many-to-one map; expanding is closer to one-to-one once you pick a canonical form.

Your intuition about street names is already handled

You are right that Street / Place / Circle are common and should not dominate. Splink already down-weights frequent tokens via term frequency (tf_adjustment_column on a comparison, or term_frequency_adjustments=True on ExactMatch). After that, Main vs Sunset vs G vs 123 is what moves the match weight, whether the suffix is Street or St.

So you do not need to delete the suffix to give the name more sway. You need the suffix to be the same string on both records so an exact / levenshtein / jaro-winkler comparison on the address field does not spend its budget on Street vs St..

Why expand is the safer canonical form

1. String metrics. Jaro-Winkler / Levenshtein on 10 Main St vs 10 Main Street still disagree on several characters. After expansion they are identical, so an exact comparison can fire.
2. Tokenisation. St. with a trailing period, St without, and ST in a different case are three tokens until you normalise. Expanding after a lowercase + punctuation strip lands on one.
3. Blocking. If you block on address tokens, street as a block key is rare enough not to explode, and st as a block key is huge (and polluted by Saint). Expanding keeps the block keys cleaner.

If you already abbreviate

You do not have to rewrite your pipeline. What Splink needs is that both datasets use the same convention. If your current cleaner already maps Street → St, apply that same mapping to the other table before linking. Mixing “full words in A, abbreviations in B” is the case that actually hurts.

If you ever add a second source that does not go through that cleaner, expanding both sides to full words is the more robust default — that is all the tutorial is arguing for.

[Anders Alexandersson (aalexandersson)]

For US patient addresses, the Splink recommendation "Replace abbreviations with full words (e.g., standardize "St." and "Street" to "Street")" is misleading, if not incorrect. See the Project US@ (Project USA) Technical Specification Appendix B. In the example, the Project US@ standard for "Street" is "ST".

Discussed at: https://oncprojectracking.healthit.gov/wiki/spaces/TechLabSC/pages/180486153/Project+US

URL of the Project US@ Technical Specification: https://oncprojectracking.healthit.gov/wiki/download/attachments/180486153/Project%20US@%20FINAL%20Technical%20Specification%20Version%201.0.pdf?version=1&modificationDate=1641545329051&api=v2

[mmagoffin-sd]

Wow incredibly relevant in my use case, thank you so much!