Trim leading zeroes from house number #110
Conversation
Previously, only literal `0` house numbers were filtered out in USA/CAN. There are presently ~67k US/CA OA records with house numbers reduceable to `0` above and beyond literal `0`. This removes them.
Over 120k OA addresses have leading 0's that can cause search and display issues. This fix trims them away.
|
nitpick: it looks like #108 is included in this PR, you could target the merge of this code in to something happened to Travis although it seems to be their fault 👍 |
The functional tests were a combination of two OA data files containing several thousand entries and a 300k line JSON file of the expected results. This made test maintenance very tedious and the tests only exercised a small part of the pipeline. I lovingly crafted 2 artisanal data files in Brooklyn with fictitious entries that contained all the cases that the overall stream should affect. The functional test also had deduplication and admin lookup turned off, so this PR makes those two pieces of behavior injectable.
|
So the intention behind the massive functional test is that it would catch interesting issues now and then. After a few months I don't think we've seen any case other than "almost every expected value has to be updated to reflect a known change we made". Should we get rid of them in the other importers too? 👍 |
|
I think we should keep them until we have time to revisit and create smaller, representative functional test files for each, like Stephen has done here. |
There are OA house numbers that are prefixed with
0that can cause lower scores for inputs that wouldn't contain these leading0's. For example:07741 West Crocus Driveand02 West Dawn Drive.This PR also refactors the functional tests to be less "real world" dependent which really didn't exercise much of the import pipeline.
Fixed #109