update address regex in the standard pii policy

[JustEmrick]

Description

This update is a change to the templated Standard PII policy that will more accurately find US based addresses

Problem

1. In doing some testing with a small test dataset, I noticed that NO address was being discovered by the standard PII policy template. I created my own template with updated regex and it correctly identified each variation of the address from the test set.

Testing

Using the test dataset below which is contained in my Mantium application, I ran both the template standard PII policy , which includes an address scan, and my own custom policy. In comparison, I was able to correctly identify the addresses in the test dataset using my regex whereas in the template, none were identified.

Standard Policy

Custom Policy

This contains the new regex

Test Dataset

[zimventures]

Just need to get that test to pass and this looks good - thanks for the fix!

[JustEmrick]

Looks like the test that's failing is the one doing the Regex check on the address and making sure there isn't a match on the incorrect address examples. Not sure why the first would be considered incorrect. Recommendation would be a test that matches all or a portion of a correct address as I'd personally want to catch potential mistakes via fat fingering or the way a system may input the data from a system. An example would be if Street address, city, state and postal code are in different columns, it probably wouldn't get loaded in to a database correctly.

Failing Test:

def test_address_pattern_invalid(self):
        """Verify that the Address regex pattern does not match invalid addresses."""
        invalid_addresses = [
            '123 Main Street, Los Angeles, CA',
            '987 Elm Avenue, New York 10001',
            '001 Oak Court, San Francisco 94102',
        ]
        self.verify_pattern('Address', invalid_addresses, False)

[zimventures]

I would agree - the fields shown as invalid addresses are still "address-like" and as such, should be flagged -> which apparently they now are! We have two options here:

1. remove the test entirely
2. Change the list of invalid addresses to be truly invalid ("Joe Cool", "123 Marbles", "Place of Business", etc...)

Personally, I feel as though option 2 keeps the spirit of the test alive.

[JustEmrick]

I like the second idea. I updated the test criteria to be more incorrect as you recommended.

[JustEmrick]

Updated so regex tests pass. The old descriptions were too address adjacent and likely that users could have those in an actual dataset. The changes are more obviously fake.

[JustEmrick]

Updated the test that was failing for identifying an "incorrect address" but was landing on a valid portion of it.