Skip to content
R code to scan for obvious PII.
R
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
example_files
.gitignore Ignore PII_output.csv Apr 24, 2017
LICENSE.txt
README.md
scan.R

README.md

Synopsis

Scan.R searches all Stata (.dta), SAS (.sas7bdat), SPSS (.sav), and comma-separated values (.csv) files found in the specified directory for variables that may contain personally identifiable information (PII) using strings that commonly appear as part of variable names or labels that contain PII. (Note: Scan.R does not search labels in .csv files.) Results are displayed to the screen and saved to a comma-separated values file in the current working directory containing the variables and data flagged as potential PII.

Instructions

To execute the script, type Rscript scan.R --path=[Target] into the command line, replacing [Target] with the folder containing the data to scan. An excel spreadsheet titled "PII_output" will be saved in the current working directory for your review.

Options
  • --quiet: Silent operation; do not display possible PII to the screen
  • --no-output: Do not output search results to CSV file
  • --output-file=output_filename.csv: Write csv of possible PII to file "output_filename.csv" instead of "PII_output.csv"
  • --strict: Use strict matching when comparing strings. For example, match "lat" but not "latin"
  • --nolabels: Do not scan variable labels for PII search terms

Requirements

  • R must be installed on the local system. Installers can be downloaded at http://cran.us.r-project.org.
  • The dplyr, purrr, optparse, rio and tools packages are required. To install missing packages, run install.packages("PACKAGE_NAME") where PACKAGE_NAME is the name of the package to install. Scan.R will warn if any of these packages are missing.

Exit Codes

scan.R will exit with a value of 10 (ten) when possible PII is found or a value of 0 (zero) when no PII is identified.

Motivation

The script was written to audit data files for personally identifiable information. It provides a solution to quickly searching a large number of files in a particular directory or searching files that contain a large number of variables. However, it does not fully replace manual detection of PII.

Search Strings

scan.R searches variables and labels for the following strings: address, bday, beneficiary, birth, birthday, block, census, child, city, community, compound, coord, country, daughter, degree, district, dob, email, father, fax, first_name, fname, gender, gps, house, husband, last_name, lat, lname, loc, location, lon, minute, mother, municipality, name, network, panchayat, parish, phone, precinct, school, second, sex, social, spouse, son, street, subcountry, territory, url, village, wife, zip

Support

Running this script on a Windows machine may result in the following error:

Error in df_parse_dta_file(spec, encoding) :
  Failed to parse [pathname]: Unable to allocate memory.
Calls: import ... standardize_attributes -> read_dta -> df_parse_dta_file -> .Call
Execution halted

If you receive this error, remove the current version of the R package "Haven" and install the 1.1.0 version as a local zip file. The older version can be downloaded at: https://mran.microsoft.com/snapshot/2017-11-01/bin/windows/contrib/3.5/.

Please use the issue tracker for all support requests.

License

See license file.

Thanks

Special thanks to IPA for their How to Search Datasets for Personally Identifiable Information document which inspired this project.

You can’t perform that action at this time.