02_assess_privacy
May 2, 2018 - Mike McMahon (mike.mcmahon@dfo-mpo.gc.ca)
At this time, data with privacy considerations must be aggregated such that each polygon has a minimum of 5 unique values for sensitive fields like Licenses, License Holders, and Vessels. This function takes a dataframe and shapefile and for each polygon in the shapefile calculates:
- Aggregate values for a number of (user-specified) fields , and
- How many unique values exist in each polygon for each of a number of sensitive fields.
The following two shapefiles can be generated:
- A shapefile looks like the original submitted shapefile, but where every polygon is populated with:
- The original shapefile fields;
- The MEAN, COUNT and SUM of all of the fields identified as
agg.fields; - One or more columns showing the number of unique values for each of the fields identified as
sens.fields; - A column ("TOTUNIQUE"), indicating the number of unique value for the least represented
sens.field. For example, if a polygon has 10 unique values for LICENSE, but only 4 for VRN, this column would display "4". The purpose is to be as conservative as possible an ensure that privacy is maintained. - A column ("CAN_SHOW") indicating whether or not more detailed, gridded data can be shown from that polygon. Values of YES and NO show that data was present, while blank fields mean there were no records in the polygon to assess.
- A shapefile of 2 min grid cells, where only the data that lay within polygons where "CAN_SHOW = Yes" is gridded. No gridded data at all is available for areas where "CAN_SHOW = No".
As an example, imagine the data below had sensitivity issues (it's actually non-sensitive RV survey data for redfish).
Now pretend that "MISSION" and "SETNO" are sensitive, personally-identifiable fields (like "LICENCE_NUMBER" and "VRN"). We want to share the data, but we have to make sure that we have enough different values of "MISSION" and "SETNO" in any given area so that no one can figure out which data is attributable to which "MISSION" or "SETNO".
To distill the data that we can show, we can run the assess_privacy() as shown below.
assess_privacy(df=data, agg.fields = c("TOTWGT", "TOTNO"),agg.poly.field = "NAFO_BEST", sens.fields = c("MISSION", "SETNO"), create.shps = T)Since no shapefile was supplied as agg.poly.shp, NAFOSubunits gets used instead, and two shapefiles are generated. The first of these ("screened_areas*.shp") is shown below. This shapefile is the same as the agg.poly.shp shapefile, but has had additional fields added corresponding with the data within each polygon. Below, it is symbolized by the field "CAN_SHOW", where open areas have "CAN_SHOW=YES", or orange hashed areas have "CAN_SHOW=NO", and grey areas have no data. Polygon labels show the different NAFO areas, and in brackets, the number of unique, private records within each (i.e. "TOTUNIQUE").
Below is a snippet of the attributes of the "screened_areas*.shp" shown above. In it, you can see:
- The MEAN, COUNT and SUM values for each of the
agg.fields; - The number of unique values for each of the
sens.fields; - "TOTUNIQUE", showing the minimum number of all of the
sens.fields;and - "CAN_SHOW", indicating whether or not data from within this polygon can be aggregated and displayed.
The second shapefile created is "2MinGrid*.shp", and is shown below (i.e. the small, blue-ish squares). This is the original point data gridded to 2 min squares for those areas where data is allowed to be shown (i.e. "CAN_SHOW = Y" in screened_areas*.shp). In this example, the grid cells are coloured by the field "TOTNO_SUM" (one of the fields calculated from an agg.field).
Note that no gridded data can be seen in the areas that were shown "CAN_SHOW=NO" (i.e. the orange, hashed areas).
Zooming in on the "2MinGrid*.shp" shows how grid cells are not even generated for those polygon areas where "CAN_SHOW = N".
The table below shows the fields that are available for each grid cell - again, note that the fields available are calculated from the fields identified in agg.field.
