Skip to content

Commit

Permalink
feat(aim): add new docs content
Browse files Browse the repository at this point in the history
  • Loading branch information
akristen committed Jan 26, 2024
1 parent a80105d commit 7f3cddd
Showing 1 changed file with 215 additions and 0 deletions.
215 changes: 215 additions & 0 deletions src/content/docs/ai-observability/filter-pii.mdx
@@ -0,0 +1,215 @@
---
title: 'Remove PII with drop filters'
metaDescription: 'Drop filters use regex to filter out personally identifying information before it reaches our databases.'
freshnessValidatedDate: never
---

When you monitor your AI-powered app, you may want to drop sensitive data before it enters the New Relic Database (NRDB). Drop rules let you create NRQL queries that filter for personally identifiable information (PII), like birthdays or social security numbers, using regex. This prevents any data from entering into NRDB, protecting you and your customers from theft.

SCREENSHOT HERE

## Build your drop filters [#sec-compliance]

Each drop filter builds off the

* `LlmChatCompletion`
* `LlmTool`
* `LlmChain events`
* `LlmChatCompletionSummary`
* `LlmChatCompletionMessage`

## Create drop filters [#create]

Drop filters make it possible to drop PII, but there are some steps you must do to stay fully protected. Read these procedures carefully as you make your first drop rule.

<Steps>
<Step>

## Go to the drop filters page

Go to **[one.newrelic.com > All Capabilities > AI Monitoring > Drop filters](https://onenr.io/0PoR8KlvYwG)**, then click **Create drop filter**.

</Step>
<Step>

## Create a drop filter

Drop filters use NRQL queries to drop sensitive data. Here are three examples of a NRQL query using regex to locate and drop sensitive data:

* EXAMPLE
* EXAMPLE
* EXAMPLE

There's a secion at the end of this procedure with regex samples to get you started with drop filters.

</Step>
<Step>

## Add drop filter to four additional tables

New Relic stores its data in five different tables. When you create a drop filter to drop one kind of data (say, a birth date), you need to repeat the process four additional times to account for those five tables.

</Step>
<Step>

## Test your drop filter

To test your drop filter, ELEPHANT

</Step>
</Steps>

## Sample expressions [#regex-ref]

Below, we've provided some sample regex expressions to obfuscate some of the most common sensitive data types. Obfuscation expressions must be created for each New Relic account where those expressions will be in use.

<Callout variant="tip">
The following examples are regular expressions you could use in the UI. To use these in GraphQL, you'd need to escape them as shown in this [example](#expressions-create).
</Callout>

<CollapserGroup>
<Collapser
id="ssn"
title="US social security number"
>
**Expression:**
```
(\d{3}[-\s\.]?\d{2}[-\s\.]?\d{4})
```
</Collapser>

<Collapser
id="ccn"
title="Credit card number"
>
**Expression:**
```
((?:(?:4\d{3})|(?:5[1-5]\d{2})|6(?:011|5[0-9]{2}))(?:-?|\040?)(?:\d{4}(?:-?|\040?)){3}|(?:3[4,7]\d{2})(?:-?|\040?)\d{6}(?:-?|\040?)\d{5})
```
</Collapser>

<Collapser
id="dob"
title="US Date of birth"
>
**Expression:**
```
((?:\d{2})?\d\d(?:\\)?(?:\/)?\d\d(?:\\)?(?:\/)?\d{2}(?:\d{2})?)
```
</Collapser>

<Collapser
id="email"
title="Email address"
>
**Expression:**
```
([a-zA-Z0-9!#$'*+?^_`{|}~.-]+(?:@|%40)(?:[a-zA-Z0-9-]+\.)+[a-zA-Z0-9-]+)
```
</Collapser>

<Collapser
id="ipv4"
title="IP address (ipv4)"
>
**Expression:**
```
^([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})\.([0-9]{1,3})$
```
</Collapser>

<Collapser
id="usstreetaddress"
title="US street address"
>
**Expression:**
```
/\d{1,}(\s{1}\w{1,})(\s{1}?\w{1,})/g
```
</Collapser>

<Collapser
id="usphone"
title="US phone number"
>
**Expression:**
```
(^[\+]?[1]?[\W]?[(]?[0-9]{3}[)]?[-\s\.]?[0-9]{3}[-\s\.]?[0-9]{4})
```
</Collapser>

<Collapser
id="uspassport"
title="US passport number"
>
**Expression:**
```
([a-zA-Z]?\d?\d{5,8})
```
</Collapser>

<Collapser
id="uknino"
title="UK national insurance number (NINO)"
>
**Expression:**
```
([a-zA-Z]{2}[-\s]?\d{2}[-\s]?\d{2}[-\s]?\d{2}[-\s]?[a-dA-D])
```
</Collapser>

<Collapser
id="spainnid"
title="Spain National ID (NIE/DNI/NIF)"
>
**Expression:**
```
([a-zA-Z]?[-\s]?\d{7,8}[-\s]?[a-zA-Z])
```
</Collapser>

<Collapser
id="indiapanid"
title="India PAN ID"
>
**Expression:**
```
^([a-zA-Z]){5}([0-9]){4}([a-zA-Z]){1}?$
```
</Collapser>

<Collapser
id="indiaaadhaar"
title="India AADHAAR ID"
>
**Expression:**
```
^([2-9]{1}[0-9]{3}\s\d{4}\s\d{4})$
```
</Collapser>

<Collapser
id="caphin"
title="Canada personal health/social insurance number (PHIN/SIN)"
>
**Expression:**
```
(\d{3}[-\s\.]?\d{3}[-\s\.]?\d{3})
```
</Collapser>

<Collapser
id="cahcn"
title="Canada health card number"
>
**Expression:**
```
(\d{10})
```
</Collapser>

</CollapserGroup>

## What's next? [#whats-next]

blahblahblah

0 comments on commit 7f3cddd

Please sign in to comment.