Skip to content

Scan directories, exports, and backups for sensitive data (like PII and API keys) with Nightfall's data loss prevention (DLP) APIs. Discover what lives at-rest in your data silos.

main
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 

Sensitive At-Rest Data Scanner

Scan directories, exports, and backups for sensitive data (like PII and API keys) with Nightfall's data loss prevention (DLP) APIs. Discover what lives at-rest in your data silos.

This service uses Nightfall's data loss prevention (DLP) APIs to scan a folder/directory, backup, or export.

For example, you can scan a backup of your Salesforce instance to detect sensitive data in Salesforce. Salesforce houses high volumes of customer information, support tickets, quotes and files, synced emails, tasks & notes, and much more. This service will (1) send Salesforce backup data to Nightfall to be scanned, (2) run a local webhook server that retrieves sensitive results back from Nightfall, and (3) write the sensitive findings to a CSV file. This output provides a comprehensive report/audit of the sensitive data at-rest in your Salesforce tenant. The same premise extends to any service that allows you to generate a backup or export.

If you'd like a more detailed tutorial or walk-through of how this service works, we recommend reviewing our file scanner tutorial, as the components are largely the same.

Prerequisites

  • Nightfall account - sign up for free if you don't have an account
  • If you are scanning a cloud backup or export, you'll need admin access to the data silos you wish to scan, in order to create a backup or export

Usage

  1. Create a cloud backup/export of the systems you wish to scan. Download the backup and extract it locally. We've compiled instructions for a handful of popular cloud apps below.
  1. Install dependencies.
pip install -r requirements.txt
  1. Create a local ngrok tunnel to point to your webhook server.
./ngrok http 8000
  1. Set your environment variables: your Nightfall API key, your Nightfall signing secret, your Nightfall detection rule UUID, your webhook server URL from ngrok, and the path to your extracted directory/export/backup that you want to crawl.

Your Nightfall detection rule UUID is optional. If not specified, the default rule will detect likely credit card numbers, US social security numbers, and API keys.

export NIGHTFALL_API_KEY=<your_key_here>
export NIGHTFALL_SIGNING_SECRET=<your_secret_here>
export NIGHTFALL_DETECTION_RULE_UUID=<your_uuid_here>
export NIGHTFALL_SERVER_URL=https://<your_subdomain_here>.ngrok.io
export SALESFORCE_BACKUP_PATH='/Users/myuser/salesforce-exports/'
  1. Start your webhook server. This runs app.py.
gunicorn app:app
  1. Run your scan.
python scanner.py
  1. Monitor your webhook server output. Once all file scan events have been received and the scan is complete, view your results in results.csv. Each row in the output CSV will correspond to a sensitive finding. Each row will have the following fields, which you can customize in your webhook server in app.py:
  • Upload ID provided by Nightfall
  • An incrementing index
  • Timestamp
  • Filepath
  • Characters before the sensitive finding (for context)
  • Sensitive finding itself
  • Characters after the sensitive finding (for context)
  • Confidence level of the detection
  • Byte range location (character indicies) of the sensitive finding in its parent file
  • Corresponding detection rules that flagged the sensitive finding

About

Scan directories, exports, and backups for sensitive data (like PII and API keys) with Nightfall's data loss prevention (DLP) APIs. Discover what lives at-rest in your data silos.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages