🎯 SecretSniper: Automated GitLab Secret Scanner

SecretSniper is a lightweight, fully automated webhook server built to catch leaked secrets (API keys, tokens, passwords) the moment they are pushed to GitLab.

Powered by Betterleaks and Python, it listens for GitLab push events, securely clones only the specific commits that were pushed, scans the diffs, and sends a direct Slack message to the developer who committed the secret.

✨ Key Features

• ⚡ Blazing Fast Batch Scanning: Uses Git's --no-walk to scan only the exact commits pushed, preventing massive repo checkouts.
• 🔒 Cryptographically Secure: Validates GitLab's new HMAC signing_token payloads to prevent replay attacks and spoofing.
• 🤖 GitOps Auto-Deployment: No manual configuration needed. SecretSniper fetches its configuration live from your GitLab repository via API. Pushing a change to watch.yaml triggers an immediate sync — no need to wait for the hourly cycle.
• 🎯 Granular Targeting & Exclusions: Automatically applies webhooks to all subgroups/projects within a target Group ID, while safely skipping any explicit exclusions.
• 🧹 Auto-Cleanup: Automatically removes orphaned webhooks from projects that are removed from the exclusion list or deleted from the config.
• 💬 Intelligent Slack Integration: Automatically maps the Git commit email to a Slack user and consolidates alerts to prevent spamming developers.

---

🏗 Architecture & Flow

1. Developer pushes code to GitLab.
2. GitLab fires an HMAC-signed webhook to SecretSniper.
3. SecretSniper validates the payload and immediately returns 202 Accepted.
4. A background thread securely fetches the specific commits using a Personal Access Token (PAT).
5. betterleaks scans the exact diffs.
6. If secrets are found, SecretSniper queries the Slack API and DMs the author with a link to the exact file and commit.
7. Smart Sync: A background loop checks watch.yaml every hour, but only runs a full sync if the file content has changed. Additionally, if a push to the config repository modifies watch.yaml, the sync is triggered immediately without waiting for the hourly cycle.

---

⚙️ Prerequisites

Before deploying, you need to configure your external APIs:

1. GitLab Setup

Create a Personal Access Token (PAT) or Group Access Token with the following scopes:

• api: To automatically configure webhooks and read the watch.yaml file.
• read_repository: To clone the target repositories for scanning. (Note: The user account tied to this token must have Maintainer or Owner access to the groups you want to monitor in order to create webhooks).

2. Slack Setup

Create a Slack App in your workspace and grant it the following Bot Token Scopes:

• users:read.email: To look up users by their Git commit email.
• chat:write: To send direct messages.

---

🚀 Deployment

SecretSniper runs as a self-contained Docker image, perfectly suited for a Proxmox VM or cloud server.

1. Build the Docker Image

Clone this repository and build the image locally. (If building on an Apple Silicon Mac for a Linux server, use the platform flag!)

docker build --platform linux/amd64 -t secretsniper .

2. Environment Variables (.env)

Create a .env file on your host server. ⚠️ CRITICAL: Do not use quotation marks around the values in this file! Docker will read them literally and crash the application.

GITLAB_TOKEN=glpat-YOUR_SINGLE_TOKEN
GITLAB_SIGNING_TOKEN=whsec_YOUR_RANDOM_SECRET
SLACK_BOT_TOKEN=xoxb-YOUR_SLACK_TOKEN
WEBHOOK_URL=https://webhook.yourcompany.com/
CONFIG_PROJECT_ID=1234567
GITLAB_URL=https://gitlab.com
CONFIG_BRANCH=main

3. Run the Container

Run the container using the secure environment file so your secrets do not appear in the host's bash history.

docker run -d \
  --name secretsniper \
  --restart unless-stopped \
  -p 5000:5000 \
  --env-file /path/to/your/.env \
  secretsniper

Networking Tip: For production, it is highly recommended to expose port 5000 to the internet using Cloudflare Tunnels or an Nginx Reverse Proxy to provide GitLab with a secure HTTPS connection.

---

📂 Configuration (GitOps)

SecretSniper automatically manages its own webhooks across your organization. To tell it which repositories to secure, edit the watch.yaml file in the root of this repository.

Note: Use the numeric ID of the group or project (found on the GitLab homepage below the title).

# watch.yaml
watch:
  groups:
    - 1234567   # Secures all projects inside this group (including subgroups)
  projects:
    - 1122334   # Secures specific standalone projects

exclude:
  groups:
    - 9988776   # Skips this specific subgroup (and its projects)
  projects:
    - 5544332   # Skips this specific project, even if its parent group is watched

Once you commit and push this file to your GitLab repository, SecretSniper will detect the change and trigger an immediate sync — as long as the config repository itself has a webhook pointing at SecretSniper (i.e. it is listed under watch in watch.yaml or has a webhook configured manually). If no webhook is present on the config repo, the sync will pick up the change within the hour.

---

🧪 Testing it Works

To verify your deployment is working and the Slack integration is correct:

1. Ensure your test repository is targeted in watch.yaml.
2. Create a test commit containing a fake AWS Access Key:

   AKIAIOSFODNN7EXAMPLE
   

3. Push to GitLab.
4. You should receive a Slack DM within seconds!

🛡️ Security Notes

• This container handles highly sensitive source code. Ensure it is deployed in a secure environment.
• The Docker container temporarily clones code to /tmp/repo_.... It aggressively deletes these folders upon completion or failure, ensuring source code does not persist on disk.