Skip to content

Latest commit

Β 

History

5 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸ›‘οΈ SENTINEL

AI-Powered Financial Crime Detection, Investigation & Intelligence Platform

Python Version FastAPI PostgreSQL Neo4j Docker License: MIT

An enterprise-grade anti-money laundering (AML), fraud detection, and case investigation intelligence platform built with modern asynchronous Python, graph analytics, and explainable AI.


πŸ“‘ Table of Contents


🎯 Executive Overview

Financial institutions face an increasingly sophisticated wave of financial crimeβ€”from distributed money mule syndicates and rapid structuring to synthetic identity bust-out fraud. Traditional rule-only legacy systems suffer from catastrophic false-positive rates (>95%), operational fatigue, and an inability to detect complex multi-hop network typologies.

Sentinel is engineered from the ground up as a hybrid AI/Graph platform that unifies:

  1. High-throughput asynchronous event processing for sub-second risk scoring.
  2. Multi-dimensional behavioral feature engineering (velocity, z-scores, counterparty diversity, device/IP risk).
  3. Declarative, YAML-driven AML rule evaluation with human-auditable condition trees.
  4. Deep Graph analytics (Neo4j) for uncovering money mule rings, pass-through entities, and shared device clusters.
  5. Ensemble Risk Fusion combining deterministic heuristics and ML anomaly scores with SHAP explainability.
  6. Full-lifecycle Case Management & Investigation Copilot allowing investigators to triage alerts, trace evidence trails, record immutable audit events, and prepare Suspicious Activity Reports (SAR).

πŸ›οΈ How It Works: End-to-End Architecture

flowchart TD
    subgraph Ingestion["1. Data Ingestion & Events"]
        TXN["Incoming Transactions"]
        CUST["Customer Profile / KYC"]
        DEV["Device & IP Telemetry"]
        BEN["Counterparty Entities"]
    end

    subgraph FeaturePipeline["2. Feature Engineering Pipeline"]
        TEMP["Temporal Features\n(Hour, Night, Gap)"]
        AMT["Amount & Stats\n(Z-Score, Log1p, 10k/50k)"]
        VEL["Velocity Windows\n(1h, 6h, 24h Ratios)"]
        GRAPH_FEAT["Graph Metrics\n(Fan-out, Degree)"]
        DEV_FEAT["Device/IP Risk\n(Sharing, VPN/Tor)"]
    end

    subgraph DetectionEngines["3. Multi-Layer Detection Engines"]
        RULE_ENG["Declarative Rule Engine\n(YAML Configured Typologies)"]
        ML_ENG["ML Anomaly Detector\n(Isolation Forest / XGBoost + SHAP)"]
        GRAPH_ENG["Neo4j Graph Engine\n(Mule Rings, Shared Fingerprints)"]
    end

    subgraph Fusion["4. Risk Fusion & Alerting"]
        RF["Risk Fusion Engine\nWeighted Aggregate (0 - 100)"]
        ALERT_GEN["Alert Generator\nPriority: CRITICAL | HIGH | MED | LOW"]
    end

    subgraph CaseMgmt["5. Investigation & Case Management"]
        QUEUE["Alert Triage Queue"]
        CASE["Case Workspace & Evidence Graph"]
        AUDIT["Immutable Audit Events"]
        SAR["SAR / STR Regulatory Package"]
    end

    Ingestion --> FeaturePipeline
    FeaturePipeline --> DetectionEngines
    RULE_ENG --> RF
    ML_ENG --> RF
    GRAPH_ENG --> RF
    RF --> ALERT_GEN
    ALERT_GEN --> QUEUE
    QUEUE --> CASE
    CASE --> AUDIT
    CASE --> SAR
Loading

πŸ” Core Detection Subsystems

1. Real-Time Feature Engineering Pipeline

Implemented in src/sentinel/features/engineering.py, this pipeline transforms raw transaction rows and account histories into ML-ready behavioral feature vectors:

  • Temporal Features: Hour of day, day of week, is_nighttime (10 PM – 6 AM), and minutes elapsed since last transaction.
  • Amount & Statistical Profiles: amount_log (log1p transform to handle power-law distributions), historical amount_zscore $(x - \mu)/\sigma$, ratio of amount to customer's expected monthly volume, round-number heuristic flags (structuring indicators), and statutory threshold violations ($10,000 / $50,000).
  • Velocity Windows: Sliding transaction counts and volume sums across 1-hour, 6-hour, and 24-hour windows, with velocity acceleration ratios.
  • Device & IP Intelligence: Device sharing multiplier (count of unique customer IDs utilizing the same fingerprint), proxy/VPN/Tor flags, and ASN reputation.
  • Network & Counterparty Diversity: New beneficiary indicators, counterparty concentration index, and historical transaction frequency with the counterparty.
  • Behavioral Drift: Deviation from customer segment baseline and dormant account reactivation flags (e.g., sudden burst after >90 days dormancy).

2. Declarative Rule Engine

Located in src/sentinel/rules/engine.py, the rule engine provides transparent, deterministic detection.

  • Why Declarative YAML? Allows compliance officers and AML analysts to modify detection thresholds and deploy new rules without code deployments or downtime.
  • Composability: Supports AND / OR Boolean logic trees with comparative operators (gt, gte, lt, lte, eq, neq, in, not_in, is_true, is_false).
  • Explainable Evidence Generation: Every triggered rule outputs an evidence dictionary detailing exact feature values versus thresholds.

Supported Typology Rules:

Rule ID Rule Name Severity Core Trigger Conditions
RULE_HIGH_VELOCITY Transaction Velocity Spike HIGH txn_count_last_6h > 5 AND velocity_6h_ratio > 3.0
RULE_NEW_BNF_LARGE New Beneficiary Large Outflow CRITICAL is_new_beneficiary == True AND amount_pct_monthly > 0.5
RULE_SHARED_DEVICE Multi-Customer Shared Device HIGH device_customer_count > 4 AND device_is_suspicious == True
RULE_GEO_DEVIATION Cross-Border High-Risk Origin MEDIUM is_home_country == False AND is_high_risk_country == True
RULE_STRUCTURING Smurfing / Structuring Detection CRITICAL $45,000 < amount < $50,000 AND txn_count_24h > 3
RULE_DORMANT_ACTIVATION Dormant Account Reactivation HIGH days_since_last_txn > 90 AND txn_count_24h > 2

3. Graph Intelligence & Network Analysis

Relational databases struggle to query multi-hop relationships. Sentinel pairs PostgreSQL with Neo4j 5.24 Community (with APOC and Graph Data Science plugins) to model entities as an interconnected graph:

  • Nodes: (:Customer), (:Account), (:Device), (:IPAddress), (:Merchant), (:Beneficiary).
  • Edges: [:OWNS], [:PERFORMED_TRANSACTION], [:USED_DEVICE], [:ACCESSED_FROM_IP], [:SENT_FUNDS_TO].
  • Detection Capabilities:
    • Mule Rings: Identifies circular transaction flows ($A \rightarrow B \rightarrow C \rightarrow A$) and high fan-in / high fan-out pass-through accounts.
    • Device Syndicate Detection: Traverses (:Customer)-[:USED_DEVICE]->(:Device)<-[:USED_DEVICE]-(:Customer) to detect coordinated account takeovers or synthetic identity rings.

4. ML Anomaly Scoring & SHAP Explainability

  • Unsupervised Anomaly Detection: Utilizes Isolation Forests and autoencoder reconstruction error to identify zero-day transaction anomalies that evade explicit rules.
  • Explainable AI (XAI): Generates SHAP (SHapley Additive exPlanations) values for every model inference, displaying which exact features drove the anomaly score.

5. Dynamic Risk Fusion Engine

Combines multi-modal inputs into a normalized risk score $[0, 100]$:

$$\text{Final Risk Score} = w_{\text{rule}} S_{\text{rule}} + w_{\text{ml}} S_{\text{ml}} + w_{\text{graph}} S_{\text{graph}} + w_{\text{device}} S_{\text{device}} + w_{\text{geo}} S_{\text{geo}}$$

  • Score Calibration & Dynamic Thresholding:
    • CRITICAL ($\ge 85$): Auto-creates high-priority Case, freezes automated outflows.
    • HIGH ($70 - 84$): Direct alert assigned to Senior AML Investigator.
    • MEDIUM ($40 - 69$): Batched in queue for standard investigator review.
    • LOW ($&lt; 40$): Logged for behavioral baselining and statistical monitoring.

πŸ’Ό Investigation & Case Management Lifecycle

Sentinel provides an investigator-first interface designed to minimize time-to-decision:

  1. Alert Triage: Real-time alert list with server-side pagination, severity filtering, status workflows (OPEN, ASSIGNED, INVESTIGATING, CLOSED_FALSE_POSITIVE, CLOSED_TRUE_POSITIVE_SAR).
  2. Case Workspace: Consolidates all related alerts, customer KYC profile, account ledger history, and graph neighborhood into a unified investigation dossier.
  3. Audit Trail: Immutable append-only investigation_events logging every investigator interaction (note added, tag updated, status transitioned, SAR generated).
  4. SAR Generation: Produces structured regulatory narratives outlining timeline, typologies triggered, total exposed amounts, and entity identities ready for FinCEN / FIU submission.

πŸ—οΈ How We Built It: Architectural & Engineering Decisions

sentinel/
β”œβ”€β”€ docker/                     # Container configurations & PostgreSQL initialization
β”‚   β”œβ”€β”€ backend.Dockerfile      # Multi-stage Python 3.12 build
β”‚   └── postgres/init.sql       # Schema DDL, indexes, and extensions
β”œβ”€β”€ docker-compose.yml          # Postgres, Neo4j, Backend, Frontend topology
β”œβ”€β”€ pyproject.toml              # Build toolchain, ruff, mypy, pytest configs
β”œβ”€β”€ scripts/
β”‚   β”œβ”€β”€ generate_data.py        # Deterministic synthetic universe generator
β”‚   β”œβ”€β”€ seed_demo.py            # Recruiter interactive demo scenario seeder
β”‚   └── test_rules.py           # Standalone rule evaluation runner
β”œβ”€β”€ src/sentinel/
β”‚   β”œβ”€β”€ api/
β”‚   β”‚   β”œβ”€β”€ main.py             # FastAPI factory, middleware, exception handlers
β”‚   β”‚   └── routers/            # Modular REST endpoints (auth, cases, alerts, etc.)
β”‚   β”œβ”€β”€ common/
β”‚   β”‚   └── logging.py          # Structlog JSON logging configuration
β”‚   β”œβ”€β”€ config/
β”‚   β”‚   └── settings.py         # Pydantic Settings with env validation
β”‚   β”œβ”€β”€ db/
β”‚   β”‚   β”œβ”€β”€ base.py             # SQLAlchemy Declarative Base
β”‚   β”‚   β”œβ”€β”€ session.py          # Async & Sync engine session factories
β”‚   β”‚   └── models/models.py    # Enterprise ORM schema definitions
β”‚   β”œβ”€β”€ features/
β”‚   β”‚   └── engineering.py      # Vector computation & statistical profiling
β”‚   └── rules/
β”‚       └── engine.py           # YAML condition evaluation engine
└── tests/
    β”œβ”€β”€ api/                    # HTTP endpoint integration tests
    β”œβ”€β”€ conftest.py             # Test fixtures & mock database
    └── unit/                   # Unit tests for settings, rules, features

Engineering Decisions:

  1. Asynchronous Architecture with FastAPI & SQLAlchemy asyncpg:
    • Avoids blocking I/O on heavy analytical queries and allows concurrent alert ingestion while investigators query case dashboards.
  2. Structured Logging & Distributed Tracing:
    • Utilizes structlog to bind unique request_id, HTTP path, latency (duration_ms), and actor identity into structured JSON logs, enabling ELK / Datadog ingest.
  3. Dual Session Architecture (async_session + sync_session):
    • Asynchronous sessions power the web API routers for maximum throughput; synchronous sessions are dedicated to high-volume CLI batch generators and offline ML feature pipelines.
  4. Pydantic v2 Settings Management:
    • Strict configuration validation with automatic .env loading and environment isolation (development, staging, production).

πŸ“Š Data Models & Schema Architecture

Sentinel utilizes a relational schema optimized with composite B-tree and GIN indexes in PostgreSQL:

  • customers: Master KYC records, date of birth, PEP (Politically Exposed Person) flags, sanctions status, risk profile (LOW, MEDIUM, HIGH, CRITICAL), and expected monthly turnover.
  • accounts: Multi-currency accounts tied to customers, balances, opening dates, and operating statuses (ACTIVE, FROZEN, DORMANT, CLOSED).
  • transactions: Double-entry ledger records with high-precision decimals (NUMERIC(18, 2)), timestamps, transaction types (TRANSFER, WIRE, CASH_DEPOSIT, CARD_PAYMENT, CRYPTO_PURCHASE), channels, and counterparty metadata.
  • devices & ip_addresses: Telemetry tracking fingerprint IDs, OS, browsers, VPN/Tor/Proxy flags, ASN, and multi-customer association counts.
  • alerts & rule_results: Stores generated alerts, risk scores, contributing rule IDs, condition evaluations, and feature snapshots in JSONB.
  • cases & investigation_events: Case dossiers with assigned investigator IDs, severity ratings, notes, tags, dispositions, and immutable event history.

πŸ”Œ API Reference & System Endpoints

The API is fully documented via interactive Swagger UI at /docs. Key endpoint namespaces include:

πŸ” Authentication (/api/v1/auth)

  • POST /login β€” Authenticate and obtain JWT bearer token.
  • GET /me β€” Retrieve current authenticated investigator profile.

πŸ“ˆ Executive & Operational Dashboard (/api/v1/dashboard)

  • GET / β€” Real-time metrics: 24h transaction volume, critical alerts count, open cases, and average portfolio risk.

🚨 Alert Management (/api/v1/alerts)

  • GET / β€” Paginated alerts query with filters for status, priority, min_score, assigned_to.
  • GET /{alert_id} β€” Full alert breakdown with triggered rule evidence and ML SHAP explanations.
  • PATCH /{alert_id}/status β€” Update alert triage state and investigator assignment.

πŸ’Ό Case Management (/api/v1/cases)

  • GET / β€” List investigation cases.
  • POST / β€” Escalate alert into an official investigation case.
  • GET /{case_id} β€” Comprehensive case dossier including timeline history.
  • PATCH /{case_id} β€” Update case severity, disposition, notes, or tags.
  • POST /{case_id}/events β€” Append investigator note or audit event to timeline.

πŸ‘₯ Customer & Entity 360 (/api/v1/customers, /api/v1/accounts, /api/v1/transactions)

  • GET /customers/{customer_id} β€” 360Β° Customer profile, linked accounts, and recent alerts.
  • GET /accounts/{account_id}/transactions β€” Historical account ledger feed.
  • GET /data-quality β€” Ingestion data quality health checks and anomaly diagnostics.

🎲 Synthetic Financial Universe & Typology Generation

To enable realistic testing without exposing PII (Personally Identifiable Information), Sentinel includes a deterministic synthetic universe generator in scripts/generate_data.py:

# Generate 1,000 customers, 2,500 accounts, and 50,000 transactions with embedded fraud typologies
python scripts/generate_data.py --seed 42 --customers 1000 --transactions 50000

Embedded Scenarios:

  1. Structuring / Smurfing Ring: Multiple cash deposits below reporting threshold ($9,500) within 48 hours followed by rapid wire transfer abroad.
  2. Mule Pass-Through: Influx of funds from multiple distinct originators immediately consolidated and transferred to a single offshore account.
  3. Account Takeover / Device Collision: Normal domestic customer account accessed from a known Tor exit node via a device shared with 10+ other compromised accounts.
  4. Dormant Awakening: An account inactive for 180 days suddenly executes high-value international wires.

πŸš€ Quickstart & Deployment Guide

Prerequisites


Running with Docker Compose (Recommended)

  1. Clone the repository:

    git clone https://github.com/dev-avneeshk/sentinel.git
    cd sentinel
  2. Configure environment variables:

    cp .env.example .env
  3. Spin up all containers:

    docker-compose up -d --build
  4. Verify container health:


Running Locally for Development

  1. Create and activate a virtual environment:

    python3 -m venv .venv
    source .venv/bin/activate
  2. Install dependencies:

    pip install -r requirements.txt
    pip install -e .
  3. Start backend with hot-reload:

    uvicorn sentinel.api.main:app --host 0.0.0.0 --port 8000 --reload

Interactive Recruiter Demo Walkthrough

We provide a dedicated seeder script in scripts/seed_demo.py that initializes pre-configured investigation cases, suspicious customer profiles, shared device rings, and investigator accounts:

python scripts/seed_demo.py

Pre-Configured Demo Credentials:

Role Username Password
Lead Investigator investigator sentinel_demo_2024
Compliance Analyst analyst sentinel_demo_2024
System Admin admin sentinel_demo_2024

Recommended Demo Exploration Flow:

  1. Navigate to http://localhost:8000/docs in your browser.
  2. Authenticate via /api/v1/auth/login using investigator / sentinel_demo_2024.
  3. Check the executive metrics at /api/v1/dashboard.
  4. Inspect the alert queue at /api/v1/alertsβ€”notice how alerts feature granular rule triggers, score contributions, and device risk metrics.
  5. Review the pre-populated case at /api/v1/cases to inspect the full chronological investigation timeline.

πŸ§ͺ Testing & Quality Assurance

Sentinel maintains a comprehensive suite of unit, integration, and API tests:

# Run complete test suite with verbose logging
pytest

# Run unit tests only
pytest tests/unit

# Run API integration tests
pytest tests/api

# Run linting and code formatting checks
ruff check .
ruff format --check .

πŸ”’ Security & Compliance Considerations

  • Audit Immutability: Investigation events cannot be modified or deleted once recorded, preserving evidential integrity for court and regulatory proceedings.
  • Role-Based Access Control (RBAC): Strict partition between Analysts (read/triage), Investigators (disposition/SAR creation), and Administrators (user/rule configuration).
  • Zero PII Exposure in Logs: All logging utilities mask credit card numbers, tax IDs, and confidential banking credentials.
  • Cryptographic Password Hashing: Utilizes salted bcrypt password hashing for all user accounts.

πŸ—ΊοΈ Future Roadmap

  • Real-time Apache Kafka Stream Integration: Ingestion of continuous SWIFT / ISO 20022 payment streams.
  • Graph Neural Networks (GNNs): PyTorch Geometric integration for automated sub-graph anomaly embeddings.
  • Automated LLM Narrative Generation: Zero-shot SAR draft generation grounded in graph traversal evidence.
  • Continuous Rule Backtesting: Automated simulation of rule threshold adjustments against 12 months of historical data.

πŸ“„ License

This project is licensed under the MIT License β€” see the LICENSE file for details.

Built with precision for high-performance financial intelligence.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages