AMLytica is an event-driven financial document processing pipeline built with FastAPI, designed to process bank statements, detect anomalies, and produce structured risk reports. The system is asynchronous, message-driven, and fully decoupled, enabling each stage to scale independently and handle failures gracefully.
- Python
- FastAPI
- aio-pika
- SQLAlchemy / Alembic
- pdfplumber
- pytest/mock
- RabbitMQ
- Redis
- Postgres
- Docker Compose
-
Ingest Service
- Handles file uploads via FastAPI.
- Performs file type validation and sanity checks (size, emptiness, allowed formats).
- Validates provided customer IDs exist using Redis cache (with DB fallback on cache miss)
- Creates a job ID, persists job metadata in PostgreSQL, and publishes a message to the Ingest output queue.
-
Extraction Service
- Consumes messages from the Ingest output queue.
- Parses PDFs using pdfplumber (OCR support should be implemented at a later date).
- Extracts and transforms transactions and customer information into structured JSON data.
- Fetches customer metadata from Redis cache (with DB fallback on cache miss, writing fetched customer data to cache)
- Publishes structured document data to the Analysis output queue.
-
Analysis Service
- Consumes messages from the Extraction output queue.
- Performs risk assessment using:
- Soft flags: statistical anomalies based on standard deviation.
- Hard flags: mathematically impossible or inconsistent transactions (e.g., balance mismatches). Additionally, mismatches with document information and customer information are flagged here (name/address mismatches).
- Publishes flagged data the analysis output queue.
-
Report Service
- Consumes messages from the Analysis output queue.
- Generates per-document risk reports in structured JSON.
- Persists reports to disk for later retrieval or downstream processing.
- PDF ingestion, validation, and job creation.
- Text extraction from native PDFs using pdfplumber.
- Structured transaction parsing with normalized dates, amounts, and balances.
- Multi-stage analysis producing hard and soft flags.
- Soft flags are typically data anomalies that, while not impossible and not raising any major issues, may raise questions about the dataset's overall validity or require further inspection to verify the validity of these datapoints
- Standard deviation outliers: some transactions may lie further outside the range of other transactions than expected - as such, they are marked as outliers but do not necessarily require inspection
- Hard flags are typically metadata errors or mathematical impossibilities.
- Metadata errors: the customer's name or address listed on the document does not match their stored metadata.
- Mathematical impossibilities: actual balance figures may not reconcile with the expected balance after adding or subtracting the amount of a given transaction from the previous balance.
- Soft flags are typically data anomalies that, while not impossible and not raising any major issues, may raise questions about the dataset's overall validity or require further inspection to verify the validity of these datapoints
- JSON report generation per document.
- Fully asynchronous, message-driven pipeline with PostgreSQL-backed job state tracking.
- OCR extraction for scanned PDFs using pytesseract.
- Multi-page compatibility with proper transaction ordering.
- Automatic debit/credit classification for OCR outputs.
- Enhanced parsing and normalization for messy OCR data.
- Improved confidence scoring and error handling for extracted data.