Skip to content

Andyeedev/data_cleansing

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Migration Validation Engine v1.0

Enterprise-grade Financial Services Migration Assurance Platform

Overview

The Migration Validation Engine provides automated, metadata-driven validation controls to assure financial system data migrations across:

Core Banking

Payments

Lending

Asset Management

Regulatory Reporting Systems

Designed for:

Tier 1–3 Banks

Financial Institutions

Regulated Enterprises

Key Features

10 structured migration controls

Metadata-driven rule execution

Entity + Control + Rule-level tracking

Exception repository

Control-level scoring

CSV audit export

Docker deployable

Architecture

Python 3.11

PostgreSQL 15+

Dockerised execution

SQL-driven validation templates

Setup (Local Development)

  1. Install

Python 3.11+

Docker Desktop

VSCode

  1. Clone Repository git clone cd migration-validation-engine

  2. Create Virtual Environment python -m venv .venv source .venv/bin/activate (Mac/Linux) .venv\Scripts\activate (Windows)

  3. Install Dependencies pip install -r requirements.txt

  4. Create PostgreSQL Databases engine_db source_db target_db

  5. Load Schema

Run:

sql/schema/01_engine_schema.sql

  1. Load Demo Data

Run:

sql/demo/01_demo_source.sql (source_db) sql/demo/02_demo_target.sql (target_db)

  1. Run Engine python app/main.py --config config.yaml --batch-id 001

Docker Deployment docker-compose up --build

Outputs

migration_validation_batch

migration_control_execution

migration_exception_register

CSV export in /exports

About

Data Cleansing

Resources

Stars

Watchers

Forks

Packages

 
 
 

Contributors