feat: Infrastructure SDKs — 12/12 components with middleware layer (Go, Python, TypeScript, Rust)#39
Open
devin-ai-integration[bot] wants to merge 12 commits into
Open
Conversation
- Python DeepFace liveness engine (passive + active challenges, anti-spoofing) - Python document OCR engine (PaddleOCR, VLM classification, Docling parsing) - Go KYC orchestrator (NIN/BVN/CAC verification, AML screening, risk scoring) - Rust identity matching engine (embedding comparison, fraud detection) - TypeScript tRPC routers + comprehensive KYC/KYB frontend pages - KYC gate integration into Claims flow - API clients for all 4 backend services Co-Authored-By: Patrick Munis <pmunis@gmail.com>
Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…e ThemeProvider) Co-Authored-By: Patrick Munis <pmunis@gmail.com>
- Revert vite.ts to use inline config spread (configFile: false) instead of configFile path - Revert vite.config.ts to remove define/dedupe/optimizeDeps additions that didn't fix React hooks issue - These reverts restore the original working configuration from previous PRs Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…t plugin double-init) Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…oral, PostgreSQL, Keycloak, Permify, Redis, Mojaloop, OpenSearch, OpenAppSec, APISix, TigerBeetle, Lakehouse Go orchestrator (8085): - PostgreSQL persistence replacing in-memory maps - Redis caching for KYC session lookups - Kafka producer for KYC completion events - Temporal client for workflow orchestration - OpenSearch auditor for compliance trail - APISix gateway with OpenAppSec WAF plugin - Mojaloop bridge for mobile money KYC-gated transfers - Keycloak/Permify authorization middleware - All 9 middleware clients wired into main.go Rust ledger service (8113): - TigerBeetle double-entry ledger with KYC-level transfer limits - Dapr sidecar for state management and pub/sub - OpenAppSec WAF validation on all requests - 10 ledger types with KYC level requirements Python services: - Lakehouse analytics (8114) with Delta Lake compliance reporting - Fluvio stream processor (8115) with WebSocket real-time events TypeScript platform integration: - KYC gate checks on claims.create, payments.process, wallet.topUp/withdraw - KYC gate on application.create/submit with level requirements - Onboarding wired to trigger KYC verification on identity step - KYB wired to Go orchestrator for CAC/TIN/director/UBO verification - Middleware integration endpoints (ledger stats, analytics metrics, stream topics, transfer limits, NDPR report) - New service clients: kycLedgerService, kycAnalyticsService, kycStreamService, checkKYCGate helper Co-Authored-By: Patrick Munis <pmunis@gmail.com>
- 6 PyTorch models: fraud detection (residual+attention), churn prediction (GLU), claims adjudication (multi-task), credit scoring (Wide&Deep), anomaly detection (VAE), GNN fraud ring detection (GraphSAGE) - Synthetic Nigerian insurance data generation (275k+ samples across 6 domains) - Real training loops with FocalLoss, OneCycleLR, early stopping, metric tracking - Trained .pt weight files for all 6 models - ONNX export for CPU-optimized inference (4 models) - Delta Lake feature store with versioning (6 tables) - MCMC Bayesian risk modeling with NumPyro/JAX (16 product lines, VaR/CVaR) - Ray distributed training infrastructure with local fallback - Neo4j graph schema for fraud ring detection with offline mode - FastAPI inference server for all models - All models run on CPU (no GPU required) Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…sioning, scheduled retraining, platform data ingestion - drift_detector.py: PSI, KS test, JS divergence for data drift + performance monitoring - model_registry.py: Champion-challenger versioning with auto-promotion - data_ingestion.py: Platform data connectors with watermarking and fallback chain - pipeline.py: 5-step orchestration (ingest → drift → retrain → validate → promote → ONNX export) - scheduler.py: Cron-based + event-driven triggers with background thread - api.py: FastAPI endpoints for CT management (/ct/retrain, /ct/drift, /ct/models, /ct/scheduler) - Fixed api_server.py imports for standalone execution - All 4 models retrained, promoted, and exported to ONNX with zero errors Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…g in CT API drift check Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…eaming ingestion, online serving, lineage, RBAC, Feature Store API, Go SDK Components implemented: - Storage: Object store abstraction (Local/S3/MinIO) with unified interface - Schema: Registry with versioning, compatibility checks (backward/forward/full), evolution tracking - Streaming: Kafka/Fluvio ingestion engine with micro-batching, DLQ, checkpointing - Computation: Real-time feature engine with sliding windows, EMA, time-decay scoring - Serving: Online feature server with L1 (LRU) + L2 (Redis) + L3 (Delta Lake) caching - API: FastAPI REST API with DuckDB SQL queries, CRUD, materialization endpoints - Lineage: Full DAG tracking (source→table→model), quality metrics, mutation audit - RBAC: Role-based access control with table/column-level policies, audit logging - Connectors: Python EventBridge + Go SDK for microservice event publishing - All components tested with functional verification (9 features computed, 3 events delivered) Co-Authored-By: Patrick Munis <pmunis@gmail.com>
Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…o, Python, TypeScript, Rust) Shared SDK libraries for all 12 infrastructure components: - PostgreSQL: connection pooling, migrations, JSONB, audit trail - TigerBeetle: KYC-level transfer limits, 6 ledger codes, batch transfers - Redis: session management, rate limiting, KYC gates, pub/sub, distributed locks - Mojaloop: mobile money interop, KYC-gated transfers, idempotency keys - Kafka: 16 platform topics, idempotent producer, DLQ support, audit events - APISix: rate limiting, OIDC, IP restriction, WAF, health checks - Keycloak: token validation, KYC level attributes, 5-min TTL caching - OpenAppSec: SQL injection, XSS, path traversal blocking - Permify: fine-grained RBAC, schema-based permissions, default-deny - OpenSearch: audit log indexing, ILM policies, structured search - Fluvio: real SDK integration, 11 platform topics, event streaming - Dapr: state management, pub/sub, service invocation Middleware layer (Go/Python/TypeScript): 1. Rate limiting (Redis) 2. Token validation (Keycloak) 3. KYC gate enforcement (Redis + Keycloak) 4. RBAC permission checks (Permify) 5. Async audit logging (OpenSearch + Kafka + Fluvio) All SDKs compile clean: - Go: go vet ./... passes - Python: py_compile all files pass - TypeScript: tsc --noEmit passes - Rust: cargo check passes Co-Authored-By: Patrick Munis <pmunis@gmail.com>
Author
Original prompt from Patrick
|
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Author
E2E Test Results — Infrastructure SDKsSession: https://app.devin.ai/sessions/0475192a778b45cea30202f85ad52b63 SummaryTested all 4 SDK languages for compile correctness, import/initialization, middleware consistency, and graceful degradation. 10/12 tests passed, 2 findings noted. Test Results Table
Finding: Middleware Fail-Open vs Fail-Closed Inconsistency
Impact: When deployed without Redis, Go services block all traffic while Python/TS services allow it through. Fix: Either change Go to |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Massive platform buildout adding shared infrastructure SDKs, an AI/ML continuous training platform, a KYC/KYB verification system, and 40+ insurance product microservices. This PR spans multiple subsystems:
Infrastructure SDKs (
infrastructure/): Shared client libraries in Go, Python, TypeScript, and Rust for all 12 platform components (PostgreSQL, Redis, Kafka, Keycloak, Permify, OpenSearch, Fluvio, Dapr, TigerBeetle, Mojaloop, APISix, OpenAppSec). Includes a middleware layer (Go/Python/TS) enforcing rate limiting → token validation → KYC gates → RBAC → async audit logging on every request.AI/ML Platform (
ai-ml-platform/): Continuous training pipeline with drift detection (KS/PSI tests), model registry (champion/challenger), scheduled retraining, data ingestion from platform services, and a FastAPI inference server. Includes Lakehouse feature store with Delta Lake, schema registry, and data lineage.KYC/KYB System (
kyc-kyb-system/): Document OCR (PaddleOCR), face verification (DeepFace), liveness detection, identity matching engine (Rust), Temporal workflow orchestrator (Go), Fluvio stream processor, and platform middleware integration.Microservices & Products: 40+ Go/Python/Rust microservices for insurance domains (parametric, peer-to-peer, livestock, crop, health, micro-insurance, climate risk, etc.) with handlers, repositories, and service layers.
Customer Portal (
customer-portal-full/): React dashboard with tRPC procedures, role-based sidebar navigation, and pages for all product categories.Review & Testing Checklist for Human
password,secret,mock,demoacross the diff. None of these should reach production without real credential injection."cac_verified": true,"risk_score": 0.15). The Rust SDK stubs log actions but don't connect to services. Verify which modules are genuinely functional vs. scaffolded.except Exception: pass). Decide whether fail-open is acceptable per component or if fail-closed is required for KYC/RBAC enforcement.Recommended test plan:
ai-ml-platform/) and hit prediction endpoints to confirm models load/api/v1/stream/publishendpointgo vet ./...ininfrastructure/go-sdk/,cargo checkininfrastructure/rust-sdk/,tsc --noEmitininfrastructure/ts-sdk/to confirm compile checks passNotes
target/directory is gitignored.Link to Devin session: https://app.devin.ai/sessions/0475192a778b45cea30202f85ad52b63