This project is an Autonomous BI Ecosystem that goes beyond simple dashboards. It leverages a Multi-Agent system to ingest raw SQL or CSV data, perform automated MLOps validation, generate time-series forecasts, and provide strategic business insights.
The Core Innovation: A strict Numerical Integrity Enforcement layer that acts as a "Straitjacket" for Large Language Models, physically preventing hallucinations by validating every metric against database ground truth before it reaches the user.
Most BI tools fail because LLMs hallucinate numbers. We solved this. This project integrates a high-performance Multi-Agent system with a rigid MLOps pipeline to ensure that every strategic insight is backed by mathematically verified ground truth.
At the center of this project is a specialized "Council of Agents" that collaborate to turn raw data into executive strategy.
- Responsibility: The "Source of Truth."
- Capabilities: Executes complex SQL queries, interacts with the Qdrant Vector DB for context, and calculates raw financial metrics.
- Power: It is physically restricted to returning raw data only—no opinions.
- Responsibility: Strategic Interpretation.
- Capabilities: Analyzes the clerk's data against the 200-day baseline, identifies 33.57% growth spikes, and warns about category monopolies.
- Power: Uses a specialized prompt to bridge the gap between MLflow metrics (MAPE) and real-world risks.
This is the project's most powerful innovation. We implemented a Deterministic Gate between the AI and the User to kill hallucinations:
- Parity Enforcement: The system extracts the Strategist's JSON and compares every digit against the Clerk's SQL result.
- Zero-Tolerance Policy: If the AI rounds 33.57% to 33.6%, the gate blocks the response and triggers a
NumericalIntegrityError. - Hybrid Parsing: We built a custom parser using
ast.literal_evalto handle LLM syntax variance (single quotes vs double quotes), ensuring the system never crashes due to "lazy" AI handwriting.
We used industry-standard tools to automate the entire data lifecycle:
- 🔄 Prefect: Orchestrates the pipeline, ensuring Deepchecks runs before the Prophet model training starts.
- 📊 MLflow: Logs every "Experiment." We tracked our model reaching a highly precise 2.82% MAPE over a 200-day history.
- 🔍 Evidently AI: Automatically generates Data Drift reports. It flags "Concept Drift" if your revenue suddenly shifts categories (e.g., from Kitchen to Electronics).
- 🛡️ Deepchecks: Acts as the "Data Policeman," scanning for duplicates in
order_idor logical inconsistencies in the 200-day ledger. - ⚡ Qdrant: Stores semantic embeddings of business contexts, allowing the agents to "remember" past trends.
- Semantic Mapping (
mapper.py): Uses AI to instantly understand your DB schema. Whether your column is namedtotal_amtorPrice_Final, the system maps it toamount. - The Pipeline: Prefect triggers:
- Deepchecks (Integrity Check)
- Evidently (Drift Report)
- Prophet (7-Day Forecast via MLflow)
- Agent Reasoning: CrewAI kicks off the Clerk and Strategist.
- Verification: The
enforcement.pygate validates the math. - Delivery: The user receives a verified, 200-day trend analysis in a clean UI.
├── app/ │ ├── flows/ │ │ └── bi_pipeline.py # 🔄 Prefect Orchestration │ ├── llm/ │ │ └── groq_client.py # 🤖 LLM Interface │ ├── logic/ │ │ ├── enforcement.py # 🛡️ THE GATEKEEPER (Numerical Integrity) │ │ ├── forecasting.py # 📈 MLflow & Prophet Logic │ │ ├── drift_monitoring.py # 🔍 Evidently AI Reports │ │ ├── data_health.py # 👮 Deepchecks Validation │ │ └── mapper.py # 🗺️ Semantic AI Column Mapper │ ├── tools/ │ │ └── revenue_tools.py # 🛠️ CrewAI SQL/Data Tools │ ├── crew_agents.py # 🧠 Agent Personality Definitions │ ├── crew_runner.py # ⚙️ Execution & Rate-Limit Logic │ ├── database.py # 🗄️ SQLAlchemy Connection │ └── main.py # 🚀 FastAPI Entrypoint ├── streamlit_app.py # 🖥️ Executive Dashboard UI ├── mlflow.db # 📊 Model Registry Database ├── requirements.txt # 📦 Dependencies └── .env # 🔑 Environment Secrets
app/crew_runner.py: The heart of the AI—manages Rate Limits, API cooling, and the Numerical Gate.app/logic/enforcement.py: The "Hallucination Killer" that validates parity.app/logic/mapper.py: Automated column-to-entity AI mapping.app/logic/forecasting.py: Facebook Prophet integration for time-series modeling.streamlit_app.py: The executive command center.
- Hallucination Rate: Reduced to 0% via Deterministic Gating.
- Forecasting Accuracy: Maintained <3% MAPE consistently over 200 days.
- Adaptability: Full "Universal Mode" support for both SQL Databases and CSV uploads.
Daniel Denision Engineering Intelligent Systems that don't just guess—they verify.