AI Rule Engine

An end-to-end pipeline that converts business-rule PDF documents into structured, executable CEL (Common Expression Language) rules — using a hybrid LLM approach that balances accuracy, determinism, and security.

Background: Three Approaches to LLM-Powered Rule Engines

When using an LLM to ingest a procedural document into a rule engine, there are three broad strategies. Each has distinct trade-offs:

Approach 1: LLM-as-Executor (Pure Agentic)

Convert the document into business-rule artifacts (structured JSON), then use the LLM at runtime to evaluate every rule — calling function tools for arithmetic and logic operations.

Pros	Cons
Flexible; handles ambiguous rules naturally	Non-deterministic — same input can produce different outputs across runs
	Very slow — each rule evaluation requires one or more LLM calls
	High token cost at scale
	Difficult to audit and reproduce results

Approach 2: LLM-Generated Code

Use the LLM to generate Python (or another language) code that implements the business rules, then execute that code directly.

Pros	Cons
Fast execution once code is generated	Cybersecurity risk — executing untrusted, LLM-generated code opens the door to injection, data exfiltration, and other exploits
Deterministic at runtime	Requires sandboxing / code review before production use
	Generated code can be brittle and hard to maintain

Approach 3: Hybrid — LLM Generates CEL Expressions (Our Adopted Approach)

Use the LLM in a compile-time-only role:

Phase 1 — Convert the document into structured business-rule artifacts (human-readable JSON with conditions, actions, data schemas, and routes).
Phase 2 — For each artifact, use the LLM to generate executable CEL expressions — a safe, strongly-typed expression language.
Runtime — Execute the CEL expressions with a deterministic CEL engine. No LLM calls at execution time.

Pros	Cons
Deterministic execution — same input always yields the same output	Requires a two-phase compilation pipeline
Secure — CEL is a sandboxed expression language; no arbitrary code execution	Complex rules may need manual review of generated CEL
Fast — rule evaluation is pure expression processing, no LLM latency
Auditable — both the artifacts and the CEL expressions are inspectable
Strongly typed — CEL enforces type safety at compile time
LLM is only used at build time, not at runtime

Why CEL over JsonLogic? CEL (Common Expression Language, created by Google) offers stronger type safety, a more expressive syntax for complex conditions, native support for ternary expressions, and better tooling for compile-time validation. Unlike JsonLogic's nested JSON syntax, CEL expressions are human-readable strings (e.g. RPS.client_access == true ? 'Compliant' : 'Non-Compliant'), making them easier for both LLMs to generate and humans to review.

Pipeline Overview

PDF Document
    │
    ▼
┌──────────────────────────────────────────────────┐
│  Phase 1: Convert PDF to Business Rule Artifacts │
│  (01_convert_pdf_to_rules.py)                    │
│                                                  │
│  PDF → Markdown → Structured Rules JSON          │
│  + Route Discovery between rules                 │
│  + Loop Detection & Route Validation             │
└──────────────────┬───────────────────────────────┘
                   │  *_routed.json
                   ▼
┌──────────────────────────────────────────────────┐
│  Visualise Rule Graph (optional)                 │
│  (02_build_graph.py)                             │
│                                                  │
│  Interactive HTML graph + starting order JSON     │
│  + Cycle detection & top-k longest paths         │
└──────────────────┬───────────────────────────────┘
                   │  *_graph.html, *_starting_order.json
                   ▼
┌──────────────────────────────────────────────────┐
│  Phase 2: Compile Artifacts to CEL               │
│  (03_generate_cel.py)                            │
│                                                  │
│  For each rule artifact, generate:               │
│    • calculation_cel  (CEL expression)           │
│    • routing_cel      (CEL expression)           │
│    • output_variable  (derived.*)                │
│  + Auto-fix & 3-layer validation                 │
└──────────────────┬───────────────────────────────┘
                   │  *_cel.json
                   ▼
┌──────────────────────────────────────────────────┐
│  Generate Mock Test Data                         │
│  (04_generate_data.py)                           │
│                                                  │
│  LLM-inferred Faker specs per variable           │
│  Type-aware: matches CEL comparison types        │
│  Golden-row injection for branch coverage        │
└──────────────────┬───────────────────────────────┘
                   │  *_mockdata.json, *_schema.json
                   ▼
┌──────────────────────────────────────────────────┐
│  Execute Rules Engine                            │
│  (05_execute_rules.py)                           │
│                                                  │
│  Compute-then-Route pattern:                     │
│    1. Evaluate calculation_cel → store result    │
│    2. Evaluate routing_cel → jump to next rule   │
│  CEL preprocessing & loop detection              │
└──────────────────┬───────────────────────────────┘
                   │  *_execution_<timestamp>.json
                   ▼
┌──────────────────────────────────────────────────┐
│  LLM Review Report                               │
│  (06_review_report.py)                           │
│                                                  │
│  Chunk-then-combine LLM review:                  │
│    • Data analysed / Outputs / Insights          │
│    • Correctness evaluation                      │
│    • Confidence score (0-100)                    │
└──────────────────────────────────────────────────┘
    *_review_<timestamp>.json / .md

Step-by-Step Guide

Step 1: Convert PDF to Business Rule Artifacts (`01_convert_pdf_to_rules.py`)

Extracts human-readable, structured business rules from a PDF document in three stages:

PDF to Markdown — Uses markitdown to convert the PDF into clean Markdown text.
Extract Rules — Sends the Markdown to an LLM (chunked at 15K characters by page markers or headings) to extract structured business rules. Each rule captures:
- rule_id / rule_name — unique identifier and descriptive name
- entity_applied — the entity type the rule applies to
- data_required — structured list of data sources with attributes (field names, types, example values)
- conditions — machine-readable predicates referencing exact DataSource.attribute_name paths
- action / outcome — what the rule does and what it produces
Discover Routes — A second LLM pass analyses the rules to find logical handoffs (outgoing routes with conditions and next_rule targets). Includes:
- Loop detection — breaks self-loops and cycles in discovered routes
- Route validation — verifies all route targets exist and resolves invalid routes using document context

python 01_convert_pdf_to_rules.py use_case_02/Conduct-of-Business-Rulebook.pdf

Output: <stem>.md, <stem>.json, <stem>_routed.json

Step 2: Build Rule Graph (`02_build_graph.py`)

Visualises the rule dependency graph as an interactive HTML file.

Builds a directed graph (NetworkX) from the rules and their outgoing routes.
Colours nodes by role:
- Green (diamond) — root nodes (entry points, in-degree = 0)
- Red (box) — terminal nodes (is_final = true)
- Grey (ellipse) — intermediate nodes
Detects and reports cycles in the graph.
Finds top-k longest root-to-leaf paths.
Exports interactive HTML visualisation (pyvis) and a starting-rule order JSON.

python 02_build_graph.py use_case_02/Conduct-of-Business-Rulebook_routed.json

Output: <stem>_graph.html, <stem>_subgraph_top3_combined.html, <stem>_starting_order.json

Step 3: Compile Artifacts to CEL (`03_generate_cel.py`)

This is the core compilation step. For each business-rule artifact from Step 1, uses the LLM to generate a Compute-then-Route CEL expression pair:

calculation_cel (CEL expression) — Computes the business value (string, number, or boolean). Handles conditional logic, gatekeeper checks, and arithmetic.
routing_cel (CEL expression) — Determines the next rule ID based on the computed value, or null if the rule is terminal.
output_variable — Where the calculation result is stored (e.g. "derived.compliance_status").

CEL Expression Constraints

The LLM is instructed to generate CEL expressions that follow strict constraints:

Constraint	Detail
Ternary syntax	`condition ? value_if_true : value_if_false`
Namespace-qualified variables	Always `Prefix.attribute` (e.g. `RPS.client_id`, `CAS.credit_amount`)
Allowed types	`string`, `double`, `int`, `bool`
Allowed functions	`double()`, `int()`, `string()`, `size()` only
No aggregation	No `map`, `filter`, `reduce`, `sum` over lists
*Output stored in `derived.`**	e.g. `derived.compliance_status`, `derived.compensation_amount`

Auto-Fix & Validation

The pipeline includes automatic correction for common LLM mistakes and three-layer validation:

Auto-fixes applied:

Unbalanced parentheses
Digit-starting identifiers
Nested ternary nesting issues
Type mixing (int/double) — converts to consistent double()
Null else-branches
Whitespace in string literals
.size() > 0 → != '' for strings

Three-layer validation:

Forbidden patterns — regex checks for banned constructs
Syntax/compile check — CEL compiler verifies expression is valid
Test-execution — runs the expression against sample data using celpy

If validation fails, the LLM retries up to 5 times with error feedback.

Example: How a Rule Becomes CEL

The compilation step transforms the human-readable rule artifact into a pair of CEL expressions. Here is a real example from the Conduct of Business Rulebook for Credit Institutions (Use Case 02).

1. Simple compliance check (final rule, no routing)

Source rule artifact (*_routed.json):

{
  "rule_id": "R003",
  "rule_name": "Digital Communication Compliance",
  "data_required": [{
    "data_source": "Regulated Person System (RPS)",
    "data_attributes": [
      { "attribute_name": "communication_medium", "data_type": "string", "example_values": ["website", "email", "mobile app"] },
      { "attribute_name": "client_access_to_internet", "data_type": "boolean", "example_values": ["true", "false"] }
    ]
  }],
  "conditions": [
    "RPS.communication_medium == 'website'",
    "Client has regular access to the internet"
  ],
  "action": "Ensure information is provided in clear and understandable language.",
  "outcome": "Information is accessible and comprehensible to clients via digital means.",
  "outgoing_routes": [],
  "is_final": true
}

Compiled CEL (*_cel.json) — the LLM translates the conditions and action into a single ternary expression:

{
  "rule_id": "R003",
  "output_variable": "derived.compliance_status",
  "calculation_cel": "RPS.communication_medium == 'website' && RPS.client_access_to_internet == true ? 'Compliant' : 'Non-Compliant'",
  "routing_cel": null
}

2. Numerical calculation (early repayment compensation)

Source rule artifact:

{
  "rule_id": "R529",
  "rule_name": "Early Repayment Compensation Limitation",
  "data_required": [{
    "data_source": "Credit Agreement System (CAS)",
    "data_attributes": [
      { "attribute_name": "credit_amount_repaid_early", "data_type": "float", "example_values": ["5000", "10000", "25000"] },
      { "attribute_name": "time_between_repayment_and_termination", "data_type": "integer", "example_values": ["12", "6", "18"] }
    ]
  }],
  "conditions": [
    "CAS.time_between_repayment_and_termination > 12",
    "CAS.credit_amount_repaid_early > 0"
  ],
  "action": "Calculate compensation as 1% of CAS.credit_amount_repaid_early.",
  "outcome": "Compensation amount for early repayment."
}

Compiled CEL — the LLM generates type-safe arithmetic with double() casts:

{
  "rule_id": "R529",
  "output_variable": "derived.compensation_amount",
  "calculation_cel": "double(CAS.time_between_repayment_and_termination) > 12.0 && double(CAS.credit_amount_repaid_early) > 0.0 ? (double(CAS.credit_amount_repaid_early) * 1.0) / 100.0 : 0.0",
  "routing_cel": null
}

Its companion rule R530 handles the shorter-period case (0.5% instead of 1%):

{
  "rule_id": "R530",
  "output_variable": "derived.compensation_amount",
  "calculation_cel": "double(CAS.time_between_repayment_and_termination) <= 12.0 && double(CAS.credit_amount_repaid_early) > 0.0 ? (double(CAS.credit_amount_repaid_early) * 0.005) : 0.0",
  "routing_cel": null
}

3. Enumeration-based rule with in operator (contract termination)

Source rule artifact:

{
  "rule_id": "R503",
  "rule_name": "Termination Conditions for Framework Contracts",
  "conditions": [
    "CR.termination_reason in ['illegal_use', 'no_transaction', 'incorrect_info', 'no_longer_resident', 'second_account']"
  ],
  "action": "Terminate framework contract under specified conditions.",
  "outcome": "Framework contract terminated under valid conditions."
}

Compiled CEL — the in operator maps directly to CEL's list membership syntax:

{
  "rule_id": "R503",
  "output_variable": "derived.contract_termination_status",
  "calculation_cel": "CR.termination_reason in ['illegal_use', 'no_transaction', 'incorrect_info', 'no_longer_resident', 'second_account'] ? 'Terminated' : 'Not Terminated'",
  "routing_cel": null
}

4. Rule with conditional routing (non-final rule in a chain)

Source rule artifact:

{
  "rule_id": "R021",
  "rule_name": "Application Form Responsibility",
  "conditions": [
    "Regulated_Person_Systems.form_completion_status == 'completed'",
    "Regulated_Person_Systems.client_responsibility == 'acknowledged'"
  ],
  "action": "Verify client acknowledged form responsibility.",
  "outgoing_routes": [
    { "condition": "Form completed and client acknowledged responsibility", "next_rule": "R022" }
  ],
  "is_final": false
}

Compiled CEL — generates both a calculation expression and a routing expression:

{
  "rule_id": "R021",
  "output_variable": "derived.client_responsibility_status",
  "calculation_cel": "Regulated_Person_Systems.form_completion_status == 'completed' && Regulated_Person_Systems.client_responsibility == 'acknowledged' ? 'Acknowledged' : 'Not Acknowledged'",
  "routing_cel": "derived.client_responsibility_status == 'Acknowledged' ? 'R022' : null"
}

The routing CEL reads the derived output of its own calculation to decide where to go next. If the client acknowledged, execution chains forward to R022; otherwise the chain terminates.

python 03_generate_cel.py use_case_02/Conduct-of-Business-Rulebook_routed.json

Output: <stem>_cel.json

Step 4: Generate Mock Test Data (`04_generate_data.py`)

Produces realistic mock data for testing the rules. For each rule:

Extracts all input variables from calculation_cel and routing_cel by parsing dotted identifiers (e.g. RPS.communication_medium).
Cross-references with data_required for context (descriptions, example values).
Uses an LLM to infer the best Faker provider and kwargs for each variable, with type-aware generation:
- Type validation — overrides LLM types based on CEL comparison operators (e.g. if CEL compares to true, type is bool)
- Branch coverage — includes all literal values from CEL == and in checks to exercise every code path
- Golden-row injection — injects a row that satisfies primary positive conditions for branch coverage
- Denominator protection — ensures division operands avoid zero
Generates mock data rows using Faker, nested into the structure expected by CEL (e.g. RPS.client_access_to_internet → {"RPS": {"client_access_to_internet": true}}).

python 04_generate_data.py use_case_02/Conduct-of-Business-Rulebook_routed_cel.json --num-rows 5

Output: <stem>_mockdata.json, <stem>_schema.json

Step 5: Execute Rules Engine (`05_execute_rules.py`)

Runs the CEL rules against mock data using the Compute-then-Route pattern:

Builds a rule repository indexed by rule_id.
For each starting rule (from the starting-order JSON or auto-detected root nodes), for each mock-data row:
- Calculate: Evaluate calculation_cel via celpy and store the result in output_variable.
- Route: Evaluate routing_cel via celpy to determine the next rule.
- Repeat until routing returns null or an unknown rule ID.
Records a full execution trace with: CEL expressions executed, context updates, context snapshots, and final context variables.

CEL Preprocessing

The execution engine applies preprocessing to handle edge cases in LLM-generated CEL:

Preprocessing	Purpose
Null-check resolution	Resolves null comparisons at Python level before CEL evaluation
Bool coercion	Maps Python bool/string to CEL `true`/`false`
Numeric coercion	Converts all numeric data to `float` for CEL `double` compatibility
Integer literal conversion	Converts int literals to double (e.g. `100` → `100.0`)
`.size() > 0` rewrite	Converts string `.size()` checks to `!= ''`

Safety Features

Loop detection — detects repeating patterns within a 6-step window
Max step limit — hard cap of 200 steps per execution to prevent runaway loops

python 05_execute_rules.py \
  use_case_02/Conduct-of-Business-Rulebook_routed_cel.json \
  use_case_02/Conduct-of-Business-Rulebook_routed_cel_mockdata.json \
  --starting-order use_case_02/Conduct-of-Business-Rulebook_routed_starting_order.json \
  --num-rows 5

Options:

--num-rows N — number of mock-data rows to execute per starting rule (default: 1)
--starting-order <file> — JSON list of root rule IDs (auto-detected if omitted)
--quiet — suppress step-by-step trace output

Output: <stem>_execution_<timestamp>.json

Step 6: LLM Review Report (`06_review_report.py`)

Reviews the execution report using an LLM with a chunk-then-combine approach to handle large reports within context-window limits:

Pre-computes global statistics from the full execution report: health metrics, input/output variable distributions, path analysis.
Chunks the run records into batches (default 50 runs per chunk).
Chunk review — sends each chunk + global stats to the LLM for a partial review.
Final aggregation — combines all chunk reviews into a single report covering:
- Data Analysed — what data sources and variables were tested
- Outputs — what derived variables were computed, value distributions
- Conclusions & Data Insights — business patterns, anomalies
- Correctness Evaluation — error rate, issues, strengths, recommendations
- Confidence Score — 0-100 overall correctness rating

python 06_review_report.py \
  use_case_02/Conduct-of-Business-Rulebook_routed_execution_20260216_123926.json \
  --rules-file use_case_02/Conduct-of-Business-Rulebook_routed_cel.json \
  --chunk-size 50

Options:

--rules-file <file> — enriched rules JSON for additional context (auto-detected from report metadata if omitted)
--chunk-size N — runs per LLM chunk (default: 50; lower for smaller context windows)

Output: <stem>_review_<timestamp>.json, <stem>_review_<timestamp>.md

The "Compute-then-Route" Pattern

The key architectural decision is decoupling calculation from routing:

                    Rule Artifact
                         │
                         ▼
              ┌─────────────────────┐
              │  calculation_cel    │   "What is the value?"
              │  (CEL expression)   │   e.g. determine compliance status
              └──────────┬──────────┘
                         │  output_variable = "derived.compliance_status"
                         ▼
              ┌─────────────────────┐
              │  routing_cel        │   "Where do I go next?"
              │  (CEL expression)   │   e.g. if Compliant → R022, else → null
              └─────────────────────┘

By separating "What is the value?" (Calculation) from "Where do I go next?" (Routing), the system becomes significantly more testable and modular:

Independent testing — You can unit-test the calculation logic in isolation, without worrying about routing.
Change isolation — If routing changes, the calculation logic is untouched, and vice versa.
Composability — New rules can be inserted into the graph by updating routing targets.
Transparency — Each rule's CEL expressions are small, self-contained, inspectable strings.

File Naming Convention

All outputs follow a consistent naming chain rooted in the input PDF stem:

<stem>.pdf                          ← Input PDF
<stem>.md                           ← Markdown conversion
<stem>.json                         ← Extracted rules (no routes)
<stem>_routed.json                  ← Rules + routes
<stem>_routed_graph.html            ← Interactive graph
<stem>_routed_starting_order.json   ← Root rule IDs
<stem>_routed_cel.json              ← Compiled CEL rules
<stem>_routed_cel_schema.json       ← Variable specs / Faker config
<stem>_routed_cel_mockdata.json     ← Generated mock data
<stem>_routed_execution_<ts>.json   ← Execution trace
<stem>_routed_review_<ts>.json      ← Review report (JSON)
<stem>_routed_review_<ts>.md        ← Review report (Markdown)

Run the Full Pipeline

Use run.sh to orchestrate all six steps in sequence:

./run.sh <path/to/input.pdf> [--num-rows N]

<path/to/input.pdf> — the input PDF document
--num-rows N — number of mock data rows per rule (default: 50)

Or run each step individually:

# Step 1: Convert PDF to structured rule artifacts (with routes)
python 01_convert_pdf_to_rules.py use_case_02/Conduct-of-Business-Rulebook.pdf

# Step 2: Visualise the rule graph
python 02_build_graph.py use_case_02/Conduct-of-Business-Rulebook_routed.json

# Step 3: Compile rule artifacts to executable CEL
python 03_generate_cel.py use_case_02/Conduct-of-Business-Rulebook_routed.json

# Step 4: Generate mock test data
python 04_generate_data.py use_case_02/Conduct-of-Business-Rulebook_routed_cel.json -n 5

# Step 5: Execute rules against mock data
python 05_execute_rules.py \
  use_case_02/Conduct-of-Business-Rulebook_routed_cel.json \
  use_case_02/Conduct-of-Business-Rulebook_routed_cel_mockdata.json \
  --starting-order use_case_02/Conduct-of-Business-Rulebook_routed_starting_order.json \
  -n 5

# Step 6: Review execution results with LLM
python 06_review_report.py \
  use_case_02/Conduct-of-Business-Rulebook_routed_execution_*.json \
  --chunk-size 50

Key Changes: JsonLogic → CEL Migration

The pipeline was originally built on JsonLogic and has been migrated to CEL (Common Expression Language). Below is a summary of all changes and bug fixes applied during the migration.

Expression Language: JsonLogic → CEL

Aspect	Before (JsonLogic)	After (CEL)
Expression format	Nested JSON objects (`{"if": [{"==": [...]}, ...]}`)	Human-readable strings (`x == 'a' ? 'yes' : 'no'`)
Type safety	Runtime (weak)	Compile-time (strong)
Compilation step	`03_generate_jsonlogic.py`	`03_generate_cel.py`
Execution engine	`json-logic-qubit` library	`celpy` library
Output files	`*_jsonlogic.json`	`*_cel.json`
Validation	Basic structure check	3-layer: forbidden patterns + syntax + test-execution

Bug Fixes Applied in CEL Compilation (`03_generate_cel.py`)

Bug	Fix
LLM generates unbalanced parentheses	Auto-fix trims/adds parens to balance
Identifiers starting with digits (e.g. `2024_metric`)	Auto-prefix or rewrite
Nested ternary expressions with incorrect grouping	Auto-fix nesting
Mixed int/double in arithmetic (`100 + 1.5`)	Normalize all numeric literals to `double()`
Null else-branches in ternaries	Auto-inject fallback values
Whitespace inside string literals	Auto-trim
`.size() > 0` on string variables	Rewrite to `!= ''`
LLM uses banned functions (aggregation, etc.)	Forbidden-pattern regex check + retry

Bug Fixes Applied in Execution Engine (`05_execute_rules.py`)

Bug	Fix
JsonLogic non-standard operators (`not`, `eq`, `sum`, etc.)	Removed — CEL uses standard syntax natively
Python `True`/`False` vs CEL `true`/`false`	Bool coercion layer maps Python bools to CEL booleans
Python `int` vs CEL `double` type mismatch	Numeric coercion converts all numbers to `float`
String range comparisons (lexicographic in JsonLogic)	Resolved — CEL handles comparisons correctly
Null-check failures	Python-level null resolution before CEL evaluation
Infinite loops from cyclic routing	Loop detection (6-step pattern window) + 200-step hard cap
Integer literals in CEL expressions	Auto-convert to double (e.g. `100` → `100.0`)

Bug Fixes Applied in Mock Data Generation (`04_generate_data.py`)

Bug	Fix
Type mismatch (LLM generates `bool` but CEL expects `string "true"`)	Type validation overrides LLM types based on CEL comparison operators
Poor branch coverage	Golden-row injection ensures positive-path execution
Division by zero in generated data	Denominator protection ensures non-zero values
Flat dict structure vs nested CEL namespace	Auto-nesting (`RPS.client_id` → `{"RPS": {"client_id": ...}}`)
Schema extraction required separate LLM call	Variable specs extracted directly from CEL expressions

Bug Fixes Applied in Route Discovery (`01_convert_pdf_to_rules.py`)

Bug	Fix
Self-referencing routes (rule routes to itself)	Loop detection breaks self-loops
Circular route chains	Cycle detection and removal
Routes pointing to non-existent rule IDs	Route validation checks all targets exist
Invalid routes	LLM re-resolution with document context

Worked Examples: Execution Traces

These examples are taken from the actual execution of Use Case 02 (Conduct of Business Rulebook for Credit Institutions) — 225 rules extracted from a regulatory PDF, compiled to CEL, and executed against mock data with 0% error rate.

Example A: Single-Step Compliance Check (R003)

Business context: A regulator requires that digital communications to clients only use the website channel when the client has internet access.

Input:  { RPS.communication_medium: "website", RPS.client_access_to_internet: true }
CEL:    RPS.communication_medium == 'website' && RPS.client_access_to_internet == true ? 'Compliant' : 'Non-Compliant'
Output: derived.compliance_status = "Compliant"

With different input the same rule produces a different outcome:

Input:  { RPS.communication_medium: "email", RPS.client_access_to_internet: true }
CEL:    (same expression)
Output: derived.compliance_status = "Non-Compliant"   ← medium is not 'website'

Benefit: The compliance check is deterministic — the same input always produces the same output. No LLM is involved at runtime, so there is no risk of hallucination or inconsistency.

Example B: Numerical Calculation — Early Repayment Compensation (R529)

Business context: When a borrower repays a credit agreement early and more than 12 months remain until the agreed termination date, the lender may charge compensation capped at 1% of the amount repaid early. This is a direct implementation of EU Consumer Credit Directive Article 16.

Input:  { CAS.credit_amount_repaid_early: 66922.56, CAS.time_between_repayment_and_termination: 16 }
CEL:    double(CAS.time_between_repayment_and_termination) > 12.0 && double(CAS.credit_amount_repaid_early) > 0.0
          ? (double(CAS.credit_amount_repaid_early) * 1.0) / 100.0
          : 0.0
Output: derived.compensation_amount = 669.2256   ← 66922.56 × 1% = 669.2256

When the remaining period is 12 months or fewer, rule R530 applies the lower 0.5% rate:

Input:  { CAS.credit_amount_repaid_early: 81026.42, CAS.time_between_repayment_and_termination: 1 }
CEL:    double(CAS.time_between_repayment_and_termination) <= 12.0 && ...
          ? (double(CAS.credit_amount_repaid_early) * 0.005)
          : 0.0
Output: derived.compensation_amount = 405.1321   ← 81026.42 × 0.5% = 405.1321

Benefit: Financial calculations are exact and auditable — the CEL expression, input data, and computed result are all recorded in the execution trace. Regulators can verify the arithmetic directly.

Example C: Enumeration Check — Contract Termination (R503)

Business context: A bank may terminate a framework contract only for specific regulatory reasons. Any other reason is rejected.

Input:  { CR.termination_reason: "illegal_use" }
CEL:    CR.termination_reason in ['illegal_use', 'no_transaction', 'incorrect_info', 'no_longer_resident', 'second_account']
          ? 'Terminated' : 'Not Terminated'
Output: derived.contract_termination_status = "Terminated"

Input:  { CR.termination_reason: "other_reason" }
Output: derived.contract_termination_status = "Not Terminated"   ← not in the allowed list

Benefit: The enumeration of valid termination reasons is explicit and inspectable in the CEL expression. Auditors can see exactly which reasons are permitted without reading the source PDF.

Example D: 20-Step Rule Chain — Client Notification Workflow (R021 → R040)

Business context: When a client completes an application form, a chain of 20 compliance checks runs end-to-end — from form acknowledgement through telephone contact verification, change-of-terms notifications, fee disclosure, statement availability, and bundled-package risk disclosure.

Step  1: R021 (Application Form Responsibility)
         CEL: ...form_completion_status == 'completed' && ...client_responsibility == 'acknowledged' ? 'Acknowledged' : 'Not Acknowledged'
         → derived.client_responsibility_status = "Acknowledged"
         → routing: 'Acknowledged' → R022

Step  2: R022 (Initial Telephone Contact Requirements)
         → derived.contact_compliance_status = "Compliant"
         → routing: 'Compliant' → R023

Step  3: R023 (Notice of Change in Terms or Conditions)
         → derived.notice_required = "Notice Required"
         → routing: always → R024

Step  4: R024 (Notice of Change in Charges or Fees)
         → derived.notice_required = "Notice Required"
         → routing: always → R025

  ...    (16 more steps covering interest rate changes, material changes,
          client refusal rights, comparable product references, fee status,
          statement availability, bundled package disclosures)

Step 19: R039 (Disclosure of Bundled Package Components)
         → derived.bundled_package_disclosure_status = "Components disclosed"
         → routing: → R040

Step 20: R040 (Disclosure of Risk Modifications in Bundled Packages)
         → derived.risk_modification_disclosure = "Applicable"
         → routing: null (chain terminates)

Final outputs — 17 derived variables computed across the chain:

Derived Variable	Value
`client_responsibility_status`	Acknowledged
`contact_compliance_status`	Compliant
`notice_required`	Notice Required
`notice_provided`	Notice Provided
`notice_heading_compliance`	Compliant
`plain_language_notice_compliance`	Compliant
`client_refusal_disclosure_status`	Refusal and consequences disclosed
`comparable_product_reference`	Comparable product reference provided
`client_assistance_status`	Assistance Provided
`notice_status`	Notice Provided
`statements_availability`	Available
`statement_provision_status`	Statement Provision on Request
`fee_status`	No Fee Charged
`interest_rate_indicated`	Interest rate indicated in statements
`bundled_package_disclosure_status`	Components disclosed
`risk_modification_disclosure`	Applicable

Benefit: A complex 20-step regulatory workflow is executed end-to-end in milliseconds with no LLM calls. Each step is independently testable, and the full execution trace is recorded for audit. If any single rule changes (e.g. a new fee disclosure requirement), only that rule's CEL needs updating — the rest of the chain is untouched.

Example E: 19-Step Rule Chain — Forbearance & NPE Workout (R541 → R560)

Business context: When a consumer credit borrower enters financial difficulty, a chain of 19 rules governs the forbearance process — from initial identification through communication, viability assessment, repossession prohibition, legal action limits, documentation requirements, workout unit establishment, and ongoing monitoring.

Step  1: R541 (Forbearance Measures Applicable to Consumer Credit)
         → derived.forbearance_applicable = "Yes"
Step  2: R543 (Client Communication in Payment Difficulties)
         → derived.communication_status = "Communication initiated"
Step  3: R544 (Forbearance Measures Viability Assessment)
         → derived.forbearance_viability = "Viable"
Step  4: R545 (Repossession Prohibition for Residential Property)
         → derived.repossession_prohibition_status = "Prohibited"
Step  5: R546 (Prohibition on Threatening Legal Action)
         → derived.financial_difficulty_status = "Financially Difficult"
  ...
Step 14: R555 (NPE Workout Unit Establishment)
         → derived.npe_workout_unit_established = "Established"
Step 15: R556 (Suspension of Legal Proceedings During Forbearance)
         → derived.suspension_status = "Suspended"
  ...
Step 18: R559 (Forbearance Contract Targets)
         → derived.target_schedule = "monthly_milestones"
Step 19: R560 (Forbearance Monitoring)
         → derived.compliance_status = "Compliant"

Benefit: A sensitive, multi-party regulatory process spanning repossession rules, legal action limits, and NPE workout procedures is codified as a deterministic, traceable rule chain. Every decision is logged, auditable, and reproducible — critical for regulatory compliance in financial services.

Summary of Benefits Demonstrated

Benefit	Evidence
Deterministic execution	969 runs, 0% error rate — same input always produces the same output
Auditable	Full execution trace records every CEL expression, input, output, and routing decision
Fast	All 969 runs (1,647 steps) complete in seconds with no LLM calls at runtime
Modular	Individual rules can be updated without affecting the rest of the chain
Covers diverse rule types	Boolean compliance checks, numerical calculations, enumeration lookups, and multi-step chains
Handles complex workflows	20-step notification workflow and 19-step forbearance chain both execute cleanly
Secure	CEL is a sandboxed expression language — no arbitrary code execution
Transparent	CEL expressions are human-readable — regulators can inspect the logic directly

Use Case Results

Use Case 02: Conduct of Business Rulebook for Credit Institutions (Latest Run)

Run date: 2026-02-16 | Confidence Score: 95/100

Metric	Value
Starting rules	225
Total execution runs	969
Total steps	1,647
Unique execution paths	257
Unique output variables	365
Data sources	45
Error rate	0%
Null output rate	0%
Runs with errors	0

Step distribution: 907 single-step (93.6%), 52 multi-step 2-9 (5.4%), 8 with 10-20 steps (0.8%), 2 hitting 200-step cap (0.2%)

Key strengths:

Zero runtime errors across all 969 runs
Broad coverage across 45 data sources and 365 derived output variables
Several healthy multi-step chains (e.g. R021→R040 spanning 20 rules, R541→R560 spanning 19 rules)
257 unique execution paths demonstrating varied branching

Known issue: 2 runs hit the 200-step safety cap due to a 4-rule routing cycle (R070 → R071 → R072 → R073 → R070). This is a rule-definition issue in the source rulebook, not an engine bug. The routing CEL on R073 unconditionally routes back to R070 when the output is "Fulfilled".

Prerequisites

Python Packages

pip install -r requirements.txt

Key dependencies:

Package	Purpose
`markitdown[all]`	PDF to Markdown conversion
`langchain-openai` / `langchain-core`	LLM integration
`pydantic`	Structured LLM output
`cel-python` / `common-expression-language`	CEL compilation and execution
`faker`	Mock data generation
`networkx` / `pyvis`	Rule graph visualisation
`markdown` / `xhtml2pdf`	Report export

Environment Variables

Create a .env file in the project root:

OPENAI_API_BASE=http://your-llm-endpoint/v1
OPENAI_API_KEY=your-api-key
MODEL_NAME=gpt-4o

The pipeline works with any OpenAI-compatible API endpoint (OpenAI, Azure, vLLM, LiteLLM, etc.).

Name		Name	Last commit message	Last commit date
Latest commit History 3 Commits
01_convert_pdf_to_rules.py		01_convert_pdf_to_rules.py
02_build_graph.py		02_build_graph.py
03_generate_cel.py		03_generate_cel.py
04_generate_data.py		04_generate_data.py
05_execute_rules.py		05_execute_rules.py
06_review_report.py		06_review_report.py
Conduct-of-Business-Rulebook-for-Credit-Institutions-Offering-Retail-Products-and-Services_routed_review_20260216_124402.md		Conduct-of-Business-Rulebook-for-Credit-Institutions-Offering-Retail-Products-and-Services_routed_review_20260216_124402.md
README.md		README.md
convert_md_to_pdf.py		convert_md_to_pdf.py
requirements.txt		requirements.txt
run.sh		run.sh

Folders and files

Latest commit

History

Repository files navigation

AI Rule Engine

Background: Three Approaches to LLM-Powered Rule Engines

Approach 1: LLM-as-Executor (Pure Agentic)

Approach 2: LLM-Generated Code

Approach 3: Hybrid — LLM Generates CEL Expressions (Our Adopted Approach)

Pipeline Overview

Step-by-Step Guide

Step 1: Convert PDF to Business Rule Artifacts (01_convert_pdf_to_rules.py)

Step 2: Build Rule Graph (02_build_graph.py)

Step 3: Compile Artifacts to CEL (03_generate_cel.py)

CEL Expression Constraints

Auto-Fix & Validation

Example: How a Rule Becomes CEL

Step 4: Generate Mock Test Data (04_generate_data.py)

Step 5: Execute Rules Engine (05_execute_rules.py)

CEL Preprocessing

Safety Features

Step 6: LLM Review Report (06_review_report.py)

The "Compute-then-Route" Pattern

File Naming Convention

Run the Full Pipeline

Key Changes: JsonLogic → CEL Migration

Expression Language: JsonLogic → CEL

Bug Fixes Applied in CEL Compilation (03_generate_cel.py)

Bug Fixes Applied in Execution Engine (05_execute_rules.py)

Bug Fixes Applied in Mock Data Generation (04_generate_data.py)

Bug Fixes Applied in Route Discovery (01_convert_pdf_to_rules.py)

Worked Examples: Execution Traces

Example A: Single-Step Compliance Check (R003)

Example B: Numerical Calculation — Early Repayment Compensation (R529)

Example C: Enumeration Check — Contract Termination (R503)

Example D: 20-Step Rule Chain — Client Notification Workflow (R021 → R040)

Example E: 19-Step Rule Chain — Forbearance & NPE Workout (R541 → R560)

Summary of Benefits Demonstrated

Use Case Results

Use Case 02: Conduct of Business Rulebook for Credit Institutions (Latest Run)

Prerequisites

Python Packages

Environment Variables

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Step 1: Convert PDF to Business Rule Artifacts (`01_convert_pdf_to_rules.py`)

Step 2: Build Rule Graph (`02_build_graph.py`)

Step 3: Compile Artifacts to CEL (`03_generate_cel.py`)

Step 4: Generate Mock Test Data (`04_generate_data.py`)

Step 5: Execute Rules Engine (`05_execute_rules.py`)

Step 6: LLM Review Report (`06_review_report.py`)

Bug Fixes Applied in CEL Compilation (`03_generate_cel.py`)

Bug Fixes Applied in Execution Engine (`05_execute_rules.py`)

Bug Fixes Applied in Mock Data Generation (`04_generate_data.py`)

Bug Fixes Applied in Route Discovery (`01_convert_pdf_to_rules.py`)

Packages