IntegratedML Custom Models

Deploy custom Python ML models directly within SQL queries using InterSystems IRIS 2025.2

---

🚀 Early Access Program (EAP)

Welcome EAP Participants! The IntegratedML Custom Models will be General Availability (GA) for the InterSystems IRIS 2026.1 release. Until then, this repository will be the source of documentation and information about the feature.

Getting Started with EAP

1. Read the EAP Guide - Understand the program, timeline, and expectations
2. Install Custom Models - Complete installation in <30 minutes (target)
3. Check Known Issues - Review current limitations before reporting bugs
4. Review Roadmap - See what's coming from EAP to GA

How to Provide Feedback

Your feedback directly shapes the final product! Choose your preferred channel:

• Survey (recommended): Survey links provided by Data Platforms Product Team
• Email: thomas.dyar@intersystems.com
• GitHub Issues (if enabled): Technical bugs and feature requests

Response time: 1-2 business days during EAP

For questions or support, see EAP FAQ or email thomas.dyar@intersystems.com.

---

Table of Contents

• Overview
• Key Features
• Demo Applications
• Quick Start
• Documentation
  • Core Documentation
  • Demo Applications
  • Feature Specifications
• Architecture
• Testing
• Development
• Contributing
• License
• Support

Overview

IntegratedML Custom Models extends InterSystems IRIS IntegratedML with a powerful new capability: deploy your own Python models directly within SQL queries. While IntegratedML has provided automated ML for years, this feature gives data scientists full control—custom preprocessing, any scikit-learn compatible model, and third-party libraries like Prophet or LightGBM—all executing in-database without data movement.

-- Train your custom Python model with a single SQL command
CREATE MODEL CreditRiskAssessment
PREDICTING (default_risk)
FROM CreditApplications
USING {
    "model_name": "CustomCreditRiskClassifier",
    "path_to_classifiers": "/opt/iris/mgr/python/custom_models/classifiers",
    "user_params": {
        "enable_debt_ratio": 1,
        "enable_risk_scoring": 1
    }
}

-- Get predictions instantly
SELECT customer_id,
       PREDICT(CreditRiskAssessment) as risk_score
FROM NewApplications

Key Features

• SQL Integration: Deploy scikit-learn compatible models directly in SQL
• Low Latency: Sub-50ms prediction latency for real-time applications
• Custom Models: Use your own Python models with domain-specific logic
• In-Database Processing: Train and predict without data exports
• Scalable: Designed for production workloads

Demo Applications

1. Credit Risk Assessment

Financial risk modeling with custom feature engineering for loan default prediction.

• Model: Custom ensemble classifier
• Test Data: 10,000 records
• Training Time: ~2.3 seconds

2. Fraud Detection

Transaction fraud detection using ensemble methods.

• Model: Multi-model ensemble (Neural + Rules + Anomaly)
• Test Data: 25,000 transactions
• Latency: <50ms per prediction

3. Sales Forecasting

Time-series forecasting combining Prophet with LightGBM.

• Model: Prophet + LightGBM hybrid
• Accuracy: 26.9% MAPE
• Features: Seasonality, holidays

4. DNA Similarity Analysis

Sequence analysis using custom similarity algorithms.

• Model: K-NN with custom distance metrics
• Test Data: 5,000 sequences
• Features: GC content, motif search

Quick Start

Prerequisites

• InterSystems IRIS 2025.2+
• Python 3.8+
• Docker & Docker Compose

Installation

# Clone the repository
git clone https://github.com/intersystems/integratedml-custom-models.git
cd integratedml-custom-models

# Setup environment (installs dependencies + starts IRIS)
make setup

# Run all demos
make demos

Running Individual Demos

# Credit Risk Assessment
make demo-credit

# Fraud Detection
make demo-fraud

# Sales Forecasting
make demo-sales

# DNA Similarity
make demo-dna

Documentation

This project's documentation is organized into three main areas:

🔶 EAP Documentation (Start Here!)

Essential guides for Early Access Program participants:

• EAP Guide - Program overview, timeline, feedback channels
• Installation Guide - Platform-specific setup (macOS primary, Linux/Windows secondary)
• Known Issues - Current limitations and workarounds
• EAP Roadmap - Features from EAP to GA (2026.1)
• EAP FAQ - Frequently asked questions
• Troubleshooting - Common issues and solutions

📚 Core Documentation (docs/)

Cross-cutting technical documentation for the entire project:

• Quick Start Guide - Get started in under 5 minutes
• User Guide - Step-by-step usage instructions
• Architecture - System design, base class hierarchy, integration patterns
• API Reference - Complete API documentation for all model classes
• Deployment - Production deployment strategies and configuration

🎯 Demo Applications (demos/)

Working examples with demo-specific setup instructions:

• Credit Risk - Financial risk modeling with custom feature engineering
• Fraud Detection - Transaction fraud detection using ensemble methods
• Sales Forecasting - Time-series forecasting with Prophet + LightGBM
• DNA Similarity - Sequence analysis with custom distance metrics

📋 Feature Specifications (specs/)

Design documents and implementation plans for new features:

• Feature specifications with user stories and acceptance criteria
• Implementation plans with architecture decisions
• Task breakdowns and validation results

Architecture

┌─────────────────────────────────────────────────────────────┐
│                        SQL Interface                         │
│  CREATE MODEL | TRAIN MODEL | VALIDATE | PREDICT()          │
└─────────────────────┬───────────────────────────────────────┘
                      │
┌─────────────────────▼───────────────────────────────────────┐
│                    IntegratedML Engine                       │
│  • Model Management  • Parameter Handling  • Serialization   │
└─────────────────────┬───────────────────────────────────────┘
                      │
┌─────────────────────▼───────────────────────────────────────┐
│                  Custom Python Models                        │
│  • Scikit-learn Compatible  • Domain-Specific Logic         │
│  • Feature Engineering      • Custom Algorithms             │
└─────────────────────────────────────────────────────────────┘

Testing

# Run all tests
make test

# Run E2E test with enhanced data volumes
python tests/test_all_demos_e2e.py

# Run specific demo tests
pytest demos/credit_risk/tests/ -v
pytest demos/fraud_detection/tests/ -v

Test Results (Latest)

Demo	Data Volume	Training Time	Performance
Credit Risk	10,000 records	2.3s	100% accuracy
Fraud Detection	25,000 transactions	11.7s	192 flagged
Sales Forecasting	365 days × 5 stores	0.4s	26.9% MAPE
DNA Similarity	5,000 sequences	1.7s	50.5% accuracy

Development

Project Structure

integratedml-custom-models/
├── demos/                    # Demo applications
│   ├── credit_risk/         # Credit risk assessment
│   ├── fraud_detection/     # Fraud detection system
│   ├── sales_forecasting/   # Time series forecasting
│   └── dna_similarity/      # DNA sequence analysis
├── shared/                  # Shared components
│   ├── models/             # Base model classes
│   ├── database/           # IRIS connection utilities
│   └── utils/              # Helper functions
├── docker/                  # Docker configuration
├── notebooks/              # Jupyter notebooks
├── tests/                  # Test suites
└── scripts/                # Utility scripts

Creating Custom Models

1. Extend the base model class:

from shared.models.base import IntegratedMLBaseModel

class MyCustomModel(IntegratedMLBaseModel):
    def fit(self, X, y, **params):
        # Your training logic
        pass

    def predict(self, X):
        # Your prediction logic
        pass

2. Deploy to IRIS:

CREATE MODEL MyModel
PREDICTING (target)
FROM MyTable
USING {
    "model_name": "MyCustomModel",
    "path_to_classifiers": "/path/to/models"
}

Contributing

We welcome contributions! Please see our Contributing Guide for details.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/AmazingFeature)
3. Commit your changes (git commit -m 'Add some AmazingFeature')
4. Push to the branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

For support with IntegratedML Custom Models:

• InterSystems Developer Community
• InterSystems Support