Confidential AI inference for regulated enterprises: keep sensitive features encrypted, enforce tenant isolation, and produce audit-ready evidence.
Most classical ML deployments still decrypt sensitive inputs in application memory. That design creates a broad blast radius across logs, caches, APM tooling, support workflows, and incident response handling. Our platform changes the trust boundary: data is encrypted client-side, processed in encrypted form, and only decrypted by the data owner.
Security and privacy guidance increasingly emphasizes governance, accountability, and resilience for AI systems:
- NIST states that “AI actors are not all equally responsible or accountable,” underscoring the need for enforceable technical controls, not policy-only promises (NIST, 2023).
- CISA and international partners note that “data security is one of the most common vulnerabilities in AI systems,” highlighting practical attack surfaces in modern AI pipelines (CISA et al., 2024).
- OWASP identifies Sensitive Information Disclosure as a recurring application risk class, which becomes harder to control when ML inference requires plaintext in server memory (OWASP Foundation, 2021).
Our platform is built to reduce exposure from that exact class of risk.
Observed in practice: sensitive features often flow through API gateways, model servers, debug traces, exception stacks, and BI pipelines.
What we do:
- Encrypt features on the client side.
- Execute inference with FHE-compatible plans in runtime.
- Return encrypted outputs so only the key owner can decrypt.
CISO benefit: materially reduces plaintext handling scope and insider-access exposure.
Observed in literature and guidance: cross-border, cross-entity data handling and retention create legal/compliance overhead.
What we do:
- Tenant-scoped keys and model boundaries.
- Strong audit-chain support and structured operational controls.
- Explicit controls for mTLS, RBAC, and key lifecycle integration.
CISO benefit: easier control mapping for GDPR/HIPAA/PCI-oriented programs and lower evidence-collection effort during audits.
Observed in media and incident reporting: dependency compromise and configuration drift are persistent operational threats.
What we do:
- CI with guardrails, security scanning, and SBOM generation.
- Service boundary hardening and traceability-ready architecture.
- Least-privilege defaults and reproducible deployment patterns.
CISO benefit: improved attestability for vendor assessments and board-level risk reporting.
Our innovation set is implemented in services/innovations/ and documented in depth in docs/INNOVATIONS.md.
-
Leaf-Centric Encoding
Parallelized leaf-indicator computation for encrypted trees to reduce sequential branching overhead. -
Gradient-Aware Noise Allocation
Precision allocation by feature importance to preserve inference quality within FHE constraints. -
Homomorphic Ensemble Pruning
FHE-friendly significance scoring to reduce computation where tree contribution is low. -
Polynomial Leaf Functions
Additive leaf-level correction terms with validation safeguards. -
MOAI-Native Tree Conversion
Rotation-minimizing conversion path for tree execution plans. -
Bootstrap-Aligned Architecture
Runtime design aligned to noise-budget and bootstrapping economics. -
Federated Multi-Key Protocol
Enables multi-party encrypted collaboration patterns. -
Streaming Encrypted Gradients
Streaming-compatible encrypted update flows for operational adaptability. -
Unified Architecture Integration
Coherent integration across compiler/runtime/ops controls.
- They target real bottlenecks in practical FHE inference (rotations, precision/noise trade-offs, per-tree cost).
- They are validated by dedicated integration and benchmark suites in this codebase.
- They provide architecture-level options to trade off latency, accuracy retention, and operational risk.
Production note: some optimization families have workload-dependent behavior. We document limitations and failure modes explicitly, including regression-sensitive cases, to support risk-informed rollout decisions rather than “one-size-fits-all” claims.
- Transport security: mTLS between services.
- Identity and authorization: API key + tenant scoping + RBAC pathways.
- Key security: evaluation-key management with envelope-encryption integrations.
- Operational assurance: telemetry, health signals, and security scan hooks.
- Governance evidence: audit-chain and control-matrix supporting artifacts in
docs/compliance/.
- Plaintext feature vectors are not required in the server-side inference path by design.
- Cryptographic context is separated from application-level model metadata.
- Tenant separation is enforced at API and model-plan boundaries.
- Gateway (Go): authn/authz, routing, rate controls, observability.
- Registry (Go + SQL): model metadata and compiled-plan references.
- Compiler (Python): model parsing and IR transformation.
- Runtime (C++): encrypted execution path.
- Keystore (Go): key lifecycle and secure retrieval boundaries.
- SDKs (Python/TypeScript): client integration for encryption and inference workflows.
- XGBoost, LightGBM, CatBoost
- Scikit-learn Decision Trees and Random Forests
- Scikit-learn Logistic Regression
- Statsmodels GLM pathways
git clone https://github.com/your-org/fhe-gbdt-serving.git
cd fhe-gbdt-serving
make docker-up
make testThen review:
CISA, U.K. National Cyber Security Centre, Australian Cyber Security Centre, New Zealand National Cyber Security Centre, & Canadian Centre for Cyber Security. (2024). Guidelines for secure AI system development. https://www.cisa.gov/resources-tools/resources/guidelines-secure-ai-system-development
NIST. (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0) (NIST AI 100-1). National Institute of Standards and Technology. https://www.nist.gov/itl/ai-risk-management-framework
OWASP Foundation. (2021). OWASP Top 10: The ten most critical web application security risks. https://owasp.org/www-project-top-ten/
Verizon. (2024). 2024 Data Breach Investigations Report. https://www.verizon.com/business/resources/reports/dbir/
European Union Agency for Cybersecurity (ENISA). (2023). ENISA threat landscape 2023. https://www.enisa.europa.eu/publications/enisa-threat-landscape-2023
Gentry, C. (2009). A fully homomorphic encryption scheme (Doctoral dissertation, Stanford University). https://crypto.stanford.edu/craig/
This section extends (not replaces) the executive brief above with a deeper, risk-program view for security leadership.
In conventional ML serving, features are decrypted in memory to perform inference. That creates exposure through:
- Debug dumps
- Tracing payload capture
- Crash diagnostics
- Privileged admin shell access
Why CISOs care: these are hard-to-detect, high-impact pathways that frequently bypass preventive DLP controls.
Platform control response: encrypted feature handling from client to inference pipeline reduces plaintext residency in server memory.
Modern reliability stacks often ingest request metadata and payload-adjacent values by default.
Why CISOs care: telemetry systems are high-value aggregators that are broadly accessible to SRE/platform roles.
Platform control response: design posture minimizes sensitive plaintext in transit through telemetry surfaces and enforces structured control boundaries.
Multi-tenant AI services can drift into keying, policy, or routing mistakes under operational pressure.
Why CISOs care: cross-tenant access is an existential trust event for regulated buyers.
Platform control response: tenant-scoped key/material boundaries plus model/plan scoping and access control enforcement.
- Before: confidentiality depends heavily on process and privileged-user restraint.
- After: confidentiality is reinforced by cryptographic execution boundaries.
- Before: proving that plaintext was not over-exposed is difficult and costly.
- After: audit-chain and control artifacts simplify evidence narratives for internal/external audit.
- Before: incidents in model-serving tiers can implicate broad data classes.
- After: reduced plaintext concentration can lower potential blast radius in compromise scenarios.
- Before: enterprise buyers require extensive compensating controls for AI data handling.
- After: stronger security architecture improves trust posture for vendor assessments and DPAs.
Security leaders increasingly require alignment to recognized frameworks and observed attack patterns.
- NIST AI RMF highlights governance/accountability complexity and emphasizes measurable risk controls (NIST, 2023).
- CISA-led secure-AI development guidance calls out data security and secure-by-design implementation needs (CISA et al., 2024).
- ENISA threat landscape reporting reinforces that systemic cyber threats continue to evolve in ways that stress data-centric controls (ENISA, 2023).
- Verizon DBIR provides ongoing empirical evidence that data compromise remains a central incident impact domain (Verizon, 2024).
- OWASP Top 10 keeps sensitive information exposure as a practical and recurring risk class in internet-facing systems (OWASP Foundation, 2021).
Board-level interpretation: the strategic issue is no longer “whether AI is valuable,” but whether security architecture can keep confidentiality risk within organizational risk appetite as AI scales.
Below is an expanded explanation of why the implemented innovations are operationally significant.
Novelty angle: replaces sequential tree traversal assumptions with parallelized encrypted indicator computation patterns, improving viability under homomorphic constraints.
Novelty angle: treats precision as a budgeted security-performance variable informed by model structure, rather than uniform static quantization.
Novelty angle: enables significance-aware compute reduction without requiring plaintext model execution at runtime.
Novelty angle: augments leaf expressiveness while preserving guardrailed validation logic for production safety.
Novelty angle: focuses on rotation-cost minimization as a first-class systems problem in encrypted inference.
Novelty angle: aligns model/runtime orchestration with practical noise-budget economics and bootstrapping realities.
Novelty angle: addresses enterprise collaboration patterns where one-party trust assumptions are unacceptable.
Novelty angle: introduces encrypted adaptation pathways needed for near-real-time operational environments.
Novelty angle: integrates innovations into an end-to-end deployable stack, reducing “research-only” fragmentation.
In this repository’s scope, SOTA means:
- The approach addresses known FHE bottlenecks,
- The implementation is integrated into deployable services,
- Bench/test artifacts exist to evaluate trade-offs,
- Limitations are explicitly documented for safe rollout decisions.
- Select one high-value, high-sensitivity inference workload.
- Enforce tenant-level key separation and logging minimization policy.
- Define quantitative acceptance criteria (latency, error rate, evidence completeness).
- Introduce canary cohorts and per-tenant policy gates.
- Require runbook validation and incident playbook updates.
- Establish routine evidence export for compliance/audit workflows.
- Integrate third-party risk and architecture review checkpoints.
- Formalize SLO/SLA and escalation pathways with platform owners.
- Continuously review innovation toggles by workload risk profile.
No. It materially reduces specific plaintext-exposure classes, but governance, IAM hygiene, software supply chain, and operational controls remain essential.
It is designed for regulated environments with control evidence in mind; final suitability depends on your legal/regulatory scope and control mapping.
Use a structured scorecard: confidentiality boundary strength, evidence quality, performance envelope, operational complexity, and incident blast-radius profile.
Treating advanced optimization modes as universally safe. Some model/task combinations require conservative settings and staged validation.
National Institute of Standards and Technology. (2024). The NIST Cybersecurity Framework (CSF) 2.0. https://www.nist.gov/cyberframework
European Union. (2016). Regulation (EU) 2016/679 (General Data Protection Regulation). https://eur-lex.europa.eu/eli/reg/2016/679/oj
U.S. Department of Health & Human Services. (2013). Summary of the HIPAA Security Rule. https://www.hhs.gov/hipaa/for-professionals/security/laws-regulations/index.html
The following section is intentionally preserved from the prior README version to ensure no previously available documentation is removed.
Production-Grade Privacy-Preserving Inference for Trees, Linear Models, and GLMs
Every day, billions of machine learning predictions are made on sensitive data—medical diagnoses, financial risk scores, fraud detection, personalized recommendations. In every case, your raw data is exposed to the server. Healthcare providers see your symptoms. Banks see your transaction patterns. Advertisers see your preferences.
This isn't a bug—it's how ML has always worked. Until now.
Fully Homomorphic Encryption (FHE) has been theoretically possible since 2009, but practical deployment has been blocked by three fundamental barriers:
| Barrier | Traditional FHE | Our Solution |
|---|---|---|
| Performance | 1,000,000x slowdown | 60ms latency (100x improvement via MOAI optimizations) |
| Complexity | PhD-level cryptography required | Drop-in SDK for XGBoost/LightGBM/CatBoost |
| Production Readiness | Research prototypes only | Enterprise-grade with mTLS, RBAC, observability |
Previous attempts failed because they treated FHE as a cryptographic problem. We solved it as a systems engineering problem—optimizing the entire stack from model compilation to encrypted execution.
1. MOAI (Model-Oblivious Architecture Interpreter) Our proprietary compiler transforms supported ML models (GBDTs, decision trees, random forests, logistic regression, and GLMs) into an FHE-optimized execution plan:
- Column Packing: Eliminates 90% of costly rotations through intelligent feature layout
- Polynomial Step Functions: Replaces lookup tables with smooth approximations
- Interleaved Aggregation: Processes all trees in parallel, not sequentially
2. N2HE-HEXL Integration We've built the first production integration of the N2HE hybrid encryption scheme with Intel HEXL acceleration:
- Hardware Acceleration: AVX-512 and GPU NTT for 10x throughput
- Noise Management: Automatic bootstrapping to prevent decryption failures
- 128-bit Security: Provably secure against quantum-resistant lattice attacks
3. Universal Model Support Production support across tree ensembles and generalized linear models:
- XGBoost / LightGBM / CatBoost for gradient-boosted trees
- Scikit-learn Decision Trees & Random Forests for classical tabular ML
- Scikit-learn Logistic Regression & Statsmodels GLM for linear/probabilistic scoring
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Your Data │ │ Our Cloud │ │ Your Result │
│ (Encrypted) │ ──────▶ │ (Blind) │ ──────▶ │ (Decrypted) │
│ │ │ │ │ │
│ features=[...] │ │ 🔒 Computes on │ │ prediction=0.87 │
│ 🔐 AES+RLWE │ │ ciphertext │ │ 🔓 Only you can │
│ │ │ only │ │ decrypt │
└─────────────────┘ └─────────────────┘ └─────────────────┘
The server never sees your data. Ever.
Your features are encrypted on your device using lattice-based cryptography. Our servers perform the entire GBDT inference—comparisons, tree traversals, aggregations—on encrypted data. Only you hold the secret key to decrypt the result.
| Use Case | Benefit |
|---|---|
| Healthcare | Run diagnostic models without exposing patient records |
| Finance | Credit scoring without revealing transaction history |
| Insurance | Risk assessment without accessing sensitive claims |
| Government | Citizen services without centralized data collection |
| Enterprise | Deploy ML models without data residency concerns |
| Model | Library | Encrypted P50 | Plaintext P50 | Overhead | Throughput |
|---|---|---|---|---|---|
| Classification | CatBoost | 62.03ms | 0.18ms | 339x | ~600 eps |
| Classification | XGBoost | 61.68ms | 0.63ms | 97x | ~500 eps |
| Regression | LightGBM | 62.47ms | 0.72ms | 86x | ~450 eps |
eps = encrypted predictions per second
CatBoost uses oblivious (symmetric) decision trees where all nodes at the same depth use identical split conditions. This maps perfectly to FHE's SIMD operations:
| Metric | CatBoost | XGBoost | LightGBM |
|---|---|---|---|
| Crypto Rotations | 8 | 12 | 28 |
| Scheme Switches | 2 | 4 | 6 |
| Memory Transfers | Low | Medium | High |
Recommendation: Use CatBoost for new models. XGBoost/LightGBM supported for migration.
| Metric | Target | Measurement |
|---|---|---|
| Latency (p50) | < 50ms | Latency profile |
| Latency (p95) | < 100ms | Latency profile |
| Latency (p99) | < 200ms | Latency profile |
| Availability | 99.9% | Monthly uptime |
| Error Rate | < 0.1% | 5xx responses |
| Queue Depth | < 100 | Pending requests |
┌────────────────────────────────────────────────────────────────────────────┐
│ CLIENT SIDE │
│ ┌──────────────┐ │
│ │ Python SDK │ • Key generation (secret + eval keys) │
│ │ │ • Feature encryption (RLWE ciphertexts) │
│ │ │ • Result decryption │
│ └──────┬───────┘ │
└─────────┼──────────────────────────────────────────────────────────────────┘
│ gRPC + mTLS
▼
┌────────────────────────────────────────────────────────────────────────────┐
│ SERVER SIDE │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Gateway │───▶│ Registry │ │ Keystore │ │
│ │ │ │ │ │ │ │
│ │ • Auth/AuthZ │ │ • Model meta │ │ • Eval keys │ │
│ │ • Rate limit │ │ • Plan store │ │ • Envelope │ │
│ │ • OTel trace │ │ • Versioning │ │ encryption │ │
│ └──────┬───────┘ └──────────────┘ └──────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ Compiler │───▶│ Runtime │ │
│ │ │ │ │ │
│ │ • GBDT libs │ │ • C++ engine │ │
│ │ • RF / DT │ │ • N2HE-HEXL │ │
│ │ • LogReg/GLM │ │ • GPU accel │ │
│ │ • MOAI optim │ │ • SIMD/AVX2 │ │
│ └──────────────┘ └──────────────┘ │
│ │
└────────────────────────────────────────────────────────────────────────────┘
| Component | Technology | Purpose |
|---|---|---|
| Gateway | Go + gRPC | Authentication, rate limiting, request routing |
| Registry | Go + PostgreSQL | Model metadata, compiled plan storage |
| Keystore | Go + Vault | Evaluation key management with envelope encryption |
| Compiler | Python | Multi-model (GBDT/RF/DT/LogReg/GLM) → ObliviousPlanIR transformation |
| Runtime | C++ | FHE execution engine with N2HE-HEXL |
| SDK | Python | Client-side crypto and API interface |
# Clone the repository
git clone https://github.com/your-org/fhe-gbdt-serving.git
cd fhe-gbdt-serving
# Start all services
make docker-up
# Run the complete cookbook (trains models, encrypts, predicts)
make cookbookfrom fhe_gbdt_sdk import FHEGBDTClient
from fhe_gbdt_sdk.crypto import N2HEKeyManager
# Initialize crypto (client-side)
key_manager = N2HEKeyManager("my-tenant-id")
key_manager.generate_keys()
# Connect to gateway
client = FHEGBDTClient(
gateway_url="localhost:8080",
key_manager=key_manager
)
# Upload evaluation keys (one-time)
client.upload_eval_keys()
# Register and compile your model
model_id = client.register_model("my_xgboost_model.json", "xgboost")
compiled_id = client.compile_model(model_id)
# Encrypt features and predict
features = [5.1, 3.5, 1.4, 0.2] # Iris flower measurements
encrypted_features = key_manager.encrypt(features)
# Server computes on encrypted data
encrypted_result = client.predict(compiled_id, encrypted_features)
# Only you can decrypt
prediction = key_manager.decrypt(encrypted_result)
print(f"Prediction: {prediction}") # e.g., [0.92, 0.05, 0.03]Detailed end-to-end examples for common use cases:
| Recipe | Description | Dataset |
|---|---|---|
| 00 Quickstart | 5-minute introduction | Synthetic |
| 01 XGBoost Classification | Binary classification | Breast Cancer |
| 02 LightGBM Regression | Continuous prediction | California Housing |
| 03 CatBoost Classification | Multi-class with oblivious trees | Iris |
| 04 Trade-offs | Performance comparison guide | - |
| 05 Troubleshooting | Common issues & solutions | - |
| Parameter | Value | Security Level |
|---|---|---|
| Scheme | N2HE (RLWE + LWE hybrid) | 128-bit |
| Ring Dimension | 4096 / 8192 | Post-quantum secure |
| Ciphertext Modulus | 2^32 | - |
| Error Distribution | Gaussian, σ=3.2 | - |
| Layer | Control | Implementation |
|---|---|---|
| Transport | mTLS | All service-to-service communication |
| Authentication | API Key + Tenant ID | Gateway enforcement |
| Authorization | RBAC | Tenant → Model → Plan scoping |
| Data at Rest | Envelope Encryption | Keystore (Vault backend) |
| Audit | Structured Logging | Request ID + Tenant ID tracing |
| Supply Chain | SBOM + Scanning | SAST, dependency, container scans |
We follow STRIDE methodology:
| Threat | Mitigation |
|---|---|
| Spoofing | API key binding to tenant_id |
| Tampering | Content-addressed plan IDs (SHA-256) |
| Repudiation | Immutable audit logs |
| Information Disclosure | FHE encryption (features never plaintext on server) |
| Denial of Service | Per-tenant rate limiting (100 req/s, burst 200) |
| Elevation of Privilege | Tenant isolation, no cross-tenant access |
# Install with Helm
helm install fhe-gbdt ./deploy/helm \
--set gateway.replicas=3 \
--set runtime.replicas=5 \
--set runtime.gpu.enabled=trueSee config/production.py for all options:
from config import ProductionConfig, SecurityLevel
config = ProductionConfig.from_env()
config.crypto.security_level = SecurityLevel.HIGH # 192-bit
config.tls.require_client_cert = True
config.rate_limit.requests_per_second = 100| Tool | Purpose | Dashboard |
|---|---|---|
| Prometheus | Metrics collection | dashboards/grafana/slo.json |
| Grafana | Visualization | SLO dashboard included |
| Jaeger | Distributed tracing | OpenTelemetry integration |
| AlertManager | Alerting | SLO-based rules |
# Unit tests (20 tests)
make test
# Integration tests - AI Engineer workflow (17 tests)
python -m pytest tests/integration/test_ai_engineer_workflow.py -v
# E2E tests with real models (4 tests)
make e2e-real
# Full test suite (44 tests)
python -m pytest tests/ -v| Category | Tests | Purpose |
|---|---|---|
| Unit | 20 | Parser, IR, kernel algebra, security patterns |
| Integration | 17 | Full AI engineer workflow (train → encrypt → predict) |
| E2E | 10 | Real XGBoost/LightGBM/CatBoost models |
| Fuzz | 2 | Parser robustness against malformed input |
| Metamorphic | 3 | Property-based testing (Hypothesis) |
Jobs:
- build-and-test # Build + unit tests + security scan + SBOM
- guardrails # Forbidden pattern detection (no plaintext logging)
- cookbook-tests # E2E with real ML models
- integration-tests # AI engineer workflow validation
- unit-tests # Fast feedback on PRs| Category | Items | Status |
|---|---|---|
| Security | mTLS, AuthZ, secrets, scanning | 8/8 ✅ |
| Correctness | E2E regression, metamorphic tests | 5/5 ✅ |
| Performance | Benchmarks, baselines, monitoring | 4/4 ✅ |
| Reliability | Load tests, soak tests, error budget | 4/4 ✅ |
| Operability | Dashboards, alerts, runbooks | 6/6 ✅ |
See PRODUCTION_READINESS.md for full details.
| Document | Description |
|---|---|
| ARCHITECTURE.md | System design and component details |
| THREAT_MODEL.md | STRIDE security analysis |
| SLO.md | Service level objectives and error budgets |
| CRYPTO_PARAMS.md | Cryptographic parameter selection |
| RUNBOOKS.md | Operational procedures |
| OPERATIONS.md | Day-to-day operational guide |
- Docker & Docker Compose
- Python 3.9+
- Go 1.21+
- (Optional) CUDA 11+ for GPU acceleration
make build # Build all Docker images
make docker-up # Start development stack
make stop # Stop all services
make test # Run unit tests
make e2e-real # Run E2E tests with real models
make cookbook # Run all cookbook recipes
make clean # Clean build artifactsfhe-gbdt-serving/
├── services/
│ ├── gateway/ # Go - API gateway
│ ├── runtime/ # C++ - FHE execution engine
│ ├── compiler/ # Python - Model compiler
│ ├── registry/ # Go - Model registry
│ └── keystore/ # Go - Key management
├── sdk/python/ # Python client SDK
├── proto/ # gRPC service definitions
├── tests/ # Test suite
├── docs/ # Documentation
├── bench/ # Benchmarking tools
├── deploy/ # Kubernetes/Helm charts
├── dashboards/ # Grafana dashboards
└── config/ # Configuration modules
We welcome contributions! Please see our Contributing Guide for details.
Please report security issues to security@example.com. We respond within 48 hours and fix critical issues within 14 days.
Apache 2.0 - See LICENSE for details.
- N2HE - Hybrid FHE library
- Intel HEXL - Hardware acceleration
- XGBoost, LightGBM, CatBoost - GBDT libraries
Privacy is not a feature. It's a right.
FHE-GBDT Serving - Inference without exposure.
For many CISOs, the core blocker in enterprise ML is not model quality—it is data-sharing risk across subsidiaries, counterparties, clinical partners, and vendors. Teams can build models, but legal, security, and regulatory constraints often stop deployment where cross-organization data is required.
This platform addresses that bottleneck by enabling inference on encrypted features so data owners do not need to disclose raw inputs to the serving environment.
- Traditional pattern: centralize or replicate sensitive data, then protect it with compensating controls.
- Confidential-compute pattern here: keep feature-level confidentiality boundary at the data owner, execute encrypted inference, and only decrypt results where policy allows.
For security leadership, this shifts control design from “who can access plaintext” to “where plaintext is ever required.”
- The World Economic Forum documents that financial crime controls depend on institution-to-institution collaboration and that data-sharing constraints are a major friction point for collective defense programs (World Economic Forum, 2020).
- The UK FCA/ICO joint guidance on privacy-enhancing technologies highlights PETs as mechanisms to enable analytics while reducing personal data exposure in regulated contexts (Information Commissioner’s Office & Financial Conduct Authority, 2023).
- Major banking privacy incidents continue to show high impact from data exposure events, reinforcing CISO focus on minimizing plaintext proliferation in analytic pipelines (Verizon, 2024).
Operational implication: encrypted inference pathways can support collaborative risk scoring and model-serving use cases with lower cross-entity disclosure pressure.
- Industry and policy literature consistently notes that healthcare AI value depends on multi-institution data breadth, while PHI constraints and breach risks impede central data pooling (ENISA, 2023; HHS, 2013).
- HHS OCR breach reporting trends and healthcare incident analyses repeatedly show the sensitivity and legal impact of unauthorized health data disclosure, increasing the burden on CISOs to reduce exposure-by-design (U.S. Department of Health & Human Services, n.d.; Verizon, 2024).
- PET guidance from regulators and standards bodies points to privacy-preserving analytics as a practical route for extracting insight without equivalent raw-data movement (Information Commissioner’s Office & Financial Conduct Authority, 2023; NIST, 2023).
Operational implication: encrypted inference supports diagnostic and risk models where hospitals/partners can consume model value without handing over raw patient features to central serving tiers.
-
Bank consortium risk scoring
Member institutions submit encrypted feature vectors to a shared model-serving fabric; only authorized members decrypt final scores. -
Cross-hospital clinical decision support
Participating providers run encrypted inference against shared models without exposing raw PHI to the central runtime operator. -
Payer-provider fraud analytics
High-risk claims are scored through encrypted workflows, reducing bilateral data transfer and plaintext handling obligations.
Evaluate readiness with a security governance scorecard:
- Plaintext minimization: Is plaintext avoided in shared serving infrastructure?
- Tenant/key isolation: Are keys and model scopes partitioned by policy boundary?
- Evidence generation: Can audit/compliance teams trace controls without disclosing sensitive data?
- Incident containment: Does architecture reduce blast radius if one control plane is compromised?
- Regulatory fit: Can control narratives map to HIPAA/GDPR/sector expectations for confidentiality and accountability?
- Start with high-sensitivity, bounded-scope models (fraud triage, readmission risk, claims routing).
- Enforce strict key lifecycle ownership and dual-control for key management changes.
- Add policy checks to prevent fallback routes that reintroduce plaintext at integration boundaries.
- Run tabletop incident exercises specifically for encrypted-inference failure modes (key loss, ciphertext replay, mis-scoped tenant access).
Information Commissioner’s Office, & Financial Conduct Authority. (2023). Privacy-enhancing technologies guidance. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/privacy-enhancing-technologies/
U.S. Department of Health & Human Services. (n.d.). Breach portal: Notice to the Secretary of HHS breach of unsecured protected health information. https://ocrportal.hhs.gov/ocr/breach/breach_report.jsf
World Economic Forum. (2020). The new frontier in financial crime: From defensive to offensive measures. https://www.weforum.org/reports/the-new-frontier-in-financial-crime/
For a code-referenced production gap assessment and remediation plan, see docs/PRODUCTION_NO_MOCKS_GAP_REVIEW.md.
- Organization A (Data Owner): A regional hospital network that has sensitive patient records.
- Organization B (Model Operator): A specialized analytics company with a validated sepsis-risk model.
- Goal: Organization B runs inference on Organization A’s patient features to generate clinical risk insights, without B ever seeing patient plaintext data.
- A does not share raw feature vectors with B.
- B does not hold A’s decryption key.
- Runtime operators only process ciphertext.
- Final predictions are only decrypted by A.
- A uses SDK + client-side encryption and key management.
- B hosts model registration/compilation and runtime services.
- Keystore stores evaluation keys only (not secret keys).
- Gateway enforces tenant/model controls and forwards encrypted inference.
A and B agree on:
- A remains data controller/owner of raw patient features.
- B operates model-serving infrastructure but cannot decrypt A data.
- Allowed model purpose, retention, and audit requirements.
- Incident response ownership and escalation paths.
Deliverable: data processing agreement + control matrix mapping to technical controls.
- B registers the model in registry.
- B compiles it to encrypted execution plan.
- B publishes model/version/profile metadata to A.
Deliverable: model_id, compiled_model_id, expected feature schema.
- A uses local SDK key manager to generate secret key + evaluation keys.
- Secret key remains in A’s controlled environment.
- A uploads evaluation keys to keystore.
Security property: B can evaluate encrypted data but cannot decrypt input or output.
- A’s EHR pipeline extracts required feature vector.
- A normalizes/transforms locally using approved preprocessing.
- A encrypts the vector client-side before network transfer.
Security property: no raw patient features cross trust boundary.
- Gateway authenticates tenant, validates model scope, forwards request.
- Runtime loads compiled plan and evaluation keys.
- Runtime computes prediction over ciphertext.
- Service returns encrypted result payload.
Operational property: runtime disconnect/failure returns explicit error, not simulated success.
- A decrypts response locally.
- A writes risk score/insight into care workflow.
- A stores only policy-approved metadata in audit systems.
Outcome: A gets model insight; B never sees patient plaintext.
- A input (plaintext):
[age, lactate, heart_rate, wbc, ...](local only) - A output (ciphertext):
C(features)-> sent to B runtime - B runtime output (ciphertext):
C(risk_score)-> returned to A - A decrypts:
risk_score = 0.82and triggers care pathway
At no point does B receive plaintext features or plaintext risk context tied to patient-level raw inputs.
- Confidentiality KPI: 0 plaintext feature payloads outside A boundary.
- Reliability KPI: >=99.9% successful encrypted inference over pilot window.
- Security KPI: 100% requests with tenant/model audit trace.
- Clinical/Business KPI: agreed threshold for precision/recall and intervention lift.
- Disable SDK simulation in production (
ALLOW_SDK_SIMULATION=false). - Enforce durable storage for keystore and registry paths.
- Require dependency-aware readiness checks for go-live.
- Add runbook drills for runtime unavailable, key retrieval failures, and rollback.
- Pin CI dependencies and require green build before model/version promotion.
- It enables collaboration value (A uses B’s model expertise) without data surrender.
- It materially reduces breach blast radius from centralized plaintext ML serving.
- It creates auditable separation of duties between data owner and compute operator.