Skip to content
development
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Build Status License PyPI version fury.io Documentation Code style: black

MLRun logo

MLRun - The Open Source MLOps Orchestration Framework

MLRun enables production pipeline design using a modular strategy, where the different parts contribute to a continuous, automated, and far simpler path from research and development to scalable production pipelines, without refactoring code, adding glue logic, or spending significant efforts on data and ML engineering.

MLRun uses Serverless Function technology: write the code once, using your preferred development environment and simple “local” semantics, and then run it as-is on different platforms and at scale. MLRun automates the build process, execution, data movement, scaling, versioning, parameterization, outputs tracking, CI/CD integration, deployment to production, monitoring, and more.

Those easily developed data or ML “functions” can then be published or loaded from a marketplace and used later to form offline or real-time production pipelines with minimal engineering efforts.

mlrun-flow


Data preparation, model development, model and application delivery, and end to end monitoring are tightly connected: they cannot be managed in silos. This is where MLRun MLOps orchestration comes in. ML, data, and DevOps/MLOps teams collaborate using the same set of tools, practices, APIs, metadata, and version control.

MLRun simplifies & accelerates the time to production.

Architecture

pipeline

MLRun is composed of the following layers:

  • Feature Store — collects, prepares, catalogs, and serves data features for development (offline) and real-time (online) usage for real-time and batch data. See also Feature store: data ingestion and Feature store: data retrieval, as well as the Feature Store tutorials.
  • ML CI/CD pipeline — automatically trains, tests, optimizes, and deploys or updates models using a snapshot of the production data (generated by the feature store) and code from the source control (Git).
  • Real-Time Serving Pipeline — Rapid deployment of scalable data and ML pipelines using real-time serverless technology, including the API handling, data preparation/enrichment, model serving, ensembles, driving and measuring actions, etc.
  • Real-Time monitoring and retraining — monitors data, models, and production components and provides a feedback loop for exploring production data, identifying drift, alerting on anomalies or data quality issues, triggering re-training jobs, measuring business impact, etc.

Get started

It's easy to start using MLRun:

  1. Install MLRun locally using Docker or over Kubernetes Cluster. Alternatively, you can use Iguazio's managed MLRun service
  2. Set up your client environment to work with the service.
  3. Follow the Quick Start tutorial and Additional Tutorials and Examples to learn how to use MLRun to develop and deploy machine learning applications to production.

For hands-on learning, try the MLRun Katakoda Scenarios. And you can watch the Tutorial on Youtube to see the flow in action.

MLRun documentation

Read more in the MLRun documentation, including: