Skip to content

BharatMLStack is an open-source, end-to-end machine learning infrastructure stack built at Meesho to support real-time and batch ML workloads at Bharat scale

License

Notifications You must be signed in to change notification settings

Meesho/BharatMLStack

Repository files navigation

BharatMLStack

BharatMLStack Logo

CI Static Badge Discord Made in India

Meesho

What is BharatMLStack?

BharatMLStack is a comprehensive, production-ready machine learning infrastructure platform designed to democratize ML capabilities across India and beyond. Our mission is to provide a robust, scalable, and accessible ML stack that empowers organizations to build, deploy, and manage machine learning solutions at massive scale.

Our Vision

๐ŸŽฏ Democratize Machine Learning: Make advanced ML infrastructure accessible to organizations of all sizes ๐Ÿš€ Scale Without Limits: Built to handle millions of requests per second with enterprise-grade reliability ๐Ÿ‡ฎ๐Ÿ‡ณ India-First Approach: Optimized for Indian market needs while maintaining global standards โšก Real-Time Intelligence: Enable instant decision-making with sub-millisecond feature serving ๐Ÿ”ง Developer-Friendly: Intuitive APIs and interfaces that accelerate ML development cycles

Running at Million Scale

BharatMLStack is battle-tested in production environments, powering:

  • 1M+ feature vector retrievals per second across distributed deployments
  • Sub-10ms latency for real-time feature retrieval
  • 99.99% uptime with auto-scaling and fault tolerance
  • Petabyte-scale feature storage and processing
  • Multi-region deployments with global load balancing

Document

Core Components

๐Ÿ“‹ Current Releases

Component Version Description
๐Ÿš€ Horizon v1.0.0 Control Plane & Backend
๐ŸŽจ Trufflebox UI v1.0.0 ML Management Console
๐Ÿ—„๏ธ Online Feature Store v1.0.0 Real-Time Features
๐Ÿน Go SDK v1.0.0 Go Client Library
๐Ÿ Python SDK v1.0.1 Python Client Library

๐Ÿš€ Horizon - Control Plane & Backend

The central control plane for BharatMLStack components, serving as the backend for Trufflebox UI.

  • Component orchestration: Manages and coordinates all BharatMLStack services
  • API gateway: Unified interface for all MLOps and workflows

๐ŸŽจ Trufflebox UI - ML Management Console

Modern web interface for managing ML models, features, and experiments. Currently it supports:

  • Feature Registry: Centralized repository for feature definitions and metadata
  • Feature Cataloging: Discovery and search capabilities for available features
  • Online Feature Store Control System: Management interface for feature store operations
  • Approval Flows: Workflow management for feature deployment and changes

๐Ÿ—„๏ธ Online Feature Store - Real-Time Features

High-performance feature store for real-time ML inference and training.

  • Real-time serving: Sub-10ms feature retrieval at scale
  • Streaming ingestion: Process millions of feature updates per second
  • Feature Backward Compatible Versioning: Track and manage feature evolution
  • Multi-source integration: Push from stream, batch and real-time sources

Key Differentiators

  • โœจ Production-Ready: Battle-tested components used in high-traffic production systems
  • ๐ŸŒ Cloud Agnostic: Kubernetes-native, so deploy on the cloud you love
  • ๐Ÿ“Š Observability: Built-in monitoring, logging

Quick Start

๐Ÿš€ Get started with BharatMLStack in minutes!

For comprehensive setup instructions, examples, and deployment guides, see our detailed Quick Start documentation:

๐Ÿ“– Quick Start Guide โ†’

What You'll Find:

  • ๐Ÿณ Docker Setup: Complete stack deployment with Docker Compose
  • ๐Ÿ“Š Sample Data: Pre-configured examples to get you started
  • ๐Ÿ” Health Checks: Verify your deployment is working
  • ๐Ÿ“ Step-by-Step Tutorials: From installation to first feature operations

TL;DR - One Command Setup:

# Clone and start the complete stack
git clone https://github.com/Meesho/BharatMLStack.git
cd BharatMLStack/quick-start
ONFS_VERSION=<version> HORIZON_VERSION=<version> TRUFFLEBOX_VERSION=<version> ./start.sh

Then follow the Quick Start Guide for detailed setup and usage instructions.

Architecture

BharatMLStack follows a microservices architecture designed for scalability and maintainability. Several components are to be open-sourced

BharatMLStack Logo

๐Ÿš€ Quick Navigation

Component Documentation Quick Start
Online Feature Store Docs Setup
Go SDK Docs Examples
Python SDK Docs Quickstart
User Guide Docs Setup

Contributing

We welcome contributions from the community! Please see our Contributing Guide for details on how to get started.

Community & Support

License

BharatMLStack is open-source software licensed under the BharatMLStack Business Source License 1.1.


Built with โค๏ธ for the ML community from Meesho
If you find this useful, โญ๏ธ the repo โ€” your support means the world to us!

About

BharatMLStack is an open-source, end-to-end machine learning infrastructure stack built at Meesho to support real-time and batch ML workloads at Bharat scale

Topics

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages