Skip to content


MemQ: An efficient, scalable cloud native PubSub system

MemQ is a new PubSub system that augments Kafka at Pinterest. It uses a decoupled storage and serving architecture similar to Apache Pulsar and Facebook Logdevice; however, it relies on a pluggable replicated storage layer i.e. Object Store / DFS / NFS for storing data. The net result is a PubSub system that:

  • Handles GB/s traffic
  • Independently scales, writes, and reads
  • Doesn’t require expensive rebalancing to handle traffic growth
  • Is 90% more cost effective than our Kafka footprint


Getting Started

Checkout our Getting Started Guide


MemQ can be a valuable PubSub choice for the following use case families:

  • Large scale data ingestion
  • Bulk uploads
  • Near Real Time Analytics


Detailed architecture docs can be found here


Please checkout the original MemQ blog for details and background.


Ambud Sharma

Ping Min-Lin

See contributors


MemQ distributed under Apache License, Version 2.0