Skip to content

memphisdev/memphis-broker

master
Switch branches/tags
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
db
Aug 24, 2022
Aug 24, 2022
Sep 13, 2022
Sep 29, 2022
Aug 24, 2022
Sep 29, 2022
Jul 31, 2022
Aug 24, 2022
Aug 22, 2022
Aug 21, 2022

Memphis light logo

Memphis light logo

Real-Time Data Processing Platform

Memphis UI

Playground - Sandbox - Docs - Twitter - YouTube

Discord Code Of Conduct GitHub release (latest by date)

Memphis{dev} is an open-source real-time data processing platform
that provides end-to-end support for in-app streaming use cases using Memphis distributed message broker.
Memphis' platform requires zero ops, enables rapid development, extreme cost reduction,
eliminates coding barriers, and saves a great amount of dev time for data-oriented developers and data engineers.

📸 Screenshots

Dashboard Station (Topic) overview CLI
Dashboard Station Overview drawing

⭐️ Why

Working with data streaming is HARD.

As a developer, you need to build a dedicated pipeline for each data source,
work with schemas, formats, serializations, analyze each source individually,
enrich the data with other sources, constantly change APIs, and scale for better performance 🥵.
Besides that, it constantly crashes and requires adaptation to different rate limits.
It takes time and resources that you probably don't have.

Message broker acts as the middleman and supports streaming architecture,
but then you encounter Apache Kafka and its documentation and run back to the monolith and batch jobs.
Give memphis{dev} a spin before.

👉 Use-cases

  • Async task management
  • Real-time streaming pipelines
  • Data ingestion
  • Cloud Messaging
    • Services (microservices, service mesh)
    • Event/Data Streaming (observability, analytics, ML/AI)
  • Queuing
  • N:N communication patterns

Features

Roadmap

v0.3.6

  • 🚀 Fully optimized message broker in under 3 minutes
  • 💻 Easy-to-use UI, CLI, and SDKs
  • 📺 Data-level observability
  • ☠️ Dead-Letter Queue with automatic message retransmit
  • SDKs: Node.JS, Go, Python, Typescript, NestJS
  • 🐳 Runs on your Docker or Kubernetes
  • 👨‍💻 Community driven

🚀 Getting Started

Sandbox
Installation videos

Helm for Kubernetes

helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && \
helm install my-memphis memphis/memphis --create-namespace --namespace memphis

Docker🐳 Compose

curl -s https://memphisdev.github.io/memphis-docker/docker-compose.yml -o docker-compose.yml && \
docker compose -f docker-compose.yml -p memphis up

connect your first app

Build an event-driven food delivery app

High-Level Architecture

memphis.dev-logo

Local access

Via Kubernetes

To access Memphis UI from localhost, run the below commands:

kubectl port-forward service/memphis-ui 9000:80 --namespace memphis > /dev/null &

To access Memphis using CLI or SDK from localhost, run the below commands:

kubectl port-forward service/memphis-cluster 6666:6666 5555:5555 --namespace memphis > /dev/null &

Dashboard: http://localhost:9000
Memphis broker: http://localhost:6666

For Production Environments Please expose the UI, Cluster, and Control-plane via k8s ingress / load balancer / nodeport

Via Docker

UI - http://localhost:9000
Broker - http://localhost:6666

Beta

Memphis{dev} is currently in Beta version. This means that we are still working on essential features like real-time messages tracing, schema registry and inline processing as well as making more SDKs and supporting materials.

How does it affect you? Well... mostly it doesn't.
(a) The core of memphis broker is highly stable
(b) We learn and fix fast

But we need your love, and any help we can get by stars, PR, feedback, issues, and enhancments.
Read more on Memphis{dev} Documentation 📃.

Support 🙋‍♂️🤝

Ask a question about Memphis{dev} or something related to us:

We welcome you to our discord server with your questions, doubts and feedback.

Create a bug 🐞 report

If you see an error message or run into an issue, please create bug report. This effort is valued and it will help all Memphis{dev} users.

Submit a feature 💡 request

If you have an idea, or you think that we're missing a capability that would make development easier and more robust, please Submit feature request.

If an issuewith similar feature request already exists, don't forget to leave a "+1". If you add some more information such as your thoughts and vision about the feature, your comments will be embraced warmly :)

Contributing

Memphis{dev} is an open-source project.
We are committed to a fully transparent development process and appreciate highly any contributions.
Whether you are helping us fix bugs, proposing new features, improving our documentation or spreading the word - we would love to have you as part of the Memphis{dev} community.

Please refer to our Contribution Guidelines and Code of Conduct.

Contributors

Thanks goes to these wonderful people :

License 📃

Please check out License to read the full text.