-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Application: Machine Messaging on IPFS pubsub #54
Comments
@chrismatthieu has been paving the path with https://computes.io 🙌🏽 |
Thanks @diasdavid. @flyingzumwalt our blog has a bunch of cool videos of our IPFS-powered global neural net supercomputer. https://blog.computes.io |
A few months ago I forgot to add some notes back here: Machine Messaging on IPFS pubsubMachine messaging is a very wide topic ranges from traditional Message-Oriented middlewares, such a AMQP implementations or XMPP, to novel approaches to instant messaging and pubsub in distributed sytems. In very simple terms and making an wide definition, machine messaging [middleware] provides a layer which allows a set of producers to publish messages in the system. These messages are consumed or delivered to clients which are fully independent. They usually use queues (but not necessarily) to store the messages while they have not yet been consumed or delivered. Usecases
Which foundational features & functionality do the use cases need or rely on (ie. libp2p, private networks, etc.)?
Related products or projects that are addressing those use cases
Related organizations who either 1) have those use cases or 2) are building solutions for those use cases
|
I believe that @mcollina would be interested in this convo! :) Hi Matteo! Are you still doing stuff with MQTT and Mosca? Have you heard about IPFS? Familiar with the work that @chrismatthieu is doing with Computes.io? I'm interested in learning how pluggable MQTT over libp2p would be? At first glance it seems it is very dependable of the low level transport, but if we could mount MQTT control and message streams over any duplex stream, then we can use libp2p to establish the P2P connection and MQTT to do the brokering system. Thanks in advance :) |
Hey @diasdavid, MQTT needs a binary duplex channel to run. Can this be achieved on top of libp2p? |
Absolutely it can! IPFS uses it that way to transfer files as blocks of data and other apps have been using it in the same way for other purposes. We can mount any protocol in libp2p using Protocol Multiplexing, saving the hurdle of open extra connections between peers which are quite expensive in a P2P network. @pgte has also been doing a lot of work of using libp2p pubsub to distribute state events for CRDTs and one of the demos I did in the past was loading the Ethereum Blockchain on the browser using libp2p. |
Folks at IDEO have been using libp2p too. @ReidWilliamsm, @gavinmcdermott you might be interested in following this thread :) |
Protocol multiplexing seems message oriented. Can it be mapped 1-1 with a Unix socket? Or code-wise, can it be implemented as a |
We still use @mcollina Mosca MQTT broker on Octoblu and love it. I would gladly incorporate his MQTT/IPFS implementation into Computes! |
@mcollina IPFS supports pubsub. Nodes can listen and broadcast to channels and even setup P2P channels by their IPFS node IDs. It doesn't have the concept of QoS. This would be very important to Computes. There's a very interesting IPFS database project underway called OrbitDB (https://github.com/orbitdb/orbit-db). You may be able to use this DB for tracking delivery like an MQTT broker. |
Are there any Golang examples of this? |
Following with interest :) |
Hush blockchain integration may be an idea. https://github.com/MyHush Hushlist: Censorship Resistant Metadata-Minimizing Multi-Blockchain Communication |
Note: Discussion on applications of IPFS are happening over in the IPFS Forums now ... please continue the discussion there! This issue is being moved over to the archived repo https://github.com/ipfs/apps/ for reference. |
Work in progress - please contribute. See #40.
The basic concept: use IPFS + ipfs-based pubsub to manage things like queues that orchestrate machine workers instead of using centralized systems like redis or Amazon's queue + messaging services.
The text was updated successfully, but these errors were encountered: