Home
Welcome to the controlling-flow-ofRequests-toBlockchain-using-Redis-and-RabbitMQ wiki!
Leveraging Cache and Messaging Queue to scale Blockchain Network
Control flow of requests to Blockchain network using RabbitMQ and Redis Cluster.
Emerging Tech - Blockchain
Congrats! You have made it to the fourth in a series of patterns that teaches a developer how to get up and get going creating a Blockchain application using Fabric-Node-SDK. Now that you know how to create the network and define operations that can be done on the network you are ready to learn how to integrate RabbitMQ and Redis Custer to scale your Blockchain application. Let's get started with optimizing our blockchain application to handle a large number of requests.
By Ishan Gulhane
https://github.com/IBM/controlling-flow-ofRequests-toBlockchain-using-Redis-and-RabbitMQ
N/A
In this pattern, we will configure Redis and RabbitMQ cluster in our application architecture. With the direct use of REST API calls, it is not possible to control the number of requests sent to blockchain network, this might cause errors such as read/write conflicts etc. In order to control the flow of request sent to blockchain network and scale our application, we will do 2 things 1) use RabbitMQ cluster with 3 nodes consisting of mirrored queues to order the user requests and then 2) use Redis cluster with 6 nodes (3 master and 3 slaves) where results of execution are stored for a short duration. In the architecture diagram, we have the RabbitMQ producer present in API containers that queue the requests to the RabbitMQ cluster. The RabbitMQ consumers is also configured with an instance of Fabric-Node-SDK in Task execution containers to consume the requests from users and send it blockchain network for execution.
- Creating and Starting the Blockchain Application
- Performing transactions on Blockchain Network
- Viewing results and transaction blocks created in blockchain network
- Create your first Blockchain network using Hyperledger Composer
- Integrate your Blockchain with anything using Hyperledger Composer and NodeRed
- Hyperledger Fabric
- Accelerating time to value
- Create a To-do list using Hyperledger Fabric V1.0
- Deploy an Asset Transfer App on Hyperledger Fabric V1.0
- Decentralized energy with Hyperledger Composer
Imagine you have built a Blockchain application that you anticipate over 40K people will use. Naturally, you will think about performance testing (stay tuned on that topic for the next pattern in this composite series). But even better – what can you do to proactively scale and successfully accommodate these large numbers of expected transactions. Let’s think about the bottleneck considerations. Take a real use case that occurred recently with an application called “The Secret Map”. In this scenario, there was the potential for over 40K attendees at a conference to want to sign up for an IoS application that would track their steps at the conference and issue a “fitcoin” after every 100 steps. The carrot that stood in front of them was “cash X amount of fitcoins” in for really cool prizes such as a bandana and long sleeve conference t-shirts. This was all managed on the Blockchain network! Who wouldn’t jump at that? Well we had to think quickly – how could we control the speed of large numbers of transaction request blocks being written on the blockchain network. Lightbulb time! Let’s take a look at Redis and RabbitMQ. Redis is an open source in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, and lists. RabbitMQ is a lightweight open source message broker. Using these two tools helped to scale your Blockchain application. RabbitMQ controls the input by ordering user requests while Redis controls the output by holding Blockchain transaction results; storing them for a short duration. This partnership of technologies is able to space out the requests over time; hence managing the load that the Blockchain application had to handle at any one time. Check out the pattern to see the code that implements this use case. It may help you manage your next Blockchain application that requires supporting thousands or even millions of transactions.