MiniQ

Just trying to write an http message queue in less than 1000 lines.

Inspired by minikeyvalue by @geohot

Features

Security

MiniQ takes a secret string as an argument on start up. This secret should be provided in Authorization request headers when interacting with the queue to ensure access. We recommend using MiniQ behind an API gateway or custom API, as your security token may be vulnerable to man in the middle attacks otherwise. A user could obtain your key by intercepting your communications with the Queue from your front end.

Read guarantee

MiniQ requires users to hit the /confirm endpoint once a message has been consumed so that is can be removed from the Queue. This is to ensure all messages are successfully consumed.

Fast and lightweight.

Because MiniQ is less than 1000 lines and written in Go, it is fast and incredibly lightweight. We take advantage of the features of LevelDB to support this.

Persistent

MiniQ uses LevelDB as its datastore. This means that even if the queue crashes or the host machine unexpectedly shuts down, all messages in the queue are preserved and can be picked up once the error has been resolved.

Minimal complexity

This project was written to be easy to use and understand. Removing a lot of the complexity and unnecessary features of larger message queues and giving users only what they need for asynchronous messaging.

Usage

Docker

Using docker commands

First, open the dockerfile:

Replace the secret highlighted in the screenshot above with a strong, secure string of your choice. If you would like to use the queue without consumption guarantees (not recommended) then add -c to the CMD line in the dockerfile like this:

In a shell of your preference:

• Run: Docker build -t miniq .
• Run: Docker run -p :8080 -d miniq
• Example
  • 
  • 

Docker Compose

First, open the dockerfile:

Replace the secret highlighted in the screenshot above with a strong, secure string of your choice. If you would like to use the queue without consumption guarantees (not recommended) then add -c to the CMD line in the dockerfile like this: Run docker-compose up in your terminal from the project root. If you would like to run it detatched (no active terminal output, best when running unsupervised or on a server) run docker-compose up -d The queue will be available on http://localhost:9000, when running on a server route requests here using something like NGINX or Apache Tomcat

Once MiniQ is running here are the steps for consumption and publishing of messages:

1: To push a message to the Queue

• Make a Post request to "queueurl:port/message" with your message in the request body and the Authorization header set to the secret which you configured.
  

2: To read a message

• Make a get request to "queueurl:port/message" with the Authorization header set to the secret which you configured.
  

3: After consuming a message

• Make a post request to "queueurl:port/confirm" with the Authorization header set to the secret which you configured and send the message key (returned in the request to read the message) to confirm that the message has been consumed and can be removed from the queue.
  

Todo

• Create topics? Or some kind of queue separation
• Timestamp queue messages
• Distribute and replicate queue for higher fault tolerance
• add functionality to read entire queue at once.