Skip to content

Commit

Permalink
Update Readme and server run scripts. Add ability to change server se…
Browse files Browse the repository at this point in the history
…ttings by execution flags
  • Loading branch information
valinurovam committed Oct 18, 2018
1 parent fa619f1 commit ef1df87
Show file tree
Hide file tree
Showing 217 changed files with 54,158 additions and 115 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
cmd/server
bin/garagemq
db/
node_modules
.idea
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ RUN GOOS=linux GOARCH=amd64 make build

EXPOSE 5672 15672

CMD ["cmd/server"]
CMD ["bin/garagemq"]
89 changes: 88 additions & 1 deletion Gopkg.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@ deps:
dep ensure && cd admin-frontend && yarn install

build.all: deps
go build -o cmd/server cmd/server.go && cd admin-frontend && yarn build
go build -o bin/garagemq main.go && cd admin-frontend && yarn build

build:
go build -o cmd/server cmd/server.go
go build -o bin/garagemq main.go

run:
go build -o cmd/server cmd/server.go && cmd/server
run: build
bin/garagemq

vet:
go vet github.com/valinurovam/garagemq...
Expand Down
154 changes: 101 additions & 53 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,54 +1,131 @@
# GarageMQ [![Build Status](https://travis-ci.org/valinurovam/garagemq.svg?branch=master)](https://travis-ci.org/valinurovam/garagemq) [![Coverage Status](https://coveralls.io/repos/github/valinurovam/garagemq/badge.svg)](https://coveralls.io/github/valinurovam/garagemq) [![Go Report Card](https://goreportcard.com/badge/github.com/valinurovam/garagemq)](https://goreportcard.com/report/github.com/valinurovam/garagemq)

AMQP-server implemented with #golang

## Goals of this project:
GarageMQ is a message broker that implement the Advanced Message Queuing Protocol (AMQP). Compatible with any AMQP or RabbitMQ clients (tested streadway/amqp and php-amqp lib)

#### Table of Contents
- [Goals of this project](#goals-of-this-project:)
- [Installation and Building](#installation-and-building)
- [Docker](#docker)
- [Go get](#go-get)
- [Execution flags](#execution-flags)
- [Default config params](#default-config-params)
- [Performance tests](#performance-tests)
- [Persistent messages](#persistent-messages)
- [Transient messages](#transient-messages)
- [Internals](#internals)
- [Backend for durable entities](#backend-for-durable-entities)
- [QOS](#qos)
- [Admin server](#admin-server)
- [TODO](#todo)
- [Contribution](#contribution)

#### Goals of this project:

- Have fun and learn a lon
- Implement RabbitMQ in Go
- Protocol compatible with RabbitMQ and standard AMQP 0-9-1.

## Compatibilities
GarageMQ compatible with any AMQP or RabbitMQ clients (tested streadway/amqp and php-amqp lib)
- Implement AMQP message broker in Go
- Make protocol compatible with RabbitMQ and standard AMQP 0-9-1.

## Building
## Installation and Building
### Docker

The quick way to start GarageMQ is by using `docker` as shown below.
The quick way to start with GarageMQ is by using `docker`. You can build it by your own or pull from docker-hub
```shell
docker pull amplitudo/garagemq
docker run --name garagemq -p 5672:5672 -p 15672:15672 amplitudo/garagemq
```
or

```shell
go get -u github.com/valinurovam/garagemq/...
cd $GOPATH/src/github.com/valinurovam/garagemq
docker build -t garagemq .
docker run -d --name garagemq -p 5672:5672 garagemq
docker run --name garagemq -p 5672:5672 -p 15672:15672 garagemq

```
... or ...
### Go get
You can also use [go get](https://golang.org/cmd/go/#hdr-Download_and_install_packages_and_dependencies): ```go get -u github.com/valinurovam/garagemq/...```
```shell
go get -u github.com/valinurovam/garagemq/...
cd $GOPATH/src/github.com/valinurovam/garagemq
make run
make build.all && make run
```
### Execution flags
| Flag | Default | Description | ENV |
| :--- | :--- | :--- | :--- |
| --config | [default config](#default-config-params) | Config path | GMQ_CONFIG |
| --log-file | stdout | Log file path or `stdout`, `stderr`| GMQ_LOG_FILE |
| --log-level | info | Logger level | GMQ_LOG_LEVEL |
| --hprof | false | Enable or disable [hprof profiler](https://golang.org/pkg/net/http/pprof/#pkg-overview) | GMQ_HPROF |
| --hprof-host | 0.0.0.0 | Profiler host | GMQ_HPROF_HOST |
| --hprof-port | 8080 | Profiler port | GMQ_HPROF_PORT |

### Default config params
```yaml
# Proto name to implement (amqp-rabbit or amqp-0-9-1)
proto: amqp-rabbit
# User list
users:
- username: guest
password: 084e0343a0486ff05530df6c705c8bb4 # guest md5
# Server TCP settings
tcp:
ip: 0.0.0.0
port: 5672
nodelay: false
readBufSize: 196608
writeBufSize: 196608
# Admin-server settings
admin:
ip: 0.0.0.0
port: 15672
queue:
shardSize: 8192
maxMessagesInRam: 131072
# DB settings
db:
# default path
defaultPath: db
# backend engine (badger or buntdb)
engine: badger
# Default virtual host path
vhost:
defaultPath: /
# Security check rule (md5 or bcrypt)
security:
passwordCheck: md5
connection:
channelsMax: 4096
frameMaxSize: 65536
```

## Performance
## Performance tests

Performance tests with load testing tool https://github.com/rabbitmq/rabbitmq-perf-test on machine:
Performance tests with load testing tool https://github.com/rabbitmq/rabbitmq-perf-test on test-machine:
```
MacBook Pro (15-inch, 2016)
Processor 2,6 GHz Intel Core i7
Memory 16 GB 2133 MHz LPDDR3
```

### Persistent messages
```shell
./bin/runjava com.rabbitmq.perf.PerfTest --exchange test -uri amqp://guest:guest@localhost:5672 --queue test --consumers 10 --producers 5 --qos 100 -flag persistent
...
...
id: test-145413-796, sending rate avg: 75123 msg/s
id: test-145413-796, receiving rate avg: 70337 msg/s
id: test-235131-686, sending rate avg: 53577 msg/s
id: test-235131-686, receiving rate avg: 51941 msg/s
```
### Transient messages
```shell
./bin/runjava com.rabbitmq.perf.PerfTest --exchange test -uri amqp://guest:guest@localhost:5672 --queue test --consumers 10 --producers 5 --qos 100
...
...
id: test-235231-085, sending rate avg: 71247 msg/s
id: test-235231-085, receiving rate avg: 69009 msg/s
```

## Internals

### Backend for durable entities and persistent messages
### Backend for durable entities
Database backend is changeable through config `db.engine`
```
db:
Expand All @@ -63,37 +140,6 @@ db:
- Badger https://github.com/dgraph-io/badger
- BuntDB https://github.com/tidwall/buntdb



### Implemented all AMQP methods, except:

#### connection

- [ ] connection.secure
- [ ] connection.secureOk

#### exchange

- [ ] exchange.delete
- [ ] exchange.deleteOk
- [ ] exchange.bind
- [ ] exchange.unbind

#### basic

- [ ] basic.recoverAsync
- [ ] basic.recover
- [ ] basic.recoverOk

#### tx

- [ ] tx.select
- [ ] tx.selectOk
- [ ] tx.commit
- [ ] tx.commitOk
- [ ] tx.rollback
- [ ] tx.rollbackOk

### QOS

`basic.qos` method implemented for standard AMQP and RabbitMQ mode. It means that by default qos applies for connection(global=true) or channel(global=false).
Expand All @@ -106,8 +152,10 @@ The administration server is available at standard `:15672` port and is `read on
![Overview](readme/overview.jpg)

## TODO
- [ ] Add more comments
- [ ] Optimize binds
- [ ] Replication
- [ ] Replication and clusterization
- [ ] Own backend for durable entities and persistent messages
- [ ] Migrate to message reference counting
- [ ] Migrate to message reference counting

## Contribution
Contribution of any kind is always welcome and appreciated. Contribution Guidelines in WIP
4 changes: 2 additions & 2 deletions admin/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ type AdminServer struct {
s *http.Server
}

func NewAdminServer(amqpServer *server.Server) *AdminServer {
func NewAdminServer(amqpServer *server.Server, host string, port string) *AdminServer {
http.Handle("/", http.FileServer(http.Dir("admin-frontend/build")))
http.Handle("/overview", NewOverviewHandler(amqpServer))
http.Handle("/exchanges", NewExchangesHandler(amqpServer))
Expand All @@ -22,7 +22,7 @@ func NewAdminServer(amqpServer *server.Server) *AdminServer {

adminServer := &AdminServer{}
adminServer.s = &http.Server{
Addr: fmt.Sprintf(":%d", 15672),
Addr: fmt.Sprintf("%s:%s", host, port),
}

return adminServer
Expand Down
5 changes: 5 additions & 0 deletions bin/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
cmd/server
db/
node_modules
.idea
.DS_Store

0 comments on commit ef1df87

Please sign in to comment.