Skip to content

Commit

Permalink
Add some docs
Browse files Browse the repository at this point in the history
  • Loading branch information
@buraksezer
buraksezer committed Jul 21, 2020
1 parent fa8091c commit cc24a30
Show file tree
Hide file tree
Showing 6 changed files with 174 additions and 51 deletions.
101 changes: 61 additions & 40 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,13 @@ See [Docker](#docker) and [Sample Code](#sample-code) sections to get started!
* Supports atomic operations,
* Supports [distributed queries](#query) on keys,
* Provides a plugin interface for service discovery daemons,
* Provides a locking primitive which inspired by [SETNX of Redis](https://redis.io/commands/setnx#design-pattern-locking-with-codesetnxcode).
* Provides a locking primitive which inspired by [SETNX of Redis](https://redis.io/commands/setnx#design-pattern-locking-with-codesetnxcode),
* Support [distributed topic](#distributed-topic) data structure,

## Possible Use Cases

With this feature set, Olric is suitable to use as a distributed cache. But it also provides data replication, failure detection
and simple anti-entropy services. So it can be used as an ordinary key/value data store to scale your cloud application.

## Project Status

Olric is in early stages of development. The package API and client protocol may change without notification.
With this feature set, Olric is suitable to use as a distributed cache. But it also provides distributed topics, data replication,
failure detection and simple anti-entropy services. So it can be used as an ordinary key/value data store to scale your cloud application.

## Table of Contents

Expand All @@ -49,28 +46,34 @@ Olric is in early stages of development. The package API and client protocol may
* [olric-stats](#olric-stats)
* [olric-load](#olric-load)
* [Usage](#usage)
* [Put](#put)
* [PutIf](#putif)
* [PutEx](#putex)
* [PutIfEx](#putifex)
* [Get](#get)
* [Expire](#expire)
* [Delete](#delete)
* [LockWithTimeout](#lockwithtimeout)
* [Lock](#lock)
* [Unlock](#unlock)
* [Destroy](#destroy)
* [Stats](#stats)
* [Ping](#ping)
* [Query](#query)
* [Cursor](#cursor)
* [Range](#range)
* [Close](#close)
* [Atomic Operations](#atomic-operations)
* [Incr](#incr)
* [Decr](#decr)
* [GetPut](#getput)
* [Pipelining](#pipelining)
* [Distributed Map](#distributed-map)
* [Put](#put)
* [PutIf](#putif)
* [PutEx](#putex)
* [PutIfEx](#putifex)
* [Get](#get)
* [Expire](#expire)
* [Delete](#delete)
* [LockWithTimeout](#lockwithtimeout)
* [Lock](#lock)
* [Unlock](#unlock)
* [Destroy](#destroy)
* [Stats](#stats)
* [Ping](#ping)
* [Query](#query)
* [Cursor](#cursor)
* [Range](#range)
* [Close](#close)
* [Atomic Operations](#atomic-operations)
* [Incr](#incr)
* [Decr](#decr)
* [GetPut](#getput)
* [Pipelining](#pipelining)
* [Distributed Topic](#distributed-topic)
* [Publish](#publish)
* [AddListener](#addlistener)
* [RemoveListener](#removelistener)
* [Destroy](#destroy)
* [Serialization](#serialization)
* [Golang Client](#golang-client)
* [Configuration](#configuration)
Expand Down Expand Up @@ -121,7 +124,8 @@ Olric is in early stages of development. The package API and client protocol may
* Provides a plugin interface for service discovery daemons and cloud providers,
* Provides a command-line-interface to access the cluster directly from the terminal,
* Supports different serialization formats. Gob, JSON and MessagePack are supported out of the box,
* Provides a locking primitive which inspired by [SETNX of Redis](https://redis.io/commands/setnx#design-pattern-locking-with-codesetnxcode).
* Provides a locking primitive which inspired by [SETNX of Redis](https://redis.io/commands/setnx#design-pattern-locking-with-codesetnxcode),
* Support [distributed topic](#distributed-topic) data structure,

See [Architecture](#architecture) section to see details.

Expand All @@ -130,7 +134,6 @@ See [Architecture](#architecture) section to see details.
* Distributed queries over keys and values,
* Database backend for persistence,
* Anti-entropy system to repair inconsistencies in DMaps,
* Publish/Subscribe for messaging,
* Eviction listeners by using Publish/Subscribe,
* Memcached interface,
* Client implementations for different languages: Java, Python and JavaScript,
Expand Down Expand Up @@ -428,15 +431,6 @@ When you call **Start** method, your process joins the cluster and will be respo
indefinitely. So you may need to run it in a goroutine. Of course, this is just a single-node instance, because you didn't give any
configuration.

Create a **DMap** object to access the cluster:

```go
dm, err := db.NewDMap("my-dmap")
```

DMap object has *Put*, *PutEx*, *PutIf*, *PutIfEx*, *Get*, *Delete*, *Expire*, *LockWithTimeout* and *Destroy* methods to access
and modify data in Olric. We may add more methods for finer control but first, I'm willing to stabilize this set of features.

When you want to leave the cluster, just need to call **Shutdown** method:

```go
Expand All @@ -447,6 +441,14 @@ This will stop background tasks and servers. Finally purges in-memory data and q

***Please note that this section aims to document DMap API in embedded member mode.*** If you prefer to use Olric in
Client-Server mode, please jump to [Golang Client](#golang-client) section.

### Distributed Map

Create a **DMap** instance:

```go
dm, err := db.NewDMap("my-dmap")
```

### Put

Expand Down Expand Up @@ -797,6 +799,25 @@ There is no hard-limit on message count in a pipeline. You should set a convenie

The `Flush` method returns errors along with success messages. Furthermore, you need to know the command order for matching responses with requests.

### Distributed Topic

Distributed topic implementation provides a very scalable way to distribute messages across clients and cluster members. Currently, it only supports
unordered delivery mode.

Create a **DTopic** instance

```go
dt, err := db.NewDTopic("my-topic", olric.UnorderedDelivery)
```

### Publish

```go
err := dt.Publish("my-message")
```

Publish accepts any serializable type as message.

## Golang Client

This repo contains the official Golang client for Olric. It implements Olric Binary Protocol(OBP). With this client,
Expand Down
38 changes: 30 additions & 8 deletions internal/protocol/dmap.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,13 @@ import (
"fmt"
)

// DMapMessageHeaderSize defines total count of bytes in a DMapMessage
const DMapMessageHeaderSize uint32 = 7

const (
// MagicDMapReq is a magic number which denotes DMap message requests on the wire.
MagicDMapReq MagicCode = 0xE2
// MagicDMapRes is a magic number which denotes DMap message response on the wire.
MagicDMapRes MagicCode = 0xE3
)

Expand All @@ -36,16 +39,18 @@ type DMapMessageHeader struct {
StatusCode StatusCode // 1
}

// DMapMessage is a message type in OBP. It can be used to access and modify DMap data structure.
type DMapMessage struct {
Header // [0-4]
DMapMessageHeader // [1..10]
extra interface{} // [11..(m-1)] Command specific extras (In)
dmap string // [m..(n-1)] dmap (as needed, length in Header)
key string // [n..(x-1)] key (as needed, length in Header)
value []byte // [x..y] value (as needed, length in Header)
buf *bytes.Buffer
Header
DMapMessageHeader
extra interface{}
dmap string
key string
value []byte
buf *bytes.Buffer
}

// NewDMapMessage returns a new DMapMessage with the given operation code.
func NewDMapMessage(opcode OpCode) *DMapMessage {
return &DMapMessage{
Header: Header{
Expand All @@ -58,6 +63,8 @@ func NewDMapMessage(opcode OpCode) *DMapMessage {
}
}

// NewDMapMessageFromRequest returns a new DMapMessage for the given bytes.Buffer. The caller can use
// Decode method to read message from the raw data.
func NewDMapMessageFromRequest(buf *bytes.Buffer) *DMapMessage {
return &DMapMessage{
Header: Header{
Expand All @@ -70,6 +77,7 @@ func NewDMapMessageFromRequest(buf *bytes.Buffer) *DMapMessage {
}
}

// Response generates a response message for the request. This is a shortcut function to reduce boilerplate code.
func (d *DMapMessage) Response(buf *bytes.Buffer) EncodeDecoder {
msg := &DMapMessage{
Header: Header{
Expand All @@ -90,59 +98,72 @@ func (d *DMapMessage) Response(buf *bytes.Buffer) EncodeDecoder {
return msg
}

// SetStatus sets a status code for the message.
func (d *DMapMessage) SetStatus(code StatusCode) {
d.StatusCode = code
}

// Status returns status code.
func (d *DMapMessage) Status() StatusCode {
return d.StatusCode
}

// SetValue writes the given byte slice into the underlying bytes.Buffer
func (d *DMapMessage) SetValue(value []byte) {
d.value = value
}

// Value returns the value
func (d *DMapMessage) Value() []byte {
return d.value
}

// OpCode returns operation code of the message
func (d *DMapMessage) OpCode() OpCode {
return d.Op
}

// SetBuffer sets the underlying bytes.Buffer. It should be recycled by the caller.
func (d *DMapMessage) SetBuffer(buf *bytes.Buffer) {
d.buf = buf
}

// Buffer returns the underlying bytes.Buffer
func (d *DMapMessage) Buffer() *bytes.Buffer {
return d.buf
}

// SetDMap sets the DMap name for this message.
func (d *DMapMessage) SetDMap(dmap string) {
d.dmap = dmap
}

// Returns the DMap name.
func (d *DMapMessage) DMap() string {
return d.dmap
}

// SetKey sets the key for this DMap message.
func (d *DMapMessage) SetKey(key string) {
d.key = key
}

// Key returns the key for this DMap message.
func (d *DMapMessage) Key() string {
return d.key
}

// SetExtra sets the extra section for the message, if there is any.
func (d *DMapMessage) SetExtra(extra interface{}) {
d.extra = extra
}

// Extra returns the extra section of the message, if there is any.
func (d *DMapMessage) Extra() interface{} {
return d.extra
}

// Encode writes a protocol message to given TCP connection by encoding it.
// Encode encodes the message into byte form.
func (d *DMapMessage) Encode() error {
// Calculate lengths here
d.DMapLen = uint16(len(d.dmap))
Expand Down Expand Up @@ -183,6 +204,7 @@ func (d *DMapMessage) Encode() error {
return err
}

// Decode decodes message from byte form into DMapMessage.
func (d *DMapMessage) Decode() error {
err := binary.Read(d.buf, binary.BigEndian, &d.DMapMessageHeader)
if err != nil {
Expand Down

0 comments on commit cc24a30

Please sign in to comment.