Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[docs] improve organization of ABCI docs & fix links #2749

Merged
merged 8 commits into from
Nov 9, 2018
Merged

[docs] improve organization of ABCI docs & fix links #2749

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
0a0fbd3
dedup with spec/abci/client-server
zramsay Oct 6, 2018
0de4984
Merge pull request #12 from zramsay/zach/abci-improvements
zramsay Nov 1, 2018
cc54d1f
fixup abci/readme
zramsay Nov 2, 2018
a87d044
Merge pull request #13 from zramsay/abci/readme
zramsay Nov 2, 2018
91d924b
link to getting started in abci/README
zramsay Nov 2, 2018
af2a7c0
https
zramsay Nov 2, 2018
3177087
spec/abci: some deduplication
zramsay Nov 6, 2018
596a5e7
docs: remove extraneous comment
zramsay Nov 9, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
156 changes: 11 additions & 145 deletions abci/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Application BlockChain Interface (ABCI)

[![CircleCI](https://circleci.com/gh/tendermint/abci.svg?style=svg)](https://circleci.com/gh/tendermint/abci)

Blockchains are systems for multi-master state machine replication.
**ABCI** is an interface that defines the boundary between the replication engine (the blockchain),
and the state machine (the application).
Expand All @@ -12,160 +10,28 @@ Previously, the ABCI was referred to as TMSP.

The community has provided a number of addtional implementations, see the [Tendermint Ecosystem](https://tendermint.com/ecosystem)

## Specification

A detailed description of the ABCI methods and message types is contained in:
## Installation & Usage

- [A prose specification](specification.md)
- [A protobuf file](https://github.com/tendermint/tendermint/blob/master/abci/types/types.proto)
- [A Go interface](https://github.com/tendermint/tendermint/blob/master/abci/types/application.go).
To get up and running quickly, see the [getting started guide](../docs/app-dev/getting-started.md) along with the [abci-cli documentation](../docs/app-dev/abci-cli.md) which will go through the examples found in the [examples](./example/) directory.

For more background information on ABCI, motivations, and tendermint, please visit [the documentation](https://tendermint.com/docs/).
The two guides to focus on are the `Application Development Guide` and `Using ABCI-CLI`.
## Specification

A detailed description of the ABCI methods and message types is contained in:

- [The main spec](../docs/spec/abci/abci.md)
- [A protobuf file](./types/types.proto)
- [A Go interface](./types/application.go)

## Protocol Buffers

To compile the protobuf file, run:
To compile the protobuf file, run (from the root of the repo):

```
cd $GOPATH/src/github.com/tendermint/tendermint/; make protoc_abci
make protoc_abci
```

See `protoc --help` and [the Protocol Buffers site](https://developers.google.com/protocol-buffers)
for details on compiling for other languages. Note we also include a [GRPC](http://www.grpc.io/docs)
for details on compiling for other languages. Note we also include a [GRPC](https://www.grpc.io/docs)
service definition.

## Install ABCI-CLI

The `abci-cli` is a simple tool for debugging ABCI servers and running some
example apps. To install it:

```
mkdir -p $GOPATH/src/github.com/tendermint
cd $GOPATH/src/github.com/tendermint
git clone https://github.com/tendermint/tendermint.git
cd tendermint
make get_tools
make get_vendor_deps
make install_abci
```

## Implementation

We provide three implementations of the ABCI in Go:

- Golang in-process
- ABCI-socket
- GRPC

Note the GRPC version is maintained primarily to simplify onboarding and prototyping and is not receiving the same
attention to security and performance as the others

### In Process

The simplest implementation just uses function calls within Go.
This means ABCI applications written in Golang can be compiled with TendermintCore and run as a single binary.

See the [examples](#examples) below for more information.

### Socket (TSP)

ABCI is best implemented as a streaming protocol.
The socket implementation provides for asynchronous, ordered message passing over unix or tcp.
Messages are serialized using Protobuf3 and length-prefixed with a [signed Varint](https://developers.google.com/protocol-buffers/docs/encoding?csw=1#signed-integers)

For example, if the Protobuf3 encoded ABCI message is `0xDEADBEEF` (4 bytes), the length-prefixed message is `0x08DEADBEEF`, since `0x08` is the signed varint
encoding of `4`. If the Protobuf3 encoded ABCI message is 65535 bytes long, the length-prefixed message would be like `0xFEFF07...`.

Note the benefit of using this `varint` encoding over the old version (where integers were encoded as `<len of len><big endian len>` is that
it is the standard way to encode integers in Protobuf. It is also generally shorter.

### GRPC

GRPC is an rpc framework native to Protocol Buffers with support in many languages.
Implementing the ABCI using GRPC can allow for faster prototyping, but is expected to be much slower than
the ordered, asynchronous socket protocol. The implementation has also not received as much testing or review.

Note the length-prefixing used in the socket implementation does not apply for GRPC.

## Usage

The `abci-cli` tool wraps an ABCI client and can be used for probing/testing an ABCI server.
For instance, `abci-cli test` will run a test sequence against a listening server running the Counter application (see below).
It can also be used to run some example applications.
See [the documentation](https://tendermint.com/docs/) for more details.

### Examples

Check out the variety of example applications in the [example directory](example/).
It also contains the code refered to by the `counter` and `kvstore` apps; these apps come
built into the `abci-cli` binary.

#### Counter

The `abci-cli counter` application illustrates nonce checking in transactions. It's code looks like:

```golang
func cmdCounter(cmd *cobra.Command, args []string) error {

app := counter.NewCounterApplication(flagSerial)

logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout))

// Start the listener
srv, err := server.NewServer(flagAddrC, flagAbci, app)
if err != nil {
return err
}
srv.SetLogger(logger.With("module", "abci-server"))
if err := srv.Start(); err != nil {
return err
}

// Wait forever
cmn.TrapSignal(func() {
// Cleanup
srv.Stop()
})
return nil
}
```

and can be found in [this file](cmd/abci-cli/abci-cli.go).

#### kvstore

The `abci-cli kvstore` application, which illustrates a simple key-value Merkle tree

```golang
func cmdKVStore(cmd *cobra.Command, args []string) error {
logger := log.NewTMLogger(log.NewSyncWriter(os.Stdout))

// Create the application - in memory or persisted to disk
var app types.Application
if flagPersist == "" {
app = kvstore.NewKVStoreApplication()
} else {
app = kvstore.NewPersistentKVStoreApplication(flagPersist)
app.(*kvstore.PersistentKVStoreApplication).SetLogger(logger.With("module", "kvstore"))
}

// Start the listener
srv, err := server.NewServer(flagAddrD, flagAbci, app)
if err != nil {
return err
}
srv.SetLogger(logger.With("module", "abci-server"))
if err := srv.Start(); err != nil {
return err
}

// Wait forever
cmn.TrapSignal(func() {
// Cleanup
srv.Stop()
})
return nil
}
```
84 changes: 0 additions & 84 deletions docs/app-dev/app-development.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,90 +47,6 @@ The mempool and consensus logic act as clients, and each maintains an
open ABCI connection with the application, which hosts an ABCI server.
Shown are the request and response types sent on each connection.

## Message Protocol

The message protocol consists of pairs of requests and responses. Some
messages have no fields, while others may include byte-arrays, strings,
or integers. See the `message Request` and `message Response`
definitions in [the protobuf definition
file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.proto),
and the [protobuf
documentation](https://developers.google.com/protocol-buffers/docs/overview)
for more details.

For each request, a server should respond with the corresponding
response, where order of requests is preserved in the order of
responses.

## Server

To use ABCI in your programming language of choice, there must be a ABCI
server in that language. Tendermint supports two kinds of implementation
of the server:

- Asynchronous, raw socket server (Tendermint Socket Protocol, also
known as TSP or Teaspoon)
- GRPC

Both can be tested using the `abci-cli` by setting the `--abci` flag
appropriately (ie. to `socket` or `grpc`).

See examples, in various stages of maintenance, in
[Go](https://github.com/tendermint/tendermint/tree/develop/abci/server),
[JavaScript](https://github.com/tendermint/js-abci),
[Python](https://github.com/tendermint/tendermint/tree/develop/abci/example/python3/abci),
[C++](https://github.com/mdyring/cpp-tmsp), and
[Java](https://github.com/jTendermint/jabci).

### GRPC

If GRPC is available in your language, this is the easiest approach,
though it will have significant performance overhead.

To get started with GRPC, copy in the [protobuf
file](https://github.com/tendermint/tendermint/blob/develop/abci/types/types.proto)
and compile it using the GRPC plugin for your language. For instance,
for golang, the command is `protoc --go_out=plugins=grpc:. types.proto`.
See the [grpc documentation for more details](http://www.grpc.io/docs/).
`protoc` will autogenerate all the necessary code for ABCI client and
server in your language, including whatever interface your application
must satisfy to be used by the ABCI server for handling requests.

### TSP

If GRPC is not available in your language, or you require higher
performance, or otherwise enjoy programming, you may implement your own
ABCI server using the Tendermint Socket Protocol, known affectionately
as Teaspoon. The first step is still to auto-generate the relevant data
types and codec in your language using `protoc`. Messages coming over
the socket are proto3 encoded, but additionally length-prefixed to
facilitate use as a streaming protocol. proto3 doesn't have an
official length-prefix standard, so we use our own. The first byte in
the prefix represents the length of the Big Endian encoded length. The
remaining bytes in the prefix are the Big Endian encoded length.

For example, if the proto3 encoded ABCI message is 0xDEADBEEF (4
bytes), the length-prefixed message is 0x0104DEADBEEF. If the proto3
encoded ABCI message is 65535 bytes long, the length-prefixed message
would be like 0x02FFFF....

Note this prefixing does not apply for grpc.

An ABCI server must also be able to support multiple connections, as
Tendermint uses three connections.

## Client

There are currently two use-cases for an ABCI client. One is a testing
tool, as in the `abci-cli`, which allows ABCI requests to be sent via
command line. The other is a consensus engine, such as Tendermint Core,
which makes requests to the application every time a new transaction is
received or a block is committed.

It is unlikely that you will need to implement a client. For details of
our client, see
[here](https://github.com/tendermint/tendermint/tree/develop/abci/client).

Most of the examples below are from [kvstore
application](https://github.com/tendermint/tendermint/blob/develop/abci/example/kvstore/kvstore.go),
which is a part of the abci repo. [persistent_kvstore
Expand Down
35 changes: 35 additions & 0 deletions docs/spec/abci/implementations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# ABCI Implementations
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great but seems its still largely a duplicate of https://github.com/tendermint/tendermint/blob/develop/docs/spec/abci/client-server.md

Can we just make sure that file contains anything from here it might be missing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed


We provide three implementations of the ABCI in written in Go:

- In-process (Golang)
- ABCI-socket
- GRPC

Note the GRPC version is maintained primarily to simplify onboarding and prototyping and is not receiving the same
attention to security and performance as the others

## In Process

The simplest implementation just uses function calls within Go.
This means ABCI applications written in Golang can be compiled with TendermintCore and run as a single binary.

## Socket (TSP)

ABCI is best implemented as a streaming protocol.
The socket implementation provides for asynchronous, ordered message passing over unix or tcp.
Messages are serialized using Protobuf3 and length-prefixed with a [signed Varint](https://developers.google.com/protocol-buffers/docs/encoding?csw=1#signed-integers)

For example, if the Protobuf3 encoded ABCI message is `0xDEADBEEF` (4 bytes), the length-prefixed message is `0x08DEADBEEF`, since `0x08` is the signed varint
encoding of `4`. If the Protobuf3 encoded ABCI message is 65535 bytes long, the length-prefixed message would be like `0xFEFF07...`.

Note the benefit of using this `varint` encoding over the old version (where integers were encoded as `<len of len><big endian len>` is that
it is the standard way to encode integers in Protobuf. It is also generally shorter.

## GRPC

GRPC is an rpc framework native to Protocol Buffers with support in many languages.
Implementing the ABCI using GRPC can allow for faster prototyping, but is expected to be much slower than
the ordered, asynchronous socket protocol. The implementation has also not received as much testing or review.

Note the length-prefixing used in the socket implementation does not apply for GRPC.