Skip to content
Æterrnal--the caching and reporting layer for the æternity blockchain
Rust Vue JavaScript PLpgSQL TSQL Shell Other
Branch: develop
Clone or download
Pull request Compare This branch is 931 commits ahead of jsnewby:develop.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github add issue template Sep 23, 2019
conf Moved conf to example Nov 8, 2019
docker
docs Merge branch 'develop' into hotfix/rename_issue Nov 21, 2019
examples
frontend lazy load contract tx and calls Dec 4, 2019
migrations
scripts
src Removed associated accounts, which never worked Dec 19, 2019
swagger Updated swagger def Dec 17, 2019
tests Started new testing framework Dec 12, 2019
.dockerignore build(docker): add migrations folder to the build context Jun 7, 2019
.env.example
.gitignore Removed chashmap, which was causing errors Nov 13, 2019
CHANGELOG.md
Cargo.lock
Cargo.toml Updated version number to 0.13.0 Dec 17, 2019
LICENSE.txt
Makefile
README.md feat(docker): improve docker-compose experience (#253) Dec 20, 2019
Rocket.example.toml moved Rocket.toml to Rocket.example.tonl for better deploy Jan 5, 2019
Rocket.toml Release/0.9.0 (#134) Aug 13, 2019
diesel.toml bar Jul 4, 2018
docker-compose.yml feat(docker): improve docker-compose experience (#253) Dec 20, 2019

README.md

æternal

Overview

The middleware is a caching and reporting layer which sits in front of the nodesof the aeternity blockchain. Its purpose is to respond to queries faster than the node can do, and to support queries that for reasons of efficiency the node cannot or will not support itself.

On startup, the middleware reads the entirety of the blockchain, and stores a denormalized version of the data in a PostgreSQL database.

Full documentation is available in the manual.

Features

Caching

The middleware answers a set of queries (listed below) on behalf of the node. Everything it doesn't itself answer it will forward to the node to handle. The goal here is to answer more quickly than the node, without sacrificing correctness.

Aggregation

A set of queries will list, for example, all transactions for a given account, all which transfer from one user to another, all between certain block heights, and so on

Contracts

The middleware unpacks and stores all contract calls for which it has the bytecode, with the function called and the arguments. In the near future we will also store the return type and value. You can see all contract calls for a given contract.

State channels

You can see all active state channels, and for a given state channel, all of the related on-chain transactions

Names

You can see all names registered, and also which names refer to a given address (reverse lookup).

Oracles

You can see all active oracles. In the near future we will create an endpoint which lists all questions and responses from an oracle

Websocket

(see below for more information).

The middleware permits you to subscribe to events via a websocket, and receive updates when the chain state changes. Currently you can subscribe to key blocks, micro blocks, and soon to all events involving a particular on-chain object (contract, account, ...).

How to use the middleware

Use ours!

There is a hosted middleware for the æternity mainnet at https://mainnet.aeternal.io/, and one for the testnet at https://testnet.aeternal.io.

Install your own

  • Install a postgresql DB somewhere. Version 11.2 or greater are supported.
  • as the postgresql admin user, execute scripts/prepare-db.sql which will create the DB and user
  • copy 'Rocket.example.toml' to 'Rocket.toml' and edit as you see fit
  • copy .env.example to .env
  • if you want to use a different DB name, edit scripts/prepare-db.sql, .env and Rocket.toml

Run via Docker

This setup runs node, compiler, postgres and aeternal together.

In the examples folder there are 2 setup for docker:

  • examples/docker-compose.yml that includes also the node configuration
  • examples/docker-compose.existing-node.yaml that relies on an existing node

Run docker including the node

  • Install Docker and Docker Compose.
  • Update examples/node/config.yaml as per your requirement.
  • From the project examples folders run docker-compose up

Run docker including relying on an existing node

  • Install Docker and Docker Compose
  • Make sure you can reach the node external port from your machine
  • Edit the file examples/docker-compose.existing-node.yaml and set the url of your node as explained in the file
  • From the project examples folder run docker-compose up -f docker-compose.existing-node.yaml

Tips and tricks

You can run several instances of the middleware simultaneously, with different options. A sensible way of doing this would be one or more using the -s option to serve clients, and one (and only one) with the -p option, populating the database.

If you don't want to interrupt service, want to update the database with new features, and can live with short-term (possible) inconsistencies, use the -H option with the whole chain to force a reload, but serve from the old version soon.

**DON'T USE diesel migration run! **

We now update migrations automatically on application start--and the command diesel migration run overwrites important files.

How to build

You need a nightly rust build

rustup default nightly

then

cargo build

The middleware will automatically set up its DB on initialization, and run migrations after an update, if they are necessary.

On Ubuntu 18 and 19 the following packages are needed: libpq-dev, libssl-dev and zlib1g-dev. YMMV depending on whether you've e.g. gcc installed.

How to run

Development mode

cargo run -- + flags below

Release mode

cargo build --release # make a release build--this will take a long long time
./target/release/mdw # + flags below

Flags

        --help        Prints help information
    -H, --heights     Adds or replaces a set of heights, or ranges of heights separated by
    		      commas to the database. i.e. -H1,3-4,6,100-200
    -d, --daemonize   If set, the middleware will daemonize on startup
    -p, --populate    Populate DB
    -s, --server      Start server
    -v, --verify      Check the DB against the chain
    -V, --version     Prints version information

Environment variables

NODE_URL - the URL of the æternity node AESOPHIA_URL - if present, the middleware will attempt to use this to decode contract calls, storing the function called, and its parameters PID_FILE - if present, and the -d option is set, the middleware stores its pid in this file LOG_DIR - if present, this directory is used for logs, otherwise stdout is used DATABASE_URL - PostgreSQL connection URL STATUS_MAX_BLOCK_AGE

Supported queries

GET /middleware/channels/active
GET /middleware/channels/transactions/address/<address>
GET /middleware/compilers
GET /middleware/contracts/all
GET /middleware/contracts/calls/address/<address>
GET /middleware/contracts/transactions/address/<address>
POST /middleware/contracts/verify
GET /middleware/generations/<from>/<to>?<limit>&<page>
GET /middleware/height/at/<millis_since_epoch>
GET /middleware/names/<name>
GET /middleware/names?<limit>&<page>
GET /middleware/names/active?<limit>&<page>
GET /middleware/names/reverse/<account>?<limit>&<page>
GET /middleware/oracles/list?<limit>&<page>
GET /middleware/oracles/<oracle_id>?<limit>&<page>
GET /middleware/reward/height/<height>
GET /middleware/size/current
GET /middleware/size/height/<height>
GET /middleware/status
GET /middleware/transactions/account/<account>/count
GET /middleware/transactions/account/<sender>/to/<receiver>
GET /middleware/transactions/account/<account>?<limit>&<page>
GET /middleware/transactions/interval/<from>/<to>?<limit>&<page>
GET /middleware/transactions/rate/<from>/<to>

GET /v2/generations/current
GET /v2/generations/height/<height>
GET /v2/key-blocks/current/height
GET /v2/key-blocks/hash/<hash>
GET /v2/key-blocks/height/<height>
GET /v2/micro-blocks/hash/<hash>/header
GET /v2/micro-blocks/hash/<hash>/transactions
GET /v2/micro-blocks/hash/<hash>/transactions/count
GET /v2/transactions/<hash>

Websocket support

The websocket is exposed by default on 0.0.0.0:3020 but can be overridden with the environment variable `WEBSOCKET_ADDRESS`. On our hosted sites it is available as wss://mainnet.aeternal.io/websocket and wss://testnet.aeternal.io/websocket, which are the recommended URLs.

Message format:

{
"op": "<operation to perform>",
"payload": "<message payload>"
}

Supported ops:

  • subscribe
  • unsubscribe

Supported payload:

  • KeyBlocks
  • MicroBlocks
  • Transactions
  • TxUpdate
  • object, which takes a further field, 'target'--see below

Object subscriptions

You may subscribe to any æternity object type, and be sent all transactions which reference the object. There is an example of this below.

Returned data

Subscriptions return the array of subscriptions (possibly empty):

{"op":"Subscribe", "payload": "KeyBlocks"}
["KeyBlocks"]
{"op":"Subscribe", "payload": "MicroBlocks"}
["KeyBlocks","MicroBlocks"]
{"op":"Unsubscribe", "payload": "MicroBlocks"}
["KeyBlocks"]
{"op":"Subscribe", "payload": "Object", "target": "ak_2eid5UDLCVxNvqL95p9UtHmHQKbiFQahRfoo839DeQuBo8A3Qc"}
["KeyBlocks","MicroBlocks", "ak_nv5B93FPzRHrGNmMdTDfGdd5xGZvep3MVSpJqzcQmMp59bBCv"]

Actual chain data is wrapped in a JSON structure identifying the subscription to which it relates:

{"payload":{"beneficiary":"ak_nv5B93FPzRHrGNmMdTDfGdd5xGZvep3MVSpJqzcQmMp59bBCv","hash":"kh_rTQLY9ymNL9MSop3RJtySjxPreuowhejHJYu4KSPBfSSZTerh","height":38263,"miner":"ak_bCCis9P7hTGCjfhDPZh3KQUSDXd5V7thyxYAznFcsr1iLn1Uv","nonce":15041902164601590529,"pow":[11019073,17901996,22830222,28569083,49447859,49462425,49519772,87145865,99866878,102237553,123920118,132218818,155018387,180362216,198154991,201321139,222409093,280306327,281351614,283186475,285893935,287852747,323975163,324943621,371752985,397769028,399438380,400871099,414481284,419637507,437183279,446297926,458204825,460525976,468647042,479097364,479884030,491013280,503534491,505981207,522008475,527020581],"prev_hash":"kh_dxbyza4hSY3Ly5U7HJnfn748Po5pJ1rv3dSgisrexcQ5Nna6p","prev_key_hash":"kh_dxbyza4hSY3Ly5U7HJnfn748Po5pJ1rv3dSgisrexcQ5Nna6p","state_hash":"bs_2mnjZYzxN23QpUx7MT2f5RRx8sXcZqvEq7GtqE28LCowq91k4u","target":503824559,"time":1550244690796,"version":1},"subscription":"KeyBlocks"}

Testing the websocket

Here is some magic you can run in your Javascript console

var exampleSocket = new WebSocket("ws://127.0.0.1:3020");
exampleSocket.onopen = function (event) {  // when connection is open, send a subscribe request
    exampleSocket.send('{"op":"Subscribe", "payload": "KeyBlocks"}');
    //to unsubscribe: exampleSocket.send('{"op":"Unsubscribe", "payload": "KeyBlocks"}')
}

exampleSocket.onmessage = function (event) {
   	console.log(event.data); // you get data here when it arrives
}

You can’t perform that action at this time.