Steem 0.19.10

Steem Equality 0.19.10 (Appbase) Release Notes

0.19.10 is an optional release that includes significant architectural changes to steemd. These architecture changes lay the framework for future improvements and as of 0.19.10 allows for fully parallel API execution.

Reindexing

There were a number of optimizations to improve reindex performance. Because of this you will need to reindex from all previous versions to run 0.19.10.

API changes

We restructured our APIs to allow greater flexibility for future upgrading and maintaining. The APIs now take in a single object as an argument and return a single object as a return type. All the existing APIs have been updated to match this standard.

Because we no longer need to stick to C++ style parameter semantics, we can use more varied default argument types as well as variadic parameters. We can also extend the functionality of a call without impacting the existing apps by extending the returned object.

The database_api has undergone significant changes to allow the querying of all consensus objects with any ordering that steemd uses already. This will provide more flexibility for services to find the objects they need. database_api is still a work in progress and could be changed.

New condenser_api

To help with this transition, we created condenser_api, which contains all of the API methods that currently exist and uses the existing argument formatting. The easiest way to get your app to work with Appbase is to change the api to condenser_api.

APIs must be called by name

If you are used to calling an API using the API id, that method of invocation is no longer supported. All the APIs must now be called by name.

Removed login_api

The login_api was designed as a way to map the API names to numeric ids. Because the APIs are no longer called via id, there is now no need for the login_api, and so it has been removed.

API methods list

If you call jsonrpc.get_methods, a list of all available API methods will be returned.

Argument and return object prototypes

If you call jsonrpc.get_signature passing an API method name, it will return the argument and return the object prototypes.

For example,

{"jsonrpc":"2.0", "method":"jsonrpc.get_signature", "params":{"method":"database_api.get_active_witnessess"}, "id":1}

returns
{"jsonrpc":"2.0","result":{"args":{},"ret":{"witnesses":[]}},"id":1}

{} is the void type argument and it returns a list in the witnesses field.

Using condenser_api

All calls in condenser_api will return [] as the argument, as the array argument passing is opaque and implemented in the API calls themselves. They follow the current argument formatting. Existing apps should only need to skip using login_api and send all of their calls to condenser_api without any other changes required to use Appbase.

For example, calling get_dynamic_global_properties with condenser_api vs database_api:

{"jsonrpc":"2.0", "method":"condenser_api.get_dynamic_global_properties", "params":[], "id":1}

{"jsonrpc":"2.0", "method":"database_api.get_dynamic_global_properties", "id":1}

Because the method has no arguments, the params field can be omitted when not using condenser_api. However, it can optionally be included as the void type (e.g. "params":{}) but it is not required.

Streamlined syntax

You might have noticed that the previous examples used a different format to call the API. Previously, there was an outdated call syntax that looked like this:

{"jsonrpc":"2.0", "id":0, "method":"call", "params":["api","function",[ARGS]]}.

However, we now support a more streamlined call syntax:

{"jsonrpc":"2.0", "id":0, "method":"api.function", "params":[ARGS]}

Both formats work, but with the new format being preferred for readability.

Enhanced JSON-RPC compliance

Some other changes were made to make steemdcompliant with the widely-used JSON-RPC specification.

Backward compatibility

Our reverse proxy service, jussi, handles the method translation. This means that even after deploying Appbase to our production environment, the old API calls should still work.

Future updates required

Note that the new APIs introduced in this release are still Work in Progress. There are a number of serialization changes that are still being made to them and hence they are in various states of completeness. We will post again in the near future explaining the nature of those changes. Feel free to play around with those APIs, but know that they will be changed. Once we finalize those APIs, we will deprecate condenser_api and begin migration to the new APIs. The good news here is that if there is some data that your app needs that you cannot get from a new API either through multiple calls or local calculations you can create an issue and we will consider adding it before the initial version of the API is finalized.

RocksDB Account History

There is a new account history plugin that is backed by RocksDB instead of Boost memory mapped files. If you want use this plugin instead you will need to reindex with the account-history-rocksdb plugin enabled. The account-history-api plugin requires one of the two plugins (account-history-rocksdb or account-history) be enabled and will expose the same interface regardless of the chosen backend plugin.

Config file changes

The logging config has always been a sore spot in the config file because of the number of options available. It was so complex that it required a different parser. We have changed to using a json format, which allows us to use only one parser. The default logging config is the following:

# Console appender definition json: {"appender", "stream"}
log-console-appender = {"appender":"stderr","stream":"std_error"}

# File appender definition json:  {"appender", "file"}
log-file-appender = {"appender":"p2p","file":"logs/p2p/p2p.log"}

# Logger definition json: {"name", "level", "appender"}
log-logger = {"name":"default","level":"debug","appender":"stderr"}
log-logger = {"name":"p2p","level":"debug","appender":"p2p"}

Most options are now namespaced with the plugin they belong to. For example, seed-node is now p2p-seed-node. Old config options still work and will print a warning on startup so that you can update your config before the old options are removed.

Plugins

Plugins are enabled via the plugin option. There is no more public-api option and all the APIs are now enabled via plugin as well.

Most config options are now namespaced by the plugin they belong to. Config options that were not namespaced are still supported but will now log a deprecation warning when used. contrib/config-for-docker.ini and contrib/fullnode.config.ini are example configs that are used in the Docker images. You can use these as example config files. As usual, we strongly recommend using the Docker images to deploy your node.

Auto-Scaling State Files

New options exist to automatically increase the size of the memory mapped state file when it is full.

shared-file-full-threshold is a 2-precision percentage (0-10000) that defines the threshold for when to autoscale the shared memory file. Setting this to 0 disables autoscaling. The recommended value for consensus node is 9500 (95%). Full node is 9900 (99%).

shared-file-scale-rate is a 2-precision percentage (0-10000) that defines how quickly to scale the shared memory file. When autoscaling occurs, the file's size will be increased by this percentage. Setting this to 0 disables autoscaling. The recommended value is between 1000 and 2000 (10-20%).

New Data Directory

To conform to blockchain standards, the steemd data directory has moved from $PWD/witness_node_data_dir to $HOME/.steemd. If you want to continue using your old data directory you can continue to do so by specifying --data-dir at runtime. Otherwise, a warning will be printed on startup prompting you to move your existing data directory. All new data directories will be created at $HOME/.steemd.