[enhancement] [2.1rc] ACTION_RETURN_VALUE enabling read only functions

[theblockstalk]

See Telegram December 16-19 https://t.me/c/1139062279/252476

Issue
Blockchain clients can only use get_tables API to read blockchain data. This has several problems for application developers:

1. only the primary key can be used for searches with no range searches
2. efficient, aggregated cross-table queries can't be made with a guarantee that table data is from the same block height (unless using dfuse state table API with block height option). This is because the client must perform aggregation and does not have a consistent view of the EOSIO database.
3. programmable and conditional queries cannot be efficiently and with a guarantee that table data is from the same block height

A few workarounds exist for the above problem:

• LiquidApps's Liquid Lens service
• Greymass API might have a feature for this (said someone in Telegram, but I could not find it)

With ACTION_RETURN_VALUE
With the new functionality of ACTION_RETURN_VALUE #8327 in EOSIO 2.1rc, there is a way to create such complex queries on the server, guaranteeing consistency with the EOSIO database. An example query would be to return all accounts with a balance > 1000 SYS and that have more than 10 SYS staked that currently has an active name bid. An action can now be created which returns a vector of these accounts. This functionality works when the action is executed on the node by adding it to the blockchain, which may be valid on private EOSIO networks but is not feasable on public networks such as EOS where CPU costs prohibit this. A workaround exists to add an additional action to the tx with an action that always fails, which is not neat, a familiar or obvious strategy for application developers.

This issue highlights that it may be feasible to build the following features to enhance ACTION_RETURN_VALUE for the benefit of application developers.

Proposals:

1. Implement read only actions (good) - define some actions that when called, will never add the tx to the blockchain, only execute on the server and return the result. this is similar to Ethereum's view functions
2. Implement no broadcast executable txs (better) - provide an additional parameter to push_transactions to specify that the tx should not be added to the blockchain. This would let clients send the tx to the node, the node would execute and provide the execution result but not add it to the blockchain. This is a more general approach, in addition to allowing for tx results to be read and hence txs to query the blockchain, this would additionally support write txs and give developers the ability to easily get an expectation on the consequences of a tx before it is sent. (this is different to the --dont-broadcast parameter of cleos's push transaction API (or similar parameters in eosjs), which doesn't even send the tx to the node, just prints the serialized data)

An additional benefit of either of the above would mean that the write AND read code for each smart contract would be logically coupled together. While not a functional requirement, this would make code management easier for application developers.

An additional benefit, useful for private networks, is that read transactions can have authorization checks using require_auth and allow data privacy at the node API level. this is currently only possible with a middleware layer above the nodeos API, or nodeos plugins.

[xia256]

This is personally the approach I use as well and I think some of Liquid's products uses it as well:

A workaround exists to add an additional action to the tx with an action that always fails, which is not neat, a familiar or obvious strategy for application developers.

It feels hacky to do this and having a proper way like Ethereum view fns would be ideal.

[theblockstalk]

As explained by Aaron from greymass:

https://github.com/greymass/eos/commits/skip_all_checks

Apply those 4 commits to nodeos, start nodeos with --skip-all-checks. Then to sample a transaction, send the transaction without any signatures to the push_transaction endpoint.

Without the signatures, it’ll execute locally, return the results (including speculative cpu usage/etc), and the transaction ends up being rejected by peers without the signature so it never makes it on chain. It’s a crude solution, but it works well.
We have it running at the following url if you want to give it a shot without compiling:

https://eos.greymass.com/v1/chain/test_transaction

[theblockstalk]

As Michael pointed out on Telegram, large query transactions may reach the hard transaction limit of 30ms, or the accounts CPU limits. As query transactions don't enter the blockchain, there should be a mechanism to allow large query txs (strategy 1) or long txs that don't enter the blockchain (strategy 2) while also considering the security of the API and prevention of DDOS attacks. API nodes should always be able to choose if these features are exposed.

[aclark-b1]

This is really helpful input, thanks guys! We're working on this, although I don't yet have an ETA to share. Please keep the feedback coming!

[theblockstalk]

From Aarin Hagerty

Here is a suggestion for proposal 2. The transaction could have a special extension to ensure it will never be included in the real blockchain. But if sending it to an endpoint to run in a pure-speculative mode (for example to read data or to do a dry-run for CPU estimation purposes), it will not reject the transaction early because of that extension. That way even if you put signatures on it to authorize a read from a private blockchain you still don’t have to worry about it potentially being rerouted to the producer node. Or if one were using some future contract-managed system for resource billing and authorization (which doesn’t leverage first authorizer or native transaction signatures), there would still be a guarantee that the dry-run transaction cannot go on the blockchain.

[theblockstalk]

From @abourget

I would have loved to have an "alwaysfail" method either be native or in all system contracts.

I had proposed something in here years ago: https://eeps.io/EEPS/eep-5 (search for alwaysfail). So many things become possie with that guarantee of non-propagation (like that proposed alerting system).

With only that, we were able to implement return values through emitting an inline action before the alwaysfail action is reached.