Rethinking Built-in Oracle Proposal

[doubiliu]

#1584
Currently, @shargon has submitted a implementation of built-in Oracle.The content is basically consistent with our discussion at the time, and of course, appropriate adjustments have been made.But we also found some problems in the review process.As @erikzhang said,"Maybe we should rethink each aspect of the two solutions and find out which suits us best."

Review

Synchronization

Flow Chart

Process:

1. User sends a request transaction,including URL.
2. Memory pool caches request transaction.
3. Oracle node monitors the memory pool.
4. When a new request transaction occurs in the memory pool, the Oracle node executes the script and accesses the corresponding url to obtain data.
5. Oracle node sends a response transaction, including data cache and node signature.
6. Memory pool caches response transactions.
7. When the response has enough node signatures, the consensus node packages the request and response transactions together into the block.
8. When the ordinary node persists the block, it directly uses the dataCache in the response.

Advantage

1. Permission and charging mechanism are consistent with existing interoperability.
2. Synchronization, no need to use callback.

Disadvantages

1. Memory pool spam attack.
2. Highly related to the state of the block, it must be executed within a block.

ASynchronization

Flow Chart

Process:

1. User send a tx to invoke Oracle contract to register a request, including the url and callback function.
2. Oracle node monitors the main chain.
3. When a new request is generated in the Oracle contract, the Oracle node accesses the corresponding url to obtain data according to the request.
4. Oracle node send a tx to invoke the Oracle contract to submit the data, including the data and the node signature.
5. When there are enough signatures, the Oracle contract will call the corresponding callback function.

Advantage

1. No spam attack.
2. Weak association with block status.

Disadvantages

1. Complex implementation of callback permission check.
2. Charging mechanism, Oracle node needs to pay the execution fee of the callback function.

Compare

Mechanism	Synchronization	ASynchronization
Block state association	Strong	Weak
Total execution time	Strong limit	Weak limit
Permission	No additional mechanism	Additional mechanism
Charging	No additional mechanism	Additional mechanism
Originality	High	Low

[doubiliu]

Additional

Both solutions only solve the problem of data handling, but there is not much thinking about trusted data sources.We recommend introducing the concept of a trusted data source DataProvider.

DataProvider
Trusted data provider. The data source can be registered as a DataProvider in the contract, and also provide a method of data audit. Anyone can audit the data of the data source that has been chained according to the corresponding data audit method.

[Tommo-L]

option3 Verifiable Oracle

Why do we need an Oracle like this?

1. Transfer valuable data on chain has become a trend, the most popular is asset price information now.
2. As a data provider, they are willing to provide data for external use, and they can also get revenue. At present, like Coinbase, Binance has begun to try to provide verifiable data, and we believe it will become a trend in the future.
3. If the data source does not provide verifiable data, no matter what solution Oracle adopts, the reliability of the data cannot be guaranteed.

Because of this, we consider introducing the DataProvider in the Oracle solution, which is mainly responsible for:

1. Provide data verification function，
2. Store data on chain [optional]
3. Config data fee, etc. [optional]

At this time, any node can become an Oracle miner, only responsible for transfering data on the chain.

For non-verifiable data, oracle miner will also transfer it, but Oracle does not provide data verification, the developer should bear the responsibility. We advocate and encourage to transfer verifiable data.

In this way, whether synchronous or asynchronous, I think the biggest difference between us and traditional Oracle is that we have built-in verifiable Oracle.

[shargon]

Disadvantages

• Memory pool spam attack.
• Highly related to the state of the block, it must be executed within a block.

I think that the sync oracle doesn't include a mempool attack, it can have the same checks than the currently memory pool, so there are no difference.

According to the state of block, my opinion: if your request use any block data, you can be lucky and receive the right response, or a FAULT, there are no problem with this for me, users should not use this data for compose the url, but if they did it, oracle nodes will deals with it, usually with a FAULT.

For me we don't need to do an extra work for oracle nodes in each block, otherwise you can have a spam attack, with sync or async version. Oracles should do his work, download the data, the rest of the work should be made by mempool when the receive the response or the request TX.

[Tommo-L]

According to the state of block, my opinion: if your request use any block data, you can be lucky and receive the right response, or a FAULT, there are no problem with this for me, users should not use this data for compose the url, but if they did it, oracle nodes will deals with it, usually with a FAULT.

I agree. I think this is not a oracle problem, it's the blockchain problem. For a contract, which use block height for switch, also face the problem. The tx may be valid and execute successful before packaged, but after that it may fail.

if(height == xxx)
   ...
else if (height == yyy)
   ...
else
   ...

[roman-khimov]

Synchronization
Disadvantages

Memory pool spam attack.

I thought this one had already been discussed in #1527, but I can reiterate the main point here --- having the same level of fee payment guarantees as for regular transactions (and we do have it) should be enough for Oracle transactions.

Highly related to the state of the block, it must be executed within a block.

And that one actually looks more like a feature to me, because long unpredictable delays between different requests and especially between initial blockchain state and blockchain state when the transaction will finally be executed will easily lead to distorted reality seen by the contract. It's almost like a rolling shutter effect, not something I'd like to see when executing a contract, especially if it somehow relates to any kind of asset.

DataProvider
The data source can be registered as a DataProvider in the contract, and also provide a method of data audit

I would probably not mix this (trust) issue in here, verifiable data is good, but it only works if your data provider signs it and you have some way to verify it on-chain (basically, there is a corresponding contract that knows the appropriate keys) and even then this data needs to be brought into the chain by Oracles and we're better concentrate on the way they gonna do that here.

[JohndeVadoss]

+1 - yes, this is akin to a feature from my perspective as well. Agree with Roman's comments wrt stale reference data.

[doubiliu]

Got it

[shargon]

Any conclusion?

[erikzhang]

I already have a perfect asynchronous solution.

[shargon]

Asynchronous solution it's something that any dapps can make, in any time, and probably better than us because they will be focused in that. What it's the sense of offer a core solution that can be improved in any time by third parties? There are no syncronous solutions in other blockchains only for one reason, they can't do that althought they want to, it's required to be integrated in the core of the blockchain, so... we only the chance now, with neo3. We can be unique.

[erikzhang]

1. The synchronization solution requires special ordering or delay of transactions. This process requires the participation of consensus nodes, which seems a bit weird.

2. The synchronization solution cannot access the URLs related to the current time, and this may be a very common scenario.

3. The asynchronous solution needs to solve the problem of fees and witnesses. These problems can only be solved by the core, not by the contract.

[shargon]

The synchronization solution requires special ordering or delay of transactions. This process requires the participation of consensus nodes, which seems a bit weird.

Agree, it's weird, but it only affect to the sort of the mempool. Both solutions will have weird things like onPostPersit xD

The synchronization solution cannot access the URLs related to the current time, and this may be a very common scenario.

You will have the same problem in async version, you will need to invalidate the original request if the time changed. Because if OracleCN sign the transaction at block 1, and now it's block 2, it doesn't care if it's async or sync, it will require to be invalidated, if you want to be able to that, I think that it's not required.

The asynchronous solution needs to solve the problem of fees and witnesses. These problems can only be solved by the core, not by the contract.

This can be done by a smart contract. We have dapps in ethereum that currently are working without this issue, you need to deposit your token, and the bridge will take the reward. (https://witnet.io/about#bridges)

[Tommo-L]

We already have CreateCallback syscalls, can we add CreateTask syscall? This can be applied in oracle, like the following:

1. User send oracle request tx, the script is:

    var trigger = oracle contract
    var contract = request contract 
    var extraData = oracleResponseFee
    CreateTask(trigger, contract, method, params, extraData)

2. CreateTask syscall will call the RegisterTask(contract, method, params, extraData) method of OracleContract, which will save the task(=contract.method(params)) and call AddGas(oracleResponseFee).
3. Oracle nodes send the response tx, and oracle contract executes the request at last.

And the CreateTask can also be used to collect signatures onchain, to solve this problem #1573

[erikzhang]

What's the difference between it and the asynchronous oracle?

[Tommo-L]

No big difference, it's just pseudo-synchronization, and user does not need to set responseFee. The only concern is that our previous propaganda is the synchronization scheme.

[erikzhang]

I don't think it is a good idea to create a new SYSCALL for just pseudo synchronization. And it is the same as calling the RegisterTask method of the oracle contract directly. The only difference now is that the name of this method is called Request.

[Tommo-L]

The only difference now is that the name of this method is called Request.

If so, why not we use the pseudo synchronization, very little code needs to be changed, and it can also be different from traditional asynchronous oracle.