Oracles

[shargon]

Wait for #1579

Close #1272 #1273 #1274 #1275 #1276 #1277 #1278 #1279 #1263 #1260 #1259

You can use this example for testing (neo-project/neo-devpack-dotnet#248 (comment))

[vncoelho]

Very good, the same could be done for neo-node branch.

[erikzhang]

With a sync oracle you always can build an async one, but it will require three tx, and watch the blockchain in order to send the callback, otherwise, the user won't pay for callback.

Currently, we need two transactions, one for the request, and the other for the response. If we use asynchronous mode, we also need two transaction, not three. One for the request, and the other for the callback. We don't even need attributes. We can deduct the callback fee directly from the account that sent the request. If the balance is insufficient, the callback transaction will fail.

[shargon]

I think going to the asynchronous oracle when we have a synchronized one is taking a step back, other blockchains only have asynchronous because they can't change the core, they need to build everything in smart contracts and offchain, and it's something that complicates the wheel and the developer experience, now we have the opportunity with neo3 of creating an sync oracle, for me have no reason to do it as async.

If you create more than one download and it produce a FAULT oracle request, it's a developer problem, ussually it will be because they use dynamic URL with block numbers, so they should expect a FAULT, if they are lucky and OracleNodes make consensus in a block, they will have his result, but it's something under control.

[erikzhang]

now we have the opportunity with neo3 of creating an sync oracle, for me have no reason to do it as async.

In my opinion, the current synchronization mode is not a real synchronization. After the transaction is sent, the waiting time is the same as asynchronous.

If you create more than one download and it produce a FAULT oracle request, it's a developer problem, ussually it will be because they use dynamic URL with block numbers, so they should expect a FAULT, if they are lucky and OracleNodes make consensus in a block, they will have his result, but it's something under control.

But in asynchronous mode, using dynamic URL with block index or current time is allowed. Because the requests are created after block persisted.

[Tommo-L]

For contract, it's realy synchronize. If we use asynchronous mode, the callback can only perform tasks that don't need to be signed by users. Therefore, we can't build arbitrarily complex applications.

[erikzhang]

If we use asynchronous mode, the callback can only perform tasks that don't need to be signed by users.

You can check the witnesses in the request transaction first.

[Tommo-L]

Some dapps can do it by that way, some may not. For example, in a gambling game, it needs to request the prize number before calling the nep5 transfer method.

[Tommo-L]

Another thing I'm thinking about is that what's really needed behind oracle is valuable data.

Oracle's main role:

1. Transfer data
2. Credit endorsement for data, when the data source does not provide a credible proof

If the data source can provide trusted data directly, follow certain criteria, such as signing the data. Then, any node can become a lightweight Oracle to fulfill the first responsibility - transfer data.

In this case, we only need to provide:

1. a signature specification (maybe a NEP) that is as good as it can be
2. lightweight Oracle
3. a data source registry on the chain.

One popular phenomenon these days is the willingness of exchanges to offer links to their data, such as coinbase and binance, which have started offering signature data sources.

[doubiliu]

Well we discussed the difference between synchronous and asynchronous modes in the discussion above.This was discussed in a similar issue in our assembly last year.

Let's make a summary

	Synchronous	Asynchronous
Block status	High correlation	Low correlation
Execution time	Strong restrictions	Unlimited
Permissions/Charge Mechanism	Simple	Complex
Originality	High	Low

These two modes are very useful for implementing a reliable built-in Oracle.

In my opinion:These two are not contradictory, just as there are synchronous / asynchronous calling functions in the front-end technology ajax.Developers will use the correct function according to actual needs.This is the same in our built-in Oracle, we do not necessarily have only one mode.

We can extend the current implementation. We only need to let the existing Request method declare the mode and callback function we want to use (optional).If the developer chooses synchronous mode, then everything will be the same.If the developer chooses asynchronous mode, the Oracle node will trigger a callback function or some other feasible operation

But we should explain the limitations of these two modes in the API documentation and give a certain demo.
E.g:
Using synchronous mode, the overall timeout time will be limited to 2/3 of the block time, after the timeout all execution will be filled with fault by default.
Using asynchronous mode, you need to declare the callback function or the user to extract the data.

@shargon 's current implementation has done a lot of work for us.Save us a lot of time.
I think we can complete the design and implementation of the extension first, and then complete the overall implementation of the asynchronous mode

[erikzhang]

In my opinion:These two are not contradictory

I don't think we should build both synchronous mode and asynchronous mode. I don't want to over complicate neo's architecture.

Maybe we should rethink each aspect of the two solutions and find out which suits us best.

[shargon]

A callback it's not possible because you create the callback in the request and must be triggered in response, when the context it's different, it's the same as filters, they call other contract with other parameters, but it could not be done with callback syscall, I think, the pointer will be lost.

I think that async can be done by wallets or dapps if they need it, but sync it's only possible to build if it was build in the core.

[erikzhang]

You don't need a "callback syscall". When I was talking about "callback" in oracle, I mean a string methodName.

[doubiliu]

What you mean is that the user declares a callback function name when calling the Oracle service, and then when the Oracle node commit the data, it will record a message in the Oracle native contract, such as xxx: FunctionA result is xxxxxxx, and then when user use data, query it from Oracle's native contract?

[erikzhang]

The oracle node send a transaction and call that method.

[erikzhang]

I have a good idea. We can add a timestamp or block height to the URL when the actual request is made, so that non-determinacy can be eliminated, and the URL has cross-block availability. After solving this problem, we seem to be able to use the synchronization solution.

The user or contract can choose to add a time stamp URL parameter, or block height. Everything is specified in OracleRequest.

@shargon If you think this approach is feasible, please resolve the conflicts.

[shargon]

The conflicts was resolved, but OracleContract require a new review.

[erikzhang]

@shargon Do you think that the oracle nodes can reach a consensus on the request timestamp of a transaction? Maybe a node is needed to write the unified timestamp into the contract storage first?

[shargon]

Do you think that the oracle nodes can reach a consensus on the request timestamp of a transaction?

I think that is possible, sometimes, but if don't, we have two possibilities: the first one, it doesn't care because the response was the same, and the worst one, it was different, currently they return a FAULT but we can change this to have more retries.

[erikzhang]

@shargon What is the consensus protocol of oracle?

[shargon]

A multisignature of all Oracle nodes

https://github.com/neo-project/neo/pull/1584/files#diff-b312bfff946ee7c97448f2b22414f873R180-R184

if they don't receive enough signatures in a certain time, they will sign as a fault download.

https://github.com/neo-project/neo/pull/1584/files#diff-671e72f8dccae549eb16aa4d94679168R335-R342