[NEO 3.0] neon-js should support the coming NEO 3.0

[MorganDream]

NEO 3.0 road map is released. And hopefully NEO 3.0 testNet will come out in this July.
NEO plans to draw lots of communities and developers to develop tools and dApps on 3.0 testNet, in which SDK will act an important role.

Now NGD members are developing 3.0 C# SDK in project neo: neo-project/neo#850. I think neon-js should be upgraded for 3.0 at the same time, because neon-js developers could request support from developers of C# SDK, as they could modify core codes properly to support SDK if necessary.

[MorganDream]

UPDATED ON 26/07/19

NEO3 Proposal for NEON-JS

Current Structure of neon-js

Targets:

To support NEO3 in neon-js.

NEO3 features

Documents: https://github.com/Tommo-L/NEO3-Development-Guide
NEO3 are still in development phase. Below I list some already-merged change.

• No UTXO, no inputs and outputs in transaction, transaction can be constructed statically.validUntilBlock property may need to be specified according to current block height. But you need to pay attention to transaction hash collision. You can generate a random nonce in transaction to avoid hash collision.
• All transactions are invocationTransaction, property version will be 0.
  • Property type removed
  • Properties nonce, sender, systemFee, networkFee, validUntilBlock added
  • validUntilBlock is used to specify which blocks can the transaction be included.
  • Attributes now only have Cosigner & Url
• NEO and GAS will be native contracts. Both will meet nep5 standard.
• No Claim transaction, gas bonus distribution would be handled by node automatically. SDK could provide a API to send NEO to self address to "claim gas".
• About 3.0 VM, the design pattern is to exploit InteropService.
  • APPCALL TAILCALL is removed, contract invocation is by SYSCALL - sc module needs to be changed
  • Crypto is removed, also CHECKSIG, VERIFY, CHECKMULTISIG - wallet module multisig needs to be modifed
  • Stack isolation is removed
• As in last feature, contract invocation will use InteropService. All contract invocation can use System.Contract.Call, Native Contracts(NEO, GAS) can also use Neo.Native.Tokens.NEO

Timeline

• Release a standalone version of neon-js tagged with neo3 to support NEO3

Plan

• SDK can be supported just by RPC queries after neo 2.10.2, in case of delay in neoscan updating to NEO3, I’ll just implement the CLI/RPC api provider.
• Modules that need refactoring or rewriting for NEO3:
  • tx -- simplify tx structure, remove types, old attributes, remarks. Add new properties and related methods.
  • wallet -- multisig function needs to be modified; Balance, Coin classes are now useless
  • sc -- scriptBuilder needs to use SYSCALL, remove tailcall. Enrich/Rewrite module sc to have method to construct invocationTransaction and method to send it through RPC methods. Or Add a module SmartContract to have such API.
  • nep5 -- refer or extends module sc for code reuse
  • NEO/GAS -- extends nep5
  • new High level API(optional) -- like what neon-api does.

Structure of neon-js for NEO3

Any suggestions and comments would be appreciated.

[snowypowers]

I think depending on how the native assets are reimplemented, we might consider merging nep5 into the core set of modules.

Also, with regards to publishing, we can either use a new namespace (cityofzion/neon3-js?) or just do a breaking change with v5 being the the compatible version with neo 3.x.

I doubt smartContract will end up under neon-api as that is a module for data integration.

neon-js itself actually contains the fluent method naming structure, not neon-api.

[MorganDream]

@snowypowers Thanks for your reply.

I think depending on how the native assets are reimplemented, we might consider merging nep5 into the core set of modules.

Core part of NEO3 is already implemented. NEO and GAS are nep5 for sure.
NEO & GAS are native contract, which meets NEP5 standard. And they have extra methods.

Also, with regards to publishing, we can either use a new namespace (cityofzion/neon3-js?) or just do a breaking change with v5 being the the compatible version with neo 3.x.

I suggest the latter. Here's my plan:

• Release a version of neon-js that would support NEO2 and NEO3 before or short after NEO3 test net going live.
• Release a version of neon-js that is simplified to only support NEO3 before or short after NEO3 main net going live.

I doubt smartContract will end up under neon-api as that is a module for data integration.
neon-js itself actually contains the fluent method naming structure, not neon-api.

I plan to integrate sc, tx and rpc in SmartContract module. Every smart contract can be a SmartContract instance. To invoke a smart contract function, just call SmartContractInstance.invoke(method, params).
NEP5 module will extend SmartContract. Invoke it like NEP5Instance.transfer(from, to, amount);
NEO and GAS will be NEP5 instances for user to invoke.

So SmartContract is kind of high level API, so I put it into neon-api.

[snowypowers]

1. Will there be a need to support neo2 and neo3 together in a single release? The alternative here is to continue publish neo2 under latest tag and neo3 under neo3 tag until we do the flip. It is possible to install 2 versions of the same package with npm for those who wish to support both chains together.

2. From the way you describe it, SmartContract sounds like a low level module but I guess I can see what is inside before I comment.

[MorganDream]

@snowypowers

1. The idea of using tags to separate neo2 and neo3 sounds good.
2. Yes. SmartContract is kind of combination of high level and low level, because NEP5 will extend it. And instance design is not fit for multiple intents transaction. I'll think more and start from core level API.

[MorganDream]

neon-js/packages/neon-api/src/funcs/main.ts

Lines 42 to 50 in 8972f6f

	return fillSigningFunction(config)
	.then(fillUrl)
	.then(fillBalance)
	.then(createContractTx)
	.then(addAttributeIfExecutingAsSmartContract)
	.then(signTx)
	.then(addSignatureIfExecutingAsSmartContract)
	.then(sendTx)
	.then(applyTxToBalance)

@snowypowers How about employing generator to replace this promise style in NEO3 high level API? So that user can have more control over the flow(like using different param according to certain middle state), and he can add parameters gradually instead of passing params all at once.

[MorganDream]

Now that low level APIs are merged into next branch. And I have now several design points which need suggestions:

• About fee(systemFee and networkFee) calculation, do we need a offline fee calculation or we just use invokeScript to get the to-be-consumed gas?
• About High Level API style, I have a proposal which is quite different from what it were, but I think it's better: feat(api): TransactionBuilder #469

Any suggestions would be welcomed.

[snowypowers]

For fees, since its going to be inaccurate to calculate the fees, I rather not do anything offline. We can add another step after constructing the transaction to add the fees.

With regards to 469, its too many features. Lets start with the TransactionBuilder which is pretty essential and build up the workflow first.

Not clear on the structure of NetProvider, it looks a little weird that you have Entity Provider inherit off it. Having a facade is a good use case though.

[MorganDream]

@snowypowers A NetProvider instance is a net-specific(mainnet, testnet, privatenet) provider that can deal with some read request(e.g. balance, height) with neo, in which case an account(private key) is not needed.
An EntityProvider is not just net-specific, but also account-specific, which can also write data on neo - sending transactions.

Considering most cases where user use sdk with specific net and specific accounts, it is not necessary to isolate all the requests, the network info and account info are usually fixed. That's the point for this design.

[Tommo-L]

It's recommend to set the contact information about the contract in manifest file, as neo3 have moved those properties, replaced by Extra field in manifest file.

neo-project/neo#1246