v1.1.24

Release v1.1.24 is out 🚀

Release description:
Binary Update v1.1.24

What's new:
#2375 New Arwen version along with fixes for ESDT and relayed transaction processes:

• #2357 Added unit tests for the ESDT system smart contract.
• #2373 Improved the gas usage in case of relayed transactions. The unused gas will be sent back to the relayer itself.
• #2362 Implemented ESDT pause functionality. It will set a flag in all the shards to pause the transactions of a certain token.
• #2377 Added multiple unit tests for the ESDT feature.
• #2411 Created storage and cache for compiled smart contract code also called Ahead of Time (AoT) compiled storage.
• #2412 Implemented gas lock mechanism in system VM.
• #2416 Implemented ESDT view functions to the system smart contract.
• #2424 Integrated new Arwen version which has Ahead of Time compilation feature.
• #2430 Integrated new Arwen version that contain several fixes around the getCompiledCode functionality.
• #2448 Implemented gas schedule soft fork mechanism with the same style as epoch notifier. Every component registers to the gas schedule notifier and will get the complete new gas map when that is changed.
• #2467 Moved vm-common functionality to core package. Removed the dependencies to the old repository.
• #2470 Set the PrevTxHash field of `VMInput whenever instantiating it.
• #2455 Implemented ESDT ticker and the creation of new identifier.
• #2460 Added gas schedule tests.
• #2473 Chenaged the BLS verify gas cost. Updated existing tests.
• #2476 Added asynchronous callback with last output transfer functionality.
• #2482 Refactored paths and responses for ESDT related REST API endpoints.
• #2479 Integrated new Arwen version that supports dynamic gas locking mechanism.
• #2485 Added gas schedule notifier unit tests.
• #2488 Return empty response on the REST API endpoint when the token(s) data is not found.
• #2499 Implemented a fix for blockchain hook to return blocks from current epoch only. Otherwise different nodes might have different response and roothash in case some smart contracts use this API.
• #2494 Added integration tests for gas limit usage.
• #2500 Fixed a bug in the scProcessor.createSCRForSenderAndRelayer function related to the previous transaction hash.

#2593 Gas price modifier feature used when executing smart contracts in order to make smart contracts calls cheaper while preserving the ability of a shard to execute smart contract calls in the bounded time.

• #2571 Initial gas price modifier changes: boiler plate code that will be used in the existing structures. Added new parameters in the config files.
• #2572 Added gasPrice modifier to the protocol. Any transaction will consume only processing fee for extra gas on top of a move balance transaction cost.
• #2592 Fixed the gas used in SmartContractsResults and UserTransaction: make moveBalanceCost take into consideration SmartContractResults vs move balance transaction. SmartContractResult does not consume moveBalance.

#2370 Implemented staking v2 mechanism with top up feature, greater value of maximum number of nodes per shard and system smart contract delegation:

• #2392 Implemented rewards distribution at end of epoch and implemented claimRewards function for delegators. Added a set of unit tests and integration tests.
• #2405 Improved the delegation address creation method.
• #2393 Integrated stake top-up functionality: added getOwner function in staking SC and getTopUp functionality in auction SC, added rewardsStakingProvider that interface and fetches data from system VM in order to be used when computing rewards, added unit tests/semi-integrated tests
• #2410 Added transfer of deposit funds to the auction smart contract.
• #2418 Implemented view functions for delegation system smart contract.
• #2423 Added delegation system smart contract integration tests.
• #2429 Fixed a situation in which the reward computation could have panicked. Added an integration tests which tests the delegation reward computation.
• #2427 Added unit tests for the view functions of the delegation system smart contract.
• #2440 Delegation system smart contract internal audit & fixes.
• #2451 Added extra checks and test for adding and removing nodes from delegation.
• #2463 Removed duplicated code and implemented other small optimizations.
• #2438 Implemented top-up mechanism for variable stake per node: nodes still receive higher return (configurable) for the base stake (minimum nodes price), but now any extra stake can earn rewards as well. The rewards formula for any excess stake is not linear but asymptotic to a configurable maximum, and also the starting gradient is configurable. Backwards compatibility was implemented through a rewards creator proxy that delegates the functionality to the correct rewards model depending on the activation epoch.
• #2466 Optimized getValidatorData function in stakingDataProvider.go to call only one system SC function.
• #2389 Changed max nodes waiting list that will enable the maximum nodes in the network to be changed through a config change. Also the maximum number of nodes that can be swapped between eligible and waiting lists is also configurable. Both config options have been added in the same config structure as they will come together and are functionally connected. Implemented backwards compatibility components.
• #2491 Move nodes from queue to staked when max number of nodes is changed.
• #2477 Top up rewards enhancements: added the rewards creator proxy for the V1 and V2 economics, in order to cleanly separate the behavior of phase 3 staking implementation from the previous behavior. There are some fixes for the phase 3 protocol rewards and the addition of backwards compatibility.
• #2492 Added read only system smart contract container. Pushed system sc container from normal processing to SCQuery. Fixed registration data creation in auction smart contract when only staking value. Fixed scenario when delegation owner could retract initial owner funds before activating nodes, thus not presenting any skin in the game.
• #2504 Fixed the number of blocks considered at the end of epoch for rewards v2 (previously End of Epoch economics was computed after the rewards miniblocks on consensus validators while for block proposer it was computed before), use own computed economics for validation instead of the one coming from metablock header, reject negative adjustments for protocol sustainability.
• #2523 Made the delegate/undelegate process much more robust by separating the handling of nodes from those of the tokens.
• #2525 Renamed the auction smart contract to a more appropriate validator smart contract.
• #2528 Fixes after external audit on the system smart contract environment.
• #2531 Added new function for unStake and unBond with value and with nodes. Deleted the initial slashing implementation from staking smart contract as it needs to be implemented to validator smart contract later on.
• #2541 Implemented selection at the end of epoch.
• #2548 Added protection against mass unstake events that might cripple a shard in extreme situations.
• #2586 Cleanup of unused functions.
• #2569 Added some fixes for End of Epoch, e.g computation of nodes that are no longer qualified due to not enough stake. One other change fixes a wrong dirty trie check while processing a block.
• #2589 Fixed the merge of output accounts in case of system VM EEI, fixed the order and creation of SmartContractResults in case of staking smart contract.
• #2591 Added fix of checking already unstaked nodes because of jail.
• #2599 Fixed system smart contract processing at End of Epoch.
• #2603 Added unit tests for the new economics functions (GetTotalStakeEligibleNodes, GetTotalTopUpStakeEligibleNodes, GetNodeStakedTopUp, PrepareStakingDataForRewards, FillValidatorInfo)

#2273 Elasticsearch indexer improvements: modified how the mechanism that index data in elasticsearch database works. Now all method that want to write data in elasticsearch use a buffered chan and a gouroutine writes data that is stored in chan sequentially. Added the ability to index accounts. Added the ability to select which objects should be indexed.
#2390 Added the possibility to sign the transaction on the transaction hash rather than the serialized form of the transaction. This will enable wallet hardware devices to sign on large transactions. To be able to differentiate transactions (normal signed transactions and transactions signed hash) added a new field in transaction structure Options uint32 and if first octet from options has the value of 1 we consider the transaction is signed with hash, also transaction version must be 2 for transaction to be accepted.
#2471 Implemented the correct execution of the pending transactions in case of a hardfork event.
#2428 Added new storer events hashes by transaction hash: extend REST API endpoint route /transaction/:hash with a query parameter ?withResults=true in order to return also the results that are generated by a transaction.
#2490 Optimized the trie get all leaves process. Replaced GetAllLeaves() with GetAllLeavesOnChannel() throughout the whole project. The new implementation creates a new trie instance and runs getAllLeavesOnChannel(), meaning that once a node has been fully visited, it's children can become nil. This way, no matter how big the state is, it will not fill the memory.
#2527 Refactored keygenerator app: added the possibility to output data on the console (through the extra -console-out flag), create a single file with all the generated keys (based on type) (through the extra -no-split flag)
#2511 Refactored the post processor execution after a hardfork event: implemented a mechanism in Hardfork, which will execute recursively all post process transactions/mini blocks generated after import execution, until no new one will be generated. Only after execution of all post process transactions, the final block body and block header of the genesis hard fork block will be created, saved and pushed in the corresponding pools.
#2542 Added sync tries progress information prints.
#2517 Trie sync optimization: instead of keeping the whole trie in memory during the syncing process, and committing the trie when it is fully synced, only keep the collapsed root and commit each node individually as it is synced.
#2521 Added maximum number of shards for genesis nodes config. This will allow more flexible initial test-net nodes config.
#2532 Added real test examples for transaction correct construction.
#2558 Bump the Arwen version to v0.5.3 that support building wasmer on MacOs.
#2582 Added test examples for address conversion. Added BLS signing example.
#2588 Added the number of decimals when issuing a new ESDT token. It should be the 4th parameter, right after the initial supply. Also exported it in the view function.
#2600 Added integration tests that verify if the transaction processor works as exepected.
#2596 Optimizations of the sleep times in integration tests.
#2608 Optimized the process of setting the owners of the BLS public keys by calling the functionality on the staking smart contract directly. The update should be done in "bulk mode", improving by 10 times the performance of the update process.
#2613 Integrated Arwen RC1 to elrond-go.
#2610 Updated the scoring function to consider change in pricing for smart contract execution.
#2614 Added new integration tests. The new tests verify if the cross-shard transactions are executed as expected on the source shard and the destination shard.
#2625 Added unit tests for topup rewards and End of Epoch economics.
#2627 Added delegation system smart contract integration test with complex scenario.
#2626 Re-delegate rewards feature: added function to re-delegate rewards, added function to call unStake at end of epoch in case of system delegation smart contracts, added function the reStakeUnStakedNodes function.
#2634 Added test for the unqualified node situation.
#2638 Added unit tests for the system delegation smart contract view functions.
#2642 Added an integration test which deploys a SC, and calls that SC using an ESDT token.
#2640 Added unit tests for rewards with different inflation rates and accumulated fees and developer fees.
#2643 Added rewards top up gradient value in REST API endpoint route /network/config.
#2639 Improved code coverage for different packages / files.
#2644 Added top up factor in REST API endpoint route /network/config.
#2646 Added ESDT multisig integration test.
#2645 Added tests with ESDT and SC interaction intra and cross shard, added small fix in smart contract processor for gasRemaining, integrated new arwen fix.
#2637 Added new view functions in system delegation smart contract, added protection on unStaking / unBonding on all tokens when there are active nodes, take out warm instance from config, added integration tests for different scenarios.
#2654 Added integration tests in which a smart contract calls another smart contract using ESDT, but the second smart contract returns an error. All state changes should be reverted.
#2656 Allow the possibility to stake nodes after unstake.
#2657 Added scQueryServiceDispatcher functionality that can launch a specified number of VMs used in query service and will route each and every request to such generated SC query service in a round-robin fashion.
#2655 Added integration test for stake delegation simulation with time printing. This test was used in the early benchmark processes.
#2660 Made some functions deprecated in staking system smart contract and validator system smart contract.
#2662 Small improvement of return message in smart contract processor.
#2653 Added network/total-staked REST API endpoint route.
#2658 Added fee in transaction that is indexed in elasticsearch database. Added gasPriceModifer in the REST API endpoint route /network/config.
#2671 Made code a little less verbose in syncHeaderByHash.go.
#2669 Removed dead code from newSnapshotNode() function. There is no need to create a new trie with the new created node as the root node, instead just return the new node.
#2613 Integrated Arwen v1.0.1 to elrond-go.
#2687 Added a benchmark for getCodeEntry function from accountsDB.
#2693 Optimized LoadAccount function: separated getCode from loadAccount - code is only fetched from trie when it is needed. Added fixes on ESDT callbacks and return of values. Integrated new Arwen v1.0.2
#2702 Added some check when creating a transaction on the REST API endpoints. In this way, a transaction is accepted only if the lengths of the fields do not pass certain invariants.
#2690 Adjusted CryptoAPICost gas costs & AoT gas cost for the new gas model map.
#2675 Created a new test indexer for the semi-integration tests in order to be able to verify if the transaction fee and the gas used field is correctly calculated. This new indexer is used in the semi-integration tests.
#2712 Added more asynchrnous smart contracts call tests.
#2717 Optimized gas computation using new transaction type handler.
#2647 Added unit tests for scProcess - Deploy, Execute and some BuiltInFunctions.

Bug fixes:
#2598 Fixed a backwards compatibility bug related to transaction fees.
#2459 Fixed the transaction proto definition to ensure backwards compatibility. Changed 2 blacklist messages in interceptors
#2513 Fixed the resolvers creation in the hardfork process as to request the data on proper topics. Added more prints regarding the node status when doing the export process.
#2534 Fixed epoch start trigger when epoch start previous header is missing.
#2540 Fixed a log message. Added protection against staking same key two times in the same stake call.
#2533 Fixed some resolver-request issues: ixed reporting of missing transactions and miniblocks in resolvers to the debugging component, replaced function that gets the data from persister in miniblock resolver from Get to SearchFirst as to align with the rest of the resolvers, removed unused flags in main.go, fixed a wrong iteration on an int slice.
#2561 Fixed the problem with the gasUsed field of transactions that are indexed in elasticsearch.
#2562 Fixed accumulated fees for built in functions: built in functions consumes gas at sender only. Otherwise all gas should be used - gas refund was already resolved at the sender shard. The gas at the destination should not be added to devfees / accumulated fees.
#2567 Added check on vmInput and vmOutput for gasProvided and gas used. Smart contract results do not need to consume move balance gas - as that part was already taken out at the sender shard.
#2570 Added 2 fixes that will further improve trie synchronization when the node is in the bootstrap process: added dynamic timeouts when syncing trie nodes and made the trie syncer search in the trie database before requesting missing trie nodes.
#2557 Reduced the number checks on added SmartContractResult as those checks are not required.
#2607 Fixed the original sender encoding in nodeTransactions.go, prepareUnsignedTx function.
#2609 Fixed relevant issues found by linters. (GoLand built-in linters + GolangCI)
#2612 Fixed backwards compatibility issues: fixed a bug when saving validator data in validatorSaveLoad.go, fixed the situation when the genesis process constructs the system SC container that will lead into creating more accounts than expected on metachain, fixed a receipt bug related to gas consumption, made the developer rewards fully backwards compatible by having the old code into a new file, turned off the consensus watchdog when running in import-db mode.
#2617 Fixed a backwards compatibility issue when computing the transaction fee.
#2618 Fixed token name length condition on ESDT issuance.
#2620 Removed persister creation test from short tests.
#2619 Fix delegation manager deploy of another smart contract. Added invariants for balance delta to not be negative in case of cross shard accounts.
#2622 Fixed log debug prints on invalid transactions, fixed merging of accounts in case of execute on destination context on system VM, added checks for non-payable functions in delegation.go, added unit tests.
#2628 Fixed the situation when the waiting queue BLS key had to become eligible but the BLS key owner was not set, fixed the situation when more-than-expected waiting queue BLS keys became eligible, optimized some prints when displayValidatorsForRandomness was called.
#2629 Fixed a mismatch between validator statistics accumulation of block proposer fees and End of Epoch economics computation. In validator statistics, the block proposer fees were accumulated considering 10% out of (feesPerBlock - devFees) while the End of Epoch economics was considering the fee as 10% out of feesPerBlock.
#2630 Fixed the wrong owner key hex encoding, fixed the getUnStakedTokensList view function.
#2633 Fixed the wrong error return when trying to unstake an already unstaked node at End of Epoch.
#2632 Fixed compute unqualified nodes when there are no nodes to unstake - it can happen when someone unstake tokens but all of its nodes are in jail, optimize get total topup and total stake, set totUp if it is less than 0 to 0.
#2635 Fixed the problem when we have a cross-shard smart contract call with too much gas provided. The value of gasUsed field was wrong because on destination shard when the transaction is indexed in elasticsearch gasUsed field was not updated.
#2641 Fixed the problem with the gasUsed field of the relayed transactions that are indexed in the elasticseach database.
#2650 Fixed the return message in "too much gas provided" situation.
#2661 Fixed local testnet scripts.
#2664 Backwards compatibility fix on the metachain order of execution of some operations.
#2676 Fixed the use of an enable flag in Arwen.
#2677 Fixed the situation when an unBond operation call would have created a wrong entry in the peer state.
#2666 Fixed wrong gas used value for the fail cross-shard transaction (invalid sc call or sc deploy), fixed the formula that calculates gas used, fixed relayed transaction gas used that was not equal with the gas limit, fixed the invalid transaction receiver shard id that was not equal with the receiver address shard id.
#2670 Added missing continue statements on for loops in sync package.
#2667 Fixed a possible context leak on network messenger.
#2668 Fixed trusting Message.From that could cause denial of service. Added protection on received direct messages so that fromConnectedPeer should be equal to Message.From field.
#2681 Fixed delete from slice in the systemSCs.go file.
#2689 Fixed misleading info about number of txs processed for a cross shard miniblock when an error is occured ("time is out" or any other error).
#2695 Marked transactions with wrong username as invalid, added sender and receiver username functions on the node's REST API endpoints.
#2683 Deleted an extra save of the state in the epoch start metachain trigger that could prevent the easy restarting of the metachain nodes if the epoch start metablock could have not been poposed due to a possible bug.
#2705 Fixed gas refund to developer balance, added statistics prints for load code process in account adapter implementations.
#2709 Fixed backwards compatibility issue in adjustProtocolSustainabilityRewards function.
#2711 Fixed 2 backwards compatibility issues: removed unnecessary backwards compatibility flag from stakingToPeer as the v1.1.10 does not change the peer state on unBond operation and removed an unnecessary backwards compatibility from Arwen. Integrated Arwen v1.0.7 to elrond-go.
#2715 Fixed all log.LogIfError calls to contain an odd number of arguments so the messages will be displayed properly.
#2716 Renamed a file with incorrect name: errors..go -> errors.go.
#2718 Fixed max block size reached edge case for transactions with populated username.
#2720 Fixed max block size reached situation when more than one SmartContractResult is generated
#2728 Fixed gas computation on edge case. Added unit test.