A pass at upgradeability #75

yondonfu · 2017-09-21T20:02:08Z

ManagerProxy.sol contains the logic for forwarding method calls to target contracts such as BondingManager, JobsManager, etc. Target contract methods are invoked using delegatecall thereby allowing the target contract to execute methods and modify state in the context of the proxy contract
The file tests/ManagerProxy.js runs through a simple scenario of setting/getting storage types that are 32 bytes or less and swaps out an old version of a target contract with a new version
Added Minter.sol which owns the LivepeerToken contracts. When bonding or making deposits for jobs, tokens are transferred to this Minter contract. The Minter can mint new tokens if it is called by the BondingManager. The Minter can also transfer tokens in its possession to a recipient if called by the BondingManager or JobsManager
Register all contracts with the controller so that a user can always get the most up to date version of a contract
Minter.sol and LivepeerToken.sol are not proxy target contracts and are not meant to be upgradeable
Cleaned up the test suite a little bit so we actually have unit tests that rely on mocked contracts. I commented out the integration test for now because I'm still trying to figure out the best strategy for that - ideally we can run our integration tests on any chain/client and not rely solely on TestRPC features (i.e. mine a block per tx, ability to fast forward time/blocks)

I'm going to leave some clarifying comments, but there is a lot in this one so it might be easier to review by reading over it once and then we can sit down and run through it

…ndingManager/JobsManager

…est strategy

yondonfu · 2017-09-21T20:04:22Z

contracts/ManagerProxy.sol

+import "./ManagerProxyTarget.sol";
+
+
+contract ManagerProxy is ManagerProxyTarget {


The proxy contract inherits from the ManagerProxyTarget base contract to ensure that it starts off with the same storage layout as the target contract

yondonfu · 2017-09-21T20:06:20Z

contracts/bonding/BondingManager.sol

-        token = LivepeerToken(_token);
+    function BondingManager(address _controller) Manager(_controller) {}
+
+    function initialize(uint64 _unbondingPeriod, uint256 _numActiveTranscoders) external beforeInitialization returns (bool) {


When we instantiate a target contract using its constructor we update the storage of the target contract. However, we want to update the storage of the proxy contract. So, the logic of the constructor is now moved into this initialize function that the proxy contract can call and execute it its own context to modify the storage of the proxy contract.

yondonfu · 2017-09-21T20:07:07Z

contracts/bonding/BondingManager.sol

+    function BondingManager(address _controller) Manager(_controller) {}
+
+    function initialize(uint64 _unbondingPeriod, uint256 _numActiveTranscoders) external beforeInitialization returns (bool) {
+        finishInitialization();


It's important to call this to set the initialized flag (inherited from Initializable) to true so that the initialize method cannot be called more than once

yondonfu · 2017-09-21T20:10:28Z

contracts/bonding/BondingManager.sol

@@ -441,7 +430,7 @@ contract BondingManager is IBondingManager, Manager {
     * Returns address of elected active transcoder and its price per segment
     * @param _maxPricePerSegment Max price (in LPT base units) per segment of a stream
     */
-    function electActiveTranscoder(uint256 _maxPricePerSegment) external constant returns (address, uint256) {
+    function electActiveTranscoder(uint256 _maxPricePerSegment) external constant returns (address) {


I changed this to just return the address of the elected transcoder. Originally it returned the address of the elected transcoder and the transcoder's price. It was returning the price before because the default behavior was to use the transcoder's price which might be lower than the max price per segment set by a broadcaster when calling job from the JobsManager.

There were some points made here #55 that I thought made sense. Defaulting to the max price per segment set by a broadcaster would mean we do not need to return the transcoder's price here. This also helps with using a proxy contract to return values when using the BondingManager as a target contract - the proxy contract cannot return tuples right now. Since we are now just returning a single address the proxy contract can return that value just fine

Agree that just using the maxPricePerSegment is better since it's essentially the broadcaster indicating their willingness to pay a certain price and have a higher probability of being matched with an available transcoder.

yondonfu · 2017-09-21T20:11:28Z

contracts/bonding/BondingManager.sol

-        return initialTokenSupply.mul(initialYearlyInflation).div(100).div(roundsManager().roundsPerYear()).mul(transcoderActiveStake).div(totalActiveTranscoderStake);
+    // Transcoder getters
+
+    function getTranscoderDelegatorWithdrawRound(address _transcoder) public constant returns (uint256) {


We need these getters to individually retrieve values for a transcoder or delegator struct (the proxy contract cannot return tuples at the moment)

Any indication on the performance hit of querying individually for each field vs being able to return a struct directly? To populate the transcoder table with all it's accompanying data could take 100s of calls.

Not sure, I'll make a note to test this out. Since constant getters are using the eth_call RPC request for a node maybe we can batch together the RPC requests to make it faster

yondonfu · 2017-09-21T20:12:14Z

contracts/bonding/BondingManager.sol

@@ -613,6 +658,7 @@ contract BondingManager is IBondingManager, Manager {
     */
    function increaseTranscoderStake(address _transcoder, uint256 _totalAmount, uint256 _transcoderShare, uint256 _round) internal returns (bool) {
        delegators[_transcoder].bondedAmount = delegators[_transcoder].bondedAmount.add(_transcoderShare);
+        delegators[_transcoder].delegatedAmount = delegators[_transcoder].delegatedAmount.add(_totalAmount);


This was not updated properly before. When we increase a transcoder's stake we should not only increase the transcoder's stake in the transcoder pools, but also update the total amount delegated to it

yondonfu · 2017-09-21T20:13:15Z

contracts/jobs/JobsManager.sol


        return true;
    }

    /*
     * @dev Withdraw deposited funds
+     * FIXME: Attacker can withdraw funds before transcoder has a chance to claim them


Going to need logic to make sure this method is not used to withdraw funds before a transcoder has a chance to claim them. Will track with an issue

yondonfu · 2017-09-21T20:14:15Z

contracts/jobs/JobsManager.sol

        job.broadcasterAddress = msg.sender;
        job.transcoderAddress = electedTranscoder;

-        NewJob(electedTranscoder, msg.sender, numJobs);
+        NewJob(electedTranscoder, msg.sender, numJobs, _streamId, _transcodingOptions);


We currently cannot return streamId or transcodingOptions when using a proxy contract because the delegatecall mechanism cannot return dynamic length types (i.e. strings). Instead we can include these fields in events and a node could retrieve the values from the event

ericxtang · 2017-09-21T21:46:56Z

contracts/ManagerProxy.sol

+    }
+
+    // Based on https://github.com/AugurProject/augur-core/blob/develop/src/libraries/Delegator.sol
+    function() public payable {


Dose this make every call payable?

Discussed offline and the fallback function needs to be payable for the case when a user is calling the verify() function in JobsManager which is payable. If we really wanted to prevent anyone from accidentally sending ether to this contract except for when verify() is called, we could add a check for the method signature and if it is not the signature for verify() and msg.value > 0 we throw

dob

Overall it seems like this approach gives us the benefit of being able to easily upgrade contract logic, at the expense of hacking around solidity and the EVM in the following ways:

Requiring getters and a single read for each property of a struct.
Needing to read certain dynamic values from LOGs instead of the chain (which may go away with Byzantium).
Dropping down to assembly and delegating calls from proxy to target contracts. Not that this is explicity unsafe, but we're fiddling around with memory and addresses where we place call data and read return values.
Needing to keep storage values in the same order within contracts as we add new ones during upgrades. This is potentially prone to bugs, programmer error, or even breakage due to underlying tools or EVM updates.

I think that, especially due to this last point, the way to look at this is: it gives us the ability to upgrade if we need to using this mechanism for now...but we may not be able to rely on it for all cases and forever into the future.

dob · 2017-09-22T18:31:39Z

contracts/bonding/BondingManager.sol

@@ -441,7 +430,7 @@ contract BondingManager is IBondingManager, Manager {
     * Returns address of elected active transcoder and its price per segment
     * @param _maxPricePerSegment Max price (in LPT base units) per segment of a stream
     */
-    function electActiveTranscoder(uint256 _maxPricePerSegment) external constant returns (address, uint256) {
+    function electActiveTranscoder(uint256 _maxPricePerSegment) external constant returns (address) {


Agree that just using the maxPricePerSegment is better since it's essentially the broadcaster indicating their willingness to pay a certain price and have a higher probability of being matched with an available transcoder.

dob · 2017-09-22T18:33:12Z

contracts/bonding/BondingManager.sol

-        return initialTokenSupply.mul(initialYearlyInflation).div(100).div(roundsManager().roundsPerYear()).mul(transcoderActiveStake).div(totalActiveTranscoderStake);
+    // Transcoder getters
+
+    function getTranscoderDelegatorWithdrawRound(address _transcoder) public constant returns (uint256) {


Any indication on the performance hit of querying individually for each field vs being able to return a struct directly? To populate the transcoder table with all it's accompanying data could take 100s of calls.

dob · 2017-09-22T18:35:06Z

contracts/bonding/BondingManager.sol

-        // Mint token reward and allocate to this protocol contract
-        token.mint(this, mintedTokens);
+        // Mint token reward
+        uint256 mintedTokens = minter().mint(activeTranscoders[activeTranscoderPositions[msg.sender]].key, totalActiveTranscoderStake);


what's the reason for getting rid of mintedTokensPerReward()?

With the addition of the Minter contract I thought it might make sense to move the logic for calculating how many new tokens to mint into the Minter contract. Now, the BondingManager tells the Minter what the current transcoder's stake and the total active transcoder stake is. The Minter then uses that information to decide how many tokens to mint.

yondonfu · 2017-09-22T21:09:36Z

Agreed on how to look at the upgradeability related additions in this PR. I think it makes sense to have a working version of this approach in this codebase for now and press on with other tasks knowing that we have a basic upgradeability solution to work with. And perhaps we will get more information in time (i.e. feedback, Byzantium on the mainnet, Solidity development) that changes how we look at this solution (for better or for worse)

yondonfu added 9 commits September 19, 2017 11:49

Cleaner controller interface. Added proxy contract

34b7f74

Initializable proxy target contracts

0572f53

Tests for ManagerProxy and Controller

050d996

Additional mock contracts. Remaining unit tests to be cleaned up = Bo…

85422d5

…ndingManager/JobsManager

Fixed JobsManager tests

33b7791

Fixed BondingManager tests

a022265

Added back missing Faucet contract. Fix Minter tests. Fix solium errors

e183919

Comment out integration test for not while figuring out integration t…

7c4cd74

…est strategy

Fixed migration script

a6ee2e5

yondonfu requested review from dob and ericxtang September 21, 2017 20:02

yondonfu commented Sep 21, 2017

View reviewed changes

Update migration config for verifier

48777e7

ericxtang reviewed Sep 21, 2017

View reviewed changes

dob reviewed Sep 22, 2017

View reviewed changes

yondonfu mentioned this pull request Sep 22, 2017

Additional restrictions on how many tokens the Minter can mint in a round #77

Closed

dob mentioned this pull request Sep 25, 2017

Moving tokens internally across contracts #56

Closed

yondonfu merged commit 70e22b0 into develop Sep 28, 2017

yondonfu deleted the yf/upgradeability branch September 28, 2017 15:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

A pass at upgradeability #75

A pass at upgradeability #75

yondonfu commented Sep 21, 2017

yondonfu Sep 21, 2017

yondonfu Sep 21, 2017

yondonfu Sep 21, 2017

yondonfu Sep 21, 2017

dob Sep 22, 2017

yondonfu Sep 21, 2017

dob Sep 22, 2017

yondonfu Sep 22, 2017

yondonfu Sep 21, 2017

yondonfu Sep 21, 2017

yondonfu Sep 21, 2017

ericxtang Sep 21, 2017

yondonfu Sep 22, 2017

dob left a comment

dob Sep 22, 2017

dob Sep 22, 2017

dob Sep 22, 2017

yondonfu Sep 22, 2017

yondonfu commented Sep 22, 2017

		import "./ManagerProxyTarget.sol";


		contract ManagerProxy is ManagerProxyTarget {

A pass at upgradeability #75

A pass at upgradeability #75

Conversation

yondonfu commented Sep 21, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dob left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yondonfu commented Sep 22, 2017