Refactor library usage #2055

abandeali1 · 2019-08-11T22:23:32Z

Description

Refactors LibMath, LibFillResults, LibOrder, LibZeroExTransaction, and LibSafeMath to be actual libraries. Note that the old SafeMath still exists so that contracts that weren't yet refactored don't break. This can be removed in a separate PR.
Simplifies LibEIP712ExchangeDomain to only calculate the domain hash in its constructor and nothing else.
Moves calculateFillResults and calculateMatchedFillResults to LibFillResults. All functions in LibFillResults were also refactored to be pure. This refactor should probably be one of the focus areas of this review.
Removes all selector constants in favor of the IExchange(address(0)).fillOrder.selector pattern.
Refactors LibEIP712.hashEIP712Domain to use assembly. This was actually written pre-vuln, but might be a good discussion to have for all of the hashing functions.
Refactors contracts-gen to accept a compiler.json with no specified contracts. In this case, boilerplate for all contracts in the package will be generated. The contracts:gen command is also added into the build command for each package, so it no longer needs to be explicitly called.
Removes the _ prefix from internal function names in actual libraries.
Moves LibExchangeRichErrors to the exchange-libs package.
Random cleanup.

Testing instructions

Types of changes

Checklist:

Prefix PR title with [WIP] if necessary.
Add tests to cover changes as needed.
Update documentation as needed.
Add new entries to the relevant CHANGELOG.jsons.

coveralls · 2019-08-12T02:00:17Z

Coverage remained the same at 78.686% when pulling 8402d21 on feat/3.0/optimizeConstants into 434d027 on 3.0.

jalextowle

Awesome job on this one! I've got a couple of questions and nits, but after that it looks good to go.

jalextowle · 2019-08-12T17:12:30Z

contracts/coordinator/contracts/src/MixinCoordinatorApprovalVerifier.sol

-            selector == FILL_ORDER_SELECTOR ||
-            selector == FILL_ORDER_NO_THROW_SELECTOR ||
-            selector == FILL_OR_KILL_ORDER_SELECTOR
+            selector == IExchange(address(0)).fillOrder.selector ||


This change is 🔥. I'm so glad we're getting rid of LibExchangeSelectors.

jalextowle · 2019-08-12T17:18:48Z

contracts/dev-utils/contracts/src/LibAssetData.sol

+                tokenId
+            );
+
+            (bool success, bytes memory returnData) = tokenAddress.staticcall(ownerOfCalldata);


I'm wondering if it would make sense to make a staticcall library that you could use to abstract away some of this logic. For example, this staticcall could be written as:

balance = tokenAddress.staticcallReturnsAddress(ownerOfCalldata) == ownerAddress ? 1 : 0;

This is probably out of scope for this PR, but I think that it would be a good thing to add to the backlog.

jalextowle · 2019-08-12T17:23:26Z

contracts/dev-utils/contracts/src/OrderValidationUtils.sol

                    transferableMakerFeeAssetAmount,
                    makerFee,
                    takerAssetAmount
                );
-                transferableTakerAssetAmount = _min256(transferableMakerToTakerAmount, transferableMakerFeeToTakerAmount);
+                transferableTakerAssetAmount = LibSafeMath.min256(transferableMakerToTakerAmount, transferableMakerFeeToTakerAmount);


Why not use transferableMakerToTakerAmount.min256(transferableMakerFeeToTakerAmount)?

I felt like that was less readable, but I could go either way.

I don't have a huge preference either way.

jalextowle · 2019-08-12T17:24:22Z

contracts/dev-utils/contracts/src/OrderValidationUtils.sol

            }
        }

        // `fillableTakerAssetAmount` is the lower of the order's remaining `takerAssetAmount` and the `transferableTakerAssetAmount`
-        fillableTakerAssetAmount = _min256(
-            _safeSub(takerAssetAmount, orderInfo.orderTakerAssetFilledAmount),
+        fillableTakerAssetAmount = LibSafeMath.min256(


Same as above.

jalextowle · 2019-08-12T17:24:35Z

contracts/dev-utils/contracts/src/OrderValidationUtils.sol

@@ -171,7 +172,7 @@ contract OrderValidationUtils is
        returns (uint256 transferableAssetAmount)
    {
        (uint256 balance, uint256 allowance) = getBalanceAndAssetProxyAllowance(ownerAddress, assetData);
-        transferableAssetAmount = _min256(balance, allowance);
+        transferableAssetAmount = LibSafeMath.min256(balance, allowance);


Same as above.

jalextowle · 2019-08-12T17:43:51Z

contracts/exchange/contracts/src/MixinExchangeCore.sol

-        uint256 remainingTakerAssetAmount = _safeSub(order.takerAssetAmount, orderInfo.orderTakerAssetFilledAmount);
-        uint256 takerAssetFilledAmount = _min256(takerAssetFillAmount, remainingTakerAssetAmount);
+        uint256 remainingTakerAssetAmount = order.takerAssetAmount.safeSub(orderInfo.orderTakerAssetFilledAmount);
+        uint256 takerAssetFilledAmount = LibSafeMath.min256(takerAssetFillAmount, remainingTakerAssetAmount);


Same as earlier.

jalextowle · 2019-08-12T17:53:15Z

contracts/utils/contracts/src/LibEIP712.sol

-        ));
+        bytes32 schemaHash = _EIP712_DOMAIN_SEPARATOR_SCHEMA_HASH;
+
+        // Assembly for more efficient computing:


jalextowle · 2019-08-12T18:12:14Z

contracts/exchange-libs/contracts/src/LibFillResults.sol

+    /// @param fillResults1 The first FillResults.
+    /// @param fillResults2 The second FillResults.
+    /// @return The sum of both fill results.
+    function addFillResults(


I'm a fan of this not being a destructive function. I think that we should make this the norm (if it isn't already).

jalextowle · 2019-08-12T18:13:31Z

contracts/exchange-libs/contracts/src/LibFillResults.sol

+        //   If the left maker can buy exactly as much as the right maker can sell, then both orders are fully filled.
+        if (leftTakerAssetAmountRemaining > rightMakerAssetAmountRemaining) {
+            // Case 1: Right order is fully filled
+            matchedFillResults = _calculateCompleteRightFill(


Same as above. Really glad this is non-destructive now.

jalextowle · 2019-08-12T18:14:54Z

contracts/exchange-libs/contracts/src/LibFillResults.sol

+            // leftTakerAssetAmountRemaining == rightMakerAssetAmountRemaining
+            // Case 3: Both orders are fully filled. Technically, this could be captured by the above cases, but
+            //         this calculation will be more precise since it does not include rounding.
+            matchedFillResults = _calculateCompleteFillBoth(


hysz

This is solid! I'm happy we were able to fit this refactor into 3.0 (~‾▿‾)~

hysz · 2019-08-13T16:49:36Z

contracts/exchange-libs/contracts/src/LibExchangeSelectors.sol

-pragma solidity ^0.5.9;
-
-
-contract LibExchangeSelectors {


RIP level 💯

hysz · 2019-08-13T16:58:45Z

contracts/exchange-libs/contracts/test/TestLibFillResults.sol

+import "../src/LibFillResults.sol";
+
+
+contract TestLibFillResults {


We have a lot of these public-to-internal wrappers now. It could be worth looking into auto-generating them. This would also alleviate some of the code bloat. I've created a task on our backlog to look into this.

hysz · 2019-08-13T20:09:32Z

contracts/utils/contracts/src/LibSafeMath.sol

+import "./LibSafeMathRichErrors.sol";
+
+
+library LibSafeMath {


hysz · 2019-08-13T20:12:50Z

contracts/utils/contracts/src/LibEIP712.sol

+        //     uint256(verifyingContractAddress)
+        // ))
+
+        assembly {


This looks good to me ~ do you know roughly how much gas this saves?

I don't think it's super significant, maybe 1000 or so. Also note that this function only gets called once in the Exchange contract. I mostly did this for consistency pre-vuln since all of the other hashing is done in assembly, but it would probably be a good time to formalize what we want to do with all of the hashing as a whole.

dorothy-zbornak

Yup, these are all much cleaner changes! 💯
Some small requests then probably g2g after rebasing against 3.0.
Not approving yet because I wanna take another look after the rebase.

dorothy-zbornak · 2019-08-13T18:49:22Z

contracts/exchange-libs/package.json

@@ -35,7 +35,7 @@
        "generate-exchange-selectors": "node lib/scripts/generate-exchange-selectors.js ../../../exchange/generated-artifacts/Exchange.json ./contracts/src/LibExchangeSelectors.sol"


We can get rid of this and the associated script now. 🎉

dorothy-zbornak · 2019-08-13T19:20:52Z

contracts/exchange-libs/contracts/src/LibFillResults.sol

-contract LibFillResults is
-    SafeMath
-{
+library LibFillResults {


Digging this refactor.

dorothy-zbornak · 2019-08-13T19:28:58Z

contracts/exchange-libs/contracts/src/LibOrder.sol

-        public
-        view
+    function getOrderHash(Order memory order, bytes32 eip712ExchangeDomainHash)
+        internal


Hmm, I wonder if we should expose a getOrderHash() function in MixinExchangeCore to make up for this being turned into an internal library. getOrderInfo() might be overkill for some users.

I discussed this with Fabio and we agreed that we can just put this in the DevUtils contract. Do you think that is sufficient?

dorothy-zbornak · 2019-08-13T19:39:29Z

contracts/exchange-libs/contracts/src/LibZeroExTransaction.sol

-        public
-        view
+    function getTransactionHash(ZeroExTransaction memory transaction, bytes32 eip712ExchangeDomainHash)
+        internal


Same. Consider exposing getTransactionHash() in MixinTransactions.

dorothy-zbornak · 2019-08-13T19:46:36Z

contracts/exchange-libs/contracts/src/LibOrder.sol

        return orderHash;
    }

    /// @dev Calculates EIP712 hash of the order.
    /// @param order The order structure.
    /// @return EIP712 hash of the order.
-    function _hashOrder(Order memory order)
+    function hashOrder(Order memory order)


Maybe just rename this to getHash() now since order.hashOrder() is stuttering.

dorothy-zbornak · 2019-08-13T20:06:27Z

contracts/exchange-libs/test/lib_zero_ex_transaction.ts

+            env.provider,
+            env.txDefaults,
+        );
+        chainId = await providerUtils.getChainIdAsync(env.provider);


Again, getting a legitimate chainId doesn't seem to matter?

dorothy-zbornak · 2019-08-13T20:08:33Z

contracts/exchange/contracts/src/MixinExchangeCore.sol


        // Compute proportional fill amounts
-        fillResults = _calculateFillResults(order, takerAssetFilledAmount);
+        fillResults = LibFillResults.calculateFillResults(order, takerAssetFilledAmount);


dorothy-zbornak · 2019-08-13T20:12:46Z

contracts/exchange/contracts/test/TestValidatorWallet.sol

 import "@0x/contracts-erc20/contracts/src/interfaces/IERC20Token.sol";
 import "@0x/contracts-utils/contracts/src/LibBytes.sol";
 import "@0x/contracts-utils/contracts/src/LibEIP1271.sol";


-interface ISimplifiedExchange {


dorothy-zbornak · 2019-08-13T20:19:40Z

contracts/utils/contracts/test/TestConstants.sol

@@ -1,57 +0,0 @@
-/*


Didn't even know this contract existed.

dorothy-zbornak · 2019-08-13T20:21:00Z

packages/contracts-gen/src/contracts-gen.ts

-        }
-    });
-    compilerJSON.contracts = _.sortBy(compilerJSON.contracts);
+    if (compilerJSON.contracts !== undefined && compilerJSON.contracts !== ALL_CONTRACTS_IDENTIFIER) {


NOICE! 👍

…n hash

… contracts by default

…n hashing

dorothy-zbornak

Just one small nit then g2g! 👍 💃

dorothy-zbornak · 2019-08-15T22:56:55Z

contracts/exchange-libs/test/lib_fill_results.ts

+        makerFeePaid: constants.ZERO_AMOUNT,
+        takerFeePaid: constants.ZERO_AMOUNT,
+    };
+    const EMPTY_MATCHED_FILL_RESULTS: MatchedFillResults = {


Can we move the matched fill results stuff closer to the tests that use them?

…ized

abandeali1 requested a review from hysz as a code owner August 11, 2019 22:23

abandeali1 force-pushed the feat/3.0/optimizeConstants branch from d6cbff0 to c9f51f4 Compare August 11, 2019 22:53

abandeali1 changed the title ~~[WIP] Refactor library usage~~ Refactor library usage Aug 12, 2019

abandeali1 requested review from dorothy-zbornak and jalextowle August 12, 2019 06:34

abandeali1 mentioned this pull request Aug 12, 2019

@0x:contracts-exchange-libs Added unit tests to exchange-libs #2051

Merged

4 tasks

jalextowle approved these changes Aug 12, 2019

View reviewed changes

hysz approved these changes Aug 13, 2019

View reviewed changes

dorothy-zbornak suggested changes Aug 13, 2019

View reviewed changes

abandeali1 added 20 commits August 13, 2019 16:00

Optimize LibEIP712

2251e5e

Remove LibExchangeSelectors.sol

7d5276a

Remove LibAssetProxyIds and TestLibConstants

2e97cfa

Make remaining functions public

d153ac0

Remove remaining hard coded selectors

58ee444

Make LibEIP712 a library

65f17fd

Do not inherit libs

2427152

Move FillResults calculations into LibFillResults

b05a2a9

Cleanup imports

a8cd168

Add library implementation of SafeMath

52dcd99

Convert LibMath to library

6ca9d4e

Convert LibFillResults to library

28e781d

Simplify LibEIP712ExchangeDomain to be used only for generating domai…

74a5c8c

…n hash

Convert LibOrder and LibZeroExTransaction to libraries

f45014f

Update test contracts

7eb64eb

Update Exchange contract to use libraries

ae859fa

Update test contracts

9b786df

Fix compile errors

0df360c

Create test contracts for each exchange lib

245956c

Fix build errors in exchange package contracts

31c2b36

abandeali1 added 21 commits August 13, 2019 16:11

Fix build in OrderValidationUtils

2e519b5

Move all exchange-libs tests to separate files

24eaf93

Move calculateFillResults tests to exchange-libs package

5016d50

Fix linting errors

d6772b4

Move LibEIP712ExchangeDomain tests to own file

45f229c

Add rrevert to safeDiv in LibSafeMath

3afce21

Make function public method of Compiler class

88d055c

Get contracts list from Compiler class to add support for reading all…

fb7b51d

… contracts by default

Do not overwrite all contracts identifier in compiler.json

e91ba07

Compile all contracts by default, regenerate boilerplate

ccce7e0

Rename _rrevert to rrevert

0e2616f

Remove unused ABIEncoderV2 pragma

7cfceeb

Remove unimplemented functions from interface

26e4d66

Fix TestValidatorWallet by using new LibOrder and LibZeroExTransactio…

d3b8070

…n hashing

Move LibExchangeRichErrors to exchange-libs package

dc31294

Remove coordinator from CI tests

1aec5e4

Regenerate boilerplate for exchange-libs package

5543651

Fix build errors

070147d

Remove generate-exchange-selectors script

c5d4559

Hard code chainId in tests

2da996f

Rename hashing functions in LibOrder and LibZeroExTransaction

1dae1d2

abandeali1 force-pushed the feat/3.0/optimizeConstants branch from be5107f to 1dae1d2 Compare August 14, 2019 20:38

Merge branch '3.0' into feat/3.0/optimizeConstants

7ac30c5

dorothy-zbornak approved these changes Aug 15, 2019

View reviewed changes

abandeali1 added 2 commits August 15, 2019 16:51

Move MatchedFillResults constants closer to tests where they are util…

cc3be94

…ized

Update CHANGELOGs

8402d21

abandeali1 merged commit 4c78b7d into 3.0 Aug 16, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor library usage #2055

Refactor library usage #2055

abandeali1 commented Aug 11, 2019 •

edited

coveralls commented Aug 12, 2019 •

edited

jalextowle left a comment

jalextowle Aug 12, 2019

jalextowle Aug 12, 2019

jalextowle Aug 12, 2019

abandeali1 Aug 12, 2019

jalextowle Aug 12, 2019

jalextowle Aug 12, 2019

jalextowle Aug 12, 2019

jalextowle Aug 12, 2019

jalextowle Aug 12, 2019

jalextowle Aug 12, 2019

jalextowle Aug 12, 2019

jalextowle Aug 12, 2019

hysz left a comment

hysz Aug 13, 2019

hysz Aug 13, 2019

hysz Aug 13, 2019

hysz Aug 13, 2019

abandeali1 Aug 13, 2019

dorothy-zbornak left a comment

dorothy-zbornak Aug 13, 2019

dorothy-zbornak Aug 13, 2019

dorothy-zbornak Aug 13, 2019

abandeali1 Aug 13, 2019

dorothy-zbornak Aug 13, 2019

dorothy-zbornak Aug 13, 2019

dorothy-zbornak Aug 13, 2019

dorothy-zbornak Aug 13, 2019

dorothy-zbornak Aug 13, 2019

dorothy-zbornak Aug 13, 2019

dorothy-zbornak Aug 13, 2019

dorothy-zbornak left a comment

dorothy-zbornak Aug 15, 2019

		import "../src/LibFillResults.sol";


		contract TestLibFillResults {

		@@ -35,7 +35,7 @@
		"generate-exchange-selectors": "node lib/scripts/generate-exchange-selectors.js ../../../exchange/generated-artifacts/Exchange.json ./contracts/src/LibExchangeSelectors.sol"

Refactor library usage #2055

Refactor library usage #2055

Conversation

abandeali1 commented Aug 11, 2019 • edited

Description

Testing instructions

Types of changes

Checklist:

coveralls commented Aug 12, 2019 • edited

jalextowle left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hysz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dorothy-zbornak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dorothy-zbornak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abandeali1 commented Aug 11, 2019 •

edited

coveralls commented Aug 12, 2019 •

edited